This error occurs when the operating system runs out of buffer space for network operations, typically during heavy socket operations or large data transfers. It indicates system-level resource exhaustion rather than a code-level bug.
The ENOBUFS error (errno 105) means "No buffer space available" and is raised when the operating system can no longer allocate buffer space for incoming or outgoing network messages. This is a system-level error that occurs when the kernel fails to allocate necessary internal memory for socket operations. This error commonly appears during network-intensive operations such as writing large amounts of data through sockets, maintaining many concurrent connections, or performing high-volume HTTP requests. The operating system has finite buffer space for network operations, and when this space is exhausted, it cannot accept new data until existing buffers are cleared. Unlike application-level memory issues, ENOBUFS specifically relates to kernel-level networking buffers. The error typically manifests during syscalls like write(), send(), or connect() when the system's networking subsystem has reached its capacity limits.
Node.js streams provide backpressure signals that indicate when to pause writing. Always check the return value of write() operations:
const stream = getWritableStream();
function writeData(data) {
const canContinue = stream.write(data);
if (!canContinue) {
// Buffer is full, wait for drain event
stream.once('drain', () => {
console.log('Buffer drained, can write more');
// Continue writing
});
}
}For large data transfers, break them into smaller chunks:
const CHUNK_SIZE = 64 * 1024; // 64KB chunks
function writeLargeData(socket, buffer) {
let offset = 0;
function writeChunk() {
while (offset < buffer.length) {
const chunk = buffer.slice(offset, offset + CHUNK_SIZE);
const canContinue = socket.write(chunk);
offset += CHUNK_SIZE;
if (!canContinue) {
socket.once('drain', writeChunk);
return;
}
}
socket.end();
}
writeChunk();
}On Linux, increase kernel networking buffer sizes by editing /etc/sysctl.conf:
# Backup current configuration
sudo cp /etc/sysctl.conf /etc/sysctl.conf.backup
# Edit the file
sudo nano /etc/sysctl.confAdd or modify these parameters:
# Maximum socket receive buffer (16MB)
net.core.rmem_max = 16777216
# Maximum socket send buffer (16MB)
net.core.wmem_max = 16777216
# Default socket receive buffer (256KB)
net.core.rmem_default = 262144
# Default socket send buffer (256KB)
net.core.wmem_default = 262144
# TCP read buffer sizes: min, default, max
net.ipv4.tcp_rmem = 4096 87380 16777216
# TCP write buffer sizes: min, default, max
net.ipv4.tcp_wmem = 4096 65536 16777216Apply the changes:
sudo sysctl -pVerify the new values:
sysctl net.core.wmem_max net.core.rmem_maxSet socket buffer sizes explicitly in your Node.js application:
const net = require('net');
const socket = new net.Socket();
// Increase send buffer to 1MB
socket.setNoDelay(true);
socket.bufferSize = 1024 * 1024;
// For HTTP agents, configure socket settings
const http = require('http');
const agent = new http.Agent({
keepAlive: true,
maxSockets: 50, // Limit concurrent connections
maxFreeSockets: 10,
timeout: 60000,
});
// Use the agent in requests
http.get({
hostname: 'example.com',
agent: agent
}, (res) => {
// Handle response
});For socket servers, set buffer sizes on accepted connections:
const server = net.createServer((socket) => {
// Set buffer sizes for each connection
socket.setKeepAlive(true);
socket.setNoDelay(true);
// Handle socket events
socket.on('error', (err) => {
if (err.code === 'ENOBUFS') {
console.error('Buffer exhausted, implementing backoff');
// Implement retry logic
}
});
});Limit concurrent connections to prevent buffer exhaustion:
const http = require('http');
// Configure agent with connection limits
const agent = new http.Agent({
keepAlive: true,
maxSockets: 50, // Max concurrent connections per host
maxTotalSockets: 100, // Max total connections
maxFreeSockets: 10, // Max idle connections
scheduling: 'lifo', // Reuse recent connections
});
// Use with fetch or http requests
const options = {
agent: agent,
timeout: 30000,
};Implement request queuing to control traffic:
class RequestQueue {
constructor(concurrency = 10) {
this.concurrency = concurrency;
this.running = 0;
this.queue = [];
}
async add(fn) {
while (this.running >= this.concurrency) {
await new Promise(resolve => this.queue.push(resolve));
}
this.running++;
try {
return await fn();
} finally {
this.running--;
const next = this.queue.shift();
if (next) next();
}
}
}
const queue = new RequestQueue(10);
// Use the queue
async function makeRequest(url) {
return queue.add(() => fetch(url));
}ENOBUFS can occur if file descriptor limits are too low. Check current limits:
# Check soft limit
ulimit -n
# Check hard limit
ulimit -HnIncrease limits temporarily:
ulimit -n 65536For permanent changes, edit /etc/security/limits.conf:
* soft nofile 65536
* hard nofile 65536For systemd services, add to your service file:
[Service]
LimitNOFILE=65536Edit /etc/sysctl.conf to increase system-wide file limit:
fs.file-max = 2097152Apply changes:
sudo sysctl -pImplement graceful error handling for ENOBUFS errors:
async function writeWithRetry(socket, data, maxRetries = 3) {
let retries = 0;
let delay = 100; // Start with 100ms delay
while (retries < maxRetries) {
try {
return await new Promise((resolve, reject) => {
const canWrite = socket.write(data, (err) => {
if (err) reject(err);
else resolve();
});
if (!canWrite) {
socket.once('drain', resolve);
} else {
resolve();
}
});
} catch (err) {
if (err.code === 'ENOBUFS' && retries < maxRetries - 1) {
console.warn(`ENOBUFS error, retry ${retries + 1}/${maxRetries}`);
await new Promise(resolve => setTimeout(resolve, delay));
delay *= 2; // Exponential backoff
retries++;
} else {
throw err;
}
}
}
throw new Error('Max retries exceeded for ENOBUFS');
}
// Usage
socket.on('error', (err) => {
if (err.code === 'ENOBUFS') {
console.error('Buffer space exhausted:', err.message);
// Implement recovery strategy
}
});Buffer Size Considerations: The optimal socket buffer size depends on your application's needs. For high-throughput applications transferring large files, larger buffers (4-16MB) reduce overhead. For applications with many small messages, smaller buffers with more frequent writes may be more efficient. Monitor your application's memory usage when increasing buffer sizes.
Platform Differences: Windows and macOS have different default buffer limits and tuning mechanisms. On macOS, use sysctl -a | grep kern.ipc to view socket buffer limits. Windows users should adjust socket buffer sizes programmatically as kernel tuning is more limited.
Monitoring Buffer Usage: Use tools like ss -m (socket statistics) on Linux to monitor socket buffer usage in real-time. This can help identify which connections are consuming the most buffer space and whether your tuning changes are effective.
HTTP/2 and HTTP/3: These protocols have built-in flow control mechanisms that help prevent buffer exhaustion. Consider upgrading from HTTP/1.1 if you frequently encounter ENOBUFS errors with HTTP traffic.
Container Environments: In Docker or Kubernetes, ensure that container resource limits (memory and network) are sufficient. Container orchestrators may impose additional buffer limits beyond system defaults. You may need to adjust both host system settings and container resource allocations.
Error: EMFILE: too many open files, watch
EMFILE: fs.watch() limit exceeded
Error: Middleware next() called multiple times (next() invoked twice)
Express middleware next() called multiple times
Error: Worker failed to initialize (worker startup error)
Worker failed to initialize in Node.js
Error: EMFILE: too many open files, open 'file.txt'
EMFILE: too many open files
Error: cluster.fork() failed (cannot create child process)
cluster.fork() failed - Cannot create child process