This error occurs when a Buffer is converted to a string using an encoding that differs from the original encoding used to create the Buffer. Node.js supports specific encodings (utf8, hex, base64, etc.), and mismatches produce garbled output or replacement characters.
This error indicates that a Buffer object is being decoded to a string using an incorrect character encoding. In Node.js, Buffers represent raw binary data and must be explicitly converted to strings using a character encoding like 'utf8', 'hex', 'base64', 'ascii', or 'utf16le'. When the encoding specified during the toString() conversion doesn't match the encoding used when the data was originally encoded, you get either garbled text, Unicode replacement characters (�), or in some cases an explicit error. Node.js only supports a limited set of encodings natively. If you attempt to use an unsupported encoding like 'latin1', 'ISO-8859-1', or 'shift_jis' without additional libraries, Node.js will either throw an "Unknown encoding" error or fall back to UTF-8, causing the mismatch. The most common scenario is reading file data with fs.readFile() and assuming UTF-8 encoding when the file was actually saved in a different encoding, or receiving binary data (like images or encrypted content) and attempting to convert it to a string without specifying the correct encoding format.
First, determine what encoding your data actually uses. For files, check the file properties or how it was originally saved. For network data, check response headers or API documentation.
Common encodings:
- Text files: usually 'utf8'
- Binary data (images, PDFs): should not be converted to string
- Base64 encoded data: use 'base64'
- Hexadecimal data: use 'hex'
You can verify supported encodings:
const encoding = 'latin1';
if (Buffer.isEncoding(encoding)) {
console.log('Supported encoding');
} else {
console.log('Unsupported - need external library');
}When using fs.readFile() or fs.readFileSync(), always explicitly specify the encoding if you want a string:
const fs = require('fs');
// Correct - specify encoding
fs.readFile('data.txt', 'utf8', (err, data) => {
if (err) throw err;
console.log(data); // String with correct encoding
});
// For non-UTF8 files with iconv-lite
const iconv = require('iconv-lite');
fs.readFile('legacy.txt', (err, buffer) => {
if (err) throw err;
const text = iconv.decode(buffer, 'windows-1252');
console.log(text);
});If you omit the encoding parameter, you'll get a Buffer object instead of a string, which is safer for binary data.
When converting Buffers to strings manually, always specify the encoding:
// Wrong - defaults to utf8
const buffer = Buffer.from('aGVsbG8gd29ybGQ=', 'base64');
const wrong = buffer.toString(); // Garbled output
// Correct - specify encoding
const buffer = Buffer.from('aGVsbG8gd29ybGQ=', 'base64');
const correct = buffer.toString('utf8'); // "hello world"
// For hex encoding
const hexBuffer = Buffer.from('68656c6c6f', 'hex');
const text = hexBuffer.toString('utf8'); // "hello"Match the encoding to your data type: 'utf8' for text, 'base64' for base64 data, 'hex' for hexadecimal, etc.
If you need encodings beyond Node.js native support (like latin1, shift_jis, ISO-8859-1), use the iconv-lite package:
npm install iconv-liteThen use it to decode Buffers:
const iconv = require('iconv-lite');
// Reading a file with legacy encoding
const fs = require('fs');
const buffer = fs.readFileSync('legacy.txt');
const text = iconv.decode(buffer, 'ISO-8859-1');
console.log(text);
// Encoding text to a specific format
const encoded = iconv.encode('Hello', 'win1252');
fs.writeFileSync('output.txt', encoded);The iconv-lite library supports 450+ encodings and is widely used in the Node.js ecosystem.
Ensure consistency when encoding and decoding data:
// Correct - same encoding throughout
const original = 'Hello World';
const encoded = Buffer.from(original, 'utf8');
const decoded = encoded.toString('utf8');
console.log(decoded === original); // true
// Wrong - encoding mismatch
const text = 'Hello';
const buffer = Buffer.from(text, 'utf8');
const wrong = buffer.toString('base64'); // "SGVsbG8="
const garbled = Buffer.from(wrong, 'utf8').toString('utf8'); // Wrong!
// Correct round-trip
const correctDecoded = Buffer.from(buffer.toString('base64'), 'base64').toString('utf8');
console.log(correctDecoded === text); // trueBinary Data Handling: Never convert binary data (images, PDFs, executables) to strings. Always keep them as Buffers and write them directly to files or streams. Converting binary data to strings with any encoding will corrupt the data.
UTF-8 Validation: When decoding Buffers that may contain invalid UTF-8 sequences, Node.js replaces invalid bytes with the Unicode replacement character U+FFFD (�). This is silent behavior - no error is thrown. If you see replacement characters, your data likely wasn't UTF-8 encoded.
Default Encoding Behavior: If you call Buffer.toString() without arguments, it defaults to 'utf8'. Similarly, fs.readFile() without an encoding option returns a Buffer, not a string. Being explicit about encodings prevents subtle bugs.
Encoding Case Sensitivity: Node.js accepts encoding strings in any case: 'utf8', 'UTF8', 'Utf8', 'UTF-8' are all valid and equivalent. However, for consistency and standards compliance, use lowercase versions.
BOM (Byte Order Mark): Some UTF-8 files include a BOM (EF BB BF bytes at the start). Node.js doesn't automatically strip this. If you see weird characters at the start of files, check for and remove the BOM: text.replace(/^\uFEFF/, '').
Stream Encoding: When working with streams, set encoding on the stream itself rather than converting chunks manually: readStream.setEncoding('utf8'). This handles multi-byte characters that might be split across chunk boundaries.
Error: Listener already called (once event already fired)
EventEmitter listener already called with once()
Error: EACCES: permission denied, open '/root/file.txt'
EACCES: permission denied
Error: Invalid encoding specified (stream encoding not supported)
How to fix Invalid encoding error in Node.js readable streams
Error: EINVAL: invalid argument, open
EINVAL: invalid argument, open
TypeError: readableLength must be a positive integer (stream config)
TypeError: readableLength must be a positive integer in Node.js streams