Buffer & Streams

Buffers and Streams are what make Node.js so efficient at handling large amounts of data, such as video processing or reading massive log files.

1. Buffers

A Buffer is a temporary storage spot for a chunk of data that is being moved from one place to another. Essential for handling binary data (images, video, etc.).

// Create a buffer from a string
const buf = Buffer.from('Hello');

console.log(buf); // <Buffer 48 65 6c 6c 6f>
console.log(buf.toJSON()); // { type: 'Buffer', data: [ 72, 101, 108, 108, 111 ] }
console.log(buf.toString()); // Hello

// Create an empty buffer of 10 bytes
const emptyBuf = Buffer.alloc(10);

2. Streams

Streams are objects that let you read data from a source or write data to a destination in a continuous fashion. Instead of reading an entire 2GB file into memory (which would crash your app), you read it piece by piece.

Types of Streams:

• Readable: Used for read operations (e.g., fs.createReadStream).
• Writable: Used for write operations (e.g., fs.createWriteStream).
• Duplex: Can be used for both read and write.
• Transform: A type of duplex stream where the output is computed based on input.

const fs = require('fs');

const readableStream = fs.createReadStream('./largeFile.txt', { encoding: 'utf8' });
const writableStream = fs.createWriteStream('./copy.txt');

// Pipe the read data directly to the write stream (Highly Efficient)
readableStream.pipe(writableStream);

readableStream.on('data', (chunk) => {
  console.log('Received chunk of data...');
});