Node.js FS Module : Files, Directories, Streams

Hey everyone,

In this article, we’re going to dive deep into the fs module in Node.js and explore its capabilities in detail. The fs module, short for File System, is a built-in module in Node.js that provides a range of functionalities to work with files and directories. Whether it’s reading and writing files, managing directories, or even dealing with file metadata, the fs module is a core utility for building file-handling applications.

With fs, you can perform both synchronous and asynchronous file operations, but for better performance and cleaner code, we’ll focus on the asynchronous (promisified) versions. These allow us to use the modern async/await syntax, making file operations more intuitive and error handling much simpler.

Let’s walk through the fs module step by step and uncover its power!

---

Getting Started: Setting Up

First, import the fs/promises module. Since it's built into Node.js, there's no need to install anything extra:

import { promises as fs } from 'fs';

For file watching, you'll use the classic fs module instead of the promises API:

import { watch } from 'fs';

---

File Operations

Reading Files

Reading the content of a file is one of the most basic operations. Use fs.readFile for asynchronous reading:

const readFile = async (fileName: string): Promise<void> => {
    try {
        const data = await fs.readFile(fileName, 'utf8');
        console.log('File content:', data);
    } catch (err) {
        console.error('Error reading file:', err);
    }
};

readFile('example.txt');

• Parameters:

  • fileName: The name or path of the file to read.

  • utf8: Specifies that the file should be read as a UTF-8 encoded string.

Writing Files

To write data to a file, use fs.writeFile. This overwrites the file if it already exists:

const writeFile = async (fileName: string, content: string): Promise<void> => {
    try {
        await fs.writeFile(fileName, content);
        console.log('File written successfully!');
    } catch (err) {
        console.error('Error writing to file:', err);
    }
};

writeFile('example.txt', 'Hello, Node.js!');

Appending to Files

To add content to an existing file, use fs.appendFile:

const appendFile = async (fileName: string, additionalContent: string): Promise<void> => {
    try {
        await fs.appendFile(fileName, additionalContent);
        console.log('Content appended successfully!');
    } catch (err) {
        console.error('Error appending to file:', err);
    }
};

appendFile('example.txt', '\nAppending this line.');

Deleting Files

To delete a file, use fs.unlink:

const deleteFile = async (fileName: string): Promise<void> => {
    try {
        await fs.unlink(fileName);
        console.log('File deleted successfully!');
    } catch (err) {
        console.error('Error deleting file:', err);
    }
};

deleteFile('example.txt');

---

Directory Operations

Creating Directories

Use fs.mkdir to create directories. The { recursive: true } option allows nested directories to be created in one go:

const createDirectory = async (): Promise<void> => {
    try {
        await fs.mkdir('newDir', { recursive: true });
        console.log('Directory created successfully!');
    } catch (err) {
        console.error('Error creating directory:', err);
    }
};

createDirectory();

Reading Directories

To list the contents of a directory, use fs.readdir:

const readDirectory = async (): Promise<void> => {
    try {
        const files = await fs.readdir('newDir');
        console.log('Directory contents:', files);
    } catch (err) {
        console.error('Error reading directory:', err);
    }
};

readDirectory();

Removing Directories

To delete a directory, use fs.rmdir. Again, { recursive: true } allows removing non-empty directories:

const removeDirectory = async (): Promise<void> => {
    try {
        await fs.rmdir('newDir', { recursive: true });
        console.log('Directory removed successfully!');
    } catch (err) {
        console.error('Error removing directory:', err);
    }
};

removeDirectory();

---

File Metadata

You can access metadata about files and directories using fs.stat. This provides details like size, creation time, and modification time.

const getFileStats = async (fileName: string): Promise<void> => {
    try {
        const stats = await fs.stat(fileName);
        console.log('File Stats:', stats);
    } catch (err) {
        console.error('Error getting file stats:', err);
    }
};

getFileStats('example.txt');

---

Watching Files

To monitor changes to a file or directory, use fs.watch:

const watchFile = (): void => {
    watch('example.txt', (eventType, filename) => {
        if (filename) {
            console.log(`File ${filename} changed: ${eventType}`);
        }
    });
};

watchFile();

This logs changes such as modifications, deletions, or renaming.

---

Working with File Streams

For scenarios involving large files, where reading or writing the entire file at once would be inefficient or memory-intensive, streams provide a powerful solution. Streams process data chunk by chunk, making them ideal for efficient file operations.

Reading with Streams

The createReadStream method allows you to read large files piece by piece:

import { createReadStream } from 'fs';

const readStream = createReadStream('largeFile.txt', 'utf8');

readStream.on('data', (chunk) => {
    console.log('Chunk received:', chunk);
});

readStream.on('end', () => {
    console.log('Finished reading the file.');
});

readStream.on('error', (err) => {
    console.error('Error reading file:', err);
});

• Events:

  • data: Triggered when a chunk of data is available.

  • end: Triggered when the stream finishes reading.

  • error: Triggered if an error occurs during the read operation.

Writing with Streams

Similarly, the createWriteStream method is used for writing data incrementally:

import { createWriteStream } from 'fs';

const writeStream = createWriteStream('output.txt');

writeStream.write('Writing to the file in chunks.\n');
writeStream.end('This is the end of the stream.');

writeStream.on('finish', () => {
    console.log('Finished writing to the file.');
});

writeStream.on('error', (err) => {
    console.error('Error writing to the file:', err);
});

• Events:

  • finish: Triggered when the writing operation is completed.

  • error: Triggered if an error occurs during the write operation.

Piping Streams

The pipe method is one of the most powerful features of streams, enabling you to directly connect the output of one stream as the input to another. This is particularly useful for operations like copying files or processing data as it flows.

Example: Copying a File Using Streams

import { createReadStream, createWriteStream } from 'fs';

const sourceFile = 'largeFile.txt';
const destinationFile = 'copiedFile.txt';

const readStream = createReadStream(sourceFile);
const writeStream = createWriteStream(destinationFile);

readStream.pipe(writeStream);

writeStream.on('finish', () => {
    console.log('File copied successfully using streams!');
});

writeStream.on('error', (err) => {
    console.error('Error during file copying:', err);
});

Explanation:

• readStream.pipe(writeStream) transfers data from the read stream to the write stream.

• Data flows seamlessly from the source to the destination without loading the entire file into memory.

Transform Streams

Transform streams allow you to modify or transform data as it passes through. For example, you could compress a file while reading it using the zlib module:

Example: Compressing a File

import { createReadStream, createWriteStream } from 'fs';
import { createGzip } from 'zlib';

const sourceFile = 'largeFile.txt';
const compressedFile = 'largeFile.txt.gz';

const readStream = createReadStream(sourceFile);
const writeStream = createWriteStream(compressedFile);
const gzip = createGzip();

readStream.pipe(gzip).pipe(writeStream);

writeStream.on('finish', () => {
    console.log('File compressed successfully!');
});

writeStream.on('error', (err) => {
    console.error('Error during file compression:', err);
});

• createGzip: Creates a transform stream that compresses data using Gzip.

• The pipeline connects the read stream, compression stream, and write stream.

Using Stream Pipelines (Recommended for Complex Operations)

For better error handling and managing complex stream workflows, Node.js provides the pipeline utility:

import { createReadStream, createWriteStream } from 'fs';
import { createGzip } from 'zlib';
import { pipeline } from 'stream/promises';

const compressFile = async (source: string, destination: string): Promise<void> => {
    try {
        await pipeline(
            createReadStream(source),
            createGzip(),
            createWriteStream(destination)
        );
        console.log('File compressed successfully using pipeline!');
    } catch (err) {
        console.error('Pipeline error:', err);
    }
};

compressFile('largeFile.txt', 'largeFile.txt.gz');

• Advantages of pipeline:

  • Simplified error handling.

  • Cleaner and more maintainable code for complex stream workflows.

By leveraging the flexibility and efficiency of streams, you can handle large-scale file operations and even integrate data processing seamlessly. Whether you're copying, compressing, or transforming files, streams are your go-to solution in Node.js.

---

Copying and Renaming Files

Copying Files

Use fs.copyFile to copy files:

const copyFile = async (source: string, destination: string): Promise<void> => {
    try {
        await fs.copyFile(source, destination);
        console.log('File copied successfully!');
    } catch (err) {
        console.error('Error copying file:', err);
    }
};

copyFile('example.txt', 'copy.txt');

Renaming Files

To rename or move a file, use fs.rename:

const renameFile = async (oldPath: string, newPath: string): Promise<void> => {
    try {
        await fs.rename(oldPath, newPath);
        console.log('File renamed successfully!');
    } catch (err) {
        console.error('Error renaming file:', err);
    }
};

renameFile('copy.txt', 'renamed.txt');

---

Conclusion

The fs module in Node.js is incredibly versatile, offering everything you need to manage files and directories. By using the promises API, we can write cleaner and more maintainable code with async/await.

Whether you’re building a file manager, logging system, or data processor, the fs module is your trusted companion. Start experimenting with these functions, and happy coding!

---

Feel free to let me know if you'd like additional examples or clarifications! 😊