Node.js In-Depth
Module A-4·22 min read

The raw network stack: how a transaction payload travels from NIC to Node.js runtime boundary via epoll and libuv.

Written by Jatin Jain Saraf · Architect of SupraScan

Module 3 — Kernel-Level I/O Multiplexing: epoll, kqueue, IOCP

What this module covers: Node.js's non-blocking I/O is not magic — it is a precise coordination between libuv and OS-level kernel interfaces. epoll on Linux, kqueue on macOS/BSD, and IOCP on Windows are the actual mechanisms that make thousands of concurrent connections possible without thousands of threads. Understanding them at the system call level lets you predict Node.js network behavior under load, tune kernel parameters correctly, and diagnose failures that are invisible from the JavaScript layer. This module traces a transaction payload from the network interface card to your callback with no hand-waving.

Why Kernel I/O Interfaces Matter

When a Node.js blockchain indexer handles 50,000 concurrent TCP connections from blockchain full nodes, each pushing new block data, two questions matter:

  1. How does the OS tell Node.js that one of those 50,000 connections has new data ready to read?
  2. How does Node.js read that data without blocking on connections that have no data?

The answer to both questions is the I/O multiplexer — a kernel interface that watches many file descriptors simultaneously and notifies the application when any of them are ready.

The three implementations differ in important ways. Knowing which one you're on and how it works determines which kernel parameters you can tune and how Node.js behaves under connection-heavy loads.

The Evolution: select → poll → epoll

select: The Original (and Still Worst)

select is the POSIX I/O multiplexer available on every Unix system. It watches a set of file descriptors and returns when any of them are ready.

// select signature (simplified)
int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds,
           struct timeval *timeout);

Critical limitation: O(N) at both submission and return. The fd_set is a fixed-size bitmap. The kernel scans every bit to find ready descriptors. You scan every bit again to find which ones fired. At 50,000 file descriptors, this is 50,000 bit checks twice, every time you call select. And select's maximum nfds is typically 1,024 on Linux — you literally cannot watch more than 1,024 file descriptors.

Node.js does not use select.

poll: Slightly Better

poll removes the 1,024 limit but keeps the O(N) scan:

int poll(struct pollfd *fds, nfds_t nfds, int timeout);

You pass an array of pollfd structs. The kernel fills in revents for each one that's ready. You still scan the entire array to find which descriptors fired. At 50,000 connections, this is 50,000 struct scans on every call.

Node.js does not use poll for network sockets (though libuv falls back to it on some platforms for specific operations).

epoll: Linux's Answer (What Node.js Uses)

epoll was introduced in Linux 2.5.44 (2002) and is the foundation of Node.js's I/O on Linux. It solves the O(N) problem:

// Create an epoll instance
int epfd = epoll_create1(0);
 
// Register a file descriptor to watch
struct epoll_event ev;
ev.events = EPOLLIN;        // watch for readable data
ev.data.fd = socket_fd;
epoll_ctl(epfd, EPOLL_CTL_ADD, socket_fd, &ev);
 
// Wait for events — returns only READY descriptors
int n = epoll_wait(epfd, events, MAX_EVENTS, timeout_ms);
// n = number of ready events, events[] contains only the ready ones

epoll_wait is like a hotel concierge with a single buzzer board instead of walking every hallway checking each room's light — the room itself lights the board the instant it needs attention; the concierge only ever looks at lit bulbs. select and poll are the concierge walking every hallway on every round, checking each door whether it's lit or not.

The key difference: epoll_wait returns only the file descriptors that are actually ready. If 50,000 connections are registered but only 3 have new data, epoll_wait returns 3 — not 50,000. The kernel maintains an internal ready-list and adds to it when descriptors become ready via interrupt-driven notification.

The result: epoll_wait is O(ready events), not O(total registered). Registering 50,000 connections costs 50,000 epoll_ctl calls at setup time, but each subsequent epoll_wait scales with activity, not connection count.

Beyond epoll: io_uring

epoll is a readiness API — it tells you a file descriptor is ready, then you still make a separate syscall (read, write, accept4) to actually move the data. Every ready event costs at least two syscalls: one to learn about it, one to act on it.

io_uring, added to the Linux kernel in 5.1 (2019), is a completion API built around two shared ring buffers (a submission queue and a completion queue) mapped into both kernel and userspace. You submit a batch of I/O requests — reads, writes, accepts — without a syscall per request, and the kernel posts completions to the other ring as they finish. This collapses the two-syscalls-per-event pattern of epoll + read/write into something closer to zero syscalls per event once a batch is submitted, and it uniformly supports both socket I/O and file I/O (epoll never worked reliably for regular files).

libuv has been incrementally integrating io_uring as an internal backend on Linux for several years — it is not yet a full replacement for the epoll-based poll phase across all of libuv's operations, but it is the direction the kernel and libuv are both moving for reducing per-I/O-event syscall overhead. None of this is exposed directly to JavaScript; it's an internal libuv implementation detail, but it explains why "epoll is the final word on Linux async I/O" is already out of date at the kernel level.

The Three epoll System Calls

Understanding these three calls precisely demystifies libuv's I/O poll phase.

epoll_create1

int epfd = epoll_create1(EPOLL_CLOEXEC);

Creates an epoll instance — a kernel data structure that maintains the watched set and the ready list. Returns a file descriptor. EPOLL_CLOEXEC ensures the descriptor is closed if the process exec's (security hygiene).

libuv calls this once at startup. The single epfd watches all sockets, timers, pipes, and signals for the Node.js process.

epoll_ctl

// Add a new file descriptor to watch
epoll_ctl(epfd, EPOLL_CTL_ADD, fd, &event);
 
// Modify the events to watch on an existing fd
epoll_ctl(epfd, EPOLL_CTL_MOD, fd, &event);
 
// Remove a file descriptor
epoll_ctl(epfd, EPOLL_CTL_DEL, fd, NULL);

Called by libuv when:

  • A new TCP connection is accepted → EPOLL_CTL_ADD with EPOLLIN | EPOLLOUT
  • A socket transitions from write-interested to read-only → EPOLL_CTL_MOD
  • A connection closes → EPOLL_CTL_DEL

For a blockchain indexer receiving 50,000 concurrent connections: 50,000 EPOLL_CTL_ADD calls at connection time. One epfd watching all 50,000.

How the connection gets accepted in the first place: when the listening socket becomes readable (a pending connection in the accept queue), libuv doesn't call plain accept() — it calls accept4(listen_fd, &addr, &addrlen, SOCK_NONBLOCK | SOCK_CLOEXEC). The two flags matter: SOCK_NONBLOCK sets the new connection's file descriptor to non-blocking mode atomically, as part of the same syscall that creates it — avoiding a separate fcntl(fd, F_SETFL, O_NONBLOCK) call and the race condition where another thread could operate on the fd in blocking mode between accept() and the follow-up fcntl(). SOCK_CLOEXEC atomically marks the descriptor to close on exec(), the same security hygiene EPOLL_CLOEXEC provides for the epoll instance itself. Only after accept4 returns does libuv register the new fd with EPOLL_CTL_ADD.

epoll_wait

struct epoll_event events[MAX_EVENTS];
int n = epoll_wait(epfd, events, MAX_EVENTS, timeout_ms);
for (int i = 0; i < n; i++) {
  handle_event(events[i].data.fd, events[i].events);
}

This is called by libuv in the poll phase of the event loop. It blocks until:

  • At least one registered fd is ready, OR
  • The timeout expires (calculated by libuv based on pending timers)

When it returns, events[0..n-1] contains only the ready descriptors. libuv processes each one, dispatching the appropriate callback.

Level-Triggered vs Edge-Triggered

epoll supports two notification modes, set per file descriptor via the EPOLLET flag in the epoll_event struct:

Sign in to keep reading

The rest of this module is free — sign in with Google to unlock it and track your progress.

Sign in & Register
PreviousModule A-3: Event Loop Saturation & Thread Pool OffloadingNext Module A-5: The HTTP/TCP Subsystem & Ingestion Backpressure

Discussion

0

Join the discussion

Loading comments...
PortfolioBlog

© 2026 Jatin Jain Saraf (JJS). All rights reserved.