Module P-13·20 min read

JSON.parse/stringify edge cases, streaming JSON for large payloads, JSON Schema with Ajv, MessagePack as a binary alternative, and the serialization cost at scale.

JJS

Written by Jatin Jain Saraf · Senior Software Engineer

Module P-13 — JSON Internals, Serialization, and Schema Validation

Q: When dealing with a large result set from a database in Node.js, what is the best way to return it as JSON to avoid high memory consumption?

Use a streaming JSON stringifier like `JSONStream` paired with a database stream. — Loading a large query result into memory and calling `JSON.stringify` can cause memory spikes and out-of-memory errors. Using a tool like `JSONStream` combined with a stream from the database processes rows as they arrive, maintaining constant memory usage regardless of the result size.

Q: Which statement correctly describes the difference between Zod and Ajv in Node.js?

Ajv uses JSON Schema which is more portable, while Zod natively infers TypeScript types. — Ajv is an implementation of JSON Schema, making it highly portable and widely used in standards like OpenAPI. It compiles schemas to highly optimized validation functions, making it extremely fast. Zod, on the other hand, is specifically designed for TypeScript projects and provides robust type inference out of the box.

Q: What is a primary reason to consider MessagePack over JSON in a Node.js microservice architecture?

It is a binary format that reduces payload size by 20–50% and supports types like binary data and timestamps. — MessagePack is a binary serialization format that generates payloads 20–50% smaller than standard JSON and allows native serialization for types not supported by JSON (e.g., timestamps and binary buffers). This makes it excellent for high-frequency internal microservice communication where latency and bandwidth are critical.

What this module covers: JSON is everywhere in Node.js — request bodies, response payloads, configuration files, inter-service communication. Most developers treat it as a black box and pay for it later with silent data loss, unexpected type coercions, and performance problems at scale. This module covers what JSON.parse and JSON.stringify actually do under the hood, the edge cases that corrupt data silently, streaming JSON for payloads too large to hold in memory, fast serialization with fast-json-stringify, JSON Schema validation with Ajv, and when MessagePack is worth the switch.

What JSON.parse and JSON.stringify Actually Do

JSON is a text format. JSON.parse converts a string into a JavaScript object. JSON.stringify does the reverse. The key word is string — everything must pass through text.

This matters because JavaScript has types that JSON doesn't:

JavaScript type	JSON result	What happens
`undefined`	omitted	property disappears
`NaN`	`"null"`	silently becomes null
`Infinity`	`"null"`	silently becomes null
`Date`	`"2024-05-22T14:32:01.123Z"`	becomes a string, not a Date on parse
`BigInt`	throws `TypeError`	crashes your process
`Map`, `Set`	`{}`	serialised as empty objects
`Symbol`	omitted	property disappears
Circular reference	throws `TypeError`	crashes your process

typescript

// Silent data loss
const obj = {
  value: undefined,     // omitted
  notANumber: NaN,      // → null
  infinite: Infinity,   // → null
};
JSON.stringify(obj);
// → '{"notANumber":null,"infinite":null}'
// 'value' is gone entirely

// Date loses its type
const date = new Date('2024-05-22');
const roundTripped = JSON.parse(JSON.stringify({ date }));
typeof roundTripped.date; // 'string', not 'object'
roundTripped.date instanceof Date; // false

// BigInt crashes
JSON.stringify({ id: 9007199254740993n });
// TypeError: Do not know how to serialize a BigInt

// Circular reference crashes
const a: any = {};
const b: any = { a };
a.b = b;
JSON.stringify(a);
// TypeError: Converting circular structure to JSON

JSON.stringify silently dropping undefined and turning NaN/Infinity into null is like a customs form that quietly discards anything it doesn't have a box for — nothing errors, the paperwork just arrives incomplete.

Silent Precision Loss: Large Numbers Without BigInt

The table above shows BigInt throwing on JSON.stringify — a loud, immediate failure you can't miss. A plain number above Number.MAX_SAFE_INTEGER is the dangerous version of the same problem, because it doesn't throw at all:

typescript

const wei = 9007199254740993; // one more than Number.MAX_SAFE_INTEGER (2^53 - 1)
JSON.parse(JSON.stringify(wei));
// → 9007199254740992 — silently rounded to the nearest representable double, no warning

A token-transfer indexer once stored raw wei amounts as a plain JS number instead of BigInt or a string. Every time a transfer record round-tripped through an internal message queue as JSON, any amount above the safe-integer threshold silently rounded to the nearest representable double. The drift was only a few hundred wei per affected value — invisible in unit tests built around small fixture numbers — until a production reconciliation audit found balances that didn't match the chain, well after the rounding had been happening quietly in production for a while.

Large numeric IDs and monetary amounts (wei, satoshis, cents at scale) should be BigInt or a string from the moment they're read out of the database — never a plain number that happens to fit in dev with smaller test fixtures.

Buffer Serialization: The Accidental Leak

Buffer doesn't throw and doesn't silently vanish like undefined — it silently expands into a verbose, easy-to-miss shape:

typescript

const buffer = Buffer.from('hello');
JSON.stringify({ avatar: buffer });
// → '{"avatar":{"type":"Buffer","data":[104,101,108,108,111]}}'

Every byte becomes a decimal number in an array. This bites in practice when a field that's a Buffer under the hood — a Prisma Bytes column, the output of crypto.randomBytes(), a file read with fs.readFileSync() without an encoding — passes straight into res.json() without anyone noticing its real type:

typescript

const user = { id: 1, name: 'Jatin', apiKeyHash: crypto.randomBytes(32) };
res.json(user);
// apiKeyHash silently becomes {"type":"Buffer","data":[12,45,201,...]} —
// a 32-byte key balloons into a 100+ element array in the response body,
// and the raw key material is now sitting in an HTTP response

Nothing errors. The response just gets bigger, uglier, and — if the buffer held anything sensitive (key material, a password hash, an internal token) — leakier than intended. The fix is the same discipline as any other field you don't want serialised as-is: convert explicitly (apiKeyHash.toString('base64')) or exclude it via the replacer whitelist pattern shown below.

structuredClone(): The Modern Alternative to JSON Round-Tripping

Deep-cloning with JSON.parse(JSON.stringify(x)) inherits every problem in this section — Dates become strings, Map/Set become {}, undefined disappears. Node 17+ ships a built-in structuredClone() that performs a real deep clone using the structured clone algorithm instead of a JSON round-trip:

typescript

const original = {
  createdAt: new Date(),
  tags: new Set(['a', 'b']),
  metadata: new Map([['key', 'value']]),
  count: undefined,
};

const jsonClone = JSON.parse(JSON.stringify(original));
// jsonClone.createdAt is a string; jsonClone.tags and jsonClone.metadata are {}; count is gone

const realClone = structuredClone(original);
// realClone.createdAt instanceof Date   === true
// realClone.tags instanceof Set         === true
// realClone.metadata instanceof Map     === true
// 'count' in realClone                 === true (preserved, value undefined)

structuredClone even handles circular references natively — something JSON.stringify can't do at all without the WeakSet workaround below. It does have limits: it throws on functions, and cloning a custom class instance gives you back a plain object with the same own properties, not an instance of that class — the prototype chain doesn't survive. For plain data (API payloads, config objects, anything already JSON-shaped), it's the better default; reach for a real cloning library only when you need class instances or functions preserved too.

Safe Patterns

Handling BigInt

BigInt appears frequently when working with PostgreSQL's BIGSERIAL primary keys or blockchain IDs that exceed Number.MAX_SAFE_INTEGER (2⁵³ - 1 ≈ 9 quadrillion):

typescript

// Option 1: Serialise BigInt as string
JSON.stringify({ id: 9007199254740993n }, (key, value) =>
  typeof value === 'bigint' ? value.toString() : value,
);
// → '{"id":"9007199254740993"}'

// Option 2: Custom replacer — handles nested BigInts
function jsonStringifyBigInt(value: unknown): string {
  return JSON.stringify(value, (_, v) =>
    typeof v === 'bigint' ? v.toString() : v,
  );
}

// Option 3: Override toJSON on a class
class OrderId {
  constructor(public readonly value: bigint) {}
  toJSON() { return this.value.toString(); }
}

Handling Dates

JSON.stringify serialises Dates as ISO strings. JSON.parse gives you strings back. To restore Dates you need a reviver:

typescript

function parseWithDates(json: string): unknown {
  const ISO_DATE = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}/;

  return JSON.parse(json, (key, value) => {
    if (typeof value === 'string' && ISO_DATE.test(value)) {
      return new Date(value);
    }
    return value;
  });
}

const obj = parseWithDates('{"createdAt":"2024-05-22T14:32:01.123Z"}');
obj.createdAt instanceof Date; // true

Detecting circular references

typescript

function safeStringify(value: unknown): string | null {
  const seen = new WeakSet();
  try {
    return JSON.stringify(value, (key, val) => {
      if (typeof val === 'object' && val !== null) {
        if (seen.has(val)) return '[Circular]';
        seen.add(val);
      }
      return val;
    });
  } catch {
    return null;
  }
}

The replacer and reviver Parameters

JSON.stringify(value, replacer, space) — replacer controls what gets serialised:

typescript

const user = {
  id: 1,
  name: 'Jatin',
  passwordHash: '$2b$12$xxx',
  email: 'j@example.com',
  internalFlag: true,
};

// Array replacer — whitelist fields
JSON.stringify(user, ['id', 'name', 'email']);
// → '{"id":1,"name":"Jatin","email":"j@example.com"}'
// passwordHash and internalFlag never leave the server

// Function replacer — transform values
JSON.stringify(user, (key, value) => {
  if (key === 'passwordHash') return undefined; // omit
  return value;
});

JSON.parse(text, reviver) — reviver transforms values after parsing:

typescript

// Convert snake_case API responses to camelCase
function camelizeReviver(key: string, value: unknown) {
  if (typeof value === 'object' && value !== null && !Array.isArray(value)) {
    return Object.fromEntries(
      Object.entries(value as object).map(([k, v]) => [
        k.replace(/_([a-z])/g, (_, c) => c.toUpperCase()),
        v,
      ]),
    );
  }
  return value;
}

JSON.parse('{"user_id":1,"created_at":"2024-01-01"}', camelizeReviver);
// → { userId: 1, createdAt: '2024-01-01' }

Prototype Pollution via JSON.parse

JSON.parse itself is not the vulnerability. Parsing '{"__proto__":{"isAdmin":true}}' produces an object with an own, enumerable property literally named __proto__ — it does not walk up and mutate Object.prototype:

typescript

const parsed = JSON.parse('{"__proto__":{"isAdmin":true}}');
Object.getPrototypeOf(parsed) === Object.prototype; // true — untouched
parsed.__proto__;                                    // { isAdmin: true } — a plain data property here
({}).isAdmin;                                        // undefined — not polluted by JSON.parse alone

The danger shows up one step later, in code that takes that parsed object and merges it into something else — a config loader, an options-merging helper, any hand-rolled "deep extend":

typescript

// A common, naive deep-merge — the kind hand-rolled in a dozen internal utils
function merge(target: any, source: any) {
  for (const key in source) {
    if (typeof source[key] === 'object' && source[key] !== null) {
      target[key] = target[key] || {};
      merge(target[key], source[key]);   // recurses straight into __proto__
    } else {
      target[key] = source[key];
    }
  }
  return target;
}

const attackerInput = JSON.parse('{"__proto__":{"isAdmin":true}}');
merge({}, attackerInput);

({}).isAdmin; // true — every plain object in the process now inherits isAdmin

target[key] = ... with key === '__proto__' triggers the real prototype accessor — bracket-notation assignment behaves differently from the object construction JSON.parse does internally — so the merge walks up and mutates Object.prototype itself. From that point on, every plain object in the process, including ones created in completely unrelated request handlers, inherits isAdmin: true, which can silently defeat an if (user.isAdmin) check anywhere in the codebase. This is the exact class of bug behind real CVEs in lodash.merge, jQuery.extend, and several config-merging npm packages.

Mitigations:

Validate before merging. A schema with additionalProperties: false (the Ajv section below) rejects unexpected keys like __proto__ and constructor before they ever reach a merge function.
Strip dangerous keys in a reviver, as a defense-in-depth layer on any JSON.parse of untrusted input:

typescript

JSON.parse(untrustedJson, (key, value) => {
  if (key === '__proto__' || key === 'constructor' || key === 'prototype') return undefined;
  return value;
});

Use Object.create(null) or a Map for objects built from untrusted keys — there's no prototype to pollute.
Prefer structuredClone or a patched merge library over hand-rolled recursive merges like the one above.

JSON.stringify Performance at Scale

At low volume, JSON.stringify is invisible. At 10,000 requests/second returning 50-field objects, it becomes measurable.

typescript

// Benchmark comparison (approximate — varies by payload size)
// JSON.stringify (built-in):      ~200ns per call for a simple object
// fast-json-stringify (precompiled): ~50ns per call — 4× faster

fast-json-stringify pre-compiles a serialiser from a JSON Schema — it knows the shape of the output ahead of time and skips runtime introspection:

bash

npm install fast-json-stringify

typescript

import fastJson from 'fast-json-stringify';

const serializeUser = fastJson({
  type: 'object',
  properties: {
    id: { type: 'integer' },
    name: { type: 'string' },
    email: { type: 'string' },
    role: { type: 'string', enum: ['user', 'admin'] },
    createdAt: { type: 'string' },
  },
  required: ['id', 'name', 'email', 'role', 'createdAt'],
});

// In your controller — faster than JSON.stringify for high-throughput routes
res.setHeader('Content-Type', 'application/json');
res.end(serializeUser(user));

Use this on high-traffic endpoints where you know the exact response shape. Not worth the complexity for infrequently called routes.

Streaming JSON for Large Payloads

Loading a 100,000-row query result into memory, calling JSON.stringify, and writing the result is a memory spike waiting to happen. Streaming processes rows as they arrive from the database:

bash

npm install JSONStream

typescript

// src/controllers/export.controller.ts
import { asyncHandler } from '../utils/asyncHandler.js';
import { Readable } from 'stream';
import JSONStream from 'JSONStream';
import { pipeline } from 'stream/promises';
import prisma from '../db/prisma.js';

export const exportUsers = asyncHandler(async (req, res) => {
  res.setHeader('Content-Type', 'application/json');
  res.setHeader('Content-Disposition', 'attachment; filename="users.json"');

  // Prisma Client has no built-in streaming query method — there is no
  // `findManyStream`. Prisma does not expose a database-level cursor you can
  // pipe from directly. The real workaround is cursor-based pagination:
  // fetch fixed-size pages with `take` + `cursor`, and push each page's rows
  // into a Readable manually so the full result set is never held in memory.
  const PAGE_SIZE = 500;
  let cursor: number | undefined;

  const userStream = new Readable({
    objectMode: true,
    async read() {
      const page = await prisma.user.findMany({
        select: { id: true, name: true, email: true, role: true, createdAt: true },
        orderBy: { id: 'asc' },
        take: PAGE_SIZE,
        ...(cursor ? { skip: 1, cursor: { id: cursor } } : {}),
      });

      if (page.length === 0) {
        this.push(null); // no more rows — end the stream
        return;
      }

      cursor = page[page.length - 1].id;
      for (const row of page) this.push(row);
    },
  });

  // Wrap individual objects into a JSON array, streamed
  const jsonStringifier = JSONStream.stringify('[', ',', ']');

  await pipeline(userStream, jsonStringifier, res);
  // Memory usage: bounded by PAGE_SIZE, not the total row count —
  // only one page of rows is ever held in memory at a time.
});

Unlike raw pg, Prisma doesn't give you a true database-level cursor to pipe from — the Readable above simulates one by paginating. When a true DB-level stream matters (very large exports, tight memory budgets), reach for pg-query-stream with raw SQL instead:

bash

npm install pg-query-stream

typescript

import QueryStream from 'pg-query-stream';
import { pool } from '../db/pool.js';
import JSONStream from 'JSONStream';
import { pipeline } from 'stream/promises';

export const exportOrders = asyncHandler(async (req, res) => {
  res.setHeader('Content-Type', 'application/json');

  const client = await pool.connect();

  try {
    const query = new QueryStream(
      'SELECT id, user_id, total, status, created_at FROM orders ORDER BY id',
      [],
      { batchSize: 100 },
    );

    const stream = client.query(query);
    await pipeline(stream, JSONStream.stringify('[', ',', ']'), res);
  } finally {
    client.release();
  }
});

JSON Schema Validation with Ajv

Ajv (Another JSON Schema validator) is the fastest JSON Schema validator for Node.js. JSON Schema is the standard for describing the structure of JSON data — more portable than Zod (which is TypeScript-specific) and used widely in OpenAPI and form validation.

bash

npm install ajv ajv-formats

typescript

import Ajv from 'ajv';
import addFormats from 'ajv-formats';

const ajv = new Ajv({ allErrors: true }); // collect all errors, not just the first
addFormats(ajv); // adds 'email', 'date-time', 'uri', 'uuid' formats

const createUserSchema = {
  type: 'object',
  properties: {
    name: { type: 'string', minLength: 1, maxLength: 100 },
    email: { type: 'string', format: 'email' },
    password: { type: 'string', minLength: 8 },
    role: { type: 'string', enum: ['user', 'admin'], default: 'user' },
  },
  required: ['name', 'email', 'password'],
  additionalProperties: false,  // reject unknown fields
};

// Compile once — reuse the compiled validator
const validateCreateUser = ajv.compile(createUserSchema);

// Use in middleware or directly
function validate(schema: object) {
  const validate = ajv.compile(schema);
  return (req, res, next) => {
    if (validate(req.body)) return next();

    const errors = validate.errors!.map(err => ({
      field: err.instancePath.slice(1) || err.params?.missingProperty,
      message: err.message,
    }));
    res.status(400).json({ error: 'Validation failed', issues: errors });
  };
}

Ajv vs Zod: Zod infers TypeScript types and is designed for TypeScript projects. Ajv follows the JSON Schema standard and is faster — it compiles schemas to optimised validation functions. For Node.js APIs already using Zod (as in P-4), stick with Zod. Ajv shines when you need JSON Schema portability (shared schemas between frontend/backend) or raw performance.

MessagePack: Binary Alternative to JSON

MessagePack is a binary serialisation format that is 20–50% smaller than JSON and faster to serialise/deserialise. It supports types that JSON doesn't: integers, floats, binary data, timestamps.

When to consider it:

High-frequency inter-service communication (microservices)
Real-time applications where payload size affects latency
Storing structured data in Redis (smaller footprint)

bash

npm install @msgpack/msgpack

typescript

import { encode, decode } from '@msgpack/msgpack';

// Serialise
const data = { userId: 42, timestamp: new Date(), values: [1, 2, 3] };
const packed = encode(data);   // Uint8Array — binary, smaller than JSON string

// Deserialise
const unpacked = decode(packed) as typeof data;

// Size comparison (approximate)
const json = JSON.stringify(data);
console.log('JSON size:', Buffer.byteLength(json), 'bytes');
console.log('MessagePack size:', packed.byteLength, 'bytes');
// JSON:        ~60 bytes
// MessagePack: ~40 bytes — ~33% smaller

MessagePack in Express

typescript

// Custom middleware to accept and return MessagePack
app.use((req, res, next) => {
  if (req.headers['content-type'] === 'application/msgpack') {
    const chunks: Buffer[] = [];
    req.on('data', chunk => chunks.push(chunk));
    req.on('end', () => {
      req.body = decode(Buffer.concat(chunks));
      next();
    });
  } else {
    next();
  }
});

// res.msgpack() helper
res.msgpack = (data: unknown) => {
  res.setHeader('Content-Type', 'application/msgpack');
  res.end(encode(data));
};

For most REST APIs serving browsers, JSON is the right choice — browsers speak JSON natively, the bandwidth difference is negligible, and debuggability is far better. MessagePack is the right choice for Node-to-Node communication where you control both ends and need to squeeze out latency.

The JSON.parse Cost in Hot Paths

JSON.parse parses the entire string synchronously. On the main thread. A 1MB request body takes ~5ms. At 1,000 req/s, that's 5 seconds of JSON parsing per second on one thread.

Strategies for hot paths:

Validate size before parsing — reject oversized bodies early (express.json({ limit: '10kb' }))
Use fast-json-stringify for serialisation on high-throughput routes
Cache parsed results when the same JSON is parsed repeatedly (e.g., static configuration)
Move heavy parsing to a worker thread for CPU-intensive transformations

typescript

// Naive version — spawns a brand-new OS thread on every single call
import { Worker } from 'worker_threads';

function parseInWorker(json: string): Promise<unknown> {
  return new Promise((resolve, reject) => {
    const worker = new Worker(__filename, { workerData: { json } });
    worker.on('message', resolve);
    worker.on('error', reject);
  });
}

Spawning a Worker per call looks fine in a demo but costs real time on every request — a new thread means a new V8 isolate, a new event loop, and module re-evaluation, typically single-digit milliseconds of pure startup overhead before any actual parsing happens. At any real request volume, that overhead dominates the time you were trying to save. The fix is a small, fixed pool of workers that stay alive for the process's lifetime, with each request handed to whichever worker is free — startup cost is paid once at boot, not per request:

bash

npm install piscina

typescript

// src/workers/json-transform.worker.ts — the actual work, run inside pool threads
module.exports = (json: string) => {
  return expensiveTransformation(json);
};

typescript

// src/utils/jsonWorkerPool.ts — one pool, created once at module load, reused for every call
import Piscina from 'piscina';
import path from 'path';

const pool = new Piscina({
  filename: path.resolve(__dirname, '../workers/json-transform.worker.js'),
  minThreads: 2,
  maxThreads: 4,   // size to CPUs available to the container, not the host
});

export function parseInWorker(json: string): Promise<unknown> {
  return pool.run(json); // dispatched to an already-running worker — no thread spawn per call
}

Piscina manages the pool's lifecycle and queues tasks when every worker is busy. A hand-rolled fixed pool (an array of pre-spawned Worker instances with a round-robin or least-busy dispatcher) works too — the point is reusing threads across calls, not creating one per request.

Summary

undefined, NaN, Infinity, BigInt, Date, Map, Set all behave unexpectedly with JSON.stringify. Know the conversion table.
BigInt throws; serialise it as a string. Dates become strings; use a reviver to restore them. Circular references throw; use a replacer with a WeakSet.
replacer as an array is the cleanest way to whitelist response fields — prevents sensitive data leaking from the serialisation layer.
fast-json-stringify pre-compiles serialisers from schemas — 4× faster than built-in JSON.stringify for high-throughput, fixed-shape responses.
Stream JSON for large exports using JSONStream.stringify piped through Prisma/pg stream results — constant memory regardless of result set size.
Ajv is fastest for JSON Schema validation; portable across languages and tools. Use it when you need JSON Schema portability. Use Zod when you want TypeScript inference.
MessagePack is 20–50% smaller and faster than JSON. Worth it for Node-to-Node communication. Not worth it for browser-facing APIs.

Next: Dockerizing Node.js applications for production — multi-stage builds, Alpine vs slim images, non-root users, health checks, and the production readiness checklist for containerised deployments.

Knowledge Check

When dealing with a large result set from a database in Node.js, what is the best way to return it as JSON to avoid high memory consumption?

Which statement correctly describes the difference between Zod and Ajv in Node.js?

What is a primary reason to consider MessagePack over JSON in a Node.js microservice architecture?

Test your knowledge with more question sets

Sign in to keep reading

The rest of this module is free — sign in with Google to unlock it and track your progress.

PreviousModule P-12: Background Jobs and Task Queues with BullMQ Next Module P-14: Dockerizing Node.js Applications for Production

Discussion

Join the discussion

Loading comments...