Build a Mini CEX with Bun, Redis, Prisma & PostgreSQL

Architecture of Centralized Exchange

Before diving into the implementation details, let's look at the overall architecture of the exchange. This high-level view shows how the different services interact, how requests flow through the system, and where the matching engine fits into the picture.

The matching engine acts as the source of truth, while Redis is used for asynchronous communication between services. PostgreSQL stores persistent application data, and periodic snapshots ensure the in-memory state can be recovered after failures.

Link: https://excalidraw.com/#json=HUQD7tEI-J8uU8c1-IgCp,JbifdE1b30P-q964hgRMvg

Throughout the rest of this article, we'll explore each of these components in detail, starting with the core concepts behind a centralized exchange and the functionality our matching engine needs to support.

Functional Requirements of a Centralized Exchange

Before implementing the matching engine, let's understand how a real centralized exchange works and what functionality we need to support.

What is a Centralized Exchange?

A Centralized Exchange (CEX) is a platform where buyers and sellers trade assets through a central authority.

Examples:

Binance
Backpack Exchange
Coinbase
Bombay Stock Exchange (BSE)
National Stock Exchange (NSE)

The exchange acts as a trusted middleman and is responsible for:

Maintaining user balances
Managing order books
Matching buy and sell orders
Recording trades
Settling balances

Most centralized exchanges require KYC (Know Your Customer) before users can trade.

Brokers vs Exchanges

An exchange is where actual matching happens.

Examples:

Binance
Backpack
BSE
NSE

Broker

A broker provides a user-friendly interface to the exchange.

Examples:

Zerodha
Groww

Think of it this way:

User
  |
  v
Broker (Zerodha)
  |
  v
Exchange (NSE)

In this project, we're building the exchange itself, not the broker.

Understanding the Order Book

The order book is the heart of every exchange.

Think of it like a real estate broker's notebook.

The broker keeps track of:

People willing to buy land
People willing to sell land

Similarly, an order book keeps track of:

Bids

People willing to buy.

Buy SOL at $82.85
Buy SOL at $82.84
Buy SOL at $82.83

Asks

People willing to sell.

Sell SOL at $82.86
Sell SOL at $82.87
Sell SOL at $82.88

The order book constantly changes as new orders arrive and existing orders get matched.

Limit Orders

A limit order specifies the exact price at which a user wants to trade.

Example:

Buy 10 SOL at $82.85

The order will only execute if someone is willing to sell at $82.85.

Otherwise, it remains in the order book.

Market Orders

A market order executes immediately at the best available price.

Example:

Buy 20 SOL at market price

The exchange automatically matches against the cheapest available sell orders.

Suppose the order book contains:

Ask Side

82.86 -> 159.71 SOL
82.87 -> 100 SOL
82.88 -> 50 SOL

If a user places:

Buy 20 SOL at Market

The exchange immediately fills:

20 SOL @ 82.86

Partial Fills

Not every order can be completely matched.

Example:

Buy 1000 SOL @ 82.85

Available liquidity:

82.85 -> 337.15 SOL

Only part of the order gets filled.

Filled = 337.15 SOL
Remaining = 662.85 SOL

The remaining quantity stays on the order book waiting for future sellers.

Maker vs Taker

Maker

Creates liquidity.

Example:

Buy 10 SOL @ 82.80

No seller exists.

Order sits in the order book.

This user is a Maker.

Taker

Consumes liquidity.

Example:

Buy 10 SOL at Market

Immediately matches with existing sellers.

This user is a Taker.

Exchanges usually charge lower fees to Makers because they provide liquidity.

What Should Be Stored in Memory vs Database?

In-Memory Data

The matching engine needs extremely fast access to data.

Because orders are created, matched, and cancelled continuously, storing this information directly in PostgreSQL would introduce:

High latency
Frequent database writes
Lock contention
Concurrency issues

Therefore, the matching engine keeps the following data structures in memory:

User Balances

balances = {
  user1: {
    usd: {
      available: 20,
      locked: 10
    }
  }
}

Order Books

orderBooks = {
  SOL: {
    bids: [],
    asks: [],
    lastTradedPrice: 0
  },

  BTC: {
    bids: [],
    asks: [],
    lastTradedPrice: 0
  }
}

This allows order matching to happen in microseconds without querying the database.

Database Schema

Even though matching happens in memory, we still need PostgreSQL for persistence.

Users


model User {
  id       String @id @default(uuid())
  username String @unique
  password String
}

APIs to Implement

Create Order

POST /order

Get Order Details

GET /order/:orderId

Returns:

Order information
Fill history
Filled quantity
Remaining quantity
Order status

Cancel Order

DELETE /order/:orderId

Cancels the unfilled portion of an order.

Example:

Original Order = 100 SOL

Filled = 40 SOL

Remaining = 60 SOL

Only the remaining 60 SOL is removed from the order book.

Get Market Depth

GET /depth/:symbol

Returns:

Top 20 bids
Top 20 asks

Example:

SOL Order Book

Bids:
82.85
82.84
82.83

Asks:
82.86
82.87
82.88

Architecture Overview

Our Mini Centralized Exchange follows an event-driven architecture:

Frontend / API Client
        |
        v
Backend API (Express)
        |
        v
Redis Queue (backend-to-engine-broker)
        |
        v
Matching Engine Process
        |
        v
Backend-Specific Response Queue
        |
        v
Backend API Response

Tech Stack

TypeScript
Bun
Express
Redis
Prisma
PostgreSQL
JWT Authentication
Zod Validation

Step 1: Initialize the Backend Project

Create a new backend directory and initialize a Bun project:

mkdir backend
cd backend
bun init

This generates the basic project structure along with a package.json file.

Step2: Set Up Prisma with PostgreSQL

We'll use Prisma as our ORM and PostgreSQL as the primary database.

Instead of manually configuring everything, follow the official Prisma Bun guide:

https://www.prisma.io/docs/guides/runtimes/bun

The guide covers:

Installing Prisma
Initializing Prisma
Connecting PostgreSQL
Creating your first schema
Running migrations
Generating the Prisma Client

Step3: Install Initial Dependencies

Before writing any code, let's install only the dependencies required to start our Express server and manage environment variables.

bun add express cors dotenv
bun add -d typescript @types/bun @types/express @types/cors

We'll install additional dependencies such as Prisma, PostgreSQL, Redis, JWT, Zod, and bcrypt as we need them throughout the project.

Project Structure

At this stage, our project structure looks like this:

backend/
├── src/
│   ├── index.ts
│   └── utils/
│       └── env.ts
├── .env
├── .gitignore
├── bun.lock
├── package.json
├── prisma.config.ts
├── README.md
└── tsconfig.json

All application code will live inside the src directory.

Step4: Creating the Express Server

Create src/index.ts:

import cors from "cors";
import express, {
  type NextFunction,
  type Request,
  type Response,
} from "express";
import { env } from "./utils/env.js";

const app = express();

app.use(cors());
app.use(express.json());

app.get("/health", async (_req, res) => {
  res.json({ ok: true });
});

app.use(
  (err: unknown, _req: Request, res: Response, _next: NextFunction) => {
    console.error(err);

    res.status(500).json({
      error: err instanceof Error ? err.message : "internal_server_error",
    });
  },
);

app.listen(env.port, () => {
  console.log(`Backend running on http://localhost:${env.port}`);
});

This gives us:

CORS support for frontend requests
JSON request parsing
A health check endpoint
Global error handling middleware
A configurable server port

Step 5: Managing Environment Variables

Create src/utils/env.ts:

import "dotenv/config";

function readRequiredEnv(name: string): string {
  const value = process.env[name];

  if (!value) {
    throw new Error(`Missing required env variable: ${name}`);
  }

  return value;
}

export const env = {
  port: Number(process.env.PORT ?? "3000"),
};

Loading environment variables through a dedicated module keeps configuration centralized and makes it easier to validate required values as the application grows.

Step 6: Create a .env File

PORT=3000

Step 7: Start the Server

Run the development server:

bun run dev

If everything is configured correctly, you should see:

Backend running on http://localhost:3000

You can verify the server is working by visiting:

http://localhost:3000/health

Expected response:

{
  "ok": true
}

Now that our Express server is running and connected to PostgreSQL through Prisma, let's implement the first authentication endpoint: Signup.

The signup flow will:

Validate incoming request data using Zod.
Hash the user's password using bcrypt.
Store the user in PostgreSQL.
Generate a JWT token.
Return the authenticated user information.

Register the Application Router

First, update src/index.ts and register the application's root router.

import { appRouter } from "./routes/index.js";

app.use(appRouter);

This keeps route definitions separate from server initialization, making the project easier to scale as new features are added.

Create the Root Router

Create src/routes/index.ts:

import { Router } from "express";
import { authRouter } from "./auth-routes.js";

export const appRouter = Router();

appRouter.use(authRouter);

The root router acts as a central place where all feature-specific routers are registered.

Create the Authentication Router

Create src/routes/auth-routes.ts:

import { Router } from "express";
import { signup } from "../controllers/auth-controller.js";
import { asyncHandler } from "../utils/async-handler.js";

export const authRouter = Router();

authRouter.post("/signup", asyncHandler(signup));

Instead of placing business logic directly inside route handlers, we delegate it to a controller.

Create an Async Handler

Express does not automatically catch errors thrown inside async functions. To avoid repetitive try/catch blocks, create a reusable async wrapper.

Create src/utils/async-handler.ts:

import type {
  NextFunction,
  Request,
  RequestHandler,
  Response,
} from "express";

export function asyncHandler(
  handler: (
    req: Request,
    res: Response,
    next: NextFunction,
  ) => Promise<void>,
): RequestHandler {
  return function wrappedHandler(req, res, next) {
    void handler(req, res, next).catch(next);
  };
}

Any unhandled error will automatically reach Express's global error middleware.

Define the Request Schema

Before creating users, we should validate incoming data.

Create src/types/auth-schema.ts:

import { z } from "zod";

export const authSchema = z.object({
  username: z.string().trim().min(1, "username is required"),
  password: z.string().min(1, "password is required"),
});

Using Zod ensures invalid requests never reach our database layer.

Create a Validation Helper

Create src/utils/validation.ts:

import type { Response } from "express";
import type { ZodError } from "zod";

export function sendValidationError(
  res: Response,
  error: ZodError,
): void {
  res.status(400).json({
    error: "validation_error",
    issues: error.issues.map((issue) => ({
      path: issue.path.join("."),
      message: issue.message,
    })),
  });
}

This gives clients a consistent error format whenever validation fails.

Create JWT Utilities

Create src/utils/auth.ts:

import jwt from "jsonwebtoken";
import { env } from "./env.js";

export interface TokenPayload {
  userId: string;
}

export function createToken(payload: TokenPayload): string {
  return jwt.sign(payload, env.jwtSecret, {
    expiresIn: "7d",
  });
}

We'll use this helper throughout the application whenever a JWT token needs to be generated.

Create the Signup Controller

Create src/controllers/auth-controller.ts:

import bcrypt from "bcryptjs";
import type { Request, Response } from "express";
import { prisma } from "../db.js";
import { authSchema } from "../types/auth-schema.js";
import { createToken } from "../utils/auth.js";
import { sendValidationError } from "../utils/validation.js";

export async function signup(
  req: Request,
  res: Response,
): Promise<void> {
  const parsedBody = authSchema.safeParse(req.body);

  if (!parsedBody.success) {
    sendValidationError(res, parsedBody.error);
    return;
  }

  const { username, password } = parsedBody.data;

  const hashedPassword = await bcrypt.hash(password, 10);

  try {
    const user = await prisma.user.create({
      data: {
        username,
        password: hashedPassword,
      },
    });

    res.status(201).json({
      token: createToken({
        userId: user.id,
      }),
      userId: user.id,
      username: user.username,
    });
  } catch {
    res.status(409).json({
      error: "username already exists",
    });
  }
}

This controller validates the request, hashes the password, creates the user, and immediately returns an authentication token.

Required Environment Variables

Update your .env file:

PORT=3000
JWT_SECRET=super-secret-key

Also expose the secret in src/utils/env.ts:

export const env = {
  port: Number(process.env.PORT ?? "3000"),
  jwtSecret: readRequiredEnv("JWT_SECRET"),
};

Testing the Endpoint

Send a request to:

POST /signup

Request body:

{
  "username": "shubham",
  "password": "password123"
}

Successful response:

{
  "token": "<jwt-token>",
  "userId": "user-id",
  "username": "shubham"
}

At this point, users can successfully create accounts and receive a JWT token that can be used for authenticated requests throughout the exchange.

With user registration complete, let's implement the Signin API.

The signin flow will:

Validate the request body.
Find the user by username.
Verify the password using bcrypt.
Generate a JWT token.
Return the authenticated user's information.

Register the Signin Route

Update src/routes/auth-routes.ts:

import { Router } from "express";
import { signin, signup } from "../controllers/auth-controller.js";
import { asyncHandler } from "../utils/async-handler.js";

export const authRouter = Router();

authRouter.post("/signup", asyncHandler(signup));
authRouter.post("/signin", asyncHandler(signin));

We now expose two authentication endpoints:

POST /signup
POST /signin

Create the Signin Controller

Update src/controllers/auth-controller.ts:

export async function signin(
  req: Request,
  res: Response,
): Promise<void> {
  const parsedBody = authSchema.safeParse(req.body);

  if (!parsedBody.success) {
    sendValidationError(res, parsedBody.error);
    return;
  }

  const { username, password } = parsedBody.data;

  const userExists = await prisma.user.findFirst({
    where: {
      username,
    },
  });

  if (!userExists) {
    res.status(401).json({
      error: "username not exists",
    });

    return;
  }

  const correctPassword = await bcrypt.compare(
    password,
    userExists.password,
  );

  if (!correctPassword) {
    res.status(403).json({
      error: "password is invalid",
    });

    return;
  }

  res.status(201).json({
    token: createToken({
      userId: userExists.id,
    }),
    userId: userExists.id,
    username: userExists.username,
  });
}

How the Signin Flow Works

Client
  |
  | POST /signin
  v
Validate Request (Zod)
  |
  v
Find User (Prisma)
  |
  v
Compare Password (bcrypt)
  |
  v
Generate JWT
  |
  v
Return Token

Unlike the signup endpoint, we do not create a new user. Instead, we verify the supplied credentials and issue a new JWT token if authentication succeeds.

Testing the Endpoint

Send a request to:

POST /signin

Request body:

{
  "username": "shubham",
  "password": "password123"
}

Successful response:

{
  "token": "<jwt-token>",
  "userId": "user-id",
  "username": "shubham"
}

If the username does not exist:

{
  "error": "username not exists"
}

If the password is incorrect:

{
  "error": "password is invalid"
}

At this point, users can register, sign in, and receive JWT tokens that will be used to access protected exchange APIs in the upcoming sections.

Step10: Connecting the Backend and Engine with Redis

Now that authentication is working, it's time to connect our backend service to the matching engine.

Instead of calling the engine through HTTP, we'll communicate through Redis queues.

This gives us an event-driven architecture where the backend and engine are completely independent services.

Architecture

Frontend
    |
    v
Backend API
    |
    v
Redis Queue
(backend-to-engine-broker)
    |
    v
Matching Engine
    |
    v
Response Queue
(response-queue-123)
    |
    v
Backend API
    |
    v
Frontend

When a user places an order:

Backend validates the request.
Backend sends a message to Redis.
Engine consumes the message.
Engine processes the order.
Engine sends a response back.
Backend returns the response to the client.

Install Redis

In the backend project:
backend/

bun add redis

In the engine project:
engine/

bun add redis

Create a Redis Database

You can run Redis locally or use a managed provider.

For this project we'll use:

https://upstash.com/

Create a Redis database and copy the connection URL.

Configure Environment Variables

Update your .env file:

REDIS_URL="your-redis-url" 
JWT_SECRET="your-secret" 
PORT=3000 
INCOMING_QUEUE="backend-to-engine-broker" BACKEND_QUEUE_ID="1234" 
ENGINE_TIMEOUT_MS=30000

Update src/utils/env.ts:

export const env = {
  port: Number(process.env.PORT ?? "3000"),
  redisUrl: readRequiredEnv("REDIS_URL"),
  jwtSecret: readRequiredEnv("JWT_SECRET"),

  incomingQueue:
    process.env.INCOMING_QUEUE ??
    "backend-to-engine-broker",

  responseQueue:
    `response-queue-${
      process.env.BACKEND_QUEUE_ID ??
      crypto.randomUUID()
    }`,

  engineTimeoutMs: Number(
    process.env.ENGINE_TIMEOUT_MS ?? "30000",
  ),
};

Why do we need two queues?

Incoming Queue

Used by the backend to send requests.

backend-to-engine-broker

Response Queue

Used by the engine to send responses back.

response-queue-1234

Every backend instance gets its own response queue.

Create the Redis Client

Create: src/utils/engine-client.ts

We'll create two Redis connections.

const publisher = createClient({
  url: env.redisUrl,
});

const subscriber = createClient({
  url: env.redisUrl,
});

Publisher

Used to send messages to the engine.

await publisher.lPush(
  env.incomingQueue,
  JSON.stringify(message),
);

Subscriber

Used to wait for engine responses.

await subscriber.brPop(
  env.responseQueue,
  0,
);

Why two Redis clients?

The subscriber uses a blocking operation:

await subscriber.brPop(...)

While waiting, that connection is blocked.

Using a separate publisher allows us to continue sending messages normally.

Connect Redis During Startup

Update src/index.ts.

Before starting Express:

await connectRedis();

void listenForEngineResponses();

Full startup sequence:

Start Backend
      |
      v
Connect Redis
      |
      v
Start Listening For Responses
      |
      v
Start Express Server

We also update our health endpoint:

app.get("/health", async (_req, res) => {
  await pingRedis();

  res.json({
    ok: true,
  });
});

Now health checks verify Redis connectivity.

Creating Request-Response Communication

When we send a message to Redis we need a way to know which response belongs to which request.

For that we use a Correlation ID.

const correlationId = crypto.randomUUID();

Example message:

{
  "correlationId": "123",
  "responseQueue": "response-queue-1234",
  "type": "create_order",
  "payload": {
    "symbol": "BTC",
    "qty": 1
  }
}

Then we push it to Redis:

await publisher.lPush(
  env.incomingQueue,
  JSON.stringify(message),
);

Tracking Pending Requests

After sending a request, the backend must wait for the engine's response.

Create:

src/store/pending-responses.ts

We store pending requests in memory.

const pendingResponses = new Map<string, PendingResponse>();

Structure:

Correlation ID
       |
       v
Promise Resolver

When a request is sent:

waitForEngineResponse(
  correlationId,
  timeoutMs,
);

we save its promise resolver.

Later, when the engine responds, we resolve the correct promise.

This allows multiple requests to be processed simultaneously.

Listening for Engine Responses

The backend continuously waits for messages.

for (;;) {
  const response =
    await subscriber.brPop(
      env.responseQueue,
      0,
    );
}

BRPOP means:

Block until a message arrives.

When a response arrives:

resolveEngineResponse(
  parsedResponse,
);

the correct pending request is completed.

Using the Engine from an API

Let's connect everything to our order endpoint.

update src/routes/index.ts

import { Router } from "express";
import { authRouter } from "./auth-routes.js";
import { exchangeRouter } from "./exchange-routes.js";

export const appRouter = Router();

appRouter.use(authRouter);
appRouter.use(exchangeRouter);

create src/routes/exchange-routes.ts

import { Router } from "express";
import {
  cancelOrder,
} from "../controllers/exchange-controller.js";
import { requireAuth } from "../utils/auth.js";
import { asyncHandler } from "../utils/async-handler.js";

export const exchangeRouter = Router();

exchangeRouter.post("/order", requireAuth, asyncHandler(createOrder));

Controller: src/controllers/exchange-controller.ts

export async function createOrder(
  req: Request,
  res: Response,
): Promise<void> {
  const engineResponse =
    await sendToEngine(
      "create_order",
      {
        userId,
        type,
        side,
        symbol,
        price,
        qty,
      },
    );

  res.status(
    engineResponse.ok ? 200 : 400,
  ).json(
    engineResponse.ok
      ? engineResponse.data
      : {
          error:
            engineResponse.error,
        },
  );
}

Notice something important:

The backend does not contain any matching logic.

Its only responsibilities are:

Authenticate the user
Validate the request
Send a message to Redis
Wait for a response
Return the result

The engine handles all trading logic.

Building the Engine Service

Create a separate engine project.

Install dependencies:

bun add redis

Configure Environment Variables

Create .env:

REDIS_URL="your-redis-url"

INCOMING_QUEUE="backend-to-engine-broker"

Create src/utils/env.ts:

import "dotenv/config";

function readRequiredEnv(name: string): string {
  const value = process.env[name];

  if (!value) {
    throw new Error(
      `Missing required env variable: ${name}`,
    );
  }

  return value;
}

export const env = {
  redisUrl: readRequiredEnv("REDIS_URL"),

  incomingQueue:
    process.env.INCOMING_QUEUE ??
    "backend-to-engine-broker",
};

Notice that we only configure INCOMING_QUEUE.

There is no RESPONSE_QUEUE here.

Why?

Because the engine doesn't have a fixed response queue.

Instead, the backend tells the engine which queue to respond to.

How the Backend Sends Requests

When the backend publishes a message, it includes:

{
  "correlationId": "123",
  "responseQueue": "response-queue-1234",
  "type": "create_order",
  "payload": {}
}

Notice the responseQueue field.

The backend is saying:

Process this request and send the response back to response-queue-1234.

This allows multiple backend instances to share the same engine.

Redis Connections

Inside the engine we create two Redis clients:

const brokerClient =
  createClient({
    url: env.redisUrl,
  });

const responseClient =
  createClient({
    url: env.redisUrl,
  });

Why Two Redis Clients?

brokerClient

Used to consume messages from:

backend-to-engine-broker

Example:

await brokerClient.brPop(
  env.incomingQueue,
  0,
);

This client continuously waits for requests coming from the backend.

responseClient

Used to publish responses back to the backend.

Example:

await responseClient.lPush(
  responseQueue,
  JSON.stringify(response),
);

We keep publishing and consuming on separate connections because BRPOP is a blocking operation.

Listening For Requests

The engine continuously waits for messages:

for (;;) {
  const item =
    await brokerClient.brPop(
      env.incomingQueue,
      0,
    );
}

Flow:

Backend
    |
    v
backend-to-engine-broker
    |
    v
brokerClient

Whenever a message arrives:

{
  "type": "create_order"
}

the engine begins processing it.

Processing the Request

After receiving a message:

const data = handleEngineRequest(message);

the engine executes the appropriate business logic.

For example:

create_order
      |
      v
Order Book
      |
      v
Matching Logic
      |
      v
Trade Result

The result is stored in data.

Sending Responses Back

After processing finishes:

await sendResponse(
  message.responseQueue,
  {
    correlationId:
      message.correlationId,
    ok: true,
    data,
  },
);

Notice that we are not sending the response to a hardcoded queue.

Instead we use:

message.responseQueue

which came from the backend request.

For example:

{
  "responseQueue":
    "response-queue-1234"
}

The response is therefore pushed into:

response-queue-1234

Why Send the Correlation ID Back?

The engine also returns:

{
  correlationId:
    message.correlationId
}

Example:

{
  "correlationId": "123",
  "ok": true,
  "data": {}
}

The backend uses this correlation ID to identify which pending request should receive the response.

Without it, the backend wouldn't know which API request is waiting for this result.

Complete Flow

User
 |
 | POST /order
 v
createOrder()
 |
 | validate request
 v
sendToEngine()
 |
 | create correlationId
 | store Promise resolver
 v
pendingResponses Map
(correlationId -> Promise)
 |
 | LPUSH
 v
Redis Queue
backend-to-engine-broker
 |
 | BRPOP
 v
brokerClient (Engine)
 |
 v
handleEngineRequest()
 |
 | process order
 v
sendResponse()
 |
 | LPUSH
 v
Response Queue
response-queue-1234
 |
 | BRPOP
 v
listenForEngineResponses()
 |
 | resolveEngineResponse()
 v
pendingResponses Map
 |
 | resolve Promise
 v
sendToEngine() returns
 |
 v
createOrder()
 |
 v
HTTP Response
 |
 v
User

This gives us a full request-response pattern over Redis while keeping the backend and engine completely decoupled.

Step 11: Implementing Deposits in the Exchange

Before implementing order matching, I needed a way for users to add funds to their trading accounts.

When a user places a buy order, the engine must verify that the user actually has enough balance available.

For example:

User deposits $1000

Balance:
USD
├─ Available: 1000
└─ Locked: 0

Now suppose the user creates a limit buy order:

Buy 2 SOL @ $100

The engine needs to reserve (lock) funds for that order.

USD
├─ Available: 800
└─ Locked: 200

Because balance validation happens on every order request, balances need to be available instantly.

Querying PostgreSQL for every order would add unnecessary latency and create concurrency issues.

That's why most centralized exchanges keep balances in memory inside the matching engine.

A Question About Architecture

Initially, my idea was:

POST /deposit
    |
    v
Backend updates PostgreSQL
    |
    v
Publish balance_updated event
    |
    v
Engine updates in-memory balance

The backend/database would be the source of truth, while the engine would simply consume balance updates.

I asked others whether this approach was correct.

The feedback was:

The engine must be the source of truth. Send updates to the engine first, then persist them asynchronously.

At first this seemed confusing, but after thinking through a real example it made sense.

Why the Engine Must Be the Source of Truth

Imagine a user deposits:

$1000

Database:

Balance = $1000

Now the user creates an order worth:

$200

The engine locks those funds:

Available = $800
Locked = $200

But if the database hasn't been updated yet:

Database = $1000
Engine   = $800 available

Now another deposit arrives.

If the backend reads the stale database value and calculates the new balance from it, the balance becomes incorrect.

The engine and database are no longer synchronized.

This is exactly why the matching engine must own all balance changes.

Final Architecture

The flow now becomes:

POST /deposit
    |
    v
Backend
    |
    v
Redis Queue
    |
    v
Engine
    |
    v
Update In-Memory Balance
    |
    v
Send Response
    |
    v
Persist Snapshot Later

The engine becomes responsible for:

Deposits

Every balance mutation happens in one place.

This prevents race conditions and keeps balances consistent.

Implementing the Deposit API

Added a new route:

exchangeRouter.post(
  "/deposit",
  requireAuth,
  asyncHandler(depositMoney),
);

Validation schema:

export const depositSchema = z.object({
  asset: z.string().trim().min(1, "asset is required"),
  amount: z.number().positive("amount must be positive"),
});

Controller:

export async function depositMoney(
  req: Request,
  res: Response,
): Promise<void> {
  const userId = getUserId(req);

  const parsedBody = depositSchema.safeParse(req.body);

  if (!parsedBody.success) {
    sendValidationError(res, parsedBody.error);
    return;
  }

  const { asset, amount } = parsedBody.data;

  const engineResponse = await sendToEngine("deposit", {
    userId,
    asset,
    amount,
  });

  res.status(engineResponse.ok ? 200 : 400).json(
    engineResponse.ok
      ? engineResponse.data
      : {
          error: engineResponse.error,
        },
  );
}

Handling Deposits Inside the Engine

A new engine command was added:

export type EngineCommandType =
  | "deposit"
  | "create_order"
  | "get_depth"
  | "get_user_balance"
  | "get_order"
  | "cancel_order";

When the engine receives a deposit request, it updates its in-memory balance store:

export function handleDeposit(
  payload: Record<string, unknown>,
) {
  const userId = payload.userId as string;
  const asset = payload.asset as string;
  const amount = Number(payload.amount);

  let balances = BALANCES.get(userId);

  if (!balances) {
    balances = {};
    BALANCES.set(userId, balances);
  }

  const balance = balances[asset] ?? {
    available: 0,
    locked: 0,
  };

  balance.available += amount;

  balances[asset] = balance;

  return {
    userId,
    asset,
    balance,
  };
}

Deposit Request Flow

POST /deposit
    |
    v
depositMoney()
    |
    v
sendToEngine("deposit")
    |
    | LPUSH
    v
Redis Queue
backend-to-engine-broker
    |
    v
Engine
    |
    v
handleDeposit()
    |
    v
BALANCES Map Updated
    |
    v
Response Queue
response-queue-1234
    |
    v
Backend Listener
    |
    v
HTTP Response

With deposits working, the engine now owns balance management and is ready for the next step: validating balances and locking funds during order creation.

Step 12: Lets Implement Limit Order Matching Engine from Scratch

1. Creating an Order Record

Every order gets a unique ID and is stored before matching begins.

const { orderId, order } = createLimitOrderRecord(input);

Implementation:

export function createLimitOrderRecord(input: CreateOrderInput) {
  const orderId = crypto.randomUUID();

  const order: OrderRecord = {
    orderId,
    userId: input.userId,
    side: input.side,
    type: input.type,
    symbol: input.symbol,
    price: input.price,
    qty: input.qty,
    filledQty: 0,
    status: "open",
    fills: [],
    createdAt: Date.now(),
  };

  return { orderId, order };
}

2. Locking User Balances

Before matching, funds/assets are moved from available balance to locked balance.

lockLimitBalance(input);

Implementation:

if (input.side === "buy") {
  usdBalance.available -= requiredAmount;
  usdBalance.locked += requiredAmount;
} else {
  assetBalance.available -= input.qty;
  assetBalance.locked += input.qty;
}

Why?

Without locking:

Available USD = 1000

Order #1 -> BUY 5 SOL
Order #2 -> BUY 5 SOL

Both orders could spend the same $1000.

3. Matching Against the Order Book

The engine looks at the opposite side of the book.

const oppositeSide =
  input.side === "buy"
    ? book.asks
    : book.bids;

Then prices are sorted.

const prices = [...oppositeSide.keys()].sort(
  (a, b) => input.side === "buy"
    ? a - b
    : b - a
);

This guarantees:

BUY  -> cheapest sellers first
SELL -> highest buyers first

4. Preventing Self-Trading

A trader should never match against their own order.

if (restingOrder.userId === input.userId) {
  continue;
}

Example:

Shubham places:

BUY 5 SOL @ 100

Then:

SELL 5 SOL @ 100

Engine skips the order.

5. Partial Fill Support

The engine calculates how much can actually trade.

const matchedQty = Math.min(
  remainingQty,
  availableQty,
);

Example:

Incoming BUY = 10 SOL

Available SELL = 3 SOL

Matched = 3 SOL
Remaining = 7 SOL

6. Creating Trade Executions (Fills)

Every successful trade creates a fill.

const fill = createFill({
  input,
  orderId,
  restingOrder,
  price,
  matchedQty,
});

Implementation:

const fill: Fill = {
  fillId: crypto.randomUUID(),
  symbol: input.symbol,
  price,
  qty: matchedQty,
};

7**. Maintaining Trade History**

Every fill is stored globally.

FILLS.push(fill);

This enables:

Recent Trades
Trade History
Market Statistics
Volume Calculation

8. Updating Maker Orders

When a trade occurs, the resting order in orderbook must be updated.

updateMakerOrder({
  orderId: restingOrder.orderId,
  matchedQty,
  fill,
});

Implementation:

makerOrder.filledQty += matchedQty;
makerOrder.fills.push(fill);

9. Balance Settlement

After matching, assets and funds are exchanged.

settleLimitTrade({
  input,
  restingOrder,
  matchedQty,
  price,
});

Example:

Buyer:
+2 SOL

Seller:
+$200

10. Removing Filled Orders

Completed orders should disappear from the book.

removeFilledOrders({
  side: oppositeSide,
  price,
  restingOrders,
});

Implementation:

side.set(
  price,
  restingOrders.filter(
    (o) => o.status !== "filled"
  )
);

11. Average Execution Price

Orders can execute across multiple price levels.

Example:

2 SOL @ 100
3 SOL @ 110
5 SOL @ 120

Calculation:

totalTradedValue += matchedQty * price;
totalFilledQty += matchedQty;

averagePrice = totalTradedValue / totalFilledQty;

Result:

((2×100)+(3×110)+(5×120))/10 = 113

12. Adding Remaining Quantity to the Order Book

If the order isn't fully filled, the remaining quantity becomes a maker order.

addRestingLimitOrder({
  input,
  orderId,
  remainingQty,
});

Implementation:

sideMap.set(input.price!, level);

Example:

BUY 10 SOL @ 100

Matched = 6
Remaining = 4

Explore the Complete Source Code

Above we we focused on the core concepts behind a limit order matching engine:

Order Creation
Balance Locking
Order Matching
Partial Fills
Trade Execution (Fills)
Order Book Management
Balance Settlement
Average Price Calculation

To keep the article readable, I only included the most important snippets.

If you'd like to explore the complete implementation, including all helper functions, data structures, and detailed inline comments, check out the full project on GitHub.

src/
├── orders/
│   ├── handleCreateOrder.ts
│   ├── handleLimitOrder.ts
│   ├── limit/
│   │   ├── createLimitOrderRecord.ts
│   │   ├── lockLimitBalance.ts
│   │   ├── matchLimitOrder.ts
│   │   ├── settleLimitTrade.ts
│   │   └── addRestingLimitOrder.ts
│   │
│   └── shared/
│       ├── createFill.ts
│       ├── updateMakerOrder.ts
│       └── removeFilledOrders.ts
│
├── store/
│   └── exchange-store.ts
│
└── utils/
    ├── getBalance.ts
    └── getOrderBook.ts

GitHub Repository:

https://github.com/shubhamsinghbundela/centralized-exchange

I've added detailed comments throughout the codebase explaining the matching process, balance movements, settlement logic, and order book updates step by step.

Step 13: Let's Implement a Market Order Matching Engine from Scratch

Unlike limit orders, market orders do not specify a price.

The goal is simple:

Execute immediately against the best available liquidity in the order book.

This means a market order can consume multiple price levels until the requested quantity is completely filled or liquidity runs out.

Let's build it step by step.

1. Validating and Locking Balances

Before matching begins, the exchange must verify that the user has sufficient funds/assets.

validateAndLockMarketBalance(input);

Implementation:

if (input.side === "sell") {
  assetBalance.available -= input.qty;
  assetBalance.locked += input.qty;
}

if (input.side === "buy") {
  validateMarketBuyBalance(input);
}

Why?

Without locking, the same funds could be spent multiple times before matching completes.

Example:

Available SOL = 10
Order #1 -> SELL 10 SOL
Order #2 -> SELL 10 SOL

Both orders could attempt to sell the same assets.

Locking prevents this.

2. Simulating Market Buy Cost

Market buy orders are unique because they don't specify a price.

The engine must determine how much USD will be required before execution.

validateMarketBuyBalance(input);

The engine walks through the ask side of the book.

const prices = [...book.asks.keys()].sort(
  (a, b) => a - b,
);

Starting from the cheapest ask:

Example:

ORDERS BOOK
SELL 2 SOL @ 100
SELL 3 SOL @ 110
SELL 5 SOL @ 120

Market BUY 10 SOL

Required USD:

(2 × 100) + (3 × 110) + (5 × 120) = 1130 USD

If the user doesn't have enough balance, the order is rejected before matching starts.

3. Creating the Market Order Record

Every market order receives a unique identifier.

const order: OrderRecord = {
  orderId,
  userId: input.userId,
  side: input.side,
  type: "market",
  symbol: input.symbol,
  price: null,
  qty: input.qty,
  filledQty: 0,
  status: "open",
  fills: [],
};

4. Finding the Opposite Side of the Book

Market orders always consume existing liquidity.

const oppositeSide =
  input.side === "buy"
    ? book.asks
    : book.bids;

Meaning:

Market BUY → consumes asks

Market SELL → consumes bids

5. Matching Best Prices First

Prices are sorted according to exchange priority rules.

const prices = [...oppositeSide.keys()].sort(
  (a, b) =>
    input.side === "buy"
      ? a - b
      : b - a,
);

This guarantees:

BUY orders get the cheapest sellers first

SELL orders get the highest buyers first

6. Preventing Self-Trading

A trader should never execute against their own resting order.

if (restingOrder.userId === input.userId) {
  continue;
}

Example:

Shubham places:

BUY 5 SOL @ 100

Then submits:

Market SELL 5 SOL

The engine skips his own order.

7. Supporting Partial Fills

The engine calculates how much quantity can actually trade.

const matchedQty = Math.min(
  remainingQty,
  availableQty,
);

Example:

Incoming Market BUY = 10 SOL

Available SELL = 3 SOL

Matched = 3 SOL

Remaining = 7 SOL

The engine continues searching for liquidity.

8. Creating Trade Executions (Fills)

Every successful match creates a fill.

const fill = createFill({
  input,
  orderId,
  restingOrder,
  price,
  matchedQty,
});

Implementation:

const fill: Fill = {
  fillId: crypto.randomUUID(),
  symbol: input.symbol,
  price,
  qty: matchedQty,
};

Each fill represents an executed trade.

9. Maintaining Trade History

Every fill is stored globally.

FILLS.push(fill);

This powers:

Recent Trades
Trade History
Market Statistics
Volume Tracking

10. Updating Maker Orders

The resting order must be updated after every match.

updateMakerOrder({
  orderId: restingOrder.orderId,
  matchedQty,
  fill,
});

Implementation:

makerOrder.filledQty += matchedQty;
makerOrder.fills.push(fill);

This ensures accurate order status tracking.

11. Balance Settlement

After a trade executes, assets and funds are exchanged.

settleMarketTrade({
  input,
  restingOrder,
  matchedQty,
  price,
});

Example:

Buyer: +2 SOL

Seller: +200 USD

The exchange updates balances immediately after execution.

12. Removing Filled Orders

Orders that are completely filled should disappear from the book.

removeFilledOrders({
  side: oppositeSide,
  price,
  restingOrders,
});

Implementation:

side.set(
  price,
  restingOrders.filter(
    (o) => o.status !== "filled"
  )
);

This keeps the order book clean.

13. Calculating Average Execution Price

A market order may execute across multiple price levels.

Example:

2 SOL @ 100
3 SOL @ 110
5 SOL @ 120

Calculation:

totalTradedValue += matchedQty * price;
totalFilledQty += matchedQty;

Average Price:

averagePrice =
  totalTradedValue /
  totalFilledQty;

Result:

((2×100)+(3×110)+(5×120))/10 = 113

This is the final execution price reported to the trader.

14. Handling Unfilled Quantity

Unlike limit orders, market orders never rest on the order book.

Example:

Market BUY 10 SOL
Available liquidity = 6 SOL

Matched = 6
Remaining = 4

The remaining quantity is cancelled.

For sell orders, locked assets are returned.

assetBalance.locked -= remainingQty;
assetBalance.available += remainingQty;

This ensures users never lose balances for unfilled market orders.

Explore the Complete Source Code

Above we focused on the core concepts behind a market order matching engine:

Balance Validation
Fund Locking
Market Order Execution
Liquidity Consumption
Partial Fills
Trade Execution (Fills)
Trade History
Balance Settlement
Average Execution Price
Order Book Cleanup

To keep the article readable, I only included the most important snippets.

If you'd like to explore the complete implementation, including helper functions, data structures, and detailed inline comments, check out the full project on GitHub.

src/
├── orders/
│   ├── handleMarketOrder.ts
│   │
│   └── market/
│       ├── validateAndLockMarketBalance.ts
│       ├── validateMarketBuyBalance.ts
│       ├── matchMarketOrder.ts
│       └── settleMarketTrade.ts
│
├── orders/shared/
│   ├── createFill.ts
│   ├── updateMakerOrder.ts
│   └── removeFilledOrders.ts
│
├── store/
│   └── exchange-store.ts
│
└── utils/
    ├── getBalance.ts
    └── getOrderBook.ts

GitHub Repository:

https://github.com/shubhamsinghbundela/centralized-exchange

I've added detailed comments throughout the codebase explaining market order execution, liquidity consumption, balance settlement, fill generation, and order book updates step by step.

Step 14: Testing a Centralized Exchange with Bun Test and Supertest

After implementing the matching engine, order book, market orders, and balance settlement logic, I wanted confidence that new changes wouldn't break existing functionality.

To achieve this, I started writing automated tests using:

Bun Test
Supertest

The goal is simple:

Whenever code changes, the exchange should automatically verify that critical workflows still work correctly.

Why Testing Matters

Imagine making a small change in the authentication layer.

Without tests:

Signup may stop working.
Signin may break.
Existing users may no longer authenticate.

You may not discover the issue until production.

Automated tests catch these problems immediately.

Test Structure

For API testing, I created a dedicated test directory inside the backend.

backend/
└── test/
    └── auth.test.ts

The authentication test suite covers:

Successful signup
Duplicate username validation
Invalid request validation
Successful signin
Unknown user signin
Invalid password signin

Repository:

https://github.com/shubhamsinghbundela/centralized-exchange

File:

backend/test/auth.test.ts

The tests are written using Bun Test and Supertest, allowing HTTP endpoints to be tested without manually sending requests from Postman.

This gives fast feedback during development and helps prevent regressions whenever authentication logic changes.

Testing Exchange APIs

After authentication tests, I started testing exchange-specific APIs.

The first endpoint was deposits.

backend/
└── test/
    ├── auth.test.ts
    └── exchange.test.ts

The deposit test suite covers:

Successful deposits
Multiple deposits accumulating balance
Invalid deposit requests
Unauthenticated requests

Step15: Testing the Limit Order Matching Engine

After testing the deposit, I moved on to one of the most important components of the exchange: the limit order matching engine.

Writing Unit Tests

To verify that the engine behaved correctly under different market scenarios, I created a dedicated test suite:

engine/tests/limit-order.test.ts

Check out test code in Github:

https://github.com/shubhamsinghbundela/centralized-exchange/blob/main/engine/tests/limit-order.test.ts

I've added detailed test cases covering realistic exchange behavior to ensure the matching engine remains reliable as new features are added.

Step16: Testing the Market Order Matching Engine

Writing Unit Tests

engine/tests/market-order.test.ts

Check out test code in Github:

https://github.com/shubhamsinghbundela/centralized-exchange/blob/main/engine/tests/market-order.test.ts

Step17: Implementing and Testing the Depth API

After testing limit order matching engine, I worked on the order book depth endpoint.

GET /depth/:symbol

The goal of this endpoint is to return the current state of the order book.

Example:

GET /depth/BTC

The request first reaches the backend, where the symbol is validated using Zod.

export const symbolParamSchema = z.object({
  symbol: z.string().trim().min(1, "symbol is required"),
});

Once validated, the backend forwards the request to the matching engine.

const engineResponse = await sendToEngine(
  "get_depth",
  { symbol }
);

Inside the engine, I implemented a dedicated depth service.

engine/src/depth/getDepth.ts

The service performs three important tasks:

Reads the current order book
Aggregates orders at the same price level
Returns sorted bids and asks

import {
  ORDERBOOKS,
  type DepthLevel,
  type DepthResponse,
} from "../store/exchange-store.js";

export function getDepth(symbol: string): DepthResponse {
  const orderBook = ORDERBOOKS.get(symbol);

  if (!orderBook) {
    return {
      symbol,
      bids: [],
      asks: [],
    };
  }

  const bids: DepthLevel[] = [...orderBook.bids.entries()]
    .sort((a, b) => b[0] - a[0])
    .slice(0, 20)
    .map(([price, orders]) => ({
      price,
      qty: orders.reduce(
        (sum, order) => sum + (order.qty - order.filledQty),
        0,
      ),
    }));

  const asks: DepthLevel[] = [...orderBook.asks.entries()]
    .sort((a, b) => a[0] - b[0]) // lowest ask first
    .slice(0, 20)
    .map(([price, orders]) => ({
      price,
      qty: orders.reduce(
        (sum, order) => sum + (order.qty - order.filledQty),
        0,
      ),
    }));

  return {
    symbol,
    bids,
    asks,
  };
}

After implementing the engine logic, I wrote unit tests to verify that depth calculations were correct.

engine/test/getDepth.test.ts

Writing Unit Tests

getDepth.test.ts

Step 18: Implementing and Testing User Balances

After implementing and testing the depth endpoint, I worked on the balance API.

GET /balance

The purpose of this endpoint is simple: return the current balances for the authenticated user.

The backend forwards the request to the matching engine using the authenticated user's ID.

const engineResponse = await sendToEngine(
  "get_user_balance",
  {
    userId: getUserId(req),
  }
);

Inside the engine, I implemented a dedicated balance service.

engine/src/balance/getUserBalance.ts

The implementation is intentionally simple.

import { BALANCES } from "../store/exchange-store";

export function getUserBalance(userId: string) {
  return BALANCES.get(userId) ?? {};
}

Since all balance updates occur during deposits, order creation, matching, settlement, and cancellations, the balance service only needs to return the latest state stored in memory.

I wrote unit tests for getUserBalance:

engine/tests/getUserBalance.test.ts

Writing Unit Tests

https://github.com/shubhamsinghbundela/centralized-exchange/blob/main/engine/tests/getUserBalance.test.ts

Step 19: Implementing and Testing Order Retrieval

After implementing balance tracking, I worked on the order retrieval API.

GET /order/:orderId

The purpose of this endpoint is to allow users to retrieve the current state of a specific order.

Example:

GET /order/094e0d93-0ac0-4e26-a081-6393e49eb82e

The backend validates the order ID and forwards the request to the matching engine.

const engineResponse = await sendToEngine(
  "get_order",
  {
    userId: getUserId(req),
    orderId,
  }
);

Inside the engine, I implemented a dedicated order lookup service.

engine/src/order/getOrder.ts

import { ORDERS } from "../store/exchange-store";

export function getOrder(userId: string, orderId: string) {
  const order = ORDERS.get(orderId);

  if (!order) {
    throw new Error("Order not found");
  }

  if (order.userId !== userId) {
    throw new Error("Order not found");
  }

  return order;
}

Writing Unit Tests

engine/tests/getOrder.test.ts

GitHub Repository:

https://github.com/shubhamsinghbundela/centralized-exchange/blob/main/engine/tests/getOrder.test.ts

Step 20: Implementing and Testing Order Cancellation

After implementing order retrieval, I worked on order cancellation.

The endpoint:

DELETE /order/:orderId

allows traders to cancel open or partially filled orders.

Example:

DELETE /order/094e0d93-0ac0-4e26-a081-6393e49eb82e

The backend validates the order ID and forwards the request to the matching engine.

const engineResponse = await sendToEngine(
  "cancel_order",
  {
    userId: getUserId(req),
    orderId,
  }
);

Inside the engine, I implemented a dedicated cancellation service.

engine/src/orders/cancelOrder.ts

import { BALANCES, ORDERBOOKS, ORDERS } from "../store/exchange-store";

export function cancelOrder(userId: string, orderId: string) {
  const order = ORDERS.get(orderId);

  if (!order) {
    throw new Error("order not found");
  }

  if (order.userId !== userId) {
    throw new Error("order not found");
  }

  if (order.status === "filled") {
    throw new Error("filled orders cannot be cancelled");
  }

  if (order.status === "cancelled") {
    throw new Error("order already cancelled");
  }

  // remove from orderbook
  const orderBook = ORDERBOOKS.get(order.symbol);

  const levels = order.side === "buy" ? orderBook?.bids : orderBook?.asks;

  const priceLevel = levels?.get(order.price!);

  if (priceLevel) {
    const updated = priceLevel.filter((o) => o.orderId !== orderId);

    if (updated.length === 0) {
      levels?.delete(order.price!);
    } else {
      levels?.set(order.price!, updated);
    }
  }
  // unlock balances
  const remainingQty = order.qty - order.filledQty;
  const balances = BALANCES.get(userId)!;

  if (order.side === "buy") {
    const refund = remainingQty * order.price!;

    balances.USD!.locked -= refund;
    balances.USD!.available += refund;
  } else {
    const assetBalance = balances[order.symbol];

    if (!assetBalance) {
      throw new Error(`${order.symbol} balance not found`);
    }

    assetBalance.locked -= remainingQty;
    assetBalance.available += remainingQty;
  }

  order.status = "cancelled";

  return {
    orderId,
    status: "cancelled",
    qty: order.qty,
    filledQty: order.filledQty,
  };
}

Validation Rules

Before cancelling an order, the engine performs several checks.

The order must:

Exist
Belong to the requesting user
Not already be filled
Not already be cancelled

Removing Orders From The Book

When an order is cancelled, it should no longer appear in market depth.

The engine removes the order from its price level.

const updated = priceLevel.filter(
  (o) => o.orderId !== orderId
);

If no orders remain at that price level, the level itself is removed.

Unlocking Funds

One of the most important parts of cancellation is releasing locked balances.

For buy orders:

Locked USD → Available USD

Example:

BUY 10 BTC @ 100

Locked USD = 1000

After cancellation:

Locked USD = 0
Available USD restored

For sell orders:

Locked BTC → Available BTC

Example:

SELL 10 BTC

After cancellation:

Locked BTC = 0
Available BTC restored

Partial Fill Handling

Cancellation must only release the remaining unfilled quantity.

Example:

BUY 10 BTC @ 100

After:

Filled = 4 BTC
Remaining = 6 BTC

When cancelled:

Only 6 BTC worth of funds
are unlocked.

The executed portion remains part of trade history.

Writing Unit Tests

To validate the implementation, I written test in below file:

engine/tests/cancelOrder.test.ts

GitHub Repository:

https://github.com/shubhamsinghbundela/centralized-exchange/blob/main/engine/tests/cancelOrder.test.ts

Step 21: Understanding Latency Before Load Testing

After completing all exchange functionality (deposits, withdrawals, order placement,matching, orderbook updates, fills, order retrieval, and order cancellation), I wanted to understand how fast the system actually was.

Before running any load tests, I spent some time learning what latency mean?

What is Latency?

Latency is the total time between sending a request and receiving a response.

When a browser requests a webpage, several steps happen:

DNS lookup
TCP handshake
TLS handshake (HTTPS)
Request sent to server
Server processing
Response returned to client

Each step adds delay.

Example:

Step	Time
DNS lookup	20 ms
TCP handshake	30 ms
TLS handshake	50 ms
Server processing	100 ms
Response travel back	30 ms
Total latency	230 ms

The user experiences the entire 230 ms as waiting time.

Why Latency Matters

Modern applications rarely make a single request.

A webpage might need:

1 HTML file
5 CSS files
10 JavaScript files
20 images

Total = 36 requests

If every request has 100 ms latency:

36 × 100 ms = 3600 ms

or roughly 3.6 seconds of waiting spread throughout page loading.

Even small latency improvements can noticeably improve user experience.

Types of Latency

Network Latency

Time spent moving data across the network.

Browser
   ↓
Internet
   ↓
Server

Example:

50 ms to reach server
50 ms to return

Network latency ≈ 100 ms round trip.

Server / Disk Latency

Time spent inside the server before generating a response.

Example:

Request arrives
      ↓
Database read
      ↓
Business logic
      ↓
Response generated
      ↓
Response sent

If a database query takes 300 ms, that delay is server-side latency.

Since my exchange keeps order books and balances in memory, I expected server-side latency to remain relatively low compared to systems heavily dependent on database reads.

Types of Latency

Network Latency

Time spent moving data across the network.

Browser
   ↓
Internet
   ↓
Server

Example:

50 ms to reach server
50 ms to return

Network latency ≈ 100 ms round trip.

Server / Disk Latency

Time spent inside the server before generating a response.

Example:

Request arrives
      ↓
Database read
      ↓
Business logic
      ↓
Response generated
      ↓
Response sent

If a database query takes 300 ms, that delay is server-side latency.

Since my exchange keeps order books and balances in memory, I expected server-side latency to remain relatively low compared to systems heavily dependent on database reads.

Step 22: Learning k6 for Load Testing

Once I understood latency, I started learning k6, an open-source load testing tool.

The goal was to simulate real users interacting with the exchange and measure:

API latency
Throughput
Failure rates
System behavior under concurrent load

Important k6 Concepts

Virtual Users (VU)

A Virtual User simulates a real user.

Example:

vus: 100

means 100 concurrent users are making requests.

Iteration

One complete execution of the default() function.

1 iteration = one full user workflow

HTTP Metrics

k6 automatically reports metrics such as:

http_req_duration:
avg=189ms
min=182ms
max=472ms
p(90)=189ms
p(95)=191ms

Meaning:

avg → average response time
min → fastest request
max → slowest request
p90 → 90% of requests completed below this value
p95 → 95% of requests completed below this value

http_req_duration is essentially the API latency observed by the client.

Failed Requests

http_req_failed

Shows the percentage of failed requests.

Using check()

Without checks:

http.post(...)

k6 only knows that a request was sent.

With checks:

check(res, {
  "status is 200": (r) => r.status === 200,
});

k6 can verify correctness of responses.

Understanding stdout

Everything printed in the terminal is called stdout.

Useful flags:

k6 run --quiet test.js

Hides execution details and progress bars.

k6 run --log-output=none test.js

Disables logs.

k6 run --quiet --log-output=none test.js

Shows only the final summary.

Step 23: Seeding 100 Test Users

Before performing load tests, I needed users with balances.

I generated JWT tokens for 100 unique users and used a k6 setup script to seed balances.

Each user received:

USD = 1,000,000
BTC = 1,000

The setup function runs once before the actual test and prepares the environment.

This step was not load testing itself—it was test data preparation.

Code: https://github.com/shubhamsinghbundela/centralized-exchange/blob/main/load-tests/setup.js

Step 24: Simulating Exchange Traffic

To mimic a real exchange, I created three different user groups.

Makers (30 VUs)

Makers place limit orders that provide liquidity.

30 concurrent users

Actions:

Place BUY limit orders
Place SELL limit orders

Example:

type: "limit"

Takers (30 VUs)

Takers consume liquidity by submitting market orders.

30 concurrent users

Actions:

Market BUY
Market SELL

Example:

type: "market"

Depth Readers (40 VUs)

Many traders continuously monitor market depth.

40 concurrent users

Actions:

GET /depth/BTC

This simulates users refreshing the orderbook view.

Test Distribution

User Type	VUs
Makers	30
Takers	30
Depth Readers	40
Total	100

The test generated continuous order placement, matching, and orderbook reads simultaneously, helping evaluate the exchange under concurrent trading activity.

Code: https://github.com/shubhamsinghbundela/centralized-exchange/blob/main/load-tests/cex-benchmark.js

k6 benchmark results under 100 concurrent users (30 makers, 30 takers, 40 depth readers

Step 25: Understanding Redis Persistence and Durability

After benchmarking the exchange under load, I started thinking about a different problem:

What happens if the matching engine crashes?

At that point, the exchange state was stored entirely in memory:

BALANCES
ORDERS
ORDERBOOKS
FILLS

A process restart would wipe everything.

To understand how production systems handle this, I studied Redis persistence using the Redis tutorial on persistence and durability.

https://redis.io/tutorials/operate/redis-at-scale/persistence-and-durability/

RDB Snapshots

RDB works like taking periodic photos of the database.

Imagine Redis takes snapshots of the current state:

Time 0s   → Snapshot 
Time 20s  → Snapshot 
Time 40s  → Snapshot

Example configuration:

save 20 3

Meaning:

At least 3 keys changed
Within 20 seconds

Redis creates a snapshot file:

dump.rdb

The snapshot contains the entire dataset at that moment.

Limitation of RDB

Suppose:

12:00 Snapshot created
12:01 SET d 4
12:02 SET e 5
12:03 Server crashes

The writes after the last snapshot are lost.

d = 4
e = 5

This is the trade-off of snapshot-based persistence.

AOF (Append Only File)

AOF works differently.

Instead of taking photos, Redis records every write operation.

Example:

SET a 1
SET b 2
SET c 3
SET d 4

During restart:

Start Redis
      ↓
Read AOF
      ↓
Replay Commands
      ↓
Restore Dataset

AOF provides better durability but produces larger files and slower recovery compared to RDB.

Step 26: Implementing Exchange State Snapshots

My matching engine stores data in JavaScript Maps.

BALANCES
ORDERS
ORDERBOOKS
FILLS

Redis can only persist Redis keys, not in-memory JavaScript structures.

To solve this, I implemented a snapshot layer that serializes the exchange state and stores it in Redis.

Persisting State

I created a persistence module:

persistEngineState()

which stores:

engine:balances
engine:orders
engine:orderbooks
engine:fills

inside Redis.

Example:

await redis.set(
  "engine:balances",
  JSON.stringify(Object.fromEntries(BALANCES))
);

Full Code:

src/snapshot/persistence.js

import {
  BALANCES,
  ORDERBOOKS,
  ORDERS,
  FILLS,
} from "../store/exchange-store.js";
import { createClient } from "redis";
import { env } from "../utils/env.js";

const redis = createClient({
  url: env.redisUrl,
});

await redis.connect();
export async function persistEngineState() {
  await redis.set(
    "engine:balances",
    JSON.stringify(Object.fromEntries(BALANCES)),
  );

  await redis.set("engine:orders", JSON.stringify([...ORDERS.entries()]));

  await redis.set(
    "engine:orderbooks",
    JSON.stringify(
      [...ORDERBOOKS.entries()].map(([symbol, book]) => ({
        symbol,
        bids: [...book.bids.entries()],
        asks: [...book.asks.entries()],
      })),
    ),
  );

  await redis.set("engine:fills", JSON.stringify(FILLS));
}

Step 27: Automatic Snapshot Creation

I wanted snapshots to happen automatically.

A mutation counter tracks state-changing operations:

let mutationCount = 0;

Every:

Deposit
Create Order
Cancel Order

increments the counter.

Every 5 mutations:

await persistEngineState();

await brokerClient.sendCommand(["BGSAVE"]);

This performs:

Engine State
      ↓
Redis Keys
      ↓
BGSAVE
      ↓
dump.rdb

without blocking the matching engine.

Full Code:

src/index.ts

let mutationCount = 0;

async function snapshotIfNeeded() {
  mutationCount++;

  if (mutationCount % 5 !== 0) return;

  console.log("Persisting engine state...");

  await persistEngineState();

  try {
    await brokerClient.sendCommand(["BGSAVE"]);
    console.log("Redis background snapshot started.");
  } catch (error) {
    if (
      error instanceof Error &&
      error.message.includes("Background save already in progress")
    ) {
      console.log(
        "Redis snapshot already in progress. Engine state is saved in Redis memory; skipping this snapshot.",
      );
      return;
    }

    throw error;
  }
}

async function handleEngineRequest(message: EngineRequest): Promise<unknown> {
 
  if (message.type === "create_order") {
    const result = handleCreateOrder(message.payload);

    await snapshotIfNeeded();

    return result;
  }

  if (message.type === "deposit") {
    const result = handleDeposit(message.payload);

    await snapshotIfNeeded();

    return result;
  }

  if (message.type === "cancel_order") {
    const { userId, orderId } = message.payload as {
      userId: string;
      orderId: string;
    };

    const result = cancelOrder(userId, orderId);

    await snapshotIfNeeded();

    return result;
  }

  throw new Error("TODO(student): implement this engine request type");
}

Step 28: Restoring State After Restart

Persisting data is only half the problem.

The exchange also needs to recover after a restart.

For this I implemented:

loadEngineState()

When the engine starts:

await loadEngineState();

The function reads:

engine:balances
engine:orders
engine:orderbooks
engine:fills

from Redis and reconstructs:

BALANCES
ORDERS
ORDERBOOKS
FILLS

back into memory.

For order books, nested Maps are rebuilt:

ORDERBOOKS.set(book.symbol, {
  bids: new Map(book.bids),
  asks: new Map(book.asks),
});

This allows the matching engine to resume from the previous state.

Step 29: Dockerizing Redis Persistence

To make the setup reproducible, Redis was containerized using Docker Compose.

services:
  redis:
    image: redis/redis-stack:latest
    container_name: redis-stack
    ports:
      - "6379:6379"
      - "8001:8001"
    volumes:
      - redis-data:/data
    command: >
      redis-server
      --save 60 5
      --dbfilename dump.rdb

volumes:
  redis-data:

Using a Docker volume ensures that Redis snapshots survive container restarts.

Persistence Flow

BALANCES
ORDERS
ORDERBOOKS
FILLS
        ↓
persistEngineState()
        ↓
Redis Keys
        ↓
BGSAVE
        ↓
dump.rdb
        ↓
Docker Volume

Recovery Flow:

Redis Restart
        ↓
Load dump.rdb
        ↓
loadEngineState()
        ↓
Rebuild Exchange State
        ↓
Trading Resumes

Key Learnings

Learned the difference between RDB and AOF persistence.
Understood durability vs performance trade-offs.
Implemented exchange state persistence using Redis.
Added automatic snapshots after mutations.
Implemented recovery logic for balances, orders, fills, and order books.
Used Docker volumes to preserve snapshots across container restarts.
Verified state restoration after restarting the engine.

Next step: Export Redis snapshots to AWS S3 and build a disaster recovery workflow.

I shared a quick overview of the approach and implementation related to redis persistance on X.

Tweet: https://x.com/shubhamsingh__1/status/2070360494565941303

Step 30: Eliminating Floating-Point Precision Errors with Decimal.js

After implementing Redis persistence and recovery, I discovered another issue that is critical for any financial application: floating-point precision.

JavaScript stores numbers using the IEEE-754 floating-point standard, which cannot represent many decimal values exactly.

For example:

0.1 + 0.2
// 0.30000000000000004

Although the error appears very small, it becomes a serious problem inside a trading engine where every calculation affects user balances.

Operations such as:

Deposits
Balance settlement
Order matching

must be mathematically exact.

Even tiny rounding errors can accumulate over thousands or millions of trades and eventually produce incorrect balances.

Moving to Decimal.js

To eliminate precision issues, I replaced JavaScript's native number type with Decimal.js for all financial values.

The balance model was updated from:

export interface Balance {
  available: number;
  locked: number;
}

to:

import Decimal from "decimal.js";

export interface Balance {
  available: Decimal;
  locked: Decimal;
}

Every balance calculation now uses Decimal arithmetic instead of native operators.

Example:

const price = new Decimal("110.15");
const quantity = new Decimal("0.25");

const value = price.times(quantity);

console.log(value.toString());
// 27.5375

Instead of:

110.15 * 0.25
// 27.537499999999998

Engine Changes

Migrating to Decimal.js required updating the entire matching engine.

Financial calculations now use Decimal methods such as:

.plus()
.minus()
.times()
.div()
.greaterThan()
.lessThan()
.equals()

instead of JavaScript arithmetic operators.

This guarantees that balances, prices, quantities, and settlements remain precise throughout the trading lifecycle.

Why This Matters

Financial software cannot tolerate approximation.

Unlike many web applications, an exchange processes thousands of monetary transactions where even microscopic rounding errors can result in incorrect balances.

Using Decimal.js ensures deterministic, precise, and predictable calculations throughout the engine.

Further Reading

I also wrote a detailed thread explaining why JavaScript floating-point numbers are unsafe for financial applications and how migrating to decimal.js eliminates precision errors in a matching engine.

If you're interested, you can read it here:

https://x.com/shubhamsingh__1/status/2069820565435863238?s=20

https://x.com/shubhamsingh__1/status/2069807991466111009?s=20

Step 31: Designing Data Persistence Without Slowing Down the Matching Engine

After implementing Redis snapshots and engine recovery, the next challenge was deciding how to persist executed trades, orders, and balances into PostgreSQL.

At first, the obvious solution seemed straightforward:

Order Arrives
      ↓
Matching Engine
      ↓
Update Memory
      ↓
Write to PostgreSQL

But this raised an important question.

Should the matching engine write directly to the database every time an order is matched?

The answer is no.

A matching engine has one primary responsibility:

Match orders as fast as possible.

Database writes are relatively slow compared to in-memory operations.

If every order, balance update, and trade execution waited for PostgreSQL, the engine's throughput would decrease significantly.

Understanding the Separation of Responsibilities

After discussing the architecture and researching how trading systems are designed, I realized that the matching engine should remain completely focused on matching orders.

During market hours:

All balances remain in memory.
Orders remain in memory.
Fills remain in memory.
Redis snapshots provide recovery if the engine crashes.

The architecture becomes:

Orders
      ↓
Matching Engine (Memory)
      ↓
Redis Snapshot

The engine never waits for PostgreSQL.

Where Does PostgreSQL Fit?

Initially, I was confused.

If Redis already stores the latest state and the frontend can read live data directly from Redis, what is PostgreSQL actually used for?

The answer is:

Long-term persistence.

PostgreSQL is not part of the critical trading path.

Instead, it stores historical data that must survive beyond the engine's runtime.

Examples include:

Historical orders
Executed trades
Completed fills
Account history
Reports and analytics

Why Not Write During Trading?

While the market is open:

Thousands of orders may arrive every second.
The matching engine must respond with the lowest possible latency.
Even a small delay caused by database writes can affect performance.

Therefore, speed takes priority over persistence during live trading.

Bulk Synchronization After Market Close

Once the market closes:

No new orders are arriving.
The final state of every order is already available in memory.
Redis contains the latest snapshot.

At this point, the complete market state can be synchronized to PostgreSQL in bulk.

Market Closes
      ↓
Read Final State
      ↓
Bulk Sync
      ↓
PostgreSQL

This approach keeps the matching engine fast while still maintaining permanent historical records.

Live UI During Market Hours

One question I had was how users would see newly placed orders immediately if PostgreSQL wasn't being updated continuously.

The solution is straightforward:

Live open orders are served directly from Redis (or the matching engine).
Real-time updates are delivered using WebSockets.
PostgreSQL is used only for historical records after synchronization.

This ensures users receive instant updates without introducing database latency into the matching engine.

Further Reading

While designing the persistence layer for my centralized exchange, I shared my thought process and discussed different architectural approaches with the community on X.

If you're interested in the complete discussion, you can read it here:
https://x.com/shubhamsingh__1/status/2070154350144655466

Step 32: Market Close Synchronization & Database Ownership

Before implementing the synchronization logic, I learned an important microservices principle:

Each microservice should own its own database.

Initially, both my Backend and Matching Engine shared the same PostgreSQL database.

At first, it seemed like the simplest approach. However, once I started implementing persistence for the matching engine, several problems became obvious.

The matching engine maintains the complete order book in memory throughout market hours. When the market closes, it synchronizes the final state of Orders and Fills to PostgreSQL.

Sharing the same database introduced several issues:

Prisma migrations started conflicting.
The matching engine became dependent on tables it didn't own.
Schema changes in one service could unintentionally impact another service.

The solution was to give each service complete ownership of its own data.

Backend Database
└── Users

Matching Engine Database 
├── Orders 
├── Fills

Now the architecture is much cleaner.

The backend authenticates users and forwards requests (including the authenticated userId) to the matching engine.

The engine owns everything related to:

Order matching
Order books
Trade execution
Orders
Fills
Market persistence

A simple rule that significantly improved the architecture:

A service should own both its business logic and the data behind that logic.

Designing the Engine Database

Once the matching engine had its own PostgreSQL database, the next step was defining the schema.

Unlike the backend database, which stores users and authentication data, the engine database only stores trading-related information.

The two core entities are:

Orders — every order placed in the exchange.
Fills — every trade generated when two orders are matched.

The relationship is straightforward:

Order (Buyer)   ────
                      ├──── Fill
Order (Seller)  ────┘

A single order can participate in multiple trades, so an Order has a one-to-many relationship with Fill.

The Prisma schema looks like this:

// This is your Prisma schema file,
// learn more about it in the docs: https://pris.ly/d/prisma-schema

// Get a free hosted Postgres database in seconds: `npx create-db`

generator client {
  provider = "prisma-client"
  output   = "../src/generated/prisma"
}

datasource db {
  provider = "postgresql"
}

model Order {
  orderId String @id

  userId String
  side   String
  type   String
  symbol String
  price  Decimal? 
  qty    Decimal
  filledQty Decimal
  status String
  createdAt DateTime

  buyFills  Fill[] @relation("BuyOrder")
  sellFills Fill[] @relation("SellOrder")
}

model Fill {
  fillId String @id

  symbol String
  price  Decimal
  qty    Decimal

  buyOrderId  String
  sellOrderId String

  createdAt DateTime

  buyOrder  Order @relation("BuyOrder", fields: [buyOrderId], references: [orderId])
  sellOrder Order @relation("SellOrder", fields: [sellOrderId], references: [orderId])
}

Closing the Market

To simulate market trading hours, I introduced a simple market state.

Every incoming order first checks whether the market is currently open.

export function isMarketOpen() {
  updateMarketState();
  return marketOpen;
}

Inside the order handler:

if (!isMarketOpen()) {
  throw new Error("Market is closed");
}

This prevents any new orders from entering the matching engine once trading hours have ended.

Automatically Running Market Close

Instead of manually triggering synchronization, I scheduled it using node-cron.

cron.schedule(
  env.marketCloseTime,
  async () => {
    await handleMarketClose();
  },
  {
    timezone: env.marketTimezone,
  },
);

The schedule is configurable through environment variables.

MARKET_CLOSE_CRON=30 15 * * 1-5
MARKET_TIMEZONE=Asia/Kolkata

Every weekday at 3:30 PM IST, the engine automatically starts the synchronization process.

Synchronizing Orders and Fills

When the market closes, the engine performs a bulk synchronization.

Market Closes
      ↓
Read Orders
Read Fills
      ↓
Bulk Insert
      ↓
PostgreSQL

The synchronization entry point is straightforward.

export async function handleMarketClose() {
  await syncOrders(prisma);
  await syncFills(prisma);
}

Rather than writing to the database after every trade, the engine exports the complete in-memory state once trading has finished.

Bulk Writing Orders

The engine reads every order stored in memory.

const orders = [...ORDERS.values()];

Instead of inserting one row at a time, all orders are written using createMany().

await tx.order.createMany({
  data: ...,
  skipDuplicates: true,
});

Bulk inserts dramatically reduce the number of database operations and improve synchronization performance.

Bulk Writing Trade Fills

Trade fills follow the same pattern.

Since a fill could accidentally appear more than once, duplicate entries are removed before insertion.

const uniqueFills = [
  ...new Map(FILLS.map(fill => [fill.fillId, fill])).values(),
];

The resulting list is then persisted using another bulk insert.

await tx.fill.createMany({
  data: ...,
  skipDuplicates: true,
});

This guarantees that each trade is stored only once.

Final Synchronization Flow

Market Open
      ↓
Matching Engine (Memory)
      ↓
Redis Snapshots
      ↓
Market Closes
      ↓
Read Orders & Fills
      ↓
Bulk Sync
      ↓
PostgreSQL

Throughout trading hours, the matching engine remains focused solely on low-latency order matching.

Only after trading ends does it persist historical data to PostgreSQL.

Further Reading

While designing the persistence layer and deciding how the matching engine should synchronize data with PostgreSQL, I shared my thought process and architectural decisions on X.

You can read the discussion here:

https://x.com/shubhamsingh__1/status/2070242573759955431

Step 33: Learning Real-Time Communication with WebSockets

After implementing market persistence and synchronization, the next challenge was enabling real-time communication between the matching engine and clients.

A centralized exchange isn't useful if users have to refresh the page to see order book updates or trades. Every new order, cancellation, and trade should be pushed instantly to connected clients.

Before implementing this, I wanted to understand the different approaches for building real-time systems.

Learning Socket.IO

Before building the WebSocket layer for the exchange, I spent some time learning how Socket.IO works and documenting the concepts I found most useful.

I shared my learning notes on X before diving into the implementation:

https://x.com/shubhamsingh__1/status/2069414064863019046?s=20

Understanding io vs socket

While learning Socket.IO, one concept that initially confused me was the difference between io and socket.

I documented my notes and examples on X before applying them in the project:

https://x.com/shubhamsingh__1/status/2069411978410615105?s=20

Learning Raw WebSockets (ws)

After understanding Socket.IO, I wanted to learn the WebSocket protocol itself by working with the ws library, a lightweight and widely used WebSocket implementation for Node.js.

I documented the key concepts and takeaways on X before integrating WebSockets into my exchange:

https://x.com/shubhamsingh__1/status/2070573810387153203?s=20

Socket.IO vs Raw WebSockets

After experimenting with both Socket.IO and raw WebSockets (ws), I compared their trade-offs to decide which one would be a better fit for my centralized exchange project.

I shared my comparison and reasoning on X before integrating the WebSocket layer into the exchange:

https://x.com/shubhamsingh__1/status/2070575799854633235?s=20

Why I Chose Raw WebSockets

After comparing Socket.IO and raw WebSockets (ws), I decided to use raw WebSockets for my centralized exchange project because it gives me lower protocol overhead and complete control over the communication protocol—both of which are important for latency-sensitive systems like trading engines.

I shared the reasoning behind this architectural decision on X:

https://x.com/shubhamsingh__1/status/2070577047009972618?s=20

Proof of Concept

Before integrating WebSockets into the exchange, I built a small proof-of-concept to understand client-server communication.

GitHub Repository

https://github.com/shubhamsinghbundela/websocket-poc/tree/main

Designing the WebSocket Architecture

After deciding to use raw WebSockets (ws), the next question wasn't how to write WebSocket code—it was how the architecture should work.

My matching engine was already processing orders through a Redis Queue (LPUSH + BRPOP). But broadcasting real-time order book updates is a different problem.

Before writing any code, I spent some time thinking through the architecture, discussing it with Others, and documenting my understanding.

I shared my thought process on X:

https://x.com/shubhamsingh__1/status/2071182638011060734?s=20

Step 34: Preparing the Matching Engine for Real-Time Depth Updates

Before implementing WebSocket broadcasting, I first needed to make sure the matching engine could expose the order book in a format suitable for real-time updates.

Rather than designing my own response format, I decided to follow the format used by Backpack Exchange. Using an existing API structure makes it easier for clients to consume market data and keeps the interface closer to what production exchanges expose.

The getDepth() API was updated to return:

Trading symbol
Bid levels
Ask levels
lastUpdateId

The response now looks like this:

{
  symbol: "BTCUSDT",
  bids: [
    ["101.5", "2.4"],
    ["101.4", "1.1"]
  ],
  asks: [
    ["101.6", "3.2"],
    ["101.7", "0.8"]
  ],
  lastUpdateId: 42
}

import {
  ENGINE_STATE,
  ORDERBOOKS,
  type DepthLevel,
  type DepthResponse,
} from "../store/exchange-store.js";

export function getDepth(symbol: string): DepthResponse {
  const orderBook = ORDERBOOKS.get(symbol);

  if (!orderBook) {
    return {
      symbol,
      bids: [],
      asks: [],
      lastUpdateId: ENGINE_STATE.lastUpdateId,
    };
  }

  const bids: DepthLevel[] = [...orderBook.bids.entries()]
    .sort((a, b) => b[0] - a[0])
    .map(([price, orders]) => [
      price.toString(),
      orders
        .reduce((sum, order) => sum + (order.qty - order.filledQty), 0)
        .toString(),
    ]);

  const asks: DepthLevel[] = [...orderBook.asks.entries()]
    .sort((a, b) => a[0] - b[0]) // lowest ask first
    .map(([price, orders]) => [
      price.toString(),
      orders
        .reduce((sum, order) => sum + (order.qty - order.filledQty), 0)
        .toString(),
    ]);

  return {
    symbol,
    bids,
    asks,
    lastUpdateId: ENGINE_STATE.lastUpdateId,
  };
}

Introducing lastUpdateId was especially important because it lays the foundation for streaming incremental order book updates over WebSockets.

Instead of sending the complete order book after every trade, clients can later use this update ID to apply depth deltas in the correct order and keep their local order book synchronized with the matching engine.

This was the first step toward implementing efficient real-time market data streaming.

Step 35: Building Incremental Order Book Updates (Depth Deltas)

Returning the complete order book through the getDepth() API works well when a client first loads the exchange.

However, sending the entire order book after every order would quickly become inefficient.

Imagine thousands of orders arriving every second. Broadcasting the full order book each time would waste bandwidth and force clients to repeatedly process data that hadn't changed.

Instead, I decided to send only the price levels that were modified by each order.

This is commonly known as a depth delta.

Tracking Changed Price Levels

During order matching, I introduced a new data structure:

const depthDelta: DepthDelta = {
  bids: new Set(),
  asks: new Set(),
};

As orders are matched, cancelled, or added to the order book, every affected bid or ask price is recorded in one of these sets.

Instead of scanning the entire order book later, the engine already knows exactly which price levels changed while processing the order.

Building the Depth Update

After the order has been fully processed, the engine generates a depth update.

const depthUpdate = buildDepthUpdate(
  input.symbol,
  depthDelta,
);

The buildDepthUpdate() function iterates only over the modified bid and ask levels.

For each updated price level it calculates the remaining quantity by summing the unfilled quantity of every resting order.

const qty = orders.reduce(
  (sum, order) => sum + (order.qty - order.filledQty),
  0,
);

The resulting payload contains only the information clients actually need.

{
  s: "BTCUSDT",
  b: [["101.5", "2.4"]],
  a: [["101.6", "1.8"]],
  U: 42,
  u: 42,
  T: ...
}

Where:

s → Trading symbol
b → Updated bid price levels
a → Updated ask price levels
U → First update ID
u → Last update ID
T → Event timestamp

Introducing Update IDs

Every depth update increments a global sequence number.

const updateId = ++ENGINE_STATE.lastUpdateId;

That update ID is included in every depth event.

This allows clients to process updates in order and detect if any updates were missed during transmission.

Although my current implementation increments one update ID per event (U == u), the same structure can later support batched updates where multiple sequence numbers are covered by a single message.

Returning Depth Updates

The matching engine now returns the generated depth update together with the order execution result.

return {
  orderId,
  status: order.status,
  filledQty: order.filledQty,
  averagePrice,
  fills,
  depthUpdate,
};

This became the foundation for the next step.

Instead of clients repeatedly calling the REST API to fetch the latest order book, the engine could now publish these incremental depth updates over WebSockets, allowing every connected client to stay synchronized in real time while transferring only the data that actually changed.

Step 36: Implementing the WebSocket Layer

With incremental depth updates working inside the matching engine, the next step was delivering those updates to clients in real time.

I wanted to keep the architecture loosely coupled, so instead of embedding a WebSocket server inside the matching engine, I created a dedicated WebSocket microservice.

The communication flow now looks like this:

    Client
       │
WebSocket Server
      ▲
      │ Redis Streams
      ▼
Matching Engine

The matching engine is responsible only for processing orders and generating market events.

The WebSocket service is responsible only for broadcasting those events to connected clients.

Separating these responsibilities makes each service simpler and allows multiple WebSocket servers to scale independently without affecting the matching engine.

Publishing Depth Updates from the Engine

Every successful order can potentially modify the order book.

After processing an order, the engine now returns a depthUpdate.

const result = handleCreateOrder(message.payload);

If the response contains a depth update, it is immediately published to a Redis Stream.

if ("depthUpdate" in data) {
    await publishDepthUpdate(depthUpdate);
}

Publishing is handled using XADD.

await streamClient.xAdd(
    "depth-stream",
    "*",
    {
        payload: JSON.stringify(depthUpdate),
    },
);

Unlike Redis Pub/Sub, Redis Streams persist every message until it is acknowledged or removed, making them much more suitable for reliable event delivery.

Creating a Dedicated WebSocket Microservice

Instead of mixing WebSocket logic with matching logic, I created an entirely separate service.

Its responsibilities are:

Accept WebSocket connections
Manage client subscriptions
Read depth updates from Redis Streams
Broadcast updates to subscribed clients

This keeps the matching engine completely unaware of WebSocket connections.

Websocket Server Code:

import { WebSocket, WebSocketServer } from "ws";
import { createClient } from "redis";

const STREAM = "depth-stream";

// Every websocket server has its own group
const GROUP = process.env.CONSUMER_GROUP!;
const CONSUMER = crypto.randomUUID();

const redis = createClient({ url: process.env.REDIS_URL }).on(
  "error",
  (error) => {
    console.error("Redis Stream client error", error);
  },
);

await redis.connect();

//creates a consumer group in Redis Streams.
//A consumer group can only be created once.
try {
  // startId options:
  // '$' - only new messages from this point forward
  // '0' - read all existing messages from the beginning
  // '1234567890123-0' - specific message ID
  // MKSTREAM - If depth-stream doesn't exist, create it first, then create the consumer group.
  await redis.xGroupCreate(STREAM, GROUP, "0", {
    MKSTREAM: true,
  });
} catch (error) {
  // BUSYGROUP means the group already exists
  if (error instanceof Error && error.message.includes("BUSYGROUP")) {
    console.log(`Group "${GROUP}" already exists`);
  } else {
    throw error;
  }
}

/**
 * Stores all active websocket subscriptions.
 *
 * Example:
 * {
 *   "depth.BTC": Set(ws1, ws2),
 *   "depth.ETH": Set(ws3)
 * }
 *
 * When a new depth update for BTC arrives,
 * only the sockets inside activeSubscriptions["depth.BTC"]
 * will receive the update.
 */
const activeSubscriptions: Record<string, Set<WebSocket>> = {};

const wss = new WebSocketServer({
  port: 8080,
});

console.log("WS Server Started");

poll();

async function poll() {
  while (true) {
    // Reading Messages with XREADGROUP
    // Engine -> XADD -> depth-stream -> xReadGroup()
    const result = await redis.xReadGroup(
      GROUP, // I'm reading as consumer group ws-server-1
      CONSUMER, // Inside a consumer group there can be multiple consumers.
      [
        {
          key: STREAM,
          id: ">", // The ">" ID means: give me messages never delivered to any consumer
        },
      ],
      {
        BLOCK: 0, // Wait forever until a message arrives.
        COUNT: 100, // Maximum messages to return at once.
      },
    );

    if (!result) continue;

    for (const stream of result) {
      for (const message of stream.messages) {
        console.log("Received from stream:", message);
        const depth = JSON.parse(message.message.payload);
        console.log(depth);

        const key = `depth.${depth.s}`;

        activeSubscriptions[key]?.forEach((ws) => {
          ws.send(JSON.stringify(depth));
        });

        await redis.xAck(STREAM, GROUP, message.id);
      }
    }
  }
}

wss.on("connection", (ws) => {
  console.log("Client Connected");

  ws.on("message", (data) => {
    const parsed = JSON.parse(data.toString());

    /**
     * {
     *   method:"SUBSCRIBE",
     *   params:["depth.BTC"],
     *   id:1
     * }
     */

    if (parsed.method === "SUBSCRIBE") {
      parsed.params.forEach((channel: string) => {
        // Create a subscription bucket if this is the
        // first client subscribing to the channel.
        if (!activeSubscriptions[channel]) {
          activeSubscriptions[channel] = new Set();
        }

        // Register this websocket connection
        // for the requested channel.
        activeSubscriptions[channel].add(ws);
      });

      // Acknowledge successful subscription.
      ws.send(
        JSON.stringify({
          id: parsed.id,
          result: null,
        }),
      );
    }

    if (parsed.method === "UNSUBSCRIBE") {
      parsed.params.forEach((channel: string) => {
        activeSubscriptions[channel]?.delete(ws);
      });

      // Acknowledge successful unsubscription.
      ws.send(
        JSON.stringify({
          id: parsed.id,
          result: null,
        }),
      );
    }
  });

  ws.on("close", () => {
    // Remove the socket from every subscribed channel
    Object.values(activeSubscriptions).forEach((clients) => clients.delete(ws));
  });
});

Websocket Client code:

import axios from "axios";

type Orderbook = {
  bids: Record<string, string>;
  asks: Record<string, string>;
};

const orderbook: Orderbook = {
  bids: {},
  asks: {},
};
let orderbookInitialised = false;
const ws = new WebSocket("ws://localhost:8080");

const buffer: {
  updatedBids: [string, string][];
  updatedAsks: [string, string][];
  startOffset: number;
  endOffset: number;
}[] = [];

function updateOrderbook(
  updatedAsks: [string, string][],
  updatedBids: [string, string][],
) {
  updatedAsks.forEach(([price, qty]) => {
    if (qty === "0") {
      delete orderbook.asks[price];
    } else {
      orderbook.asks[price] = qty;
    }
  });

  updatedBids.forEach(([price, qty]) => {
    if (qty === "0") {
      delete orderbook.bids[price];
    } else {
      orderbook.bids[price] = qty;
    }
  });
}

ws.onopen = () => {
  ws.send(
    JSON.stringify({
      method: "SUBSCRIBE",
      params: ["depth.BTC"],
      id: 1,
    }),
  );
};

ws.onmessage = async (event) => {
  const message = JSON.parse(event.data);

  // Subscription acknowledged by the server
  if ("result" in message && message.id === 1) {
    console.log("Subscribed successfully");

    const res = await axios.get("http://localhost:3000/depth/BTC");

    const { bids, asks, lastUpdateId } = res.data;

    bids.forEach(([price, qty]: [string, string]) => {
      orderbook.bids[price] = qty;
    });

    asks.forEach(([price, qty]: [string, string]) => {
      orderbook.asks[price] = qty;
    });

    orderbookInitialised = true;

    let expected = lastUpdateId + 1;

    buffer.forEach((msg) => {
      if (msg.endOffset < expected) {
        return;
      }

      if (msg.startOffset > expected) {
        throw new Error("Sequence gap detected. Need resync.");
      }

      updateOrderbook(msg.updatedAsks, msg.updatedBids);

      expected = msg.endOffset + 1;
    });

    buffer.length = 0;

    console.log("Orderbook initialized");

    return;
  }

  // Actual depth update
  const updatedBids = message.b;
  const updatedAsks = message.a;
  const startOffset = message.U;
  const endOffset = message.u;

  if (!orderbookInitialised) {
    buffer.push({
      updatedAsks,
      updatedBids,
      startOffset,
      endOffset,
    });
  } else {
    updateOrderbook(updatedAsks, updatedBids);
  }
};

setInterval(() => {
  const bids = Object.entries(orderbook.bids).sort(
    (a, b) => Number(b[0]) - Number(a[0]),
  ); // Highest price first

  const asks = Object.entries(orderbook.asks).sort(
    (a, b) => Number(a[0]) - Number(b[0]),
  ); // Lowest price first

  console.clear();

  console.log("===== BIDS =====");
  bids.forEach(([price, qty]) => {
    console.log(`Price: ${price} | Qty: ${qty}`);
  });

  console.log("\n===== ASKS =====");
  asks.forEach(([price, qty]) => {
    console.log(`Price: ${price} | Qty: ${qty}`);
  });
}, 1000);

Conclusion

Over the course of this series, we built a mini centralized exchange from the ground up and explored many of the core concepts used in real-world trading systems.

Starting with authentication and deposits, we moved on to designing an event-driven architecture using Redis, implemented a matching engine for both limit and market orders, built APIs for balances, orders, and market depth, wrote automated tests, benchmarked the system with k6, and finally added Redis-based snapshot persistence and recovery to make the engine resilient to crashes.

The final architecture of the exchange is shown below, bringing together all the components we've built throughout this series.

This project is still a simplified exchange, but it demonstrates many of the same principles used by production trading platforms: keeping the matching engine as the source of truth, maintaining an in-memory order book for low-latency execution, decoupling services with message queues, and ensuring durability through snapshots and recovery.

If you'd like to explore the complete implementation, source code, or contribute to the project, check out the GitHub repository:

GitHub: https://github.com/shubhamsinghbundela/centralized-exchange

Command Palette

Architecture of Centralized Exchange

Functional Requirements of a Centralized Exchange

What Should Be Stored in Memory vs Database?

APIs to Implement

Architecture Overview

Tech Stack

Step 1: Initialize the Backend Project

Step2: Set Up Prisma with PostgreSQL

Step3: Install Initial Dependencies

Step4: Creating the Express Server

Step 5: Managing Environment Variables

Step 6: Create a .env File

Step 7: Start the Server

Step8: Creating the Signup API

Step9: Creating the Signin API

Step10: Connecting the Backend and Engine with Redis

Response Queue

Step 11: Implementing Deposits in the Exchange

Step 12: Lets Implement Limit Order Matching Engine from Scratch

Explore the Complete Source Code

Step 13: Let's Implement a Market Order Matching Engine from Scratch

Explore the Complete Source Code

Step 14: Testing a Centralized Exchange with Bun Test and Supertest

Testing Exchange APIs

Step15: Testing the Limit Order Matching Engine

Step16: Testing the Market Order Matching Engine

Step17: Implementing and Testing the Depth API

Step 18: Implementing and Testing User Balances

Step 19: Implementing and Testing Order Retrieval

Step 20: Implementing and Testing Order Cancellation

Step 21: Understanding Latency Before Load Testing

Step 22: Learning k6 for Load Testing

Step 23: Seeding 100 Test Users

Step 24: Simulating Exchange Traffic

Step 25: Understanding Redis Persistence and Durability

Step 26: Implementing Exchange State Snapshots

Step 27: Automatic Snapshot Creation

Step 28: Restoring State After Restart

Step 29: Dockerizing Redis Persistence

Step 30: Eliminating Floating-Point Precision Errors with Decimal.js

Step 31: Designing Data Persistence Without Slowing Down the Matching Engine

Step 32: Market Close Synchronization & Database Ownership

Step 33: Learning Real-Time Communication with WebSockets

Step 34: Preparing the Matching Engine for Real-Time Depth Updates

Step 35: Building Incremental Order Book Updates (Depth Deltas)

Step 36: Implementing the WebSocket Layer

Conclusion

Comments

Learn Node.js with Shubham

Understanding HTTP Requests, Responses, and HTTP Methods using Node.js

More from this blog