Implementing server-side caching middleware in tRPC 10

Question

I'm working on a Next.js project and looking to implement in-memory caching for tRPC results, with each tRPC procedure being able to opt-in with a custom TTL. I think tRPC's middleware would be suitable for this purpose. Unfortunately, the current tRPC middleware documentation doesn't seem to cover this specific scenario.

How can a server-side caching middleware in tRPC 10 be implemented?

Answer 1

There's a related discussion on a Github issue for a feature request that allows to do this easily. In the meanwhile, it is still possible to do this by implementing custom logic.

The example below uses node-cache as an in-memory caching approach. It's been tested with tRPC 10.43.3. Procedures listed in cachedProcedures are configured to opt into the cache.

// src/app/api/middleware/cache.ts

import { initTRPC } from "@trpc/server";
import NodeCache from "node-cache";

const cacheSingleton = new NodeCache();

// A map of cached procedure names to a callable that gives a TTL in seconds
const cachedProcedures: Map<string, (() => number) | undefined> = new Map();
cachedProcedures.set("router0.procedure0", () => 2 * 3600); // 2 hours
cachedProcedures.set("router0.procedure1", () => 1800); // 30 minutes
cachedProcedures.set("router1.procedure0", secondsUntilMidnight); // dynamic TTL
cachedProcedures.set("router1.procedure1", () => undefined); // never expires

const t = initTRPC.create();
const middlewareMarker = "middlewareMarker" as "middlewareMarker" & {
  __brand: "middlewareMarker";
};


const cacheMiddleware = t.middleware(
  async ({ ctx, next, path, type, rawInput }) => {
    if (type !== "query" || !cachedProcedures.has(path)) {
      return next();
    }
    let key = path;
    if (rawInput) {
      key += JSON.stringify(rawInput).replace(/\"/g, "'");
    }
    const cachedData = cacheSingleton.get(key);
    if (cachedData) {
      return {
        ok: true,
        data: cachedData,
        ctx,
        marker: middlewareMarker,
      };
    }
    const result = await next();

    //@ts-ignore
    // data is not defined in the type MiddlewareResult
    const dataCopy = structuredClone(result.data);

    const ttlSecondsCallable = cachedProcedures.get(path);
    if (ttlSecondsCallable) {
      cacheSingleton.set(key, dataCopy, ttlSecondsCallable());
    } else {
      cacheSingleton.set(key, dataCopy);
    }
    return result;
  }
);
export default cacheMiddleware;

// src/server/api, or wherever you define tRPC procedures

import cacheMiddleware from "@/app/api/middleware/cache";
...
export const publicProcedure = t.procedure.use(cacheMiddleware);

A few remarks:

• The example assumes Next.js app router, but should also work with the pages router or any other framework
• The cache is only enabled for query types, not mutation or subscription
• It uses different cache keys based on the inputs provided to the procedure
• The middlewareMarker is a somewhat hacky fix, as any middleware return must include a marker
• structuredClone is only available in Node 17+, if this is not an option a different deep cloning approach is required
• It would be wise to add logging