GimmetheCache

This site runs on AWS — containerized in Docker, deployed via ECS Fargate, served through CloudFront, with a multi-layer cache architecture that keeps every page and image fast across restarts and deployments. This post is a technical walkthrough of how that infrastructure works, the problems that came up while building it, and what each piece is actually doing.

This site started on Vercel — which is excellent for most Next.js deployments. But Vercel's serverless model made one part of this site awkward: the Chatbot Mixer uses Vercel's Workflow Dev Kit, which runs durable long-lived executions on Vercel's infrastructure. That coupling, combined with wanting to own the full deployment pipeline, made self-hosting the right move.

The migration was done in two phases: move everything except the Chatbot Mixer first (Phase 1), then migrate the Chatbot Mixer separately once the main site was stable on AWS (Phase 2). Phase 1 took one session. The site now runs on AWS ECS Fargate behind an Application Load Balancer and CloudFront CDN — Sanity CMS is unchanged and doesn't care where the frontend lives.

The app runs as a Docker container built with a three-stage Dockerfile: a deps stage that installs dependencies, a builder stage that runs next build, and a lean runner stage that copies only the standalone output — no source code, no dev dependencies, no build tools. The final image is around 150MB. next.config.js uses output: 'standalone' to produce a self-contained Node.js server.

# Stage 1: install deps
FROM node:22-alpine AS deps
WORKDIR /app
COPY package.json yarn.lock ./
RUN yarn install --frozen-lockfile

# Stage 2: build
FROM node:22-alpine AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
ARG NEXT_PUBLIC_SANITY_PROJECT_ID
ENV NEXT_PUBLIC_SANITY_PROJECT_ID=$NEXT_PUBLIC_SANITY_PROJECT_ID
RUN yarn build

# Stage 3: lean runtime image
FROM node:22-alpine AS runner
WORKDIR /app
ENV NODE_ENV=production
ENV HOSTNAME=0.0.0.0
RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs
RUN mkdir -p /app/.next/cache && chown -R nextjs:nodejs /app/.next
USER nextjs
COPY --from=builder /app/.next/standalone ./
COPY --from=builder /app/.next/static ./.next/static
COPY --from=builder /app/public ./public
COPY --from=builder /app/cache-handler.js ./cache-handler.js
CMD ["node", "server.js"]

The container runs as a non-root user (nextjs, uid 1001) for security. If an attacker exploits a vulnerability in the app, they get a restricted account rather than the host machine. NEXT_PUBLIC_* vars are passed as build args — Next.js bakes them into the JS bundle at compile time, so they can't be injected at runtime.

Images are pushed to Amazon ECR (Elastic Container Registry) and deployed on ECS Fargate (Express Mode) — managed containers without needing to provision or maintain EC2 instances. ECS Express Mode handles canary deploys by alternating traffic between two target groups, enabling zero-downtime deployments. An Application Load Balancer sits in front of ECS, and CloudFront sits in front of the ALB as a global CDN.

Every push to main triggers a GitHub Actions workflow that builds the Docker image for linux/amd64 (required because the build machine is Apple Silicon — AWS ECS runs on x86), pushes it to ECR with the commit SHA as a tag, registers a new ECS task definition revision, updates the service to the new revision, waits for the deployment to stabilize, and then syncs the HTTP listener weights on the ALB to match the HTTPS listener after ECS Express Mode performs its canary swap.

There are three cache layers, each solving a different problem at a different point in the request path. They're independent — CloudFront has no idea whether your server used Redis or EFS to build a response. It just caches the HTTP response it gets back.

Next.js's default ISR cache is in-memory and dies with the container — every redeploy cold-renders all pages. cache-handler.js replaces the default with Upstash Redis, so the ISR cache persists across deploys and container restarts. When a page is requested, the server checks Redis first. On a hit, it skips Sanity entirely and serves the cached HTML. TTL is set per page via the revalidate export — blog posts revalidate every 50 minutes, the blog index every 17 minutes.

// Next.js RSC payloads include Buffer objects (rscData).
// JSON.stringify converts them to { type:'Buffer', data:[...] }
// but they aren't revived to real Buffers on parse.
// Custom replacer/reviver so Buffers survive the Redis round-trip.

function toStorable(value) {
  return JSON.parse(
    JSON.stringify(value, (_k, v) =>
      Buffer.isBuffer(v) ? { __b64__: v.toString('base64') } : v
    )
  );
}

function fromStorable(value) {
  return JSON.parse(
    JSON.stringify(value),
    (_k, v) =>
      v && typeof v === 'object' && typeof v.__b64__ === 'string'
        ? Buffer.from(v.__b64__, 'base64')
        : v
  );
}

next/image optimizes images on first request — downloads the original from Sanity CDN, resizes it, converts to WebP or AVIF, and saves the result to .next/cache/images/. Subsequent requests serve the file directly. In a container, that cache lives on ephemeral disk and wipes on every deploy. An EFS (Elastic File System) volume is mounted at /app/.next/cache in the ECS task definition, so the processed image files persist across container restarts and redeployments.

CloudFront caches the full HTTP responses — page HTML and optimized image files — at AWS edge locations globally. Once CloudFront has a response, subsequent requests for that URL don't reach the server at all. Optimized images are cached at CloudFront for 30 days via Cache-Control headers. For a rarely-requested image that falls out of CloudFront's cache after 30 days, the next request re-primes it from EFS — fast, no reprocessing needed.

EFS and Redis persist cache across deploys, but after each deploy the new container mounts fresh — the caches are still populated from before, but CloudFront is cold for any URL that wasn't recently requested. Without warming, the first real user after a deploy pays the full slow-path penalty: CloudFront miss → server → EFS/Redis hit (fast), but CloudFront then caches it for everyone after.

The cache warmer runs as the last step in the GitHub Actions deploy workflow. It makes two passes: first hitting every page route (ISR pass), then parsing all /_next/image URLs out of the rendered HTML and requesting each one (image pass). Static page routes are hardcoded in STATIC_PATHS. Blog post slugs are fetched dynamically from Sanity before each run — no manual updates needed when new posts are published.

async function main() {
  const blogPaths = await getBlogSlugs();
  const allPaths = [...STATIC_PATHS, ...blogPaths];

  // Pass 1: hit every page → populates Redis ISR cache
  console.log(`── Pass 1: ISR cache — warming ${allPaths.length} pages ──`);
  const allImageUrls = new Set();
  for (const path of allPaths) {
    const html = await warmPage(path);
    if (html) {
      // collect /_next/image URLs embedded in the SSR HTML
      extractImageUrls(html).forEach(u => allImageUrls.add(u));
    }
  }

  // Pass 2: request each image URL → triggers optimizer, writes to EFS, primes CloudFront
  console.log(`── Pass 2: Image cache — warming ${allImageUrls.size} images ──`);
  for (const imageUrl of allImageUrls) {
    await warmImage(imageUrl);
  }
}

Both of these hit on the same day. They're worth documenting because they illustrate how infrastructure that looks correct can have a silent flaw — one that only manifests under the exact right conditions.

The symptom was slow image loads. CloudWatch logs showed hundreds of EACCES: permission denied, mkdir '/app/.next/cache/images' per minute — one for every image request. The image optimizer was running, but it couldn't write the result anywhere. Every image was being reprocessed from scratch on every single request.

The Dockerfile had RUN chown -R nextjs:nodejs /app/.next at build time. That sets ownership correctly inside the image. But at container startup, EFS mounts over /app/.next/cache — replacing that directory with the EFS filesystem, which presents as owned by root. The build-time chown was completely undone at runtime. The fix was an EFS access point configured with posixUser: { uid: 1001, gid: 1001 }, which makes EFS present the mount as owned by uid 1001 regardless of the underlying filesystem.

aws efs create-access-point \
  --file-system-id fs-046d1055c95e57951 \
  --posix-user Uid=1001,Gid=1001 \
  --root-directory "Path=/,CreationInfo={OwnerUid=1001,OwnerGid=1001,Permissions=755}" \
  --region us-east-2

The EFS fix required deploying a new ECS task definition revision. That deployment was triggered manually via the AWS CLI rather than through GitHub Actions. The site immediately went 503 on every page.

ECS Express Mode uses two ALB target groups and alternates between them on each canary deploy. It manages the HTTPS:443 listener automatically — shifting weights to the new target group after each deploy. The HTTP:80 listener was created manually (CloudFront connects to the ALB over HTTP to avoid TLS hostname mismatch) and is not managed by ECS. The GitHub Actions deploy workflow has a step that reads the active target group from HTTPS and mirrors the weights to HTTP. Bypassing GitHub Actions meant that step never ran. ECS swapped HTTPS to the new target group; HTTP stayed pointed at the old empty one. CloudFront routes via HTTP — 503.

When a new build deploys, Next.js generates new JS chunks with content-hashed filenames. Any user already on the old version of the site has HTML referencing the old chunk filenames. If they navigate client-side while the deploy is in progress, the browser tries to fetch old chunk URLs that no longer exist — a ChunkLoadError. The fix is a small component mounted in the root layout that listens for this specific error and calls window.location.reload(). The hard reload fetches fresh HTML with the new chunk filenames. The user sees a brief reload instead of a broken page.

'use client';
import { useEffect } from 'react';

export default function ChunkErrorReloader() {
  useEffect(() => {
    const handler = (event) => {
      const err = event.reason ?? event.error;
      if (err?.name === 'ChunkLoadError' || err?.message?.includes('Loading chunk')) {
        window.location.reload();
      }
    };
    window.addEventListener('unhandledrejection', handler);
    window.addEventListener('error', handler);
    return () => {
      window.removeEventListener('unhandledrejection', handler);
      window.removeEventListener('error', handler);
    };
  }, []);
  return null;
}

Test volume mount write permissions before shipping. Running docker exec into the container and trying touch /app/.next/cache/test would have caught the EFS permissions bug immediately. Configure EFS access points by default for any non-root container writing to a volume — not as a post-incident fix.

Add a post-deploy smoke test that verifies both ALB listener weights match before marking the deployment successful. And treat the GitHub Actions pipeline as the only valid deployment path — not a convenience wrapper around CLI commands, but the authoritative record of how this system deploys.