TYPO3 fileadmin on S3 — without every request hitting S3

TL;DR

The fileadmin (uploads, images, their scaled variants) lives in an S3 bucket, mounted into the TYPO3 pod as a volume via GeeseFS — keyless through the EC2 instance role, no static keys in the cluster. The decisive lever isn't the mount, though: /fileadmin/* is never answered by TYPO3/PHP at all, but served by the web server as static, content-addressed files with Cache-Control: immutable. In front of that sit three cache layers: browser (one year), Souin shared cache in the reverse proxy (RAM, 1 h), and the GeeseFS page cache (512 MB RAM). The result: S3 GET requests are the smallest line item on the entire S3 bill — below plain storage. Reads are practically free.

The setup in one sentence

A single-node k3s on AWS Graviton (arm64). TYPO3 runs as a FrankenPHP pod, the fileadmin lives in an S3 bucket mounted into the pod as a regular ReadWriteMany volume via a FUSE driver. In front of it: a caddy-proxy with an HTTP cache. So far, so normal — the interesting part is the "how".

The mount: S3 as a volume, keyless and arch-correct

The fileadmin mount runs on the yandex k8s-csi-s3 driver with GeeseFS as the mounter. GeeseFS is a FUSE filesystem for S3 that — important for later — caches file contents in RAM.

# Helm values of the tenant chart
s3:
  bucket: koh-web-<site>-media
  region: eu-north-1
  mounter: geesefs
  # --iam  -> EC2 instance role via IMDS. No static keys in the cluster.
  # --memory-limit 512  -> 512 MB in-memory page cache
  mountOptions: "--iam --region eu-north-1 --memory-limit 512 --dir-mode 0777 --file-mode 0666"
  mountPath: /app/public/fileadmin

Three things you can easily trip over here.

Keyless via instance role

--iam lets GeeseFS use the EC2 instance role via IMDS. There are no S3 access keys in the cluster — an endpoint/region secret is enough, auth stays keyless. A compromised pod gets no permanent credentials to walk away with.

Building the arm64 driver yourself

The cluster is Graviton. The official driver images are amd64-only and answer that on ARM with exec format error. So I build the CSI driver myself as arm64; the standard sidecars (provisioner/registrar) are multi-arch anyway. A pattern that runs through the whole setup on Graviton: take multi-arch or build it yourself.

World-writable modes (0777/0666)

The driver hardcodes the owner to nobody (65534) and ignores --uid/--gid. For TYPO3 (www-data, UID 1000) to write scaled images to fileadmin/_processed_/, the file modes are the only way. Looks wild, is defensible in context: only TYPO3 runs in the pod, and the bucket itself is protected by IAM — the "world" ends at the pod boundary.

The core idea: fileadmin never goes through PHP

The most important lever isn't the cache, it's how an image URL gets answered. Static assets are not rendered by TYPO3 — the web server delivers /fileadmin/* with a plain file_server straight from the mount and marks them as immutable:

@fileadminDir { path /fileadmin/* }
handle @fileadminDir {
    header Cache-Control "public, max-age=31536000, immutable"
    file_server { root /app/public }
}

No PHP bootstrap, no TYPO3 frontend, no database query — just a file read. And immutable with a one-year cache lifetime is safe here because TYPO3 stores processed images content-addressed: the image content determines the filename (hash in the path). If the image changes, the URL changes — the old cache entry is simply never requested again. No invalidation needed.

That's the trick that carries everything else: because the response is static and immutable, every layer in front of it may keep it as long as it wants.

Three cache layers along the path

Browser
  │   (1) browser cache:  Cache-Control: immutable, max-age=1 year
  ▼
Ingress (Traefik, public TLS)
  │
  ▼  mTLS
caddy-proxy  ── (2) Souin shared cache (RAM, TTL 1h, stale-while-revalidate)
  │
  ▼  mTLS
FrankenPHP pod (file_server)
  │
  ▼
GeeseFS mount  ── (3) in-memory cache (512 MB RAM)
  │
  ▼
S3   ←  only for a cold object on a cold cache

Layer 1: browser with immutable

The best request is the one that never happens. A returning visitor never loads the image a second time. For that to work, the immutable header has to make it all the way to the browser: my caddy-proxy rewrites Cache-Control only for HTML pages and explicitly excludes /fileadmin/*, bundled assets, and typo3temp/*. Static files keep their "one year, immutable".

Layer 2: Souin — shared cache in the reverse proxy

My caddy-proxy is a golden image built with xcaddy, carrying rate limiting, the CrowdSec WAF, and the Souin HTTP cache. Souin caches GET responses shared across all visitors:

cache {
    ttl 1h
    stale 60m
    default_cache_control "public, max-age=600, stale-while-revalidate=3600, stale-if-error=3600"
    # never cache the TYPO3 backend & dynamic endpoints:
    regex { exclude "/typo3/*|/login|/module/.*|/ajax/.*|/install/.*" }
}

The first visitor of an image fills the cache, everyone after is served from Caddy's RAM — app pod, mount, and S3 never see those requests. stale-while-revalidate also serves an immediate (slightly stale) response even with a slow origin, while refreshing in the background.

Layer 3: GeeseFS in-memory cache

Whatever slips past Souin — right after a deploy, say, when Caddy's RAM cache is cold — gets caught by GeeseFS's 512 MB page cache. Only a truly cold object on a cold cache produces a real S3 GET.

To be honest about the third layer: this is pure RAM, not a disk cache. On a large media inventory, layer 3 is small — the long-lived caching is done by layer 1 (browser) and layer 2 (Souin). GeeseFS's cache is a shock absorber, not a reservoir.

Invalidation: almost a non-problem for fileadmin

Because fileadmin content is content-addressed and immutable, it practically never needs invalidating. A changed image gets a new URL, the old cache entry simply expires unused. Cache invalidation is therefore almost exclusively a topic for HTML/CSS/JS on deploys — not for media.

Two lessons you should know before rebuilding this.

Souin with the in-memory store cannot be reliably purged via API

I found out live that a PURGE against the default in-memory store doesn't reliably take effect. The dependable path after a deploy is a kubectl rollout restart of the caddy-proxy (zero downtime via RollingUpdate), which cleanly throws away the RAM cache — triggered as a GitOps PostSync hook. Targeted content purging would need a shared Souin store (Redis/Badger); that's deliberately not in yet.

The cache API is only reachable internally

The proxy's admin port listens on localhost only, the invalidation route sits on the internal mTLS listener and is restricted by client IP to the cluster network (external → 403). A "clear cache" endpoint never belongs on the public internet.

The 404 trap that haunts you for a year

immutable on a missing file is a trap: without a guard, the server would deliver a 404 as "immutable for one year", and Souin would hold on to that 404 accordingly — until the next restart, and after every asset rename. That's why the immutable header only applies to files that actually exist (@existing file matcher). A one-line guard whose absence really hurts.

What shows up on the S3 bill

The cost structure confirms the design: GET requests — exactly what visitors trigger most often — are the smallest line item on the S3 bill, below plain storage. Reads are practically free, even though the sites serve images all day. That's exactly the picture you expect when read caching works: what gets requested most often is almost never actually read from S3.

One observation belongs in an honest assessment: writing and listing (request tier 1) costs more on the bill than reading (tier 2). A FUSE filesystem performs many LIST/HEAD operations to map directories — and those fall into the more expensive tier 1 category. If you want to cut S3 costs, don't just think about GET caching, think about the mount's metadata operations too.

Conclusion

"S3 as a volume" only gets cheap and fast once you prevent every request from actually reaching S3. The biggest lever wasn't the FUSE cache, it was the decision to serve fileadmin past TYPO3/PHP as static, content-addressed, immutable files — and then to put three cheap cache layers in front: browser, Souin, GeeseFS RAM.

The result: read costs that practically vanish on the S3 bill — for a media-heavy TYPO3 landscape. The rest is discipline — keyless auth, arch-correct images, and a 404 guard that keeps you from caching a mistake for a year.