TYPO3 pods in twelve seconds instead of a minute

01 — Where the 54 seconds went

A fresh pod used to run a fixed chain of init containers. Two verified signature and SBOM, one downloaded the code with ORAS, the last ran the TYPO3 setup: extension:setup, the schema migration, cache:flush and cache:warmup. On the pilot tenant that added up to 54 seconds. 13 of them went to the download, 29 to the setup.

The image volume took care of the download. The setup deserved a second look: measured one by one, its steps took little more than twelve seconds. The rest was self-inflicted. The script ran extension:setup twice. Then cache:flush emptied everything the earlier steps had just built, and the warmup started from zero again.

Something else weighed more. The flush also emptied the shared page caches of every pod of the tenant – on every start, every scale-out, every restart. A pod added because of load first made more work for all the others.

02 — The setup moves into a hook

A schema has to be migrated once per release, not once per pod. Until now there was no other way: the code lived only in each pod's emptyDir, and an Argo hook would simply have had no code. With the image volume, a hook pod mounts the same image as the app pods, pinned to the same digest.

That became two hooks and a leaner pod:

• PreSync hook:extension:setup and the schema migration, once per sync, under the same database lock as before and with set -e. If a step fails, Argo stops the sync before the Deployment changes.
• App pod: only warms its own system caches.
• PostSync hook:cache:flush from a pod of its own, once the rollout is through.

The third point replaces a detour. Until now a hook restarted the app after every sync. The reason: a flush under a running FrankenPHP worker makes the loaded DI classes collide with freshly generated ones. A flush from another pod, by contrast, only clears the shared cache entries and leaves the workers' files alone. That removes the second rollout after every sync.

03 — First trap: the network rule came too late

The first sync with the hooks failed in PreSync. The hook pod waited two minutes for the database and then gave up:

wait-db: db:3306 not reachable after 120s

In the namespace, components may only talk to each other the way NetworkPolicies allow. The permission for the new hook sat in the same chart as the hook itself. Argo, however, applies NetworkPolicies only in the sync phase, after PreSync. A failing PreSync never reaches that phase – the state would not have resolved itself. Visitors noticed nothing, because nothing about the Deployment changes before PreSync.

The fix follows a rule the platform already applies to pod certificates: grant first, switch second. The hooks' network permissions now exist as soon as the code arrives as an image volume. They are in place one sync before a tenant is switched to the hooks.

04 — Second trap: pages without CSS

The second attempt looked clean. The hook took 15 seconds, a pod was ready after 23, every check reported green. Almost an hour later it turned out that several websites had lost their styling. The pages answered 200, their stylesheets under /_assets/ answered 404.

The setup had done a second job on the side that was described nowhere as such. extension:setup publishes every extension's public resources into public/_assets. In the pod that directory sat on a writable emptyDir. With the setup in the hook, the assets landed in the hook pod's volume, and the app pod saw an empty directory. They had never been in the image: the Composer hook for asset:publish was switched off in CI – on the grounds that the pod takes care of it at start.

A person noticed first, not the tooling. Every check asked whether a page answers 200, and a page without CSS does exactly that. Since then every check also fetches a file the page links. The rollback took minutes, because the image volume could stay and only the setup moved back into the pod.

05 — Assets and caches from the build

The assets belong in the image. Since TYPO3 14.2 there is the asset:publish command for that, and it needs no database. The directories under _assets/ are named after the MD5 hash of the path relative to the project directory, and their entries are relative symlinks to vendor/*/Resources/Public. A build under an entirely different path therefore produces exactly the names TYPO3 later writes into the pages under /app. The app pod no longer puts a writable directory over public/_assets. If an image brings nothing there, an init container refuses to start.

The caches were harder. A full cache:warmup took seven to eleven seconds in the pod, a TYPO3 bootstrap with the code caches present only 0.3 seconds. The shared runtime caches such as translations are warm in Valkey anyway once a tenant runs. So it is enough to build the code caches for the DI container, TCA, routes and middleware in the pipeline. Three things had to be taken care of:

• The path: the cache identifier and some entries contain the project directory. The build therefore warms a copy under /app.
• No database: without Valkey the cluster cache backend falls back to the database. During the warmup, a line appended to a throwaway copy of the configuration redirects every cache to plain file backends.
• The environment: the site configuration reads domains from environment variables that differ per tenant. It stays out and is built in the pod on the first request.

A comparison showed whether that holds. Build and running pod arrived at the same cache identifier, and a copied cache stayed byte for byte unchanged after the first bootstrap. If the warmup fails in the build, the image ships without caches, and the pod warms itself as before.

06 — What remains

A new pod is now ready after twelve seconds. The signature check costs three of them, container start and the FrankenPHP worker boot about three, the rest goes to scheduling and mounting. The init container that copies the caches from the image takes less than a second.

A sync has become simpler too. Instead of two or three rollouts there is one, the setup runs exactly once, and no new pod empties the other pods' caches any more. Sylius follows as soon as its setup no longer writes files into /app.

Twelve seconds are also the precondition for the next step. A backend that sits at zero pods outside working hours has to wake up fast – an editor should not wait long at login. With a minute of cold start that was not an option. With twelve seconds it is.

Conclusion

A setup that runs in every pod costs more than its own runtime. On the side it does things the rest quietly relies on. Here it was the public assets, and they only showed up in the hour in which they were missing. What stayed: setup once per sync, assets and code caches from the build, and a pod that is working after twelve seconds.

The second lesson is about monitoring. A check that only asks whether a page answers would have called this failure healthy too. Today it also asks whether what the page needs actually arrives.