Mail and images leave the TYPO3 request

01 — What should not wait in the request

A visitor submits a form. TYPO3 renders the confirmation mail and hands it to the mail server before the response page arrives. If the mail server is slow, the visitor waits. If it does not answer at all, the mail is gone. For us that once ended a password reset with an error 500.

Images are similar. TYPO3 produces image sizes only when a page asks for them. A page hero with a srcset in three formats needs sixteen sizes at once, AVIF included. The first view after publishing carries that work: the editor's preview, the first visitor or a crawler.

Neither belongs in the request. The visitor needs the answer, not the delivery of the mail. The page needs the finished images, not their production.

02 — Symfony Messenger in TYPO3 14, but without a net

TYPO3 14 ships Symfony Messenger. This component puts work into a queue as a message instead of doing it right away. A separate process, the consumer, picks the messages up and works through them. In TYPO3 that is the command messenger:consume. That sounds finished. It is not.

The core's consumer registers no retry and no failure queue. If the code that processes a message fails, the consumer still marks it as done, and it is gone. For a mail that means: a brief hiccup at the mail server, and the mail never arrives. Nobody notices, because nothing reports an error.

The extension koh/typo3-async closes that gap. Symfony already has the building blocks for retry and a failure queue. The extension wires them into TYPO3's event handling. A message gets three attempts, 30, 60 and 120 seconds apart. After that it lands in the queue failed instead of disappearing.

The mail itself is still rendered in the request. Only there does TYPO3 know the site, the language and the links. The consumer only delivers it.

vendor/bin/typo3 messenger:consume koh_async_mail koh_async_images \
  --time-limit=3600 --memory-limit=256M

03 — One consumer per tenant, and the switch belongs to the platform

Each tenant's queue lives in its own database, in the table sys_messenger_messages. It is part of the backup and needs no new service. Valkey, the platform's cache server, was out of the question. A cache may drop keys under memory pressure, and a queue must not.

The Helm chart builds the consumer from the same pod template as the app: same identity, same network rules, same code. It runs as a Deployment of its own and restarts itself every hour. A CronJob every five minutes would have let a password-reset mail wait up to five minutes.

More important than the pod is the switch. A queue nobody reads does not delay a message. It keeps it. So the extension does not decide; the environment does. Messages go into the queue only when MESSENGER_ASYNC=1 is set. The chart sets that variable only together with the consumer. Set it by hand without a consumer, and rendering the chart already fails, so nothing rolls out.

04 — Image sizes are learned, not configured

The obvious idea was a list: please produce these sizes after every upload. It failed on a detail. TYPO3 recognises a finished image by a checksum over the instructions it was made with. That checksum also covers the order of the keys and empty values.

{"width": 640}
{"width": 640, "height": null}

To TYPO3 those are two different images. A hand-kept list easily produces images no page ever finds. And it goes stale silently as soon as a template changes.

So the extension reads the instructions from the table of images already produced, key by key in their stored order. Whatever a template requested for at least ten percent of the images, the consumer now produces right after the upload. Crops are learned by the name of the crop variant, recovered from the links between the file and its content elements. Images the consumer made itself do not count. Otherwise every learned size would confirm itself with every upload.

The first run paid off. A list I had checked against the templates beforehand missed one crop variant. The learning mode found it straight away. A test image got 25 sizes in about 90 seconds, and the page then found them in 17 to 100 milliseconds.

05 — The consumer reports on itself

A consumer that fails no longer costs the visitor anything. That is exactly what makes it dangerous: it fails silently. The check Kubernetes uses to watch a pod sees a running process. It does not see whether the process works.

So the consumer writes its own figures, from its own loop, every 15 seconds. For each queue that is the number of messages, the age of the oldest due message and a timestamp. A small companion container in the same pod serves the file to Prometheus, without database access. If the timestamp stops moving, the loop hangs. No other mechanism sees that.

Two alerts are built on that:

• QueueBacklog: a due message has waited longer than 15 minutes, or the figures have not moved for 15 minutes.
• MessagesParked: the failure queue is not empty. A person decides here.

That person has three commands. messenger:failed:show shows the message and its error. koh-async:failed:retry puts it back into its queue. messenger:failed:remove drops it.

06 — Two traps on the way

The first trap was a settings template. Version 1.1.0 shipped an ext_conf_template.txt. On deploy TYPO3 runs extension:setup, and that command writes every template's defaults into settings.php. Our code, though, sits read-only in the pod. So the setup step before the rollout failed, and Argo CD stopped the rollout. Because the tenants roll out one after another, the tenant behind it stopped as well. No site went down; both kept running on the old version. Version 1.1.1 came without the template. A test keeps it from coming back.

The second trap is a command that does not exist. TYPO3 registers just one: messenger:consume. The obvious way out would be Symfony's messenger:failed:retry. But that command processes the message with Symfony's own dispatching, past TYPO3. If it fails again, no retry applies, and it is lost. So koh-async:failed:retry only puts the message back into its queue. The consumer does the rest, with all three attempts.

Conclusion

Slow work does not belong in the request. Since mail and image sizes go into a queue, no visitor waits for the mail server any more. A brief outage no longer costs a mail. New images are ready before anyone opens the page.

The effort was not in the queue itself. It was in what TYPO3 does not bring along: retry, a failure queue, a consumer that reports on itself. And in one insight about images. Sizes cannot be planned, only learned from what the page really asks for.