My publishing job ran on time and published nothing, because the queue feeding it was quietly empty

What I expected

My social-media publishing setup has two halves that run at different times.

• A nightly content job writes a few new items, renders them into images and a short video, and adds the ones that render cleanly to the end of a queue file. Items that fail to render stay out of the queue.
• A publishing job runs at fixed slots during the day. Each slot takes the next item from the queue and publishes it. If there is nothing to take, the slot logs that there was nothing to prepare and stops.

On September 8 I changed the content format. The nightly job was told to pick up only the new item type, and the renderer for the new format had been tested on five sample items. Items in the old format were still in the queue. The plan was to move that old backlog aside once the nightly job was producing the new items, so the new format would take over the slots.

So I expected a short handover: old items for a few days, then new ones, with no gap.

What actually happened

The publishing job ran every slot on time. From the 21 slot on September 13 onward, each slot found nothing in the queue and said so. Nothing crashed, and “nothing to prepare” is a normal message for that job. It is designed to stop quietly when the queue is empty.

An alert from the pipeline did reach my phone on Telegram. I found the problem’s cause on September 14 during a routine check.

When	What the records show
Sept 8	Format changed. Nightly job set to pick up only the new item type
Sept 13, 21	First slot with “nothing to prepare”. No post
Following slots	The same message, slot after slot
Sept 14	Check finds the cause: the four items written that night all failed to render, and none were queued
Sept 14	Nightly job fixed. First new item put at the top of the queue
Sept 15, 12	First slot scheduled to publish from the fixed chain

None of the items produced in the new format had reached the publisher.

Why

The nightly job’s selection step had been updated for the new format. Its render step had not. It still called the two renderers built for the previous format. Those renderers expect an item to contain a list of sections. The new items do not have that list. They are built differently. So each render stopped with a data-shape error.

The nightly job did what it was built to do with a failed render: it left the item out of the queue. That rule is right. A broken image should never be published. But it meant the failure showed up in only two places: the nightly job’s own log, and, days later, an empty queue.

Meanwhile the publishing job was working correctly too. My make-up step, which schedules one extra attempt after an empty slot, deliberately does not retry when the reason is an empty queue. Waiting fifteen minutes does not fill a queue; only the upstream job can. So every layer was behaving as designed, and the only visible symptom was downstream and polite.

Two correct parts, joined by a stale piece of wiring, produced a failure that looked like a quiet day.

What I changed

On September 14:

1. One item type, one renderer. The nightly job now picks up only the current format’s items and renders each with the single renderer written for that format. That renderer produces both the images and the video in one pass, so there is one piece to keep in step with the format instead of two.
2. The publisher recognises the new format. The step that checks whether a video matches the images it belongs to was taught the new format, so a valid item is not thrown out as an old package.
3. A dry run before trusting it. I put the first new item at the top of the queue and ran the publisher’s preparation step in dry-run mode to confirm it would prepare that item for the 12 slot the next day.

The earlier renderers already refuse to produce a broken card; they exit with an error instead of cropping text. That stayed. What was missing was a check across the join between the two jobs after the format changed.

What this has to do with hiring someone

If a freelancer builds you a pipeline with a producing step and a publishing step, each piece can pass its own test and the whole can still deliver nothing. It is the same pattern as my expired-token incident: every layer reports success because every layer is judged on its own.

Two things belong in the brief and in acceptance:

• An end-to-end test after any change. When the format, the template or a connected tool changes, the acceptance run should start at the first job and end with a published result, not a green check on each part. The acceptance checklist for unattended automations explains why the test has to be run by the real schedule.
• A signal for “less work than usual”. Ask what tells you when the queue is close to empty, or when the upstream job has rejected everything it made. A slot that says “nothing to do” is a result to look into, not a success.

Format changes are also a common source of paid fixes after delivery. Whether that is a fix or a new order is worth settling in the maintenance terms before it happens. When you brief a workflow automation, ask the freelancer to describe what happens between the two jobs when one of them rejects an item, and who hears about it.