Acceptance checklist: does it run unattended?

The short answer

A demo proves the automation can run. Acceptance has to prove it does run — on schedule, on its own, after the machine has slept, with a login that has quietly expired — and that it tells you when it doesn’t. Most delegated automations that die do not die loudly. They stop, and you find out a week later. Seller reviews will not warn you either: they are written the day the order closes, as 317 reviews of automation freelancers and 252 of chatbot developers show.

So before the final payment, run nine checks. Most of them come from failures in my own stack of scheduled jobs and bots — each of which had passed a manual test first. The rest are traps I found while building and closed before they fired.

Why “it worked when I clicked Run” is not acceptance

When you or the freelancer press Run, the job inherits everything about that moment: an open terminal, a logged-in session, the right working folder, a network connection, a machine that is awake. The scheduler gives it none of those guarantees.

Same job, two very different starts

What the job can count on in each case.

You click RunScheduler, 3 a.m. A console window is open ✓ yes? not guaranteed Someone is logged in ✓ yes? not guaranteed It starts in the right folder ✓ yes? not guaranteed The network is up ✓ yes? not guaranteed The machine is awake ✓ yes? not guaranteed Someone is watching the screen ✓ you are✗ nobody

My clearest example: a batch file that published posts on a schedule worked every single time I launched it by hand. Under the scheduler it was killed a second after it started, every time. The cause was one harmless-looking line that changes the console’s text encoding — fine in a real console window, fatal when there is no window at all. I could not reproduce it manually, which is exactly why it slipped through.

The lesson is not about that line. It is that the test has to be run by the same thing that will run it in production: the real scheduler, the real trigger, on its own.

The nine checks

The nine checks as a five-day acceptance week

Agree the week with the freelancer up front; the final payment waits for step 5.

1. Nights 1–3: leave it aloneChecks 1–3: real schedule, one sleep or restart, one slot with the network off.
2. Day 4: break itChecks 4–6: expire a login, confirm the alert reaches you, and silence on normal days.
3. Day 5: try to double itChecks 7–8: re-run a finished job, start two copies at once.
4. Day 5: check the keysCheck 9: every key and login is in your accounts.
5. Then payPlus the written run-book: how to restart it, where the logs are.

What each check is protecting you from

Sleep and restart (check 2). On a laptop or a home machine, the system will happily skip a scheduled time it slept through. Scheduled posts of mine simply did not go out this way, with nothing in the log to say so (the full incident). The fix was a “run when available” setting and, more importantly, keeping the machine awake for the slots that matter.

Network loss (check 3). One outage cost me three days of scheduled slots. The work had been prepared ahead with a 24-hour expiry, so when the connection came back, the prepared items had already lapsed. Nothing retried. Ask the freelancer: what happens to work that was queued when the network came back?

Expired login (check 4). The most common silent failure. My publishing job kept running on schedule for two days while a code-hosting token had expired; the failure sat in a log file nobody reads. The job “ran”. Nothing was published.

The job started every night. Nothing was published for two.

A scheduled publishing job of mine, after its code-hosting token quietly expired.

Night 1Night 2Night 3Night 4Night 5 ▼ token expires Job started on schedule ✓✓✓✓✓ Post actually went out ✓✓✓✗✗ Anyone told me ————— Same job, with a watchdog checking the result quietquietquietAlert✓

Alerts (checks 5 and 6). The fix for all of the above is not more careful code — it is a watchdog. A small separate check that looks at the results (did a post actually appear? did the report actually land?) rather than trusting the job to report on itself. A job that crashes hard cannot send its own error message; something outside it has to notice.

Put the alarm outside the job

A job that crashes cannot send its own error. A small separate check that looks for the result can.

Duplicates (checks 7 and 8). These two I caught while building rather than in production, and they are worth asking about precisely because a demo never shows them. My bot’s message inbox only allows one reader at a time; start a second copy and the two silently steal each other’s messages, so exactly one reader is allowed to run. Separately, re-preparing a job that had already published would have cleared its “done” marker and queued the same post again; a guard now refuses to touch anything already published.

Credentials (check 9). If the automation runs on the freelancer’s API key, it stops the day their card expires. If it runs on yours, you can hand it to the next person.

Put this in the brief, not in the argument afterwards

A freelancer can pass all nine checks — but only if they know about them before quoting. Add one field to your brief:

Acceptance. Final payment after three unattended scheduled runs, one forced failure that alerts me, and one duplicate-run test. Credentials in my accounts. Written run-book of how to restart it.

That line changes the quote, and it should. You are paying for “it keeps working”, not “it worked once”. The workflow automation brief already has an acceptance field in this shape.

Where it goes wrong anyway

Even with every check passing, two things will eventually break it:

• Something upstream changes. A site redesign, an API version bump, a login flow that adds a new step. This is why maintenance belongs in the contract, not in a favour you ask for later.
• You change the schedule or the machine. Moving to a new computer or server re-opens checks 1 to 3. Re-run them.

Neither is the freelancer’s fault. Both are cheaper when the agreement already says who fixes it and at what rate.

Wrap-up

A delegated automation is worth what it does on the nights you don’t look at it. Watch three of those nights before you pay, break it on purpose once, and make sure the keys are in your name. Then write those same checks into the next brief so the freelancer builds for them from the start.