Pilots run the same checklist every single time, precisely because the routine is where complacency lives. Scheduled jobs deserve the same courtesy. AutomationOps is the Ops Atlas’s editorial label for giving operational automation production discipline — the Atlas’s framing of the practice is the reference behind every line below — and it compresses remarkably well: a short list of checks before any job’s first real run, plus a few numbers watched afterwards. Here is that checklist in friendly, usable form, for everything from nightly backups to fleet-wide clean-ups.
The AutomationOps Pre-Ship Checklist
Twelve items, in order. On a small job they take half an hour; on a scary job they take an afternoon and are worth every minute.
- One sentence on purpose. “Refreshes the cached departures page every ten minutes so readers never see stale times.” If you cannot write the sentence, the job is not ready to run — and if you can, you have just written the first line of its documentation.
- An owner with a pulse and a pager. Individual people answer alerts; group inboxes do not. Record the person’s name, and when they move on, move the record — ownership that outlives its holder is ownership nobody holds.
- Its own identity. The job gets credentials that belong to nobody else. Six months from now, the logs should say exactly who acted, and revoking one job should never lock out its colleagues.
- A grade before a run. Ask four questions: can it change anything at all, can one target be undone, can a hundred be undone, or is the damage permanent? The answers set the job’s risk grade, and the grade decides how much ceremony the remaining items deserve.
- A rehearsal that changes nothing. Run the job in its dry mode and read the output like a sceptic. Know what your tool’s check mode skips — several leave out direct commands unless flagged — so a passing rehearsal is a measurement, not a vibe. Rehearse against realistic data, too: an empty afternoon test tells you less than one bad Friday’s worth.
- The zero-target test. Feed the job an empty list on purpose. It must stop and complain. If it carries on cheerfully as if “nothing” meant “everything”, do not ship it; that single behaviour is behind some of the most expensive deletes in industry history.
- A rollout plan in waves. First target, then a small batch, then the rest — with a health check between waves and an automatic halt when failures cross the line you chose. The line is usually one failure. Ten minutes of patience here is what stands between a burnt edge and a burnt fleet.
- Brakes you can reach. A kill switch for this job, a global one above it, both labelled, both tested. The on-call engineer should find both from memory at three in the morning, and item eleven exists to prove it.
- Registration as code. One entry — owner, tier, limits, review date — versioned beside the job, so an unregistered change fails review instead of surfacing in an incident. Approvals then follow the grade automatically: light for harmless, human sign-off for anything destructive. Once the register exists, the question “what else is running out there?” finally has an answer.
- A fallback answer. When the job fails twice, who steps in, with what instructions? Write the handoff down while everyone is calm; it is terrible reading material for a crisis.
- A booked drill. For grades 2 and up, stopping the job is practised twice a year on the calendar, not saved for the night you need luck.
- A review date. Every automation rots quietly as the world around it changes — feeds evolve, endpoints retire, assumptions calcify. Six months out, the job comes back to this checklist. Anything that cannot justify its existence at review is retired with thanks, which is the healthiest sentence in this whole list.
After the Run: Five Numbers Worth Watching
The checklist gets a job safely airborne; numbers keep it honest in cruise. Five earn their place on a dashboard:
- Run success rate — the plain percentage of runs finishing without help. A falling curve means the world changed underneath the job while it stayed the same.
- Human fallback rate — the share of runs that still end with a person cleaning up. High values mean you own a robot that needs a handler, and the hours-saved arithmetic deserves a recount.
- Guard trip rate — how often the brakes fired: batches halted, empty lists refused, budgets exhausted. Occasional trips mean the guards work; constant trips mean the job is built wrong.
- Induced-incident rate — of every hundred runs, how many leave an outage or a broken thing behind. It borrows its spirit from DORA’s change-failure rate, and it catches the awkward truth that a job can “succeed” while making everything worse.
- Time to stop — measured during drills, from “something is wrong” to “it has stopped”. Seconds here are the whole ROI of the brakes.
The First Ninety Days, Mapped on One Page
Starting from zero, or from a decade of unsupervised cron? The Atlas’s ninety-day ramp fits a small team nicely, and it asks for no reorganisation — only sequenced honesty. By day 30, inventory everything that runs unattended and register the twenty riskiest jobs with owners and grades. By day 60, consolidate the scheduled work onto a single runner so everything answers to one set of brakes, separate every job’s credentials from its neighbours’, and hold the first stop drill. By day 90, start promoting jobs with evidence — numbers, not anecdotes — and push ownership coverage past ninety percent. Three months, no reorganisation, and the estate goes from folklore to ledger.
When Your AutomationOps Runner Is an AI Agent
The newest wrinkle: the “script” increasingly has reasoning. The current wave makes this concrete: Amazon’s DevOps Agent left preview in March 2026, Azure now fields an SRE Agent of its own, and both can act on operational problems — restarts, mitigations, clean-ups — bringing judgment into the loop. The governance answer is refreshingly unchanged. Give an agent the narrowest identity that works, start it read-only until its suggestions earn trust, route its actions through the same guards as any other job, and grade anything it may do before it may do it. An AI runner is not a new discipline; it is the old discipline with a chattier front end — a theme this site’s piece on how AIOps transforms IT infrastructure and automation picks up from the monitoring side, and its guide to practical AIOps knowledge for IT teams extends for day-to-day practice.
Fly the Checklist, Keep the Smile
None of this needs a platform purchase or a committee. It needs the same charm aviation has always run on: respect for routine, written checklists, and people who practise the emergencies they hope never to meet. Teams wanting the wider engineering context behind these habits will enjoy the deep dive on enterprise automation, infrastructure and reliability at cloud scale, and the Atlas reference linked above for the full framework. Fill in the twelve items, watch the five numbers, hold the ninety-day line — and your unattended jobs will keep earning their keep, night after night, without ever making the morning news. That is the entire trick — and the checklist is now yours.
