Cron Jobs and Queues: The Work Nobody Demos

Nobody asks to see the cron jobs in a demo. Stakeholders want to see the button that works, the page that loads, the feature that ships. But across every backend I have built, from Laravel projects early on to Go and Node.js services more recently, the background jobs are where a surprising amount of actual reliability lives.

Two different problems that look similar

Cron jobs and queues get lumped together because both run code outside a request-response cycle, but they solve different problems.

A cron job is for “this needs to happen on a schedule regardless of what users are doing”: nightly report generation, expiring stale sessions, syncing a cache. It does not care whether anything triggered it. It just runs.

A queue is for “this needs to happen because something happened, but not right now, in this request.” A user signs up, and sending the welcome email should not make them wait an extra 400ms for an SMTP round trip. A report gets requested, and generating it might take twenty seconds, far too long to hold a browser tab open. The request enqueues a job and returns immediately; a worker picks it up whenever it can.

Mixing these up is a common early mistake. I have seen scheduled reports built as a queued job with no actual schedule triggering it, and rate-sensitive user actions stuffed into a cron job that only runs every five minutes, making users wait for no good reason.

Where this actually breaks

The failure modes are rarely about the happy path. They show up in the edges:

• A cron job that overlaps with itself. If a nightly job usually takes ten minutes but occasionally takes forty because of a slow query, and it is scheduled to run every thirty minutes, you eventually get two instances running at once, both writing to the same rows. A simple lock (a database row, a Redis key with a TTL) prevents this, but only if you remember to add it before the collision happens, not after.
• A queue with no dead letter handling. A job fails, retries, fails again, and either retries forever or silently disappears. Neither is acceptable. Failed jobs need a place to land where a human can see them and decide what to do, not a black hole.
• Jobs that are not idempotent. If a worker crashes after doing the work but before marking the job complete, it runs again. If “send an email” and “charge a card” are not written to tolerate running twice, that second run causes a real problem, not just a log line.

What I actually check for now

After enough of these incidents, the checklist before shipping any background job is short but non-negotiable: does it have a timeout, is it safe to run twice, and is there a way to see it failed without having to grep logs after a user complains. None of this is impressive in a demo. It is the difference between a system that degrades gracefully at 2am and one that pages someone.