How Cloudflare Workflows Handles Failure So You Don't Have To
Multi-step jobs fail. Cloudflare Workflows makes sure they don't fail twice.
Multi-step jobs fail. Cloudflare Workflows makes sure they don't fail twice.
If you’ve ever built a multi-step backend process, something like upload a file, process it, call a third-party API, write to a database, you already know the real enemy isn’t the happy path. It’s everything that can go wrong in between.
APIs time out. Rate limits kick in at the worst moment. A database connection blips for half a second and suddenly your “simple” pipeline needs retry logic, backoff logic, idempotency checks, and a way to resume from wherever it died instead of starting over from step one. Most teams end up hand-rolling this stuff over and over, and it’s rarely fun to maintain.
Cloudflare’s answer is a product called Workflows, built on Workers and Durable Objects. It’s their take on “durable execution,” the same general idea behind Temporal or AWS Step Functions, where the platform, not your app code, is responsible for tracking progress, retrying failures, and picking up exactly where things left off.
Want to just poke at it instead of reading about it? One command scaffolds a starter:
npm create cloudflare@latest workflows-starter -- \
--template "cloudflare/workflows-starter" Open the generated file and deploy it with wrangler deploy.
The building block here is the step. Each one wraps a piece of work, a DB call, an API request, a file read, and its result gets cached automatically. If something downstream fails, the Workflow doesn’t start over. It just resumes from the last step that didn’t finish.
export class MyWorkflow extends WorkflowEntrypoint {
async run(event, step) {
const files = await step.do("list uploaded files", async () => {
return await listFilesFromBucket(event.payload.bucket);
});
// more steps follow, each with access to prior results
}
} You can tune retry behavior per step too. Limit, delay, exponential backoff, timeout, all without touching anything else:
await step.do(
"call the flaky upstream API",
{ retries: { limit: 5, delay: "5 seconds", backoff: "exponential" }, timeout: "15 minutes" },
async () => {
// do the actual work here
}
); Steps don’t need to be defined ahead of time either. You can generate them on the fly, one per file, one per record, which matters more than it sounds like it should once you’re processing a batch of unknown size.
Here’s the part that actually got my attention. Cloudflare built Workflows almost entirely out of their own existing primitives. No new backend infrastructure, no separate compute layer. Just Workers and Durable Objects, including the newer SQLite-backed storage DOs picked up recently, stacked on themselves.
Roughly three layers make it up. There’s a stateless config API and binding layer that fields requests from the REST API, Wrangler, or the dashboard. There are account controllers, one Durable Object per account, tracking every workflow, version, and instance that account owns. And there are engines, one per running instance, which do the actual step execution and state persistence.
One Durable Object per account is a clever way to get account-level isolation basically for free. Data stays close to wherever the request is coming from, and if something breaks for one account, it stays contained to that account instead of taking down the whole platform. I like this pattern generally. It’s the kind of design decision that looks obvious in hindsight but clearly wasn’t the first thing anyone tried.
Internally, these Workers talk to each other over RPC instead of the older fetch-and-JSON-parse dance. It’s a small detail, but it removes a real amount of boilerplate. You just call a method on a remote object like it’s local, and TypeScript catches the mistakes you’d otherwise find in production.
This is my favorite design decision in the whole thing. Their first instinct for modeling execution internally was a classic state machine. Makes sense on paper, except a state machine needs to know the full shape of your workflow ahead of time, which falls apart the moment steps are generated dynamically. You’d need a build step just to understand what your own workflow does.
So they modeled it as a game loop instead. Keep iterating over the steps, replay cached results for anything already done, keep grinding on whatever hasn’t finished, until the sequence completes. It’s almost dumb simple as an idea, which is exactly why I think it’s the right call. It doesn’t require Cloudflare, or you, to understand your workflow’s shape in advance, and it maps naturally onto how Durable Objects already behave. They sleep, they wake up on an alarm, they pick up where they left off. That’s the same mechanism that lets a step pause for three days without anything burning compute the whole time.
Think about what happens when a customer’s subscription is about to renew. You want to charge their card, and if the charge fails (an expired card, insufficient funds, a bank momentarily rejecting the transaction), you don’t want to just give up or hammer the payment provider immediately. You want to retry with a delay, maybe email the customer to update their card, wait a few more days, try again, and only cancel the subscription if every attempt over, say, a week has failed. This is exactly the kind of job people usually reach for a scheduled task, a queue, and a polling table to solve, and it’s basically a textbook Workflow instead.
Kick off a Workflow on the renewal date. Attempt the charge as the first step, with its own retry policy in case the payment provider hiccups, which, in my experience, it always eventually does. If it still fails after retries, send the customer a “please update your card” email. Have it pause and wait for two days, then attempt the charge again. If it fails a second time, wait a bit longer and try once more before finally canceling the subscription and notifying the customer.
The logic reads like a description of what should happen, not a defense against everything that might go wrong. That’s really the whole pitch. The durability stuff fades into the background so the code says what you meant it to say.
Long-running async jobs are miserable to debug blind, so Workflows leans hard into observability by default. Every instance keeps a timestamped log of each step’s attempts, successes, and failures, queryable via REST, aggregated through a GraphQL analytics endpoint, or just browsable in the dashboard. Wrangler will also print a clean formatted summary of an instance’s status, which honestly is the thing I’d reach for first when something’s stuck.
Because Workflows uses the same CPU-time pricing as Workers generally, you’re billed for the compute your steps actually consume, not for the wall-clock time spent asleep waiting on a timer or a slow API. That matters a lot given the shape of workflow this thing is built for: stuff that might sit idle for days between a trigger and a final action. Paying for idle time would undercut the entire premise.
Durable execution isn’t a new concept. Temporal and Step Functions have been doing versions of this for a while. What’s different here is not having to run and babysit your own compute cluster just to get the retry-and-resume guarantees. If you’ve looked at Temporal before and been put off by the operational overhead, this is a noticeably lighter lift, assuming you’re already in the Workers ecosystem.
But honestly, the pattern matters more than the product. Wrap risky operations in small, retriable, idempotent units. Let the infrastructure track state instead of hand-rolling it in application code. Whether you do that with Workflows, Temporal, Step Functions, or something you build yourself, that shift is what keeps multi-step systems resilient, even when APIs time out, services become unavailable, or unexpected failures occur.
No comments yet