Durable Execution
A survey of durable execution: making a long-running program survive the death of its process by journaling the results of its effects and re-deriving everything else. The technique sits directly downstream of algebraic effects — the journaling layer is a handler over an effect boundary — but it grew up in a different literature, and the two have barely met.
The survey is deliberately application-neutral: it reads each system on its own terms and judges it by what its own users need. sparkles has no durable- execution layer today, and the gap between the state of the art and what sparkles:event-horizon's capability row already provides is recorded in the comparison's delta table — but no consumer's requirements shaped which questions were asked or which answers were judged good.
Last reviewed: September 12, 2026.
The eleven questions this survey answers
- How is an effectful step identified and matched on replay? → Comparison §1, and every deep-dive's Analysis §1.
- When the journal and the world disagree, which wins, and how is disagreement detected? → Comparison §2, ARIES, Helland.
- How is determinism between journaled steps enforced? → Comparison §3, deterministic record and replay, deterministic simulation testing.
- How are compensations registered, ordered and triggered? → Comparison §4, Sagas, compensation calculi.
- How does workflow code evolve against journals written by older code? → Comparison §5, Golem, Temporal.
- How do concurrent steps replay? → Comparison §6, compensation calculi.
- Replay or snapshot: what does each cost and rule out? → replay versus continuation snapshotting, Comparison §7.
- How is a durable program tested? → Comparison §8, deterministic simulation testing, and the runnable replay-journal example.
- How is the record written safely, and what stops two writers corrupting it? → Comparison §9, ARIES, Marten, Akka/Pekko.
- What can a person do to a run that is stuck, wrong, or diverged? → Comparison §10, Temporal, Golem, deterministic record and replay.
- How does a program wait for something it did not invoke? → Comparison §11, Restate, Effect, Trigger.dev.
The last three were added after a first pass over the finished catalog showed that every subject had an answer to them, scattered through the prose rather than asked uniformly.
Start with Concepts and Vocabulary if the terms are unfamiliar: the field has no agreed names, and the same idea is a step, an activity, a journal entry and an oplog entry depending on whose documentation you are reading.
Master catalog
Durable-execution engines and SDKs
| Subject | Language | Persistence model | Step identity | Link |
|---|---|---|---|---|
@effect/workflow + @effect/cluster | TypeScript | replay | name + attempt | effect-workflow |
| Temporal | Go server; Go + TypeScript SDKs | replay | positional counter | Temporal |
| Restate | Rust server; TypeScript SDK | replay | positional + header check | Restate |
| DBOS Transact | TypeScript (+ Python, Go, Java) | replay | positional function_id | DBOS |
| Azure Durable Functions | C# (DTFx) + JavaScript | replay; entities snapshot | positional ordinal | azure |
| Dapr Workflow + Durable Task Framework | C#, Go | replay | positional idCounter | dapr |
| Inngest | Go server; TypeScript SDK | replay | hashed name + repeat counter | Inngest |
| Cloudflare Workflows | TypeScript on workerd | replay | name + occurrence count | cloudflare |
| Vercel Workflow DevKit | TypeScript (SWC plugin) | replay | seeded deterministic ULID | vercel |
| Resonate | Rust server; TypeScript SDK | replay | derived promise id | Resonate |
| Golem | Rust runtime; any WASM guest | replay + optional state snapshot | positional + structural claim | Golem |
Contrast cases
| Subject | Language | Persistence model | Why it is here | Link |
|---|---|---|---|---|
| Trigger.dev | TypeScript | snapshot (CRIU process checkpoint) | the one system that really snapshots a computation | trigger |
| AWS Step Functions | Amazon States Language | explicit state machine | control flow as data; the rejected design alternative | asf |
Event-sourced state
| Subject | Language | Persistence model | Link |
|---|---|---|---|
| Akka / Pekko Persistence | Scala, Java | event log + snapshots | akka |
| Orleans | .NET | snapshot, event log, or both per provider | Orleans |
| KurrentDB (EventStoreDB) | .NET | append-only log + projections | kurrent |
| Marten | .NET on PostgreSQL | event log + inline/async projections | Marten |
Theory
| Subject | Year / venue | Grounds questions | Link |
|---|---|---|---|
| Sagas (Garcia-Molina, Salem) | SIGMOD 1987 | 4, 9, 10 | sagas |
| ARIES write-ahead logging (Mohan et al.) | TODS 1992 | 1, 2, 4, 8, 9, 10 | wal |
| Compensation calculi (cCSP; Bruni et al.; Helland) | 2005, 2005, 2009 | 2, 4, 6, 9, 10 | calculi |
| Idempotence and the outside world (Helland) | CIDR 2007, Queue 2012 | 1, 2, 4, 9, 11 | idempotence |
| Durable Functions: Semantics for Stateful Serverless | OOPSLA 2021 | 1, 2, 3, 6, 7, 9, 11 | burckhardt |
| Netherite: Efficient Execution of Serverless Workflows | VLDB 2022 | 2, 3, 7, 9 | Netherite |
| Deterministic record and replay (rr and ancestors) | 1987–2017 | 1, 3, 6, 7, 9, 10, 11 | replay |
| Replay versus continuation snapshotting | 2006–2026 | 5, 7, 9, 10, 11 | replay-vs-snapshot |
| Deterministic simulation testing | 2014–2026 | 3, 8, 9, 10 | dst |
| Effect handlers and record/replay | 2013–2022 | 1, 3, 4, 9, 11 | handlers |
Taxonomy
By persistence model
| Model | Subjects |
|---|---|
| Replay only | Temporal, Restate, DBOS, dapr, Inngest, cloudflare, vercel, Resonate, effect-workflow |
| Replay + state snapshot | Golem, azure (entities), Netherite, Orleans, Marten, akka, kurrent |
| Continuation snapshot | trigger; historically Termite Scheme and Unison (replay-vs-snapshot) |
| Explicit state machine | asf |
By step identity
| Scheme | Subjects | Consequence |
|---|---|---|
| Positional | Temporal, azure, dapr, DBOS, Restate, Netherite, Golem | editing the program is a compatibility event; needs patch markers |
| Named | Inngest, cloudflare, effect-workflow | free reordering; name collisions and silent renames |
| Derived | Resonate, vercel | stable without naming discipline; opaque in the journal |
| None | trigger, asf | nothing re-executes, so nothing is matched |
By determinism enforcement
| Degree | Subjects |
|---|---|
| By construction | Netherite, burckhardt |
| By sandbox | Golem, vercel, Temporal (TypeScript) |
| By substitution | Temporal (Go), azure, dapr, Restate |
| By static analysis | azure (.NET only) |
| By discipline | Inngest, DBOS, cloudflare, Resonate, Orleans, Marten |
| Not required | trigger, asf |
By journal ownership
| Owner | Subjects |
|---|---|
| A dedicated server | Temporal, Restate, Inngest, Resonate, asf, kurrent |
| The application's own database | DBOS, Marten, effect-workflow, akka |
| The platform's storage runtime | cloudflare, azure, dapr, Golem, Netherite, Orleans |
| Local files | vercel (development world) |
Milestones
Dates are those of the cited publication or release, verified on the linked page.
| Year | Milestone |
|---|---|
| 1987 | Sagas defines compensation and backward recovery for long-lived transactions |
| 1992 | ARIES fixes the write-ahead protocol, "repeating history", and compensation log records |
| 2005 | Compensation calculi give compensable processes a trace semantics and the LIFO law |
| 2006 | Termite Scheme ships serializable continuations for process migration (replay-vs-snapshot) |
| 2007 | Helland argues at CIDR that distributed transactions must be replaced by idempotent activities |
| 2009 | Building on Quicksand reframes compensation as apology (calculi) |
| 2011 | Cloud Haskell shows arbitrary closures cannot be serialized, only static labels (replay-vs-snapshot) |
| 2013 | Ramalingam and Vaswani prove fault tolerance via idempotence for a replayed effect log (handlers) |
| 2017 | rr makes whole-process record and replay deployable on commodity hardware |
| 2018 | Pyro's poutine ships the trace/replay handler pair — the mechanism, without persistence (handlers) |
| 2021 | Burckhardt et al. give durable functions a formal semantics and prove replay equivalent to direct execution |
| 2021 | FoundationDB's simulation testing is published with the SIGMOD paper |
| 2022 | Netherite shows a partitioned event-sourced engine beating per-step storage round trips |
| 2025 | Cloudflare Workflows reaches general availability, putting step.do on an edge runtime |
Suggested reading paths
"I want the shortest honest summary." Concepts, then Comparison.
"I am designing the sparkles durable layer." Comparison (especially its delta table and the closing "Questions for the sparkles design"), then ARIES and Helland for the journal-versus- world problem, then Sagas and the calculi for compensation, then Golem for the one serious answer to versioning, and Resonate for the testing bar to clear.
"I want the formal ground." Burckhardt et al., then handlers for what the effects literature does and does not have, then Netherite for the engineering the semantics permits.
"I want to see it work." The runnable replay-journal example: about 200 lines of D that journal a miniature release, crash at every index, resume, and detect a diverged program. ci --example-files compiles and runs it.
"I want the closest prior art to what we are building."effect-workflow (the design this was sketched from), Restate (the closest journal shape), and DBOS (the closest deployment shape — a library over a database, no server).
Sources
Each deep-dive carries its own primary citations: 430-plus GitHub file references, every one pinned to a commit hash and verified to resolve, plus the papers and official documentation listed on each page. The catalog reads upstream source trees cloned under $REPOS at the revisions its metadata tables name.