--- title: "Roll back a release" summary: "When to use rollback, resume and abort, what each guarantees, and why a data effect can make a rollback refuse." description: "Rollback, resume and abort — three different operations with three different guarantees." status: shipped read_when: - "A deploy failed or was interrupted" - "Deciding between resume, abort and rollback" - "Understanding why a rollback was refused" --- Three commands, three different situations. Choosing the wrong one is the usual way an incident gets worse. | Situation | Command | What it does | | --- | --- | --- | | A deploy was interrupted and you want to finish it | `ob resume` | Continues from durable phase state | | A deploy was interrupted and you want it undone | `ob abort` | Reverts the interrupted deploy | | A deploy completed but the release is bad | `ob rollback` | Activates the previous release | Start by asking the server, not your memory: ```sh ob status --output json ``` `ob status` exits non-zero on divergence, so it is safe to use as a gate in a script. ## Rollback activates the previous release ```sh ob rollback ``` Every release has a strict manifest state and an explicit predecessor. Rollback activates the current serving release's eligible predecessor, verifies it, and records the transition; it does not select a directory by timestamp or name. `deployment.retain_releases` (default `5`) bounds ordinary history while always protecting current, predecessor-chain, and checkpoint-referenced releases. ## What rollback does not do > **Application rollback is not data recovery** > > Application rollback, data recovery, and reversal of an external side effect are > different operations with different guarantees. Onebox classifies the risk and > **refuses** when evidence or a driver contract is insufficient. > > No flag turns an unsupported operation into a safe one. The override flags below > do not widen what Onebox can do — each one records that you have taken > responsibility for one specific judgement it refused to make for you. Concretely: - **Supporting services are not rolled back.** A service runs in its own Compose project and outlives every release, so no deploy and no rollback stops it or removes its volume. That protects your data; it also means a schema migration is still applied after you roll the application back. - **A job with `data_effect: migration` or `destructive` constrains the gate.** That is what the field is for. - **A job whose result evidence is missing becomes `changed=unknown`** and halts before workload replacement rather than proceeding on an assumption. ## Resume and abort work from durable state Both read the append-only journal, strict release manifests, and the durable activation checkpoint. Recovery removes only containers owned by the exact interrupted release, restores and verifies its predecessor, and clears the checkpoint last. A failed postcondition leaves the checkpoint retryable instead of hiding divergence. ```sh ob resume --output ndjson ob abort --output ndjson ``` Activation splits the two halves of a deploy. Interrupted before it, the release never took effect and `ob resume` replays the phases that remain. Interrupted after it — retention, schedule sync or the `post_deploy` hook failed while the new release was already current and healthy — there is nothing to replay: `ob resume` completes only those remaining steps, skipping any the journal already records as done, and the release keeps serving throughout. The manifest carries a failed operation outcome in the meantime, so a healthy release and an unfinished operation stay distinguishable. `ob resume` re-verifies the serving release first, then finalizes only when the recorded activation, the current release, the serving manifest and its recorded predecessor, the absence of an open activation checkpoint, and the live workload labels all agree. One disagreement returns `finalize_refused`: no step runs and no release, symlink or workload changes. Only the newest deploy is recoverable. Once a later deploy reaches a terminal state of its own — finished, aborted, or automatically rolled back — the older interrupted one is history rather than pending work, and resuming it would re-activate a release the host has already moved past. `ob status` stops reporting it; `ob audit` still shows the run. A manual `ob rollback` is journaled under the release it restores, so it does not settle an interrupted deploy that way. That deploy stays visible, and resuming it is refused on its superseded manifest before any workload is touched. ## When recovery refuses Two refusals have an override, and each grants exactly one thing. **`migration_gate_closed`** — the interrupted release ran a job or hook whose data effects are rollback-unknown, so `ob abort` will not roll the application back on its own. Fix forward and `ob resume`, or, if you know the old code reads the current data correctly, take responsibility explicitly: ```sh ob abort --break-migration-gate --output ndjson ``` **A stale operation lock** — a previous runner died holding it. Inspect the holder in `ob status` first; the lock records who took it and when. If it is genuinely abandoned: ```sh ob abort --break-lock --output ndjson ``` `--break-lock` is on six commands: `deploy`, `bootstrap`, `abort`, `job run`, `service apply` and `proxy apply`. Not every mutating command has it — `resume`, `destroy` and `secrets push` do not. It breaks the application and host locks only. The protection lock has no override and clears on TTL expiry or when the same operation returns. ## Recovery onto another host is a different workflow Rolling deployment can avoid interruption while the host is healthy. It cannot make a failed host available. Recovery onto another host is a distinct, evidence-backed workflow — not failover, and not something rollback reaches. ## Inspecting what happened ```sh ob audit # human table of recorded operations ob logs web # one workload or Onebox-run service ob status --output json ``` The journal is append-only, so the record of a failed operation survives the operation that replaced it.