---
title: "Roll back a release"
summary: "When to use rollback, resume and abort, what each guarantees, and why a data effect can make a rollback refuse."
description: "Rollback, resume and abort: three different operations with three different guarantees."
status: shipped
read_when:
  - "A deploy failed or was interrupted"
  - "Deciding between resume, abort and rollback"
  - "Understanding why a rollback was refused"
---
Rollback, resume and abort handle different situations; choosing the wrong one
is the usual way an incident gets worse.

| Situation | Command | What it does |
| --- | --- | --- |
| A deploy was interrupted and you want to finish it | `ob resume` | Continues from durable phase state |
| A deploy was interrupted and you want it undone | `ob abort` | Reverts the interrupted deploy |
| A deploy completed but the release is bad | `ob rollback` | Activates the previous release |

Start by asking the server, not your memory:

```sh
ob status --output json
```

`ob status` exits non-zero on divergence, so it is safe to use as a gate in a
script.

## Rollback activates the previous release

```sh
ob rollback
```

Every release has a strict manifest state and an explicit predecessor. Rollback
activates the current serving release's eligible predecessor, verifies it, and
records the transition; it does not select a directory by timestamp or name.
`deployment.retainReleases` (default `5`) bounds ordinary history while always
protecting current, predecessor-chain, and checkpoint-referenced releases.

Rollback reconciles each workload against the predecessor's immutable runtime
revision. A healthy fleet already at that exact revision stays running, even if
its `onebox.release` label names an older release; a missing, unhealthy, incomplete
or different revision follows the predecessor's configured rolling or recreate
strategy. Abort uses the same fail-closed rule.

## What rollback does not do

> **Application rollback is not data recovery**
>
> See the [bounded safety guarantees](/docs/reference/policies#safety-claims-are-bounded). The override flags below
> do not widen what Onebox can do; each one records that you have taken
> responsibility for one specific judgement it refused to make for you.

Concretely:

- **Supporting services are not rolled back.** A service runs in its own Compose
  project and outlives every release, so no deploy and no rollback stops it or
  removes its volume. That protects your data; it also means a schema migration
  is still applied after you roll the application back.
- **A job with `dataEffect: Migration` or `destructive` constrains the gate.**
- **A job whose result evidence is missing becomes `changed=unknown`** and halts
  before workload replacement.

## Resume and abort work from durable state

Both read the append-only journal, strict release manifests, and the durable
activation checkpoint. Recovery removes only containers owned by the exact
interrupted release, restores and verifies its predecessor, and clears the
checkpoint last. A failed postcondition leaves the checkpoint retryable instead
of hiding divergence.

```sh
ob resume --output ndjson
ob abort  --output ndjson
```

Activation splits the two halves of a deploy. Interrupted before it, the release
never took effect and `ob resume` replays the phases that remain. Interrupted
after it (retention, schedule sync or the `post_deploy` hook failed while the
new release was already current and healthy) there is nothing to replay:
`ob resume` completes only those remaining steps, skipping any the journal
already records as done, and the release keeps serving throughout. The manifest
carries a failed operation outcome in the meantime, so a healthy release and an
unfinished operation stay distinguishable.

`ob resume` re-verifies the serving release first, then finalizes only when the
recorded activation, the current release, the serving manifest and its recorded
predecessor, the absence of an open activation checkpoint, and the live workload
labels all agree. One disagreement returns `finalize_refused`: no step runs and
no release, symlink or workload changes.

Only the newest deploy is recoverable. Once a later deploy reaches a terminal
state of its own (finished, aborted, or automatically rolled back) the older
interrupted one is no longer pending work, and resuming it would
re-activate a release the host has already moved past. `ob status` stops
reporting it; `ob audit` still shows the run.

A manual `ob rollback` is journaled under the release it restores, so it does not
settle an interrupted deploy that way. That deploy stays visible, and resuming it
is refused on its superseded manifest before any workload is touched.

## When recovery refuses

Two refusals have an override, and each grants exactly one thing.

**`migration_gate_closed`**: the interrupted release ran a job or hook whose
data effects are rollback-unknown, so `ob abort` will not roll the application
back on its own. Fix forward and `ob resume`, or, if you know the old code reads
the current data correctly, take responsibility explicitly:

```sh
ob abort --break-migration-gate --output ndjson
```

**A stale operation lock**: a previous runner died holding it. Inspect the
holder in `ob status` first; the lock records who took it and when. If it is
genuinely abandoned:

```sh
ob abort --break-lock --output ndjson
```

`--break-lock` is on six commands: `deploy`, `bootstrap`, `abort`, `job run`,
`service apply` and `proxy apply`. Not every mutating command has it: `resume`,
`destroy` and `secrets push` do not. It breaks the application and host locks
only. The backup lock has no override and clears on TTL expiry or when the
same operation returns.

**An unreadable or invalid fencing epoch**: there is no override.
The application authority is `<base>/<app>/epoch`; each backup service has a
second authority at `<base>/<app>/backup/locks/<service>.epoch`. Do not delete
either file or replace it with `0` or `1`: that can recreate a fence still held
by an interrupted runner.

If the contents are intact, repair only the ownership or permissions that made
the file unreadable. Otherwise preserve the file and host evidence, stop or
isolate every runner that may still hold the old fence, and restore an epoch
known to be greater than every value previously issued. Onebox will refuse to
guess this value because a convenient reset would weaken the stale-runner
guarantee the epoch exists to provide.

## Recovery onto another host is a different workflow

Rolling deployment can avoid interruption while the host is healthy. It cannot
make a failed host available. Recovery onto another host is a distinct,
evidence-backed workflow: not failover, and not something rollback reaches.

## Inspecting what happened

```sh
ob audit          # human table of recorded operations
ob logs web       # one workload or Onebox-run service
ob status --output json
```

The journal is append-only, so the record of a failed operation survives the
operation that replaced it.