Skip to main content

Migrating a Cluster

ClusterMode.Migrating lets you retire a cluster without downtime or losing in-flight work — useful when moving to new infrastructure, changing a database provider, or consolidating clusters. A migrating cluster never locally dispatches or executes new work: held jobs and recurring schedules are continuously moved (not just referenced or proxied — physically relocated, then deleted from here) to another, healthy cluster, which does the actual dispatch/execution instead, while whatever is already in flight on this cluster drains out naturally.

Enabling it​

Use MigrateTo(targetActiveClusterId) rather than Mode(ClusterMode.Migrating) directly — it sets the mode and the mandatory target together, so there's no way to end up in Migrating mode without a target:

config.MigrateTo("payroll-cluster-v2");

What happens​

  • No new buckets are created on the migrating cluster — it never onboards new work locally again.
  • Jobs and recurring schedules held on the Master DB (OnMaster) are continuously moved to the target cluster's own Master DB instead — inserted there, then deleted here. A job's full execution history (every JobExecution attempt) and its JobExecution-category logs move with it, the same way they do when archiving a job — nothing about a held job's retry history is lost by migrating it.
  • Jobs already assigned to a bucket at the moment of the switch finish via the normal Drain path — nothing in flight is abandoned.
  • Execution-mode workers are not allowed on a migrating cluster (there's nothing left for them to execute, since no new buckets are ever created); Coordinator, Drain, and Full are all fine — Full skips its execution runners and acts as Coordinator + Drain.

Typical migration workflow​

  1. Stand up the new cluster ("payroll-cluster-v2") as Active, alongside the existing one.
  2. Point your producers/API traffic at the new cluster ID.
  3. Flip the old cluster into Migrating mode via MigrateTo("payroll-cluster-v2").
  4. Let it drain out naturally — the old cluster keeps running (so its Drain workers can finish existing buckets and forward anything held on master) until it's empty.
  5. Decommission the old cluster once it has no more buckets or held work.

Validation​

Enforced at startup:

  • A migrating cluster must have a target configured (MigrateTo(...)) — an empty target throws.
  • The target can't be the cluster itself.
  • The target must resolve to another configured cluster. That cluster must be Active — either explicitly via Mode(ClusterMode.Active), or simply never configured with a Mode at all, since Active is the framework default.
  • Like Archived, a Migrating cluster can't have any buckets explicitly configured on its workers.

See: Cluster Configuration · Archiving