Distributed Setup

In a distributed setup, coordination and execution are separated across dedicated connections. The Master database remains the source of truth for job persistence and orchestration, while Agent connections handle the transport layer for fast job ingestion and execution.

This unlocks independent horizontal scaling of workers without adding load to the Master database.

tip

If you're just getting started, begin with the Getting Started. You can move to a distributed setup at any time.

Configuration

Register the cluster with a dedicated Master database, then add one or more Agent connections and bind workers to them:

builder.Services.AddJobMasterCluster(config =>
{
    config.ClusterId("Cluster-1")
          .UsePostgresForMaster("[master-connection-string]");

    config.AddAgentConnectionConfig("Postgres-1")
          .UsePostgresForAgent("[agent-connection-string]");

    config.AddWorker()
          .AgentConnName("Postgres-1");
});

var app = builder.Build();

await app.Services.StartJobMasterRuntimeAsync();

Multiple Agent Connections

You can mix different transport providers in the same cluster. Each agent connection gets its own set of workers:

builder.Services.AddJobMasterCluster(config =>
{
    config.ClusterId("Cluster-1")
          .UsePostgresForMaster("[master-connection-string]");

    config.AddAgentConnectionConfig("Postgres-1")
          .UsePostgresForAgent("[agent-connection-string]");

    config.AddAgentConnectionConfig("SqlServer-1")
          .UseSqlServerForAgent("[agent-connection-string]");

    config.AddAgentConnectionConfig("Nats-1")
          .UseNatsJetStream("[nats-connection-string]");

    config.AddWorker().AgentConnName("Postgres-1");

    config.AddWorker()
          .AgentConnName("SqlServer-1")
          .WorkerLane("LongRun");

    config.AddWorker().AgentConnName("Nats-1");
});

await app.Services.StartJobMasterRuntimeAsync();

With a distributed setup, AddWorker() enables advanced worker configuration — parallelism factor, bucket counts, buffer sizes, and worker mode. See Workers for the full reference.

Next Steps

• Cluster Configuration — Cluster-level settings and defaults
• Agents — Agent connections, producer/consumer separation, and migration strategy
• Workers — Worker modes, lanes, parallelism, and throughput tuning
• Providers — Full configuration for each transport provider