Agent Connections

While the Cluster defines global policy, the Agent Connection and Worker Configuration define how individual instances interact with the hardware and database.

An Agent is a high-speed ephemeral transport layer (PostgreSQL, SQL Server, MySQL, or NATS JetStream) used by workers to manage their buckets. It is designed for fast job ingestion and execution — not for long-term persistence. Durable state is always owned by the Master database; the Agent layer is intentionally short-lived and replaceable.

You can define multiple Agent connections to distribute load across databases, or even mix different provider types within the same cluster.

Agent Connection Strings

Each provider has its own configuration method. Below is a PostgreSQL example — see Providers for the full list including SQL Server, MySQL, and NATS JetStream.

config.AddAgentConnectionConfig("Postgres-1")
    .UsePostgresForAgent("Host=localhost;Database=agent_1;...");

The connection name (e.g. "Postgres-1") must contain only letters, digits, hyphens, and underscores, with a maximum of 50 characters.

warning

The agent connection name (e.g., Postgres-1) permanently identifies a set of buckets in the cluster. Both the name and the underlying database it points to must remain stable once jobs have been processed — changing either orphans in-flight work. The fingerprint mechanism enforces the database constraint at startup; see Connection Protection below.

To safely remove or migrate a connection, see Decommissioning a connection.

Connection Protection

By default, JobMaster enables connection protection on every Agent Connection to guard against accidental misconfiguration.

config.AddAgentConnectionConfig("Postgres-1")
    .UsePostgresForAgent(connectionString)
    .ProtectConnectionChanges(true); // default — can be omitted

Protection controls two distinct behaviors: what happens at startup when a fingerprint mismatch is detected, and what happens at runtime when a connection stops heartbeating.

Startup — fingerprint mismatch (e.g. the connection string was silently changed to point at a different database):

ProtectConnectionChanges	Behavior
true (default)	Throws an exception and refuses to start
false	Logs a warning and continues

Runtime — connection goes silent for more than 10 minutes:

ProtectConnectionChanges	Has active buckets	Behavior
true (default)	Yes	Emits a Critical log — connection is never auto-deleted
true (default)	No	Auto-deleted after 1 hour
false	Yes	Deletion skipped — retried every 5 minutes until buckets are empty
false	No	Connection record is auto-deleted from the Master DB

For production, leave protection at its default. To remove a connection safely, follow the Decommissioning a connection procedure.

Producer-Consumer Configuration

JobMaster is designed for high-scale environments where you may want to separate the Producer from the Consumer (Worker Services) across different servers or containers.

Producer-Only Instances

To enable an instance to only schedule work, define the agent connection but omit the worker configuration. This enables the short-circuit mechanism — scheduled jobs bypass the Master DB on the hot path and land directly in the Agent transport.

config.AddAgentConnectionConfig("Postgres-1")
    .UsePostgresForAgent(connectionString);

Consumer-Only Instances

To enable an instance to only process work, define the connection and bind a worker to it.

config.AddAgentConnectionConfig("Postgres-1")
    .UsePostgresForAgent(connectionString);

config.AddWorker()
    .AgentConnName("Postgres-1");

Scaling & The Hand-off

This separation allows for independent scaling of your infrastructure. You can have 10 producer instances handing off work to 50 dedicated Worker instances.

Immediate Job Flow: When you schedule a job using SavePending:

1. The Producer writes the job to the Agent ephemeral transport — the Master DB is not touched on the hot path.
2. A background runner evaluates the job's planned start time against the TransientThreshold:
   • Within the window (YES path): The job is written to the Master DB for auditing and routed directly to an active bucket (AssignedToBucket) for near-immediate execution.
   • Outside the window (NO path): The job is flushed to the Master DB as HeldOnMaster and picked up later by the Coordinator scan.

Decommissioning a connection

To safely remove or replace a connection without losing in-flight work:

1. Remove the AddWorker() call for that connection — the connection config itself stays so existing buckets can finish.
2. Deploy. Remaining jobs drain naturally with no new work routed to that connection.
3. Once the connection has no active buckets, remove the AddAgentConnectionConfig entry in the next release.