Skip to main content

​
Horizontal scaling

Pylon is a single Rust binary. A single-machine deploy is the default setup: one machine serves HTTP, WebSocket, SSE, and job execution for the whole app. When traffic outgrows one machine, you scale horizontally by running pylon on multiple instances behind a load balancer. WebSocket broadcasts are in-process by default. A mutation handled by machine A fans out only to clients connected to machine A. Clients connected to machine B do not see it until their next reconnect, or until a visibility change triggers the client’s reconcile() backstop. Live UX with sub-second propagation needs more than this.

​
ClusterBus: cross-machine fanout

Pylon ships a ClusterBus abstraction. Configure a transport, and every change event, presence relay, or CRDT frame published locally also publishes to the bus. Subscriber threads on every peer machine receive these and re-broadcast them to their own WebSocket and SSE clients. The default transport is NoopBus. Single-machine deploys pay zero overhead. Pylon provides two production transports:
  • Redis PUB/SUB for self-hosted deployments.
  • The PylonSync Durable Object relay for Pylon Cloud.
Multi-machine deployments must use Postgres. SQLite cannot be shared safely by multiple Pylon processes.
Postgres stores the entities, sessions, global sync sequence, and persistent change log. Redis carries low-latency events between the running processes. Pylon Cloud uses the built-in relay. It does not add Redis to a customer project. The control plane gives each project a derived relay key. It never gives a customer machine the relay root key. PYLON_CLUSTER_NAMESPACE prefixes the Redis channel so multiple unrelated pylon deploys can share one Redis instance without cross-talk. It defaults to pylon if unset. Connection failures at startup are fatal. Pylon refuses to boot if PYLON_CLUSTER_BUS is set but unreachable. If it silently fell back to NoopBus on a multi-machine deploy, every machine would miss peer mutations. That produces the same “phantom row” UX failure the bus exists to fix, and it is harder to diagnose. A loud failure is the safer default. The managed relay applies backpressure when its publish queue is full. It retries a publish until the relay accepts it. Each publish has a stable message ID. The relay removes duplicate retries and keeps a bounded replay ring for WebSocket reconnects.

​
What’s fanned out

  • Change events (ChangeEvent): every mutation, action write, and entity-CRUD broadcast.
  • Presence relays: typing indicators, cursor positions, and any message sent through WsHub::broadcast_presence.
  • CRDT binary frames: Loro snapshots and updates for clients subscribed through useLoroDoc. Snapshot bytes are base64-encoded into the JSON envelope, so a single pubsub channel handles every payload shape.

​
Shared sync state

Postgres provides one global sequence for all changes. It also stores the persistent change log. Each process keeps a bounded in-memory ring for fast pulls and hydrates that ring from Postgres at startup. The cluster bus mirrors a peer event into each process’s local ring. As a result, a write on machine A is available from /api/sync/pull?since=N on machine B. If the local ring does not cover the requested range, the pull reads the shared Postgres change log. The client-side reconcile() pass remains a safety check. On reconnect or a visibility change, it compares the local replica with the authoritative entity rows and removes stale rows.

​
Shared jobs and workflows

Postgres deployments use Postgres as the job queue and workflow store. Workers claim ready jobs with row locks and short leases. One replica executes a claim at a time. Another replica can claim the job after the lease expires if the worker stops or loses its machine. Workflow transitions use the same lease model. A lease token prevents a stale worker from replacing newer workflow state. Workflow steps and the workflow state commit in one database transaction. Jobs use at-least-once execution. A worker can finish an external side effect and then stop before it records completion. A later worker can run that job again. Job and workflow handlers must be idempotent. An idempotent handler gives the same result when it runs more than once. Schedules created inside a Postgres mutation commit in the same transaction as the application writes. A rollback removes both the writes and the scheduled job. During a rolling release, each replica claims only job names for which it has a local handler. An old replica does not consume work that only the new release can execute. SQLite keeps jobs and workflows on the local machine. SQLite deployments remain single-machine deployments.

​
State that remains local

  • Per-client policy filtering. Each receiving machine re-runs the read policy before it forwards an inbound event to its connected WS and SSE clients. The bus carries raw events. The local fanout remains responsible for authorization.
  • Rate-limit counters. Limits apply per process. With N processes, the effective cluster limit is approximately N times the configured value. Adjust PYLON_RATE_LIMIT_MAX and PYLON_RATE_LIMIT_MAX_AUTHED for the number of processes.
  • SSR output cache. Each process has its own cache. Put a CDN in front of the load balancer when pages need a shared cache layer.

​
Self-event filtering

Pubsub backends deliver every published message to every subscriber, including the publisher itself. Without deduplication, this creates a feedback loop: A publishes, A’s subscriber receives, A re-broadcasts, and the event (already shipped locally) is delivered twice. Every envelope carries the publisher’s instance_id (one per pylon process, minted at startup). Each subscriber filters out events carrying its own id before it re-broadcasts them. Operators do not need to think about this; the filtering is invisible to them.

​
Diagnostics

Pylon logs the bus mode at startup:
Or, for single-machine:
If subscriber reconnects happen (for example, the Redis primary cycled, or there was a network blip), you will see them in the logs too:

​
When to enable

  • Anytime you run more than one pylon process serving the same app.
  • Fly autoscale with min_machines_running > 1.
  • K8s deployments with replicas > 1.
  • Blue/green rollouts where two versions of the binary briefly serve traffic simultaneously.
  • Local multi-process dev simulating production.

​
When to leave it disabled

  • Single-machine deploys. NoopBus is free; adding Redis only adds failure surface for no benefit.
  • Per-developer local dev. The reconcile() backstop covers the rare cases where two tabs need to see each other’s writes without a real cluster bus.

​
Pylon Cloud

Pylon Cloud supports multiple application machines for Postgres projects. The control plane configures the managed Durable Object relay, copies the current image and inline files to each new replica, and fans deploys and secret updates to all replicas. It also disables autostop while more than one machine serves the project. SQLite projects stay on one application machine. Switch the project to Postgres before you request replicas.

​
Backend choice

Use Redis for self-hosted apps. Pylon Cloud uses the managed Durable Object relay. ClusterBus remains transport-agnostic, so other transports can be added without API changes for callers.