> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pylonsync.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Realtime Shards

> Authoritative game and simulation logic in Rust, compiled to WebAssembly and run by the stock pylon binary on a fixed tick.

A shard is one running simulation: a match, a zone, a room. The server owns
its state. Clients send inputs, the shard applies them on its tick, and every
subscriber gets a snapshot after each tick.

You write the simulation in Rust with the `pylon-shard-guest` crate and
compile it to WebAssembly. The `pylon` binary loads the module at boot. You
do not build a custom server binary, so the same app runs on `pylon dev`,
`pylon start`, a Docker image, and Stack0 Cloud.

The complete example is [`examples/shard-arena`](https://github.com/pylonsync/pylon-examples/tree/main/shard-arena).

## Set up

Install the WebAssembly target once:

```bash theme={null}
rustup target add wasm32-unknown-unknown
```

Make a library crate inside the app, for example `shards/arena`:

```toml shards/arena/Cargo.toml theme={null}
[package]
name = "arena-shard"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["cdylib"]

[dependencies]
pylon-shard-guest = "0.13"
serde = { version = "1", features = ["derive"] }

[profile.release]
opt-level = "s"
lto = true
```

If the app is inside a Cargo workspace, add an empty `[workspace]` table so
the crate builds on its own.

## Write the simulation

Implement `Shard` and export it with `export_shard!`:

```rust shards/arena/src/lib.rs theme={null}
use std::time::Duration;

use pylon_shard_guest::{export_shard, Shard};
use serde::{Deserialize, Serialize};

#[derive(Deserialize)]
struct Params {
    width: f32,
    height: f32,
}

#[derive(Deserialize)]
#[serde(rename_all = "snake_case")]
enum Input {
    MoveTo { x: f32, y: f32 },
}

#[derive(Serialize, Clone)]
struct Player {
    id: String,
    x: f32,
    y: f32,
}

#[derive(Serialize)]
struct Snapshot {
    players: Vec<Player>,
}

struct Arena {
    width: f32,
    height: f32,
    players: Vec<Player>,
}

impl Shard for Arena {
    type Params = Params;
    type Input = Input;
    type Snapshot = Snapshot;

    fn init(_shard_id: &str, p: Params) -> Result<Self, String> {
        Ok(Arena { width: p.width, height: p.height, players: Vec::new() })
    }

    fn apply_input(&mut self, sid: &str, input: Input) -> Result<(), String> {
        let Input::MoveTo { x, y } = input;
        if x < 0.0 || x > self.width || y < 0.0 || y > self.height {
            return Err("outside the arena".into());
        }
        match self.players.iter_mut().find(|p| p.id == sid) {
            Some(p) => (p.x, p.y) = (x, y),
            None => self.players.push(Player { id: sid.into(), x, y }),
        }
        Ok(())
    }

    fn tick(&mut self, _dt: Duration) {}

    fn snapshot(&self) -> Snapshot {
        Snapshot { players: self.players.clone() }
    }
}

export_shard!(Arena);
```

The trait has these methods:

| Method | Called | Default |
| - | - | - |
| `init(shard_id, params)` | Once, from `ctx.shards.create`. An error refuses the create. | Required |
| `apply_input(subscriber, input)` | On the tick, once per queued input, in arrival order. An error goes back to the sender as an `apply_failed` rejection. | Required |
| `tick(dt)` | At the tick rate. `dt` is `1 / tickRate` with `fixedTimestep`. | Required |
| `snapshot()` | After each tick. | Required |
| `snapshot_for(subscriber)` | After each tick, per subscriber. Return `Some` for area of interest or fog of war. | `None`: the subscriber gets `snapshot()` |
| `is_finished()` | After each tick. `true` stops the shard. | `false` |
| `authorize_subscribe(subscriber, auth)` | When a client connects. | Admits an admin, a ticket holder, or a user whose id is the subscriber id |
| `authorize_input(subscriber, auth, input)` | Before an input is queued. | Admits all |

`auth` carries `user_id`, `is_admin`, `roles`, `tenant_id`, and the verified
ticket. Read ticket claims with `auth.claim("realm")`.

The module has no clock, no random source, and no I/O. Put a seed in
`params` when the game needs random numbers. The same inputs then give the
same state, which a replay needs. `pylon_shard_guest::log(Level::Info, "…")`
writes to the server log.

### Send each player only what is near them

In a large zone, return an `InterestConfig` from `interest`. Each tick the
host puts your entities in a grid, keeps the ones within each subscriber's
radius, and asks `snapshot_visible` for that subscriber's snapshot:

```rust theme={null}
use pylon_shard_guest::{EntityPos, InterestArea, InterestConfig, View};

impl Shard for Zone {
    // ...
    fn interest(&self) -> Option<InterestConfig> {
        Some(InterestConfig { cell_size: 150.0, margin: 10.0, shared_snapshots: true })
    }

    fn entities(&self, out: &mut Vec<EntityPos>) {
        out.extend(self.units.iter().map(|u| EntityPos { id: u.id, x: u.x, y: u.y }));
    }

    fn interest_area(&self, sid: &str) -> Option<InterestArea> {
        let me = self.player_unit(sid)?;
        Some(InterestArea { x: me.x, y: me.y, radius: 150.0 })
    }

    fn can_see(&self, sid: &str, entity: u64) -> bool {
        !self.is_stealthed(entity) || self.can_detect(sid)
    }

    fn snapshot_visible(&self, _sid: &str, view: &View) -> Option<Snapshot> {
        Some(Snapshot { units: view.visible.iter().map(|id| self.unit(*id)).collect() })
    }
}
```

* An entity leaves view only after it is `margin` past the radius, so an
  entity at the edge does not flicker in and out.
* `can_see` hides an entity in range, for stealth or fog of war.
* `view.entered` and `view.left` list the ids that came into and went out
  of view since the last tick.
* With `shared_snapshots: true`, subscribers that see the same entities get
  one snapshot, encoded once. Use it only when `snapshot_visible` reads
  nothing but `view.visible`.
* Positions are 2D. A 3D world passes its ground plane.
* For 300 subscribers and 1,000 moving entities, the grid work takes about
  0.2 ms per tick when players are spread out, and about 1.5 ms when they
  crowd together.

A Rust shard compiled into a custom binary uses the same hooks on
`SimState`: `interest_config`, `entities`, `interest_area`, `can_see` (or
`filter_visible`), `snapshot_visible`, `shares_visible_snapshots`,
`replicated`, and `replication_config`.

### Replicate entities instead of sending snapshots

A snapshot is the whole visible state every tick. For many moving entities,
replicate them instead: keep them in a `Replicated` store, and each
subscriber gets only what changed.

```rust theme={null}
use pylon_shard_guest::{Replicated, ReplicationConfig};

const HP: u8 = 1; // component ids are yours to choose

impl Shard for Zone {
    // ...
    fn tick(&mut self, dt: Duration) {
        for unit in &mut self.units {
            unit.step(dt);
            self.store.set_pos(unit.id, [unit.x, unit.y, 0.0]);
        }
    }

    fn replicated(&mut self) -> Option<&mut Replicated> {
        Some(&mut self.store)
    }

    fn replication_config(&self) -> ReplicationConfig {
        ReplicationConfig { precision: 0.01, max_bytes_per_tick: 900, y_up: false }
    }
}
```

Use the store's methods for every change: `spawn(id, [x, y, z])`,
`set_pos`, `set_component(id, component, bytes)`, `remove_component`, and
`despawn`. A change to the same value is not sent.

Each tick, each subscriber gets one frame with:

* **despawn** for entities that left its view or the store,
* **spawn** with the full state for entities that came into view,
* **update** for entities in view that changed: only the axes whose
  position moved (as a difference) and the components that changed or were
  removed.

The view is the interest view when `interest` returns a config, or every
entity otherwise. `can_see` still hides entities. `snapshot*` is not used.

| Setting | Meaning |
| - | - |
| `precision` | Position step in world units. Positions travel as `round(value / precision)`. |
| `max_bytes_per_tick` | Update budget per subscriber per tick. Spawns and despawns always go; updates go in priority order (the longer since the entity was sent and the nearer it is, the higher). What does not fit is sent later. `0` = no limit. |
| `y_up` | The ground plane interest management uses: `false` for x/y, `true` for x/z. |

A new connection, and one whose outbound queue dropped frames, gets a full
frame. Replication needs the shard WebSocket (`/shard`) or WebTransport
(see below); SSE refuses a replicating shard.

For 300 players in one area, all moving, at 20 Hz: about 23 KB/s per
client with no budget, and 15 KB/s with a 900-byte budget, where a
client's positions stay within 0.08 units of the server's at p99.

### Client: a render loop

`@pylonsync/realtime` is the client with no framework. `connectShardGame`
connects, applies each frame, and places entities for a render frame
without a React render:

```ts theme={null}
import { connectShardGame } from "@pylonsync/realtime";

const game = connectShardGame<Input>(shardId, { subscriberId, ticket, tickRate: 20 });

function frame(now: number) {
  game.frame(now);
  for (const id of game.left) removeMesh(id);
  for (const id of game.entered) addMesh(id);
  for (const e of game.entities.values()) {
    moveMesh(e.id, e.x, e.y, e.z, e.components.get(HP));
  }
  requestAnimationFrame(frame);
}
requestAnimationFrame(frame);
```

* **Interpolation.** Entities are drawn `interpolationDelayMs` (default
  100\) behind the shard's current tick, between the two frames around that
  time. Spawns, despawns, and component changes show at the tick they
  happened. `e.from`, `e.to`, and `e.t` give the samples and the fraction,
  to interpolate your own data in components (a heading). An id in both
  `left` and `entered` is a new entity that reused the id.
* **The clock.** `game.clock` estimates the shard's current tick from when
  frames arrive, anchored on the least-delayed frame. Pass `tickRate` when
  you know it; otherwise it is measured.
* **Prediction.** `game.predict(step)` returns a `Predictor` that records
  each input you `send`. After each frame, `reconcile(serverState, ack)`
  starts from the shard's state for your entity and applies the inputs it
  has not processed. Refused inputs drop out, and a reconnect clears it.
* **Round trip.** `game.rttMs` is the smoothed time from an input to the
  frame that acknowledges it.
* **View tick.** `game.send` stamps each input with the tick the last
  `frame` drew, for a shard with [lag compensation](#lag-compensation).
* **Reconnects** get a full frame. The entities it lacks leave.

`game.latest` is the newest table, not interpolated. `connectShard` is the
connection alone, with `onReplication` and `entities`. `useShard` in
`@pylonsync/react` re-renders on every frame; use it for small,
turn-based, or UI-only shards. In Swift, `ShardClient<NoSnapshot, Input>`
has `entities` and a `replication()` stream.

### WebTransport

Over a WebSocket (TCP), one lost packet holds back every frame behind it
until it is sent again. Over WebTransport (QUIC over UDP), entity updates
travel as datagrams: a lost one delays only itself, and the next update
for the entity carries its current position. Spawns, despawns, full
frames, snapshots, and inputs still travel in order on a stream.

Turn it on for the server with a UDP port:

| Variable | Meaning |
| - | - |
| `PYLON_WEBTRANSPORT_PORT` | UDP port to serve WebTransport on. Unset: off. 443 is the best choice: networks that block other UDP ports usually allow it. The Docker image lets its non-root user bind it; in a container run with no-new-privileges, use a port of 1024 or more. |
| `PYLON_WEBTRANSPORT_BIND` | Address to bind. Default: every address. On Fly, `fly-global-services`. |
| `PYLON_WEBTRANSPORT_URL` | The URL clients connect to, `https://<host>:<port>/shard`. Default `https://localhost:<port>/shard`. |

The server makes its own self-signed certificates, each valid for 13 days,
and changes to the next one every 6 days. Clients pin them by hash, so
no domain or CA is needed. `GET /_pylon/shard/webtransport` returns the
URL and the hashes, or 404 when WebTransport is off. On an app with
several machines, each machine lists the hashes of every machine, and a
session that reaches a machine without the shard is relayed to the
machine that runs it. WebTransport sessions count against
`PYLON_SHARD_WS_MAX_PER_IP` together with WebSockets.

In the client, `transport` picks the connection:

```ts theme={null}
const game = connectShardGame(shardId, { subscriberId, ticket, tickRate: 20 });
game.connection.transport; // "webtransport" or "websocket"
```

* `"auto"` (the default for `connectShardGame`): WebTransport when the
  browser has it and the app serves it, else a WebSocket. When a session
  does not open (UDP blocked, `webTransportTimeoutMs`, default 3000), the
  client uses WebSockets from then on.
* `"websocket"` (the default for `connectShard` and `useShard`).
* `"webtransport"`: WebTransport or an error.

In a browser without WebTransport, `"auto"` uses a WebSocket. The C#
client has the same three choices through a native plugin
(`ShardTransport`, see [C# and Unity](https://docs.pylonsync.com/clients/csharp.md#webtransport)). The
Swift client uses WebSockets.
`webTransportInfoUrl` sets where the client fetches the URL and hashes,
when the API is not at `baseUrl`.

The wire format: the client opens one bidirectional stream and sends a
hello, a 4-byte big-endian length and JSON
`{ shard, sid, ticket?, token? }`. Then each message on the stream is a
4-byte length and a message: from the server, a version 2 frame; from
the client, a type byte (`0` input in the shard's codec, `1` datagram
acks, `2` input as JSON) and the body. The server sends updates as bare
datagrams (`EntityTable.applyDatagram` reads them; apply stream frames
with `apply(payload, tick)` so the table knows their ticks), and the
client acks each one with a datagram: type byte `1`, a varint count, then
per ack the datagram's frame number and `EntityTable.streamTick`.

Each datagram names how many datagrams its tick has and the tick of the
last stream frame sent by then. Every tick without a full frame sends at
least one. The client applies a tick's stream frames and datagrams, and
takes its input ack, only when it has all of them, or when a later tick
is whole. So the ack always describes the table, which prediction needs.
When 5 seconds pass with no tick whole, the datagrams have stopped arriving:
the client closes the session, and `"auto"` uses a WebSocket from then
on. Acks go in datagrams no larger than the session allows. Before the server closes a session it sends a closing frame (kind
`6`, JSON `{ code, reason }`), because a browser's `closed` does not get
the code from a server close; the client closes the session when it
reads it.

### Lag compensation

A client draws other players in the past: its interpolation delay (100 ms
by default) plus the time frames take to reach it. Without lag
compensation, the shard checks an aimed hit against where the target is
when the input arrives, and players must lead every shot by their own ping.

With lag compensation, the shard checks the hit against what the shooter
drew:

1. `connectShardGame` (TypeScript) and `ShardGame` (C#) stamp each input
   with `view_tick`: the tick the last `frame` drew.
2. The shard keeps what it sent each subscriber for `history_ticks` ticks.
   This is what the client drew, including updates the byte budget
   deferred and only the entities interest management sent.
3. The shard clamps the view tick and calls `apply_input_at` with a
   `Rewind`, which places each entity where that subscriber drew it then.
   The clients interpolate the same way, so positions match exactly.

```rust shards/range/src/lib.rs theme={null}
use pylon_shard_guest::{LagCompensation, Rewind, Shard};

impl Shard for Range {
    // ...

    // Keep 10 ticks: 500 ms at 20 Hz.
    fn lag_compensation(&self) -> Option<LagCompensation> {
        Some(LagCompensation::new(10))
    }

    // An input with a view tick comes here.
    fn apply_input_at(&mut self, sid: &str, input: Input, rewind: &Rewind) -> Result<(), String> {
        match input {
            // A hitscan shot: the first target on the line, as the shooter saw it.
            Input::Shoot { aim } => {
                let shooter = self.position_of(sid)?;
                let hit = rewind
                    .entities_near(shooter, RANGE)
                    .into_iter()
                    .filter(|(id, p)| *id != self.entity_of(sid) && on_line(shooter, aim, *p, HIT_RADIUS))
                    .min_by(|a, b| dist(shooter, a.1).total_cmp(&dist(shooter, b.1)));
                if let Some((target, _)) = hit {
                    self.damage(target, 25);
                }
                Ok(())
            }
            // A projectile: spawn it where it is now on the shooter's
            // screen, then advance it to the present. The tick loop moves
            // it from there.
            Input::Throw { dir } => {
                let start = self.position_of(sid)?;
                let flown = rewind.ticks_behind() * TICK_SECS * SPEAR_SPEED;
                self.spawn_spear(add(start, scale(dir, flown)), dir);
                Ok(())
            }
        }
    }

    // Inputs without a view tick (and every input with history off).
    fn apply_input(&mut self, sid: &str, input: Input) -> Result<(), String> {
        // ... check against the current positions.
        Ok(())
    }
}
```

* `rewind.position_at(id)` gives where the subscriber drew one entity, or
  `None` when it did not draw it. `rewind.entities_near(center, radius)`
  gives every entity it drew within the radius; narrow a cone or a swing
  arc from those.
* `rewind.tick()` is the clamped view tick, and `rewind.ticks_behind()` is
  how far the shooter's picture was behind the shard.
* The shard clamps the view tick so a client cannot shoot further into the
  past than its connection explains. The tick is never after the newest
  tick sent. It is never before the newest tick minus `history_ticks`. Once
  the connection's round trip is known (from WebSocket pings, or QUIC on
  WebTransport), it is never before the round trip plus `max_view_delay_ms`
  (default 250 ms).
* An input without a view tick goes to `apply_input`. So does every input
  when `lag_compensation` returns `None`.
* In a test outside the server, build a `Rewind` with
  `Rewind::with_positions(view_tick, now_tick, positions)`.
* A native Rust shard implements `SimState::lag_compensation` and
  `SimState::apply_input_at`, which gets `Option<&Rewind>`.
* The rewind interpolates as the default client options do: `holdWhenQuiet`
  on and no `snapDistance`.

Lag compensation costs memory and time per subscriber. In
`crates/realtime/benches/replication_bench.rs`, building the frames for 300
players took about 0.8 ms more per tick with a view radius of 20, and about
3.4 ms more when every player sees all 300.

To measure it, `examples/shard-arena` has a shooting range
(`functions/joinRange.ts`). Bots aim at the target where they draw it, over
WebTransport with 50 ms of delay and 2% packet loss each way:

```bash theme={null}
pylon bench shard --join joinRange --join-args '{"history":10}' --bots 20 --rate 5 \
  --transport webtransport --net-delay 50 --net-loss 2 \
  --aim 1 --input '{"shoot":["$aim:x","$aim:y"]}'
```

With `"history":10`, 99.8% of the shots hit. With `"history":0`, none hit.

## Declare the shard kind

Add the kind to `buildManifest` in `app.ts`:

```ts app.ts theme={null}
import { buildManifest, shard } from "@pylonsync/sdk";

export default buildManifest({
  // ...
  shards: [
    shard({
      name: "arena",
      wasm: "shards/arena.wasm",
      crate: "shards/arena",
      codec: "msgpack",
      tickRate: 20,
    }),
  ],
});
```

| Field | Meaning | Default |
| - | - | - |
| `name` | The kind name `ctx.shards.create` takes. | Required |
| `wasm` | The module, relative to the app root. | Required |
| `crate` | A Cargo crate that `pylon shards build` compiles to `wasm`. | None |
| `codec` | `"json"` or `"msgpack"`, for inputs and snapshots. | `"json"` |
| `tickRate` | Ticks per second. `0` ticks only when inputs arrive. | 20 |
| `fixedTimestep` | Pass `tick` a constant `dt`. | `true` |
| `maxSubscribers` | Subscribers per shard. | 256 |
| `maxInstances` | Shards of this kind that may run at once. | 64 |
| `memoryMb` | Memory cap per shard. | 64 |
| `tickBudgetMs` | CPU time limit for the module in one tick (inputs, `tick`, snapshots), or in one authorize call. Only the module's own code counts: not the host's work, and not time the tick waits for a CPU. | 100 |
| `idleShutdownSecs` | Stop a shard with no subscribers after this long. `0` never stops it. | 90 |
| `input` | `ratePerSec`, `burst`, `maxQueued`, `maxPerTick` per subscriber. | 120, 240, 256, 32 |
| `onStop` | A function to run when a shard of this kind ends. See below. | none |

### Run a function when a shard ends

`onStop` names a function that runs when a shard of the kind ends, with `{ shardId, kind, reason }`:

* `"stopped"`: `ctx.shards.stop` or the admin API stopped it.
* `"idle"`: it had no subscribers for `idleShutdownSecs`.
* `"finished"`: the module ended it.
* `"failed"`: the module trapped, panicked, or went over its budget.

```ts theme={null}
shard({ name: "arena", wasm: "shards/arena.wasm", onStop: "raceEnded" });

// functions/raceEnded.ts
export default mutation({
  internal: true,
  handler: async (ctx, { shardId, reason }) => {
    const race = await ctx.db.lookup("Race", "shardId", shardId);
    if (race) await ctx.db.update("Race", race.id, { endedAt: new Date().toISOString(), endReason: reason });
  },
});
```

It runs as a background job with admin auth, so it can be `internal: true`, and a failed run is retried up to 3 times. It runs on the machine where the shard ended. A shard that stops because its machine shuts down, loses its lease, or hands it to another machine has not ended, so `onStop` does not run.

## Build the module

```bash theme={null}
pylon shards build
```

The command runs `cargo build --release --target wasm32-unknown-unknown` for
each kind with a `crate`, and copies the module to its `wasm` path.
`pylon dev` runs the same build at start. It builds again and restarts the
server when a `.rs` file or `Cargo.toml` in the crate changes.

Commit the `.wasm` file. `pylon build` copies it into the artifact, and
`pylon deploy` uploads it even when `.gitignore` lists it. When `cargo` is on
the path, `pylon deploy` builds the crates first. A GitHub deploy uses the
committed file, because the Cloud builder has no Rust toolchain.

## Start a shard and join it

An action starts a shard and gives the player a ticket:

```ts functions/joinArena.ts theme={null}
import { action } from "@pylonsync/functions";

export default action({
  args: {},
  async handler(ctx) {
    const id = "arena-main";
    if (!(await ctx.shards.get(id))) {
      await ctx.shards.create("arena", id, { width: 800, height: 500 });
    }
    const ticket = await ctx.shards.ticket(id);
    return { shardId: id, subscriberId: ctx.auth.userId, ticket };
  },
});
```

`ctx.shards` has these calls:

| Call | Result |
| - | - |
| `create(kind, id, params)` | Starts shard `id`. Throws `SHARD_EXISTS`, `SHARD_LIMIT_REACHED`, `SHARD_KIND_NOT_FOUND`, `SHARD_ID_INVALID`, or `SHARD_INIT_FAILED`. Actions only. |
| `stop(id)` | Stops the shard and closes its connections. Actions only. |
| `get(id)` | `{ id, kind, tick, subscribers, running, error? }`, or `null`. |
| `list()` | Every running shard. |
| `ticket(id, opts)` | A signed ticket. See [shard tickets](https://docs.pylonsync.com/concepts/functions.md#shard-tickets). |
| `transfer(from, subscriberId, to)` | Moves a player to another shard with its state. See [Move a player to another shard](#move-a-player-to-another-shard). Actions only. |
| `publish(to, topic, data)` | Sends a message to shards. See [Messages between shards](#messages-between-shards). Actions only. |
| `send(id, input)` | Sends an input to a shard. In a mutation, after the commit. See [Send an input from a function](#send-an-input-from-a-function). Mutations and actions. |

A shard id is 1 to 128 letters, digits, `-`, `_`, `.`, or `:`.

The client connects with the ticket:

```tsx theme={null}
import { useShard } from "@pylonsync/react";

const { snapshot, send, lastRejection } = useShard<Snapshot, Input>(shardId, {
  subscriberId,
  ticket,
});

send({ move_to: { x: 120, y: 80 } });
```

The client connects to `/shard` on the app's own origin, so it needs no
extra port or proxy rule. See [React client](https://docs.pylonsync.com/clients/react.md#multiplayer-shards)
for acks, rejections, and custom decoders.

A ticket expires, and a new signing key (a deploy that changes
`PYLON_SHARD_TICKET_SECRET` or `PYLON_JWT_SECRET`) makes it invalid. A
client with a fixed ticket that the shard refuses stops reconnecting and
reports the error, because the same ticket fails every time. Pass a ticket
function to get a new ticket on each attempt:

```tsx theme={null}
ticket: async () => (await callFn<{ ticket: string }>("joinArena", {})).ticket,
```

## Move a player to another shard

A player crosses a zone line, enters a dungeon, or joins a match that runs
on another shard, possibly on another machine. Its entity moves with its
state (hit points, buffs, cooldowns, position), and the client reconnects
to the new shard on its own.

The module takes the entity out and puts it in:

```rust theme={null}
fn transfer_out(&mut self, subscriber: &str) -> Result<Option<Vec<u8>>, String> {
    match self.players.remove(subscriber) {
        Some(p) => serde_json::to_vec(&p).map(Some).map_err(|e| e.to_string()),
        None => Ok(None), // no entity here
    }
}

fn transfer_in(
    &mut self,
    subscriber: &str,
    state: &[u8],
    auth: &Auth,      // the player's user and its ticket for this shard
    returning: bool,  // this shard's own player, back from a failed move
) -> Result<(), String> {
    // Always take a returning player back. Refusing keeps the player in
    // the source.
    if !returning && self.full() {
        return Err("the zone is full".into());
    }
    if !returning && auth.claim("party").is_none() {
        return Err("join a party first".into());
    }
    let p: Player = serde_json::from_slice(state).map_err(|e| e.to_string())?;
    self.players.insert(subscriber.to_string(), p);
    Ok(())
}
```

A server function moves a player:

```ts functions/moveZone.ts theme={null}
export default action({
  args: { from: v.string(), to: v.string() },
  async handler(ctx, args) {
    return ctx.shards.transfer(args.from, ctx.auth.userId!, args.to, {
      claims: { party: "north" },
    });
  },
});
```

The target's ticket names the same user (the source ticket's, else the
session's) and carries only `opts.claims`: claims issued for the source are
not copied.

Or the module asks for the move itself, after a tick, from
`transfer_requests`:

```rust theme={null}
fn transfer_requests(&mut self) -> Vec<(String, String)> {
    // (subscriber, target shard) for each player past the zone line
    std::mem::take(&mut self.leaving)
}
```

What happens:

1. The source's `transfer_out` removes the entity. The source refuses the
   player's inputs until the move ends.
2. The target's `transfer_in` adds it. When the target refuses (or is not
   running), the source's `transfer_in` gets the state back, and
   `ctx.shards.transfer` throws `SHARD_TRANSFER_REFUSED` (or
   `SHARD_NOT_FOUND`). The player stays in the source, still connected.
3. Each of the player's connections to the source gets a transfer frame:
   the target shard and a ticket for it (valid 10 minutes). A connection
   that comes back to the source within that time gets the frame again,
   unless it brings a ticket issued after the move (a new join).
   `connectShard`, `connectShardGame`, `useShard`, and the Swift
   `ShardClient` reconnect to the target at once, and `onTransfer` (or
   `transfers()` in Swift) reports it. The target sends a baseline.

A ticket function (`ticket: (shardId) => ...`) gets the current shard, so a
later reconnect asks for a ticket to the right one. The clients use the
transfer frame's ticket until the new shard answers; a fixed `ticket`,
which names the first shard, is not sent after a move.

A module asks for a move once per player: a move that failed is not tried
again for 5 seconds, and one already in progress is skipped.

On several machines a row in the directory holds the player while it is in
flight. The source saves its state without the player and writes the row
in one transaction; the target saves its state with the player and marks
the row taken in one transaction; a refusal gives the row back to the
source the same way. One of the two takes the row, so the player is never
in two shards and never lost. A step whose commit the database did not
confirm is settled from the row in the next round. When a machine dies
halfway, the machine that starts the source from saved state finishes the
move at once, and the player cannot act in the source until then. When the
source ended (it finished, or an operator stopped it), the target's machine
takes the player; a target that refuses it for 30 minutes gives up, the row
marked `dropped` with the player's state for a week.

## Messages between shards

Shards send each other messages: a realm-wide event ("the relic of Midgard
is taken") to every frontier zone, a keep-under-attack alert to the zones
of one realm, a chat channel across zones, a GM command.

A message goes to one shard (`shard:<id>`), to a group (`group:<name>`), or
to every shard (`all`), on any machine. A module lists its groups, sends
from its outbox after each tick, and receives in `on_message` at the start
of its next tick:

```rust theme={null}
fn groups(&self) -> Vec<String> {
    vec![format!("realm:{}", self.realm)]
}

fn outbox(&mut self) -> Vec<Outgoing> {
    // Filled during the tick, taken after it.
    std::mem::take(&mut self.to_send)
}

fn on_message(&mut self, from: &str, topic: &str, data: &[u8]) {
    if topic == "relic_taken" {
        self.banner = Some(String::from_utf8_lossy(data).into_owned());
    }
}
```

```rust theme={null}
self.to_send.push(Outgoing {
    to: Target::Group("realm:midgard".into()),
    topic: "relic_taken".into(),
    data: b"the relic of Midgard is taken".to_vec(),
});
```

A server function sends with `ctx.shards.publish`; the data arrives as
JSON bytes, with an empty sender. A module cannot tell who called the
function, so check the caller there (an admin or GM role) before
publishing a command:

```ts theme={null}
if (!ctx.auth.isAdmin) throw ctx.error("FORBIDDEN", "GMs only");
await ctx.shards.publish("all", "gm", { command: "restart", inSecs: 300 });
```

Between machines a message travels on the cluster bus when the app has
one (`PYLON_CLUSTER_BUS`), else as a signed call to each live machine.

Delivery is at most once. A message is dropped, with no notice, when its
shard is stopped, when the shard already holds 1024 messages for
its next ticks, or when a machine cannot be reached or is behind. A
sender never gets its own message. A shard applies at most 64 messages a
tick; more wait for the next ones. Limits: 64 messages sent per tick,
topics of 1 to 128 bytes, data up to 64 KB, 64 groups per shard; a
message over a limit is dropped with a log line.

## The app's data

A game keeps durable data in entities: characters, inventory, currency, XP.
A shard reads and changes it through the app's functions and through
buffered writes. Each has a different promise.

| Path | For | Promise |
| - | - | - |
| `calls` with a query | Loading a character when a player joins | The result arrives on a later tick. |
| `calls` with a mutation | Items, currency, XP | Runs once per key, even when the shard crashes and sends it again. |
| `writes` | Position, health, and other frequent changes | The latest value, written every 2 s and when the shard stops. A crash loses the last 2 s. |
| `ctx.shards.send` | A GM command, a keep claimed on the website | In a mutation, sent after the commit. At most once. |

### Call a function

After each tick the host takes the module's `calls`. Worker threads run
them, one shard's calls in turn with the others', so a slow function never
delays a tick. The result reaches `on_call_result` at the start of a later
tick, before its messages and inputs:

```rust theme={null}
fn calls(&mut self) -> Vec<Call> {
    // Every call still waiting, every tick. The host skips a key that is
    // in flight, so a call is sent again only after a crash.
    self.grants
        .iter()
        .map(|(key, g)| Call {
            key: key.clone(),
            function: "grantItem".into(),
            args: serde_json::json!({ "characterId": g.character, "item": g.item }),
        })
        .collect()
}

fn on_call_result(&mut self, key: &str, result: Result<serde_json::Value, String>) {
    let Some(grant) = self.grants.remove(key) else { return };
    match result {
        // The mutation committed: show the item now.
        Ok(_) => self.give(&grant),
        // The function refused, and rolled back: no item exists. The text
        // is "CODE: message".
        Err(e) => log(Level::Warn, &format!("grant {key} failed: {e}")),
    }
}
```

A mutation called with a key runs once per function and key. The key, a
hash of the arguments, and the mutation's result are written in the
mutation's own transaction. A later call with the same key and arguments
returns the stored result and changes nothing. A call with the same key and
other arguments fails with `KEY_REUSED`.

A call that fails in the database or the functions runtime (codes that
start with `PG_`, `SQLITE_`, `TX_`, or `RUNNER_`, and `FN_TIMEOUT`,
`IO_ERROR`, `HANDLER_CRASH`, `CALL_CANCELLED`, `LOCK_FAILED`,
`BEGIN_FAILED`, `COMMIT_FAILED`, `ROLLBACK_FAILED`) is made again by the
host with the same key, after 250 ms and then twice as long each time, up
to 30 s. A statement Postgres refuses for what it is (a constraint, a data
error: `PG_REJECTED`) is not made again. The module gets only the
function's answer or its own error. A result goes only to the run of the
shard that sent the call: a shard that restarts sends its calls again. So a grant has these rules:

* Save the calls in flight with the shard's state (`save`). A shard that
  restarts from its last save sends them again under the same keys.
* Apply the effect (the item in the player's inventory) only in
  `on_call_result`. Before that, the mutation may not have committed.
* Load the characters again after a restore. A grant can commit after the
  last save, and a zone restored from that save does not know it yet. Keep
  items by row id (the mutation returns it), so an item that both the
  reload and a late result bring shows once.
* Make the key unique for good. A stored result is kept for 7 days. The
  example keeps the next grant number in the character row and uses
  `loot:<character>:<n>`. On `KEY_REUSED` (a restored zone counted from an
  older row), it sends the grant again under the next number.

A query runs each time it is called. An action is refused
(`FN_NOT_TRANSACTIONAL`): it has no transaction to store the result in.
Calls run with admin rights, like a scheduled job, so a function that
shards call must check the caller (`ctx.auth.isAdmin`) if players can also
reach it. A shard has at most 256 calls in flight. Each call holds a
database connection while it runs, so the workers are a quarter of the
Postgres pool, 1 to 4 (1 on SQLite); `PYLON_SHARD_CALL_WORKERS` sets the
number. A result can arrive twice for one key; its value is the same.

### Write entity fields

After each tick the host takes the module's `writes`. It keeps the latest
value of each field of each row, whichever shard wrote it, and writes them
every 2 s (`PYLON_SHARD_WRITE_EVERY_MS`, at least 100) and when the shard
stops. Each field keeps the shard that wrote it and the lease that shard
ran under. A flush checks, in the write's own transaction and with the
shard's place in the directory locked (on SQLite, with the database's write
lock held), that this machine still holds the shard under that lease: a
copy that lost its place (its machine's lease lapsed, or another machine
runs it now) writes nothing more, even a write already on its way. A write goes
through the same path as `PATCH /api/entities/<entity>/<id>` with admin
rights: plugins run, and synced clients see the change.

```rust theme={null}
fn writes(&mut self) -> Vec<Write> {
    std::mem::take(&mut self.moved)
        .into_iter()
        .map(|id| Write {
            entity: "Character".into(),
            id: self.players[&id].character.clone(),
            set: serde_json::Map::from_iter([("x".into(), self.players[&id].x.into())]),
        })
        .collect()
}
```

A write to a row that does not exist, or that a plugin refuses, is
dropped with a log line. A write that fails in the database is tried again
on the next 4 flushes. A machine holds at most 100,000 rows to write.

### Send an input from a function

`ctx.shards.send(id, input)` queues `input` for the shard's next tick, on
any machine. The module's `apply_input` gets it from the empty subscriber
id. No connection can use that id, so a module can trust it:

```rust theme={null}
Input::Heal { who, hp } => {
    if !subscriber.is_empty() {
        return Err("from the server only".into());
    }
    self.players.get_mut(&who).ok_or("no such player")?.hp += hp;
}
```

```ts functions/gmHeal.ts theme={null}
export default mutation({
  args: { zone: v.string(), userId: v.string(), hp: v.number() },
  async handler(ctx, args) {
    if (!ctx.auth.isAdmin) throw ctx.error("FORBIDDEN", "GMs only");
    await ctx.db.insert("GmAction", { zone: args.zone, userId: args.userId, action: "heal" });
    await ctx.shards.send(args.zone, { heal: { who: args.userId, hp: args.hp } });
  },
});
```

In a mutation the input is sent after the commit, and not at all when the
mutation rolls back. In an action it is sent at once, and a shard on this
machine has it when the call returns, and one that runs nowhere fails with
`SHARD_NOT_FOUND`. Delivery is at most once: a shard that stops, a full
input queue, or a machine that cannot be reached loses it. An input is at most 64 KB as JSON. A query cannot send.

`examples/shard-arena` has all four: its zone loads a `Character` on join,
grants `Item`s through `functions/grantItem.ts`, writes x back, and takes
heals from `functions/gmHeal.ts`.

## Load test

`pylon bench shard` starts headless bots that join like players, send
inputs, and decode every frame:

```bash theme={null}
pylon bench shard --join joinArena --bots 300 --duration 30 --rate 10 \
  --input '"join"' \
  --input '{"move_to":{"x":"$rand:0:800","y":"$rand:0:500"}}'
```

* `--join <function>` signs each bot in as a guest and calls the function,
  which returns `{ shardId, subscriberId, ticket }`.
* `--shard <id>` joins a running shard with tickets the command mints. Run it
  from the app directory, so it uses the same key as `pylon dev`, or set
  `PYLON_SHARD_TICKET_SECRET` to the server's value.
* `--input` is a JSON template, and repeated templates are sent in turn. The
  strings `"$rand:A:B"`, `"$int:A:B"`, `"$bot"`, and `"$sid"` are replaced
  for each input.
* `--url` defaults to `http://localhost:4321`. An `https://` origin connects
  over `wss://`.
* `--transport webtransport` connects over WebTransport, from the URL and
  hashes at `/_pylon/shard/webtransport`. `--drop-datagrams <percent>`
  drops that share of the datagrams each bot receives, to test loss.
* `--net-delay <ms>` and `--net-loss <percent>` (with WebTransport) send
  every packet through a local UDP proxy that delays and drops packets each
  way.
* `--aim <entity>` makes each bot draw entities like a client
  (`--interp-delay`, default 100 ms, behind the shard's tick at
  `--tick-rate`, default 20). In an input template, the strings
  `"$aim:x"`, `"$aim:y"`, and `"$aim:z"` become where the bot draws the
  entity, and the input carries its view tick. The report counts the
  aimed shots the shard refused.

The report gives:

| Line | Meaning |
| - | - |
| tick rate seen | New ticks per second each bot received |
| snapshot interval | Time between frames at a bot (p50, p99, max) |
| input to ack | Time from sending an input to the first frame that acks it |
| bytes per bot | Bytes each bot received per second |
| rejections | Refused inputs, by code |

`pylon dev` lifts the per-IP connection cap. Against `pylon start`, set
`PYLON_SHARD_WS_MAX_PER_IP=0` on the server when all bots run on one
machine.

To run bots from several machines, give each machine its own
`--bot-offset` and `--report <file>`, then combine the reports:

```bash theme={null}
pylon bench shard ... --bots 300 --bot-offset 0   --report a.json   # machine 1
pylon bench shard ... --bots 300 --bot-offset 300 --report b.json   # machine 2
pylon bench merge a.json b.json
```

## Metrics and operations

`/metrics` reports every shard (Prometheus text with
`Accept: text/plain`, JSON otherwise), with `shard` and `kind` labels.
Percentiles cover the last 1200 ticks (a minute at 20 Hz); bytes per
subscriber sample four subscribers a tick, in turn, over those ticks.
Counters run from the shard's start.

A shard per match makes new series for every match. Set
`PYLON_SHARD_METRICS_LABELS=kind` to report one series per kind instead:
counters and gauges are summed, and each percentile is the highest any shard
of the kind has. The JSON (and Studio) stays per shard.

| Metric | Meaning |
| - | - |
| `pylon_shard_tick_duration_seconds` | Tick duration, p50 and p99 (a summary). |
| `pylon_shard_tick_overruns_total` | Ticks skipped because the loop fell behind. |
| `pylon_shard_phase_seconds_total{phase}` | Time in `inputs` (`apply_input`), `tick`, `interest` (interest management and building snapshots or replication frames), and `encode` (encoding and queueing frames). |
| `pylon_shard_phase_duration_seconds{phase,quantile}` | The same phases, p50 and p99. |
| `pylon_shard_subscribers`, `pylon_shard_input_queue` | Live connections, and inputs waiting for the next tick. |
| `pylon_shard_frame_bytes_total` | Bytes of frames queued for subscribers, including frames a full queue drops later. |
| `pylon_shard_tick_bytes`, `pylon_shard_subscriber_tick_bytes` | Bytes per tick for all subscribers, and per subscriber per tick, p50 and p99. |
| `pylon_shard_dropped_frames_total` | Frames dropped from full subscriber queues. |
| `pylon_shard_dropped_inputs_total{reason}` | Inputs refused before queueing: `queue_full` or `rate_limited`. |

Studio's **Shards** page (admins) lists the running shards with these
numbers. Open one to see its phase timings and subscribers, kick a
subscriber, or stop the shard. The same actions are HTTP routes, admin-only:

* `GET /api/shards` and `GET /api/shards/<id>` (the detail adds
  `subscriber_ids`),
* `POST /api/shards/<id>/kick` with `{ "subscriber": "<id>" }` closes that
  subscriber's connections,
* `POST /api/shards/<id>/stop` stops the shard and closes every connection.

## Shards across machines

Every app with a database file or Postgres has a shard directory in its
database: where each shard runs, its saved state, and players moving
between shards. On SQLite the app runs on one machine; the directory makes
shard state survive a restart and a deploy (see
[Limits and failures](#limits-and-failures)).

An app on Postgres can run on several machines. Each machine joins the
directory, and shards spread across them:

* **Placement.** `ctx.shards.create` runs the shard on the live machine with
  the most free capacity (`PYLON_SHARD_CAPACITY` shards a machine, default
  100\), or on the machine you name:
  `ctx.shards.create("arena", id, params, { machine })`. `maxInstances`
  counts every machine. `ctx.shards.get` and `list` report each shard's
  `machine`.
* **Routing.** A client can connect to any machine. The machine that gets a
  connection for a shard it does not run sends it on. Between Fly machines
  (Stack0 Cloud), it answers with `fly-replay: instance=<machine>`, and
  Fly's proxy connects the client to that machine directly, with no extra
  hop. Elsewhere, it proxies the connection over the machines' private
  addresses. The WebSocket (on `/shard` and on the shard port), the SSE
  stream, and HTTP inputs all route this way.
* **Failover.** A machine with no heartbeat for 10 seconds is dead. Each of
  its shards moves to one live machine (the same choice on every machine)
  and starts again: from its saved state when the module saves state
  (below), or from `init` with its create params when it does not. Clients
  reconnect on their own (the client libraries retry, and a ticket function
  gets a new ticket).
* **Deploys.** A machine that gets SIGTERM hands each shard to another
  live machine before it exits, and its players stay connected (see
  [Deploys](#deploys)). A shard it cannot hand over is saved and starts on
  another machine once this one leaves the directory; its clients
  reconnect after the gap.
* **Restarts.** A machine that comes back under the same id (a crash and
  restart, or a deploy on Fly, where the machine id stays) waits until the
  old process has left (a SIGTERM) or has been silent for 10 seconds, so
  two processes never hold one id. Then it gets a new lease, and the old
  process's shards move from their saved state, possibly back to it.
* **Fencing.** A machine may run shards for 5 seconds after it sent the last
  heartbeat that reached the database. When no heartbeat lands in that time
  (the database is down, slow, or unreachable, even if calls hang instead of
  failing), a separate thread stops its shards and closes their
  connections, and the module refuses every call and interrupts a running
  one. The others start them only when the heartbeat is 10 seconds old. Two copies of a shard never run at once, and a machine cut off from
  the database cannot overwrite the state of the copy that replaced it.
* A shard that cannot start (its module refuses the saved state) keeps its
  state, and `ctx.shards.get` reports why. Stopping it forgets it.

A module saves state by implementing `save` and `restore` on its `Shard`:

```rust theme={null}
fn save(&self) -> Option<Vec<u8>> {
    serde_json::to_vec(&self.players).ok()
}

fn restore(shard_id: &str, params: Params, state: &[u8]) -> Result<Self, String> {
    let mut arena = Self::init(shard_id, params)?;
    arena.players = serde_json::from_slice(state).map_err(|e| e.to_string())?;
    Ok(arena)
}
```

The host saves every 5 seconds (`PYLON_SHARD_SAVE_SECS`), between ticks, so
a failover loses at most that much. A `Replicated` store saves with
`full_changes` and loads with `apply_changes`.

| Setting | Meaning |
| - | - |
| `PYLON_REPLICA_ID` | The machine's id. On Fly, `FLY_MACHINE_ID` is used. Without either, the host name and process id: a restart is a new machine. |
| `PYLON_SHARD_ADVERTISE_URL` | The URL other machines reach this one at, for proxying and machine-to-machine calls. On Fly, `http://[FLY_PRIVATE_IP]:<port>`. |
| `PYLON_SHARD_DIRECTORY=off` | Keep each shard on the machine that created it, with no directory. |
| `PYLON_SHARD_DRAIN_SECS` | How long a machine that gets SIGTERM hands shards to other machines (default 20; 0 turns it off). |

The machines sign their calls to each other, and shard tickets, with a key
kept in the directory: there is no secret to share. A
`PYLON_SHARD_TICKET_SECRET` you set still signs tickets.

An app on SQLite runs on one machine. One process at a time runs a SQLite
file's shards: it holds the lock file `<database>.shards.lock` while it
runs, and a second `pylon start` on the same file waits 10 seconds for it,
then does not start. When the process exits or crashes, the next one takes
the lock and starts the shards from their saved state at once. An app on an
in-memory database keeps no state across a restart, shards included, and
has no directory.

### Deploys

In a rolling deploy each machine gets SIGTERM in turn. For each shard it
holds, a machine that gets SIGTERM:

1. Stops ticking the shard. Connections stay open, and the shard's
   buffered entity writes are written.
2. Saves the shard's state.
3. Asks another live machine to take the shard. That machine moves the
   shard's place in the directory to itself and starts it from the saved
   state.
4. Sends each connection a transfer frame for the same shard, with a new
   ticket, and closes it. The client libraries (JavaScript, React, Swift,
   and `pylon bench shard`) reconnect at once. A same-shard transfer does
   not call `onTransfer`.

A hand-over takes milliseconds. Inputs a client sends between steps 1 and 4
are lost; the shard's state is not. A machine that is shutting down reports
no room in the directory, so no shard is placed or handed to it. The
module's `save` and `restore` carry the state: a shard with no `save`
starts from `init` on the other machine, and its clients reconnect to it.

A shard the machine cannot hand over within `PYLON_SHARD_DRAIN_SECS` (no
other live machine, or none takes it) is saved and starts on another machine
once this one leaves. On one machine (every SQLite app), the next process
starts it from that save; its clients reconnect after the gap. Give the platform time to wait: Fly kills a machine
5 seconds after SIGTERM unless `kill_timeout` (fly.toml) or the machine's
`stop_config` says otherwise. Stack0 Cloud sets 45 seconds.

## Limits and failures

* A tick whose module code uses more CPU time than `tickBudgetMs` stops the shard.
* A module that grows past `memoryMb` stops the shard.
* A panic stops the shard. The server log shows the panic message.
* `ctx.shards.get(id)` returns the reason in `error` until the host removes
  the stopped shard (within a few seconds).
* A module may import only `pylon.log`. The server refuses a module built for
  WASI or with any other import, and the boot fails.
* A shard's state is in memory. A module that implements `save` and
  `restore` starts again from its last save after a restart, a deploy, or a
  crash (saves run every 5 seconds, `PYLON_SHARD_SAVE_SECS`, and when a
  machine stops), on SQLite and on Postgres. A player moving between shards
  when the machine crashed ends up in exactly one of them. A module with no
  `save`, or an app with no directory (an in-memory database, or
  `PYLON_SHARD_DIRECTORY=off`), loses the shard at a restart.

## Hosted on Stack0 Cloud

[Stack0 Cloud](https://docs.pylonsync.com/cloud.md) runs shards with no extra setup. Clients connect to
`wss://your-app.stack0.app/shard`, or to `/shard` on your custom domain.

* An open shard connection keeps the app's machine running.
* A machine with autostop on can stop when no client is connected. Its
  shards end, like on any restart.
* WebTransport is off by default. Turn it on in the app's **Scaling**
  settings. The app gets a dedicated IPv4 address for UDP (Fly bills it
  monthly) and must run one machine per region.
* A machine gets 45 seconds to stop, so a deploy saves its shards (and, on
  an app with several machines, hands them over) before it exits. On one
  machine, the new one starts them from that save.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.