Operating the cluster
Day-to-day operations — fleet health, publishing releases, and reloading the load balancer.
This is the operator's day-to-day guide: check fleet health, publish releases, and apply configuration changes safely.
Daily health check
A 60-second top-of-shift sweep:
Dashboard
No failed operations in the last 24h; no surprise upstream-version banners.
Workers
Every enabled host is green, on the expected API version, and not stuck on an old browser build.
Operations
The most recent manifest and install operations succeeded; nothing stuck in running.
Fleet operations
Workers are polled over HTTP every 30 seconds and each report a snapshot. The fleet view shows reachability:
| State | Meaning |
|---|---|
| Healthy | Reachable and the worker's status endpoint returned OK. |
| API offline | Reachable, but the Local API is down or misconfigured. |
| Host down | Unreachable (firewall, reboot, network). Installs are refused. |
Installing a version on one worker
When a worker is missing a build the rest of the fleet has, open the host page, find the Available to install card, and click install. The cluster manager pre-checks the manifest, probes the worker, acquires a per-host lock, and streams the install log into an audited operation.
Force re-install
Use the force toggle only when a worker's version marker is stale but the build is actually missing on disk.
Auto-update
Each worker runs an auto-update job that pulls the latest API, browser builds and profiles. The fleet UI shows whether it is enabled and its schedule, and lets an operator enable, disable, or run it on demand.
Releases
Releases come in three artifact types, each with its own page:
Worker (Local API)
Publish a new Local API build for the fleet.
Browser builds
Publish a browser build per major version, then activate it.
Profiles
Publish and activate profile bundles.
Publish
Upload or select the artifact and submit. The publish runs as a single audited operation you can tail live.
Activate
Publishing does not automatically promote. Click Activate to make a build the one workers install — the same pattern for browser builds and profile bundles.
Roll out
Workers pick up the active version on their next auto-update, or you can push it to a canary worker immediately.
Reverting a release
- Browser build: re-activate the previous build; workers downgrade on next update.
- Profiles: activate the previous bundle — the active pointer is what workers consume.
- Worker API: re-activate the previous API version and regenerate the manifest.
Render & Reload
After any cluster or host change, the load-balancer config is stale until you apply it:
Preview
Render a preview and review the unified diff against the live config.
Apply
Applying validates the config, atomically swaps it into place, and reloads the load balancer — all captured in one operation. If validation fails, the live config is left untouched.