Operating the cluster
Day-to-day operations: fleet health, publishing releases, and reloading the load balancer.
This is the operator's day-to-day guide: check fleet health, publish releases, and apply configuration changes safely.
Daily health check
A 60-second top-of-shift sweep:
Dashboard
No failed operations in the last 24h; no surprise upstream-version banners.
Workers
Every enabled host is green, on the expected API version, and not stuck on an old browser build.
Operations
The most recent manifest and install operations succeeded; nothing stuck in running.
Fleet operations
Workers are polled over HTTP every 30 seconds and each report a snapshot. The fleet view shows reachability:
| State | Meaning |
|---|---|
| Healthy | Reachable and the worker's status endpoint returned OK. |
| API offline | Reachable, but the Local API is down or misconfigured. |
| Host down | Unreachable (firewall, reboot, network). Installs are refused. |
Installing a version on one worker
When a worker is missing a build the rest of the fleet has, open the host page, find the Available to install card, and click install. The cluster manager pre-checks the manifest, probes the worker, acquires a per-host lock, and streams the install log into an audited operation.
Force re-install
Use the force toggle only when a worker's version marker is stale but the build is actually missing on disk.
Bundle sync
There is no auto-update cron, and nothing to enable, disable or schedule. The containerized worker has no cron, runs as a non-root user and mounts a read-only root filesystem, so a scheduled in-container updater was never something the image could run.
What the host page offers instead is the Browser & profile bundles card: it reports when the worker last synced and gives you Sync now. A sync downloads and verifies the active browser builds and profile bundles into the worker's install volume while the container keeps running, and it is tracked as an audited operation you can tail.
Updating the worker image is a separate axis and lives under Infrastructure → Updates. A container cannot pull and recreate itself, which is why the two are not the same button.
Releases
Releases come in three artifact types, each with its own page:
Worker (Local API)
Publish a new Local API build for the fleet.
Browser builds
Publish a browser build per major version, then activate it.
Profiles
Publish and activate profile bundles.
Publish
Upload or select the artifact and submit. The publish runs as a single audited operation you can tail live.
Activate
Publishing does not automatically promote. Click Activate to make a build the one workers install — the same pattern for browser builds and profile bundles.
Roll out
Nothing rolls out on its own. Trigger Sync now on the workers you want on the new version, starting with a canary, or install the build on one host from its page.
Reverting a release
- Browser build: re-activate the previous build, then sync the workers to pull it down.
- Profiles: activate the previous bundle — the active pointer is what workers consume.
- Worker API: re-activate the previous API version and regenerate the manifest.
Render & Reload
After any cluster or host change, the load-balancer config is stale until you apply it:
Preview
Render a preview and review the unified diff against the live config.
Apply
Applying validates the config, atomically swaps it into place, and reloads the load balancer — all captured in one operation. If validation fails, the live config is left untouched.