Troubleshooting
Diagnose launch refusals, unreachable workers, certificate failures and registry pull errors.
Symptoms grouped by where they surface, each with the underlying cause and the fix.
Launches are refused
402 with reason: "concurrency_limit"
The organization is at its ceiling. Concurrency is counted organization-wide across every worker, not per host, so a second cluster does not get its own allowance, and it counts running sessions plus live launch reservations.
{
"success": true,
"data": {
"allowed": false,
"reason": "concurrency_limit",
"current": 200,
"limit": 200
}
}The verdict is data.allowed. success only reports that the check ran.
Ask what you are being charged for
GET /v1/license/sessions answers with the worker's organization key and tells you
which rows are responsible, including a stale flag on each:
curl -fsS https://api.veilbrowser.net/v1/license/sessions \
-H "X-API-Key: $VEIL_KEY"current is runningSessions + reservedSlots. Reservations expire on their own after
two minutes, so if that is where the count is going, wait rather than act.
Reclaim what no worker will ever close
Rows flagged stale belong to a worker that was killed, OOM'd or recreated without
closing its browsers. Release them without waiting out the deadline:
curl -X POST https://api.veilbrowser.net/v1/license/sessions/reclaim \
-H "X-API-Key: $VEIL_KEY"Idempotent, and scoped to the key's organization. Add ?machineId= to limit it to one
worker, or ?olderThanMinutes= to override the deadlines with an explicit cutoff. A
session whose browser is genuinely still running can be stopped from the dashboard or
with DELETE /sessions/:id on the owning worker instead.
Prevent recurrence
Leave VEIL_SESSION_HEARTBEAT_SEC at its default of 60 on every worker. It is what
releases a killed session in seconds instead of hours: the worker reports which browsers
it actually has, and anything else attributed to that machine is released immediately.
With it disabled, an abandoned row waits out the six-hour backstop.
Then bound the sessions themselves. VEIL_IDLE_BROWSER_TIMEOUT already defaults to
900 seconds, so a browser that stops receiving CDP traffic is reaped; set
VEIL_AUTO_CLOSE=true (which is off by default) to close a browser the moment its
last CDP client disconnects.
Launches fail when the cloud is unreachable
This is intentional. Authorization is fail-closed: a worker that cannot reach
POST /v1/license/authorize does not launch. Sessions already running are never killed.
Check SAAS_API_URL on the worker, then confirm outbound HTTPS to
api.veilbrowser.net is permitted. Workers need egress to the control plane even in an
otherwise isolated network.
AUTHENTICATION_ERROR on authorize
The worker's VEIL_KEY is wrong, disabled, or belongs to a different organization.
Verify it against Settings → API keys, remembering that a deleted key cannot be
restored — issue a new one and roll the fleet.
RESOURCE_NOT_FOUND naming a browser version
The profile asks for a build that worker does not have on disk. The error's constraint
lists what it does have:
{ "field": "browserVersion", "code": "RESOURCE_NOT_FOUND",
"constraint": { "availableVersions": [135, 141, 147] } }Open the host page in the console, find Available to install, and install the missing build. If it is missing from the whole fleet, publish and activate it first — see Operating the cluster.
No profile matches the criteria
A launch spec whose os, osVersion, renderer or model matches no bundle fails
before the browser starts. Query the worker for what it can actually produce:
curl -s "http://localhost:38923/profiles?os=win&browserVersion=147" \
-H "X-API-Key: $WORKER_API_KEY"Broaden the criteria, or publish a bundle that satisfies them. See Fingerprints.
Workers look wrong in the fleet view
Workers are polled over HTTP every 30 seconds.
| State | Meaning | Where to look |
|---|---|---|
| Healthy | Reachable, status endpoint returned OK | — |
| API offline | Host reachable, Local API down or misconfigured | Container logs; a bad API_KEY or missing VEIL_INSTALL_DIR exits at startup |
| Host down | Unreachable entirely | Firewall, reboot, or the shared Docker network |
Installs are refused against a host that is down, which is deliberate — a half-applied install is worse than a deferred one.
A worker is healthy but never receives launches
Three things to check, in order:
clusterProxyEnabledis set on the Cluster row, and you have run Render & Reload since setting it. Until then the edge has no route to the worker.VEIL_WORKER_IDon the worker matches its FleetHost hostname exactly. Sticky/w/<workerId>/…routing fails silently when they diverge.VEIL_MAX_BROWSERShas not been reached. At the cap the worker returns503before the WebSocket upgrade so the load balancer retries the next one — which looks like the worker being skipped.
Sessions vanish after an update
Worker updates drain first and wait for running browsers to finish. If sessions are
dying mid-update, the drain is being bypassed — check that the host has an updater agent
registered rather than being recreated by hand with docker compose up -d.
Certificate issuance fails
The wizard reports which check failed. In order of likelihood:
| Cause | Fix |
|---|---|
| DNS does not resolve here yet | Wait for propagation, or correct the A record |
| DNS points at Cloudflare's proxy | The orange cloud intercepts the port 80 challenge, so HTTP-01 cannot complete. Use Cloudflare + Full instead |
| Port 80 or 443 blocked | Open both. Port 80 answers the ACME challenge, 443 serves the issued certificate |
| Let's Encrypt rate limit | Too many recent attempts for that domain. Wait for the window to reset |
Let's Encrypt behind Cloudflare via DNS-01 is not supported. See Edge security & TLS.
The site is unreachable after setting a hostname
Expected, if you are still browsing by IP. Once a hostname is configured the edge drops
any request whose Host header or TLS SNI does not match, closing the connection with
HTTP 444 and no response body. Browse to the hostname instead.
Cloudflare returns 521 or 522
The origin lock is refusing the connection, or the origin is not listening where Cloudflare expects.
- Flexible requires the origin to serve HTTP on port 80.
- Full requires HTTPS on port 443 with a certificate present, even a self-signed one.
A mismatch between your Cloudflare SSL/TLS dashboard setting and the mode chosen here is the usual cause. Also confirm a host firewall has not blocked a Cloudflare range that was added recently.
Image pulls fail
unauthorized or denied from the registry
The org key in the pull path is wrong or revoked. The path segment is your VEIL_KEY
lowercased:
docker pull registry.veilbrowser.net/<org-key-lower>/local-api:latestinstall.sh derives this as VEIL_ORG_KEY_LOWER in .env. If you rotated the key,
update both VEIL_KEY and VEIL_ORG_KEY_LOWER; changing only one leaves the pull
working while the runtime fails, or the reverse.
No docker login is needed — the gate authenticates the private upstream for you. If you
find yourself needing credentials, you are pointed at the upstream directly rather than
through the gate.
manifest unknown
The tag does not exist for your channel. Check RELEASE_CHANNEL and prefer a
:X.Y tag over a channel alias when you need reproducibility.
Browsers crash immediately
Almost always the Chromium sandbox or shared memory. The single-host compose template
and the docker run in Run a worker both
grant what Chromium needs; a hand-rolled command usually does not. Check that command
for the three requirements — a privileged container (or SYS_ADMIN with the supplied
seccomp profile), at least 1 GB of /dev/shm, and a writable /tmp.
On Ubuntu 24.04 and newer, also confirm the host allows unprivileged user namespaces:
sysctl kernel.apparmor_restrict_unprivileged_userns # must be 0Config changes have no effect
The load-balancer configuration is derived state. Editing a cluster or host record updates the database, but the live edge keeps its previous configuration until you Render & Reload. Preview the diff first — if validation fails, the live config is left untouched and the operation records the error. See Operating the cluster.
Next steps
- Configuration reference — every variable named above.
- Operating the cluster — the daily health sweep.
- Errors — the response envelope these codes arrive in.