Troubleshooting
Symptom, cause, fix
| Symptom | Cause | Fix |
|---|---|---|
Connection refused on SSH | You used port 22, or the instance is not running yet | Copy the command from the Connect tab; wait for the running badge |
Connection refused on a service you started | The port was never published at launch | Forward it over SSH, or declare it on the template and redeploy |
Permission denied (publickey) | No key was attached at deploy, or your client offered a different key | Redeploy with a key attached; or point at the right key with -i and IdentitiesOnly=yes |
REMOTE HOST IDENTIFICATION HAS CHANGED | A recycled host and port now belongs to a new instance | ssh-keygen -R "[host]:port", then connect again |
Instance sits in creating | A large image is still being pulled | Watch the Logs tab; if it lands in error, destroy and redeploy |
Instance is in error | The workload failed to start, failed or was lost while running, or finished on its own. What went wrong on the instance page says which, as does the API's error_code — see Why an instance failed | Read the logs, destroy it, deploy again — it cannot be started or stopped |
JupyterLab returns 403 Forbidden | The request carried no token, or a stale one | Return to the console and use Open JupyterLab again |
A template's web UI or API answers 401 | The port requires the instance access token, and the request carried neither the token nor the cookie the Open button sets | Open it with the Open button, or send Authorization: Bearer <access token> from scripts. See The instance access token |
| Browser warns the certificate is untrusted | JupyterLab serves HTTPS with a self-signed certificate | Expected; continue past the warning |
No space left on device | The disk you chose at deploy is full | Free space, or redeploy on a larger disk — disks cannot be resized |
| Instance stopped on its own | The organization's balance reached $0. That stop also released its GPUs, where the machine supports it | Add credits until the balance covers an hour of what the organization runs, then start it again |
Instance went back to running after a stop | The host did not confirm the stop within 30 minutes; the container is still up and billing. The instance page shows The last action did not complete, and the API's error_code reads stop_timed_out | Nothing — the stop is retried on its own. Destroy the instance if you do not need it |
Start refused with HOST_UNREACHABLE, START_REFUSED or START_UNAVAILABLE | The host could not take the start; the instance is still stopped | Start it again later. If START_REFUSED keeps coming back, destroy the instance and deploy again |
Start refused with CAPACITY_UNAVAILABLE | The instance's GPUs were released — the balance reached $0, or a payment dispute froze payments — and another renter is using one of them now. The instance page shows GPUs released | Start it again later. If you cannot wait, copy your data off and destroy it |
A stopped instance moved to error with workload_gone | Its container disappeared from the machine while it was stopped | Nothing bills for it any more. Destroy it and deploy again; if you do not, it is destroyed automatically about an hour later |
| Instance disappeared entirely | The balance stayed at or below $0 for 24 hours and reached −$5, and stopped instances were destroyed | The disk is gone; add credits and deploy again |
409 INVALID_STATE from the API | The instance is mid-transition | Wait for the transitional state to land, then retry |
402 INSUFFICIENT_BALANCE on deploy or start | The balance does not cover one hour of the organization's burn with this instance running — its GPU and disk plus what the others cost. Being above zero is not enough. The error carries balance_micro and required_micro | An admin adds credits; members cannot |
Connection refused
Two different problems wear the same message.
If it is SSH itself, you are almost certainly on the wrong port. The instance's sshd is published on a host port assigned at launch, never 22, and the number can change after a stop and start. Copy the command fresh from the Connect tab.
If it is a service you started inside the instance, the port is not published. Ports are fixed when the instance launches, so starting a server afterwards exposes nothing. Forward it through the SSH session you already have:
ssh -N -L 8000:localhost:8000 -p 40123 root@203.0.113.14
Also confirm the process is bound to 0.0.0.0 and not 127.0.0.1 — unless the port is behind the access proxy, where the app belongs on 127.0.0.1. See Ports and the Open button.
Permission denied (publickey)
Password authentication is disabled, so this always means the key your client offered is not the one in authorized_keys.
ssh -i ~/.ssh/id_ed25519 -o IdentitiesOnly=yes -p 40123 root@203.0.113.14
If that still fails, the instance most likely has no key at all — the SSH key selector on the Deploy page was left on No key. There is no way to attach a key to a running instance. Add the key under SSH Keys, then deploy again. See Add a key.
Stuck in creating
creating normally completes in well under a minute. What extends it is the image pull: a multi-gigabyte image on a host that has never seen it takes as long as it takes. The Logs tab shows the pull line, then image ready, creating container.
If the workload never comes up, the instance moves to error on its own. It will not sit in creating forever.
An instance in error
error means the container failed to start. The logs are the only place the reason lives, so read them before you clean up:
- An image or tag that does not exist, or a private image whose registry credentials are wrong.
- A template pinned to an image that does not match the launch mode. The
sshandjupytermodes need an image built from the Superheat base image. - A workload that never became ready inside the platform's start window.
A failing onstart is not one of the causes. The base image runs it in a subshell, writes the failure to /var/log/superheat/onstart.log, and leaves the container up so you can connect and fix it.
From error the only action available is Destroy. Fix the template, then deploy again. See Read the logs.
JupyterLab returns 403
The notebook server accepts one credential, a token minted by the platform at deploy. A 403 means the browser sent the wrong one or none at all, which usually means the tab came from a bookmark or a copied address rather than from the console.
Go back to the instance in the console and press Open JupyterLab. That button carries the token the platform minted for the instance; a hand-built address does not. See Open JupyterLab.
Out of disk
df -h /
du -sh /workspace/* | sort -h | tail
The usual culprits are accumulated checkpoints and package caches:
rm -rf ~/.cache/pip ~/.cache/huggingface
The disk cannot be grown. If the job genuinely needs more room, copy what you have off the instance, destroy it, and deploy again with a larger disk_gb. See Disk and storage.
The instance stopped by itself
Running instances are stopped automatically when the organization's balance reaches $0. Nothing is lost — the disk is kept, and you can start the instance again once the balance covers an hour of it, GPU plus disk, on top of an hour of whatever else the organization runs. Storage charges continue against the empty balance while it sits stopped, so it keeps sinking.
That stop also releases the instance's GPUs back to the machine, where the machine supports it, and so does a zero balance for instances you had stopped yourself. A later start has to get the same GPUs back: if another renter is using one of them, it is refused with 409 CAPACITY_UNAVAILABLE and the instance stays stopped with its disk. See Stop vs destroy.
If the balance stays at or below $0 for 24 hours and reaches −$5, stopped instances are destroyed and their disks go with them. Check the balance and the ledger on the billing page, and see Auto-stop and auto-destroy.
Below $0 your data is still there — for at least 24 hours, and until the balance reaches −$5. Once both have passed, stopped instances are destroyed, and there is no recovery.