> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ankra.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# A cluster or node that will not come up

> What to read and what to try when a cluster Ankra builds stops in setup, its agent never connects, or a node does not join or stays NotReady.

This page is for a cluster Ankra builds for you - on Hetzner, OVHcloud, UpCloud, DigitalOcean, Scaleway, AWS, Ankra Cloud, Proxmox VE or HPE Morpheus - that does not reach **Online**, or a node that does not become `Ready`. It tells you where Ankra records what went wrong, which reads it can run on the servers for you, and which remedies exist. You end up knowing why setup stopped and either a cluster that finished on a retry or a clear reason to remove it and create it again.

## Before you start

* The cluster's create has started. A create refused before anything was built, such as "The cluster was not created" with **Open Plan and billing**, is covered in [Create a cluster](https://docs.ankra.ai/guides/create-a-cluster.md).
* The `clusters.operate` permission on the cluster, for **Try again**, retries and the diagnose reads. See [Roles and access](https://docs.ankra.ai/guides/roles-and-access.md).
* For the CLI steps, the [Ankra CLI](https://docs.ankra.ai/reference/cli/index.md) signed in to the organisation, with the cluster selected (`ankra cluster select <cluster>`).

## Find where setup stopped

**1. Read the progress page**

After **Create cluster**, the progress page follows the setup in stages. Its lines say how far it got:

| What the page says | Where setup is |
| - | - |
| **Preparing your cluster** with "Ankra is creating the servers and network." | Servers, network and bastion are being created at the provider |
| "Ankra is installing Kubernetes and configuring the control plane." | Kubernetes is being installed over SSH through the bastion |
| **Connecting your cluster** with "Infrastructure setup is complete. Waiting for the Ankra agent to connect." | Everything is built; the Ankra agent in the cluster has not reached Ankra |
| **Your cluster is ready** | Done. Under **Infrastructure → Clusters** the status reads **Online** |

"The platform has not reported a setup stage yet. Progress updates automatically." means the operation has not started its first step. Wait for it.

**Check:** if the page shows a setup failure card, carry on with the next step. If it is still moving, leave it; a large cluster takes longer than a small one.

**2. Read the failure card**

When setup stops, the page shows what stopped and what to do. Where Ankra recognises the failure, the card names it and lists the steps to take; the common ones are under [What can go wrong](#what-can-go-wrong). Otherwise the card reads **Setup stopped at this step:** followed by the step and the server it ran on, or **Setup stopped before it finished**, with the provider's or the installer's error below it. "The platform recorded no error message for this failure." means there was none to show.

Open **Technical details** on the card and choose the failed task to read its full log.

**Check:** you can name the step that failed (for example a server create, the Kubernetes install on one control plane, or the agent install) and you have its error.

**3. Open the operation**

Every create is an operation. Choose **View operation** on the progress page, or a task under **Technical details** on the failure card, to see every step, its status and its log. The [Operations](https://docs.ankra.ai/platform/operations.md) page explains the list, retries and attention states.

From the CLI:

```bash theme={null}
# Failed operations on the selected cluster that still need someone
ankra cluster operations list --attention open

# The steps of one operation, with their status
ankra cluster operations steps <execution-id>
```

**Check:** the failed step in the operation matches the one the failure card named.

**4. Try again**

Many failures are temporary: a provider out of capacity for a few minutes, a network connection that timed out. Choose **Try again** on the failure card. It reads **Starting** until the new attempt has started. A retry starts a new operation that re-plans only the resources that did not finish.

From the CLI:

```bash theme={null}
ankra cluster operations retry <execution-id>
```

Without the permission, the card says "You do not have permission to run setup again. Ask an organisation owner to run it for you."

**Check:** the progress page moves past the step that failed. If the same step fails again with the same error, the cause is not temporary: read [What can go wrong](#what-can-go-wrong) and [Read the servers](#read-the-servers).

## Read the servers

When the operation's own log does not explain the failure, Ankra can read the servers for you. These reads change nothing. The node and bastion reads run through the bastion with the platform's own access, so you do not need the cluster's SSH key; `nodes console-log` reads the OVHcloud API instead. The platform waits up to two minutes for a diagnose, 90 seconds for `nodes cloud-init-log` and one minute for `nodes console-log`; a slower read hands back an operation id to follow with `ankra cluster operations list`. Running a `nodes` read again attaches to the one already running; a second `bastion diagnose` is refused until the first finishes.

Find node ids first:

```bash theme={null}
ankra cluster <provider> nodes list <cluster>
```

### A node that did not finish its first boot

A server is set up by cloud-init at its first boot. On the providers that offer it (see [What each provider offers](#what-each-provider-offers)), read the outcome: `cloud-init status --long` and the tail of `/var/log/cloud-init-output.log`.

```bash theme={null}
ankra cluster <provider> nodes cloud-init-log <cluster> <node-id>
```

### A node that joined but is NotReady

A node that is `NotReady`, or whose kubelet stopped posting status, is below what the cluster's own API shows. The node diagnose read snapshots the node's OS: the Kubernetes service unit and its journal, disk, memory, uptime, the kernel log and the firewall path to the API server, and names the symptoms it finds.

```bash theme={null}
ankra cluster <provider> nodes diagnose <cluster> <node-id>
```

Once the agent is connected, the cluster's events often say why a node or a pod is held back:

```bash theme={null}
ankra cluster get events --all-namespaces
```

### A node that hung

An OVHcloud node that no longer answers over SSH can still be read from its serial console, through the OVHcloud API. A kernel soft lockup, hung-task warnings, an out-of-memory kill or a root filesystem gone read-only is often the last thing it printed.

```bash theme={null}
ankra cluster ovh nodes console-log <cluster> <node-id>
```

### The bastion

Every read above except `nodes console-log` goes through the bastion. When they fail too, read the bastion itself: sshd configuration, failed logins, disk, failed units, journal errors and listening ports.

```bash theme={null}
ankra cluster <provider> bastion diagnose <cluster>
```

### What each provider offers

| Provider (`<provider>`) | `nodes cloud-init-log` | `nodes diagnose` | `bastion diagnose` | Also |
| - | - | - | - | - |
| Hetzner (`hetzner`) | Yes | Yes | Yes | `nodes restart` |
| OVHcloud (`ovh`) | Yes | Yes | Yes | `nodes console-log`, `nodes restart` |
| UpCloud (`upcloud`) | Yes | Yes | Yes | `nodes restart` |
| DigitalOcean (`digitalocean`) | Yes | Yes | Yes | `nodes restart` |
| Scaleway (`scaleway`) | Yes | Yes | No: the Public Gateway's bastion has `bastion status` | `nodes restart` |
| AWS (`aws`) | Yes | Yes | Yes | `nodes restart` |
| Ankra Cloud (`ankracloud`) | Yes | Yes | Yes | `nodes restart` |
| Proxmox VE (`proxmox`) | No | No | Yes | `nodes restart` |
| HPE Morpheus (`morpheus`) | No | No | Yes | |

On Proxmox VE and HPE Morpheus, read the failed step's log in the operation, then the server's console in Proxmox VE or in your Morpheus appliance.

## Verify

1. Under **Infrastructure → Clusters**, the cluster's status reads **Online**.
2. `ankra cluster operations list --attention open` lists no open failure for the create. A failed attempt stays in the history; once a retry finished the resources it failed on, it carries the **Resolved by a later run** label in the portal.
3. Every node is `Ready`. Point `kubectl` at the cluster through Ankra (this needs a [Cluster Access](https://docs.ankra.ai/guides/cluster-access.md) grant):

   ```bash theme={null}
   ankra cluster kubeconfig add <cluster> --use
   kubectl get nodes
   ```

## What can go wrong

The first six rows are failures the progress page names, with the steps it gives. The last two are not on the page: they are what to check when setup looks finished but the cluster is not.

| The card says | What happened | What to do |
| - | - | - |
| "`<provider>` has no free capacity for this server in this location" | "The location you chose cannot start a server of this size right now, so setup stopped." | Wait a few minutes, then choose **Try again**. If it fails again, remove this cluster and create it again in another location or with another server size |
| "Your Hetzner account has reached a limit" | Hetzner refused another server because the project is at its core limit | Remove servers this Hetzner project no longer needs. Or ask Hetzner for a higher limit: in the Hetzner Cloud console, open the account menu in the upper right, choose **Limits**, then **Request change** and **Limit increase**. Come back to the page and choose **Try again** |
| "Your `<provider>` account has reached a limit" | The provider refused to create more resources for this cluster | Remove resources this account no longer needs, or ask the provider for a higher limit, then choose **Try again** |
| "The connection to the servers failed" | "Ankra connects to the servers over the network to set them up. That connection failed or took too long, which is often temporary." | Choose **Try again**. If it fails again, check that the servers can reach the internet and that no firewall blocks the connection. Then run `bastion diagnose` |
| "Ankra does not have access to this cluster" | The cluster refused the credentials Ankra used to sign in to Kubernetes | Check that the credential you connected to Ankra is allowed to manage this cluster, then choose **Try again** |
| **Cluster setup was cancelled** | Someone cancelled the operation | "Open the operation to review what completed before setup stopped." Retry it, or remove the cluster |
| **Connecting your cluster** does not move on | The servers and Kubernetes are up, but the agent has not connected | The agent connects outbound from the cluster to Ankra. Run `nodes diagnose` on a control plane and check the cluster's network reaches the internet |
| A node stays `NotReady` after setup | The node's kubelet or its network path is unhealthy | Run `nodes diagnose` on the node. `ankra cluster <provider> nodes restart` restarts it, on every provider but HPE Morpheus |

For a cluster that cannot be saved, terminate it and create it again: see [Terminate a cluster](https://docs.ankra.ai/guides/operate-ankra-managed-clusters.md#terminate-a-cluster), and the provider guide for what terminating deletes.

## Next steps

* [Create a cluster](https://docs.ankra.ai/guides/create-a-cluster.md) - the create form, and the refusals before anything is built
* [Operations](https://docs.ankra.ai/platform/operations.md) - retries, cancels and attention states
* [Operate a cluster Ankra built](https://docs.ankra.ai/guides/operate-ankra-managed-clusters.md) - node groups, the bastion, SSH access and upgrades
* Provider guides: [Hetzner](https://docs.ankra.ai/guides/hetzner-clusters.md), [Scaleway](https://docs.ankra.ai/guides/scaleway-clusters.md), [Ankra Cloud](https://docs.ankra.ai/guides/ankra-cloud-clusters.md), [Proxmox VE](https://docs.ankra.ai/guides/proxmox-clusters.md), [HPE Morpheus](https://docs.ankra.ai/guides/morpheus-clusters.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.