> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ankra.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Watch for a problem and decide what Ankra does

> Choose a problem for Ankra to look out for, read what it will do before the watch is on, pick who hears about it, and change or remove the watch.

A watch tells Ankra a problem you care about, what to do when it happens, and who to tell. Behind it is an alert rule, and, when you let Ankra investigate, an agent that runs each time the alert fires. In about five minutes you have a watch that is on, says in one sentence what it will do, and shows its state on **Ankra AI → Watching**.

## Before you start

* **A role that manages alerts** (the `alerts.manage` permission). Without it, **Watching** reads **Someone who manages alerts can set up the first watch.** See [Roles and access](https://docs.ankra.ai/guides/roles-and-access.md).
* **An organisation admin**, if Ankra is to investigate. Anyone else can create a watch that only tells people: the form says **Only an organisation admin can let Ankra investigate. This watch will tell people.**
* **A Slack, Teams or webhook destination** under **Integrations & alerts → Notification destinations**, unless the watch only investigates. Email is not a watch destination. See [Notification routing](https://docs.ankra.ai/guides/notification-routing.md).

## 1. Choose what to watch

**1. Open Watching**

Open **Ankra AI → Watching** (`/next/ai/watching`) and press **Watch something**.

**You should see** the **Watch something** form, with a **What happens** summary beside it.

**2. Name it**

Enter a **Name**. It is what your team sees when the watch fires.

**3. Choose the clusters**

Under **Clusters**, choose **Every cluster** (clusters added later are included) or **Only the clusters I choose**, and pick them.

**4. Choose the problems**

Tick one or more **Problems**:

| Problem | What it means |
| - | - |
| **The cluster goes offline** | Ankra loses contact with the cluster agent |
| **Pods keep restarting** | More than 5 container restarts in 15 minutes |
| **Containers run out of memory** | Any out-of-memory kill in 15 minutes |
| **Pods fail** | Any pod in the Failed phase |
| **A node stops being ready** | Any node reporting NotReady |
| **A deployment fails** | An add-on, manifest or Stack job fails |
| **GitOps stops syncing** | The cluster's GitOps sync reports failing |
| **Pods can't be scheduled** | Any pod waiting in Pending |
| **CPU runs hot** | Cluster CPU above 90% |
| **Memory runs out** | Cluster memory above 90% |
| **Disks fill up** | Cluster disk above 85% |
| **An exploited vulnerability appears** | A new CVE on the CISA KEV list in a running image |
| **A fixable critical vulnerability is running** | Any critical CVE with a fix available |

**5. Choose how long it must last**

**Only when it lasts** keeps a brief blip from firing the watch: **As soon as it happens**, **After 2 minutes**, **After 5 minutes**, **After 10 minutes** or **After 15 minutes**.

## 2. Decide what Ankra does

Under **When it happens**, choose one:

| Choice | What Ankra does |
| - | - |
| **Investigate and propose a fix** | Finds the cause and proposes a change. Nothing is applied until a person approves it |
| **Investigate and explain** | Finds the cause with read-only access and reports what it found |
| **Only tell people** | No agent runs. The destinations hear about it |

An investigation runs as an agent named `Investigate: <watch name>`, started by the watch's alert and limited to 12 runs a day, 3 turns and 30 minutes a run. It runs with the agent limits set for your organisation, so **Pause autonomous actions** and **Stop all AI** stop it too. See [AI Autonomy](https://docs.ankra.ai/platform/ai-autonomy.md).

> [!NOTE]
>
> A watch's own investigation never applies a fix on its own. To let Ankra AI apply small fixes to firing alerts without asking, see [Fixing alerts automatically](https://docs.ankra.ai/platform/ai-autonomy.md#fixing-alerts-automatically).

## 3. Pick who hears about it

**1. Choose destinations**

Under **Destinations**, tick the Slack, Teams or webhook destinations to tell. A watch that only tells people needs at least one: otherwise the form says **Choose who to tell, or let Ankra investigate.**

A watch with no destination of its own is still delivered through your organisation's [notification routes](https://docs.ankra.ai/guides/notification-routing.md). The **Tell people** column on **Watching** then reads **Notification routes decide**.

**2. Choose how often it repeats**

**Repeat while it continues** sets how often a problem that goes on is reported again: **Every 15 minutes at most**, **Every hour at most**, **Every 4 hours at most** or **Once a day at most**.

## 4. Read what it will do, then turn it on

Before anything is on, **What happens** says the watch in one sentence, built from what you chose, for example: *When pods keep restarting on any cluster for 5 minutes, Ankra will investigate and propose a fix and tell platform-alerts.*, where `platform-alerts` is the name of the destination you ticked. Below it, Ankra lists what it promises: a problem that continues is reported again at most at the interval you chose, any change Ankra proposes waits for a person, and pausing a watch keeps its settings.

Read the sentence. If it is not what you meant, change the form until it is. Then press **Start watching**.

**You should see** **Watching** again, with **Watch saved** and **It is watching now. Pause it here at any time.** The watch's row shows **When this happens**, what **Ankra will** do, who it will **Tell people**, and its **State**: **Watching**.

## Verify it works

On **Watching**, the watch's **State** reads **Watching** until the problem happens. While it is happening it reads **Firing**, and the destinations you chose hear about it. **All signals** opens **Alerts** (`/next/ai/alerts`), where each alert shows the signal, its impact and the work moving it forward.

For a watch that investigates, each firing starts a run of `Investigate: <watch name>` under **Ankra AI → Activity**. Open it to read what Ankra found, and any change it proposes waits for a person. See [Follow, steer and stop an agent run](https://docs.ankra.ai/guides/agent-runs.md).

## Change, pause or remove a watch

* **Pause** - the **Pause** button on the watch's row stops it firing and keeps its settings. **Resume** turns it back on, and the **State** reads **Paused** meanwhile.
* **Change** - open the watch from its name. You can change its **Name**, what Ankra does under **When it happens**, and its **Destinations**, then press **Save changes**. What it looks out for and on which clusters are set when you create it. Choosing **Only tell people** pauses its investigator; choosing an investigation again resumes it.
* **Delete** - on the watch's page, press **Delete watch** and confirm with **Delete watch**. Its alert rules stop and nobody is told when these problems happen. Its investigating agent is removed too when you are an organisation admin; otherwise it stays until an admin removes it, but cannot start without the watch.

> [!WARNING]
>
> Deleting a watch removes its alert rules, so nobody hears about these problems any more. To stop it for a while instead, pause it.

## What can go wrong

| What you see | What it means |
| - | - |
| **Watch saved, but Ankra will not investigate yet** | The watch is on and tells the people you chose, but its investigating agent was not created. Check **Watching** before you try again |
| **Watch saved; check whether Ankra will investigate** | Whether its investigating agent was created could not be confirmed. Check **Watching** before trying again, so it is not created twice |
| **Ankra will** reads **Only tell people (its investigator is paused)** | The watch's agent is switched off. Choose an investigation under **When it happens** to resume it |
| **Ankra will** reads **Not known: the agents could not be read** | Ankra could not read the organisation's agents. Reload the page |
| **Choose at least one cluster, or watch every cluster.** | **Only the clusters I choose** is selected with none picked |
| **Choose what to look out for.** | No problem is ticked |

## Next steps

* [Follow, steer and stop an agent run](https://docs.ankra.ai/guides/agent-runs.md) - read what an investigation found
* [AI agents](https://docs.ankra.ai/platform/ai-agents.md) - the agents behind investigations
* [Alerts](https://docs.ankra.ai/guides/alerts.md) - alert rules in full, from the CLI and the API


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.