Skip to main content

Incidents

An alert is a signal; an incident is the unit of work. InfraSage groups correlated alerts into incidents automatically and gives each one a room: a single timeline of everything that happened, a lifecycle your team drives, and the evidence panes anchored to the incident's own window.

How incidents form

The causal grouper watches firing alerts and clusters them by graph adjacency: services that call each other fail together. A payment-service latency fault that drags order-service and the gateway with it becomes one incident with three member services, not three pages.

Alerts that no group claims still appear in the incident list as singletons. An ungrouped page is still an incident to the human carrying it.

Each incident carries:

FieldMeaning
servicesMember services, with the grouper's root candidates ranked separately.
severityThe highest member severity, after calibration.
statusopen → mitigated → resolved → learned, set by your team, not the machine.
detected_byWhich grouping strategy formed it (e.g. graph_adjacency).

The room

Opening an incident shows:

  • The spine: a composed timeline of alerts fired/resolved, deploys and config changes in the blast window, RCA verdicts, human decisions, and free-text notes. Comments land here too, so the narrative and the artifacts live in one place.
  • Per-service evidence: click any member service for its anomaly-score history around the incident's timestamp (not "the last hour from now"), current state, and quick stats. If a service was silent, with no telemetry at all, the pane says exactly that rather than showing empty charts. For "stopped sending data" alerts, the silence is the incident.
  • RCA panel: the latest analysis for the anchor service, with origin attribution and its confidence (or an explicit abstention with the reason), the acceptability judge's shadow review, and recommended actions. Ask the agent opens a chat pre-seeded with this incident's context.

Lifecycle

Status changes are deliberate, human actions:

  • open: firing or under investigation.
  • mitigated: impact stopped, cause may still be live.
  • resolved: done, and it feeds MTTR and the Insights analytics.
  • learned: resolved and the lesson captured, whether that is a runbook drafted, a threshold tuned, or a pattern recorded. It is the state that separates teams that improve from teams that repeat.

Assignees, title edits, and comments are all recorded on the timeline with the actor.

The decision queue

Anything the platform wants to do, most commonly execute a runbook, becomes a pending action in the decision queue (the bell in the console header, and in Slack when connected). Each card shows what will run, against which service, at which trust tier. Approve or reject in one click; either lands in the audit trail with who, when, and from which surface. Nothing tiered above auto-execute runs without this gate.

Access & API

Incidents are tenant-scoped end to end. List, detail, timeline, lifecycle changes, and comments are all filtered to services your tenant can see.

GET /api/v1/incidents?status=open|mitigated|resolved|learned|all
GET /api/v1/incidents/{incident_id}
GET /api/v1/incidents/{incident_id}/events
PATCH /api/v1/incidents/{incident_id} {"status": "...", "assignee": "...", "title": "..."}
POST /api/v1/incidents/{incident_id}/comment {"text": "..."}

:::note Singleton pseudo-incidents List entries with ids like alert:al-… are ungrouped firing alerts surfaced for completeness. They have no lifecycle of their own, so act on the alert itself. :::