Use case 03 · Incident
Production incident at 2 a.m.
At 2 a.m. the hard part is finding what changed and deciding whether rollback is safe — fast, without losing the evidence trail.
The work already crosses these systems:
Before and after
From scattered checks to one governed run.
Today
An on-call SRE gets paged, opens ServiceNow, reviews CI/CD deploy history, checks GitHub commits, then files a Jira follow-up and updates the Confluence postmortem alone.
With NopsAI
An alert-triggered pipeline collects incident, change, deploy and commit context, runs investigation goals, and executes a pre-approved rollback behind a production approval gate.
A controlled incident workflow instead of a 2 a.m. scavenger hunt.
Governed run
The run, step by step.
Deterministic work first, reasoning inside a boundary, and a named human before anything high-risk executes.
Trigger
The alerting system calls an authenticated external trigger bound to the production scope.
Collect context
Recent deploys, open changes, commit diffs and the service runbook are gathered in parallel.
Verify state
Deterministic health checks confirm which instances are affected and whether the last deploy is the suspect.
Reason
A Production SRE agent role correlates the signals and proposes rollback or a targeted mitigation.
Approve
A named approver from the platform/prod team confirms before anything touches production.
Execute and record
The pre-approved rollback runs on an eligible runner; the postmortem starts from the run record, not memory.
Evidence
What the run leaves behind.
The useful part is not only the automation. It is repeatability with proof.
Trigger and subject
What started the run and which effective identity it ran as.
Authorization snapshot
Which resources were checked, and which decision each check returned.
Tool and AI activity
Every tool call, the profile that allowed it, and the model usage it consumed.
Approvals and outputs
Who approved, when, and the deliverables the run produced.
← Sprint planning for the next two weeks
Planning starts with manual research instead of decisions. Blockers surface late and half-ready stories enter the sprint.
QA is blocked on unclear requirements →
QA loses time reconstructing intent. Testing can start from the wrong assumption and clarification cycles cost a day or more.
Map this workflow against your controls.
Bring the trigger, the tools it touches, the approvers, the runtime boundary and the evidence you need to keep.