Incident response
Severity
| Level |
What it means |
Examples |
React |
| SEV-1 |
Service down, data lost, security breach |
App unreachable, database corrupted, secret leaked |
Now |
| SEV-2 |
A main feature is broken for everyone |
Import fails, analyses crash, blank pages |
Within 2 hours |
| SEV-3 |
A minor feature is broken, there is a workaround |
Boxes misaligned, a missing translation |
Next working day |
Steps
- Detect.
/api/health shows "status": "degraded" or does not answer, a user opens an issue, CI fails on main, or a container keeps restarting. /api/health always answers HTTP 200: read the status and database fields.
- Assess. Pick the severity. Find the broken part (backend, frontend, Docker, CI) and what changed last:
git log --oneline -10 main.
- Tell people. SEV-1: warn every maintainer right away. SEV-2 and SEV-3: open an issue labeled
bug, with P0 or P1.
- Mitigate first. If it started with a deploy, roll back. If Docling itself fails in the backend, switch to the remote engine: the
-remote image with DOCLING_SERVE_URL, or CONVERSION_MODE=remote docker compose --profile remote up -d --build.
- Fix. Branch
hotfix/* from main, fix the cause, add a test that reproduces it, then follow the deployment checklist.
- Write it up for SEV-1 and SEV-2, in the incident issue.
Post-mortem template
## Post-mortem: <title>
- Date, severity, duration:
- Timeline: detected / mitigated / root cause found / fixed
- Root cause:
- Impact (who, how long, which data):
- What went well:
- What went wrong:
- Actions (owner, due date):