Case study · Hackathon team project · November 2026
OpsRelay
AI-assisted incident response for web apps. It monitors what users actually do, investigates what broke, proves a fix against the tests, and hands it to a person as a pull request.
The hosted dashboard requires an account. The README explains how to run everything locally.
- DetectA functional probe fails, such as a checkout that no longer completes.
- ReproduceThe failing tests are rerun before any model is called.
- DiagnoseEvidence must be quoted from the real logs and code.
- VerifyA candidate fix must pass the failing tests and the full suite.
- ReviewThe fix opens as a draft pull request. A person decides whether to merge.
The problem
In an outage, noticing that something is wrong is usually the quick part. The slow part is working out what actually failed, whether it’s your code or something you depend on, and getting a safe fix reviewed while people wait.
Widely reported Microsoft 365 and Outlook disruptions are a reminder that even mature platforms spend much of an incident on diagnosis, verification and communication. We wanted to see how much of that loop an agent could take on without anyone giving up control of what ships.
OpsRelay is a hackathon prototype that monitors and repairs its own demo applications. It has no connection to Microsoft’s or any other production systems.
Built at the hackathon
Four of us built OpsRelay together at a hackathon in November 2026. We split the work early. I built the two demo apps and the uptime monitor, so the agent would have real systems to watch and real bugs to find. Faiz, Mohamed and Hamza built the repair agent.
Splitting the work like that meant agreeing on the interfaces between our parts first: the probe results, the incident hand-off document the agent reads, and where each app writes its logs.
What we built
Four parts that run on their own and work together:
- NovaMart is a storefront with a Next.js front end and a FastAPI back end. Pricing is calculated on the server, an inventory guard prevents overselling, and a side-effect-free probe runs the real checkout logic.
- NovaConnect is a real-time messenger: a Next.js client with a FastAPI and WebSocket API on SQLite. It has direct and group chats, idempotent sends, delivery and read receipts, and its own functional probe.
- The uptime monitor and status dashboard is a Python worker that checks each service’s website, health and core workflow. It opens and resolves incidents and writes an evidence hand-off for the agent.
- The repair agents are a Python orchestrator that picks up failed GitHub Actions runs and live incidents, diagnoses them, tests fixes, and opens draft pull requests. Their state and memory live in MongoDB Atlas.
Both demo apps include controlled failure scenarios, such as a wrong discount calculation, a broken duplicate-message guard, or a dependency that times out. That gives the system real, reproducible bugs to find.
How it works
Detect what users would notice
A homepage that returns 200 doesn’t count as healthy. Each service has a functional probe, such as completing a checkout or sending a message. If it fails, the service is marked degraded even while the site still loads.
Open an incident with evidence
The monitor writes a structured hand-off for the agent: the failing probes, the HTTP status, likely source files, timing, and whether the issue looks like service health or a code problem.
Reproduce before reasoning
The agent reruns the failing tests at the failing commit, in a clean copy, before any model call. Flaky or unreproducible failures go to a person instead of a guess.
Diagnose from quoted evidence
The model gets logs, traceback frames, numbered source and the recent diff. Any evidence it cites must match those inputs word for word, or the diagnosis is retried and then rejected.
Prove the fix
Up to three minimal patches are each tested in a fresh copy. A patch must pass the failing tests and the full suite, and patches that touch tests, CI, dependencies or secrets are refused.
Hand it to a person
A passing patch becomes a draft pull request. The intent is saved in MongoDB first, so a crash can’t open duplicates. A person reviews and merges it; the agent never merges.
When the evidence points to a dependency timeout and no recent commit touched that code, the agent recommends an operational step and changes no code. Many real incidents look like that, and a code change would be the wrong fix.
Architecture decisions
Crash-safe orchestration
A stage machine with leases stored in MongoDB Atlas. Each incident resumes from its last completed stage, so work isn’t lost or repeated.
Repair memory
Past incidents and fixes are stored with embeddings for Atlas Vector Search. A rejected pull request switches that memory off.
Idempotent checkouts
A stable checkout ID follows the cart through confirmation, so recovery can resend a missing confirmation without creating a duplicate order.
Health separate from recovery
A service can be working again while its incident stays open until interrupted checkouts are reconciled. The dashboard shows both.
What I built
I wrote the first versions of three of the four components. My teammates then integrated and extended them.
- NovaMart (PR #2): the storefront, product and cart pages, checkout, server-side pricing and promotions, the inventory guard, the checkout health probe, a pytest suite, and the CI workflow template.
- NovaConnect (PR #9): real-time messaging over WebSockets with delivery status, read receipts, typing indicators and reactions, a functional monitoring probe, and three controlled failure scenarios with tests.
- Uptime monitoring and status dashboard (PR #13): the probe worker, incident lifecycle, uptime calculation, SQLite and MongoDB storage, the agent hand-off document, and the first Next.js dashboard.
The repair-agent pipeline and most of the later dashboard features were built by my teammates.
Where it stands
OpsRelay is a working prototype, not a production service. The repository includes a launcher script that runs the whole system locally, and repair tests for both demo apps’ failure scenarios. A hosted dashboard is deployed on Vercel.
What I’d take into a real team: automated repair is only useful when it’s narrow, verifiable and easy to say no to. Reproduce first, demand evidence, test everything, and leave the merge to a person.
Team
Areas are based on each person’s commits in the public repository.
Next project
Simply, a Flutter receipt manager I’m building with a five-person team.