One contract, many frameworks
Every adapter implements `TriageRunner`, and a shared test runs each of them against the same cases.
Code: src/supportops_agents/contracts.py
Rejected: Comparing frameworks by their own demos.
Multi-agent orchestration
Multi-agent triage with a bounded loop, compared across frameworks on the same contract.
offline verified, quality pendingLast verified: 2026-10-30 · Commit e9de403
TL;DR
- Problem
- Framework demos rarely show how to keep contracts, retries, approval pauses and termination comparable.
- My role
- I built the supervisor, the shared contract and the adapters, and the accounting for the live operation.
- Stack
- Python, LangGraph, CrewAI, AutoGen, LlamaIndex, Semantic Kernel
- Result
- One live synthetic workflow ran with four provider attempts and 1274 in, 261 out tokens. That shows the transport and the accounting work. It says nothing about answer quality.
A supervisor delegates to specialist workers with shared state, hard step limits and explicit stop conditions. The same ticket runs through several framework adapters behind one `TriageRunner` contract, so the choice of framework is a measured decision, not a preference.
See it run
The live demo runs this system against a real model and shows every step.
How it works
The whole flow. Press play or next to walk through it.
Text version of this diagram
- TriageRunner contract P01. Every framework adapter implements the same contract.
- then Supervisor
- Supervisor P01. Delegates one step at a time and owns the stop conditions.
- then Classifier
- Classifier P01. Sets the category and whether to escalate.
- then Researcher
- Researcher P01. Reads the supplied sources.
- then Billing agent
- Billing agent P01. Checks the money side.
- then Responder
- Responder P01. Writes the answer and cites a source.
- then Stop condition
- Stop condition P01. A hard step limit ends the run, whatever the model says.
Decisions
The stop condition belongs to the supervisor
A hard step limit ends the run whatever a worker returns, so a looping model cannot extend its own budget.
Code: src/supportops_agents/core.py
Rejected: Trusting the model to say it is done.
Results
Measured
| Metric | Value | Mode | Source |
|---|---|---|---|
| One live synthetic workflow, four provider attempts (tokens) Transport and accounting only. Says nothing about answer quality. | 1274 in, 261 out | live model | docs/harness/progress.mdCommit 6c8e06d · 2026-10-30 · Exit code 0 |
Not measured, so not claimed
- Which framework is best. The comparison runs offline on fixtures.
- Answer quality of the live workflow.
Failure gallery
- What happened
- A completed call with missing usage replaced its reservation with zeros.
- Cause
- Cumulative accounting treated unknown usage as no usage.
- What changed
- Unknown usage keeps the original reservation, with a regression test.
- What happened
- An older adapter rotated to another key after a transient failure and bypassed the accounting.
- Cause
- It built provider clients itself instead of using the shared receipt boundary.
- What changed
- Red tests were written first, then the adapter was moved behind the accounted boundary.
Validate it yourself
git clone https://github.com/DiegoVSouza/supportops-agents.git
cd supportops-agents
uv sync --locked --all-groups && uv run --locked python scripts/check_fast.pyExpected: The offline suite passes. One test that compares against a lab-only module is skipped, with its reason printed, outside the lab.
What I would do differently
I would write the stop condition as a separate small module with its own tests before writing any worker.