Evidence
Every number on this site is one of these records, copied from a saved file at an exact commit. If it is not here, it is not claimed.
| Metric | Value | Mode | Source |
|---|---|---|---|
| Unique cases that passed all three trials Same campaign, same status. | 9 of 12 | live model | artifacts/evals/live/free-dev-c2r1/report.jsonCommit 09d8c1e · 2026-10-28 · Command: run_live_campaign.py, campaign free-dev-c2r1, as recorded in the campaign report · Exit code 1 |
| Failures by family in the development campaign Counts of errors, not of failed trials; one trial can carry more than one. | classification 3, routing 6, tool selection 3 | live model | artifacts/evals/live/free-dev-c2r1/report.jsonCommit 09d8c1e · 2026-10-28 · Command: run_live_campaign.py, campaign free-dev-c2r1, as recorded in the campaign report · Exit code 1 |
| Protected evaluation budget | 84 held-out calls and 60 confirmation calls stayed reserved | live model | docs/harness/progress.mdCommit 6b84f9a · 2026-10-30 · Command: recorded in the P00 progress log · Exit code 0 |
| Real-model trials passed (development set) Development set, not held out. Status FAIL_PROVISIONAL; the exit code follows that status, as the P00 README defines it. | 27 of 36 | live model | artifacts/evals/live/free-dev-c2r1/report.jsonCommit 09d8c1e · 2026-10-28 · Command: run_live_campaign.py, campaign free-dev-c2r1, as recorded in the campaign report · Exit code 1 |
| Wilson 95% interval for the all-three pass rate Rounded to two decimals from the report. | 0.47 to 0.91 (Wilson, 95%) | live model | artifacts/evals/live/free-dev-c2r1/report.jsonCommit 09d8c1e · 2026-10-28 · Command: run_live_campaign.py, campaign free-dev-c2r1, as recorded in the campaign report · Exit code 1 |
| One live synthetic workflow, four provider attempts (tokens) Transport and accounting only. Says nothing about answer quality. | 1274 in, 261 out | live model | docs/harness/progress.mdCommit 6c8e06d · 2026-10-30 · Command: accounted local operation, phase 027, as recorded in progress.md · Exit code 0 |
| Locked fake gate Fixtures test control logic, not model quality. | 82 unit tests plus fixture and adversarial checks | fake: control logic | .planning/phases/016-accounted-local-operation/VERIFY.mdCommit 5b91e73 · 2026-10-29 · Command: python scripts/check_full.py · Exit code 0 |
| Full fake service gate | 70 tests, service traces, a dashboard and four firing alerts | services, fake model | .planning/phases/025-accounted-local-live/VERIFY.mdCommit 83fe067 · 2026-10-29 · Command: python scripts/check_full.py with the compose services · Exit code 0 |
| Isolated full fake gate Terraform was planned, never applied. | 127 tests, locked service scenarios, image build and Terraform plan | services, fake model | .planning/phases/043-root-snapshot-gate/VERIFY.mdCommit d3e694d · 2026-10-30 · Command: uv run --locked python scripts/check_full.py · Exit code 0 |
| demo-live-v1 ledger and failover tests against PostgreSQL 16 Fake provider transport. No model was contacted. | 17 passed | services, fake model | artifacts/evidence/demo-live-v1/pytest-demo-live-postgres.txtCommit ffcdb81 · 2026-10-31 · Command: P05_TEST_PG_DSN=... uv run --locked pytest tests/test_demo_live.py -v · Exit code 0 |
| Requester cannot approve their own refund; a tenant cannot read another tenant's ticket | 2 passed | fake: control logic | artifacts/evidence/portfolio-w7/dual-control-tests.txtCommit 6e51483 · 2026-10-31 · Command: uv run --locked pytest -v tests/test_api_security.py (two named tests) · Exit code 0 |
| P05 test suite at phase 033 | 179 tests passed | fake: control logic | .planning/phases/033-console-blockers/VERIFY.mdCommit d859a50 · 2026-10-31 · Command: uv run --locked pytest · Exit code 0 |
Feature status, as each project records it
Copied from each project's own feature list. Nothing is rounded up: PENDING stays PENDING.
| Projects | verified | claimed | partial | open |
|---|---|---|---|---|
| P00 SupportOps Foundation | 3 verified | 0 claimed | 0 partial | 2 open |
| P01 SupportOps Agents | 1 verified | 6 claimed | 0 partial | 0 open |
| P02 SupportOps AI Toolkit | 5 verified | 0 claimed | 1 partial | 2 open |
| P03 SupportOps Observability | 12 verified | 0 claimed | 1 partial | 1 open |
| P04 SupportOps Async Platform | 11 verified | 0 claimed | 2 partial | 2 open |
| P05 SupportOps Platform | 8 verified | 0 claimed | 9 partial | 1 open |
| P06 SupportOps Console | 4 verified | 0 claimed | 1 partial | 0 open |