Diego Souza · AI Engineer

All projects

The engineering harness

The engineering system that built all of the above with coding agents.

process in use, outcome per projectLast verified: 2026-10-31 · Commit 7b67edd

TL;DR

Problem
Coding agents write fast and claim more than they checked. Speed without a gate produces confident, unverified work.
My role
I wrote the rules, the phase template and the gates, and I review what the agents produce.
Stack
Markdown specs, Python gates, Git hooks, Conventional Commits
Result
The rules are in `AGENTS.md`: no result is called done without a command, an exit code and saved output, and a number comes only from a recorded run.

Specs before code, scoped write paths, gated checks, evals, budget ledgers, and saved evidence for every claim, following AI engineering best practices. Coding agents did the volume. The harness is why the result can be trusted, and every phase is in the repository to read.

See it run

The live demo runs this system against a real model and shows every step.

Live demo

How it works

From intake to a gateFlow: Intake, Spec, Plan, Execute, Verify, GateIntakeSpecPlanExecuteVerifyGateFrom intake to a gateFlow: Intake, Spec, Plan, Execute, Verify, GateIntakeSpecPlanExecuteVerifyGate

The whole flow. Press play or next to walk through it.

Text version of this diagram
  1. Intake Lab. Decisions and a profile before any code.
    • then Spec
  2. Spec Lab. A written specification with requirement ids.
    • then Plan
  3. Plan Lab. Allowed write paths and forbidden changes are declared.
    • then Execute
  4. Execute Lab. Agents write the code inside those paths.
    • then Verify
  5. Verify Lab. Commands run, exit codes and output are saved.
    • then Gate
  6. Gate Lab. No evidence means the result stays incomplete.

Decisions

Evidence or it is incomplete

A task is done when a command ran, its exit code and output are saved, and the diff was reviewed. Without that, the result is marked incomplete.

Code: AGENTS.md

Rejected: Accepting an agent's statement that it works.

Write paths are declared per plan

Each plan lists the paths it may change and the changes it may not make. A product phase cannot edit the shared kit.

Code: harness/GSD-LITE-PLUS.md

Rejected: Letting an agent edit whatever it needs.

Results

Measured

PENDING

Not measured, so not claimed

  • How much faster the harness makes delivery. I have no controlled comparison.
  • Whether the process transfers to a team.

Failure gallery

What happened
A full gate was marked incomplete after repeated attempts.
Cause
A stale literal in a test no longer matched a policy value that had changed.
What changed
The attempts are recorded, no live call was made on a red gate, and the stale value was fixed in its own task.

Validate it yourself

git clone https://github.com/DiegoVSouza/ai-engineer-lab.git
cd ai-engineer-lab
python scripts/harness/verify.py

Expected: The lab's structural checks pass.

What I would do differently

I would version the phase template and show a diff between versions, to make process changes as reviewable as code changes.

Next project

SupportOps