Pi and mini-swe-agent passed 9/9 checks each in my latest experiment. A second code review still found defects in both.

As part of my AI Studio project, I’m testing harnesses for coding.

The initial screening included 10 harnesses:
Pi, mini-swe-agent, Crush, OpenCode, Goose, Prime Agent, Oh My Pi, Qwen Code, Octomind (reduced offline profile) and Aider.

Models:
• DeepSeek V4.1 Flash
• Qwen3.8-27B
• Laguna S 2.1

All accessed through OpenRouter.

The detailed code review covered Pi, mini-swe-agent, Crush and OpenCode across all three models and tasks: 36 combinations. Two attempts produced no patch.

The three Golang tasks were deliberately different:
\- Add strict validation for an HTTP query parameter.
\- Migrate 200 logging calls while preserving behaviour and context.
\- Add bookmark tags across the API, storage migration and HTML rendering.

Pi and mini-swe-agent each passed the original acceptance checks on all nine combinations. But a second agent review, followed by isolated reproduction probes, exposed three gaps:
\- silently dropped malformed query fields
\- bookmark's task exposed a mutable tag slice from the store
\- one migration rejected a valid older store

Good news too: all logging migrations preserved behaviour in differential probes covering 100 functions and nine integer inputs, including the minimum and maximum values.
My takeaway: the evaluator and the reviewer both need testing. A green result is evidence about the checks we ran; broader correctness needs further evidence. This experiment did not establish a decisive winner between Pi and mini-swe-agent. Human correction time also is still unmeasured.

Any opinion welcome.