← r/LocalLLaMA
▲
5
+3
20👁
r/LocalLLaMA · u/litLikeBic177 · 3d ago

Best open-weight coding model + harness for an on-prem multi-agent setup? (80-200+ GB VRAM, 2-3 concurrent users)

Setup: GPU box with 1x H200-class card now; can allocate more (up to several H200s) if it clearly buys better results. Inference via vLLM or similar, OpenAI-compatible endpoint. Coding happens on a separate non-GPU Linux VM on the same network - IDE/harness there, calling models over LAN. Outbound internet is fine for packages/extensions, but inference stays on our hardware (no code to external model APIs).

Constraints: on-prem only, open weights, permissive license preferred (Apache/MIT), not Chinese-origin including base models (I understand a lot of fine-tunes are Qwen underneath), NATO-country lab preferred. Use is general software dev plus security tooling and code analysis - repo-level agentic work on existing codebases (fix / extend / refactor / test) is the core, with a human reviewing diffs. 2-3 concurrent users max.

Two things I'm trying to work out:

  1. Capability tiers vs. VRAM. On one card candidates seem to maybe be something like Cohere North Mini Code (30B MoE/3B active), Mistral Small 4 (119B MoE/6B active), Nemotron 3.5 maybe as a generalist baseline; Gemma 5? The Vibe Code Bench results suggest small open models fall over on long E2E builds, where only Large-4-class (4-8 cards) and closed models seem to hold up. Is that your experience? Where's the step-change for agentic repo work on an existing codebase - does 30B-class -> 120B-class matter much, or only the jump to 500 GB+? We could get more compute for something like Command A+ or Mistral Large 4 if it's actually better for code rather than just for E2E.
  2. Heterogeneous multi-agent. Does a big planner/reviewer (Large 4 / Command A+ class) plus small fast executors (e.g., North, Small 4) actually beat a single mid-size model, with a human approving plans and diffs? Which harnesses handle routing sub-agents to different endpoints well - OpenHands, OpenCode, Pi, VS Code extensions (Cline/Roo/Continue), Codex CLI in local-model mode?

Harness/IDE: something that supports multi-agent workflows (planner / executor / reviewer agents checking each other / etc.) but would also like humans to be able to step in, review diffs and edit by hand.

Anyone running something like this? Which model + harness combo is holding up best as of late? Thanks!

EDIT: thanks all - adding Poolside Laguna S 2.1, Muse Glimmer 30B, Inkling-Small (2x H200), K2 Horizon (checking lineage), Gemma 4 31B and Reflection Beam (501B MoE / 23B active, Apache 2.0, weights due this month) to the candidates; Ornith is Qwen-based so out. Pi added to the harness list.

6 0 5 10/6 21:47 10/9 07:48 UTC
scorecomments20 sightings
first seen 2026-10-06 21:47 UTClast seen 2026-10-09 07:48 UTCscore then 2score now 5gained +3sightings 20
open on reddit ↗ 💬 57 (+35)