← r/LocalLLaMA
▲
9
+5
13👁
r/LocalLLaMA · u/KMatysek · 3d ago

ARC-1: a 1.7B decision model (pick / score / yes-no, with probabilities) that answers in ~20 ms on a 4060 Ti

I spent the last 11 days training a small model for typed decisions: routing support tickets, intent detection, moderation, "should the agent call this tool", that kind of thing. You give it some context and a question, and it gives you back a choice, a score or a yes/no probability.

  • 1.7B parameters (Qwen3-1.7B-Base + LoRA). Each option is scored in its own branch, so the order of the options doesn't matter
  • \~16 ms for a short request and \~25 ms median on JevBench items, on a single RTX 4060 Ti, batch 1
  • JevBench public 231: 68.4% (Jev 86.6, Strands Decider 2B 72.3 self-reported, Laya 58.4). DecideBench v1.1: 75.5%
  • Several questions about the same text in one forward pass
  • Weights are CC BY-NC 4.0 because part of the training data is non-commercial. The code is Apache 2.0

To be upfront: it is clearly less accurate than hosted APIs like Jev, and the README has all the numbers, including the ones where it loses. What it has going for it is that it's fast, runs locally and is free.

GitHub: https://github.com/realslapout/arc-1 Weights: https://huggingface.co/realslapout/ARC-1 Colab (free GPU): https://colab.research.google.com/github/realslapout/arc-1/blob/main/notebooks/quickstart.ipynb

Happy to answer questions, and I'd love to hear where it breaks.

11 0 9 10/6 15:47 10/8 03:36 UTC
scorecomments13 sightings
first seen 2026-10-06 15:47 UTClast seen 2026-10-08 03:36 UTCscore then 4score now 9gained +5sightings 13
open on reddit ↗ 💬 2