← r/LocalLLaMA
▲
0
 
1👁
r/LocalLLaMA · u/jeeva1398 · 3d ago

I fine-tuned Qwen2.5-Coder-1.5B on a free Kaggle T4 to review Node.js code offline. The base model invented bugs in 9/9 clean diffs; the fine-tune in 0/9.

I wanted an AI helper for Node.js that runs fully offline and doesn't need an API key, so I built one and released it as an npm package. Model: jeeva1398/eventa-1.5b-gguf, a Qwen2.5-Coder-1.5B-Instruct QLoRA fine-tune (Unsloth, r=16, 2 epochs, responses-only loss), Q4_K_M, 986 MB. Trained on a free Kaggle T4 in about 9 minutes. Data (~740 examples, all generated and checked, nothing scraped): - 68 crash types. Small Node programs that really crash are executed, and their real stack traces are parsed. This includes TypeScript tsc errors, NestJS DI errors and Prisma error codes. - 74 review scenarios: before/after diffs with annotated issues. Half are clean diffs, so the model learns to say "No issues found." - npm audit/outdated reports built from real advisories. Eval: 54 held-out examples, whole scenarios the model never saw in training. Base and fine-tune get the exact same prompt, static-check hints included. The main win is review. On 9 clean diffs the base model invented problems in all 9; the fine-tune said "No issues found." on all 9. On the 8 buggy diffs, 48% of what base flagged was real vs 100% for the fine-tune. It's also about 2x faster on CPU (5.9s vs 13.6s), mostly because it gives shorter answers. Deps went from 93% to 100% on not inventing package versions. Small eval, I know. 8/8 and 9/9 is encouraging, not proof. Where it's worse: explaining error types it never saw in training. It gets 68% of key facts vs 75% for base. That's what the next data round is for. To be fair to the base model, the static checks do most of the actual bug finding. The fine-tune's job is to confirm them without making stuff up, and to write the fix. Getting there took 4 rounds. One round learned "no hint = no issue", the next flagged everything, and I had to rebalance the data a few times. CLI: npx u/jeeva1398/eventa explain --run "node app.js". If Ollama is running it uses it. Otherwise it installs node-llama-cpp from a pinned lockfile (CPU build only, about 80 MB) and downloads the GGUF with SHA-256 verification. The same CLI runs as a GitHub Action, so the 1.5B model reviews pull requests on a plain CPU runner (the model is cached between runs). - Repo, with dataset builder, notebook and eval: https://github.com/Jeeva1398/eventa - Model: https://huggingface.co/jeeva1398/eventa-1.5b-gguf Happy to answer questions about the data pipeline. Feedback on making a 1.5B model reason better about unseen errors is very welcome.

posted Tue, 06 Oct 2026 12:08:34 GMTseen 1 time
open on reddit ↗ 💬 5