How I stopped my local model from hallucinating bank balances
Hey everyone, I have been building Burrow, a private budget and journal desktop app for Windows. I wanted a built-in helper that runs entirely on your local PC through Ollama, using smaller models like Llama 3.2 or Qwen 2.5. We all know the problem: small local models are great at sounding natural but they are terrible at arithmetic. If you hand a 7B model a list of 40 transactions and ask "how much did I spend on dining?", it will give you a confident, well-written, and completely wrong answer. In a finance app, that is a dealbreaker. To fix this, I built the app around one hard rule: code calculates, the model summarizes. Here is how I handle it: Aggregates only: Every number the model sees is computed first in SQL or JavaScript. The model never receives a raw list of transactions. Instead, it gets pre-computed context like dining\_this\_month: 312.40, budget: 300.00, over\_by: 12.40. Prompt restrictions: Prompts never ask the model to calculate. They tell it the exact opposite: use the figures provided and do not work out new ones. The model is only used to turn the data into plain language and point out what matters. Enforced via CI: I wrote a test script that scans every system prompt. If a prompt includes words like "calculate", "compute", "add up", or "average of", the build fails. Taking the math away from the model makes it completely trustworthy for the part it is actually good at, and it keeps the responses incredibly fast even on laptops without dedicated GPUs. I would love to hear how the rest of you handle structured data and math with small local models. Do you trust the model to use tools to do the math itself, or do you take the math away from it completely like I did?