← r/LocalLLaMA
▲
2
+1
2👁
r/LocalLLaMA · u/dreamyrhodes · 14h ago

Help with EXL2/3 pls

I am trying to run EXL2/3 models using Silly Tavern. Normally I am running GGUF but I wanted to see if EXL2/3 format could provide a better lore coherence than Q4 quants.

My rig is running a 4060 with 16GB.

As an API provider I tried TabbyAPI (know a better one for EXL2/3?).

I tried it with this template (and various adjustments, tempereture etc) https://huggingface.co/Nitral-AI/Violet\_Magcap-12B/blob/main/ST%20Presets/ChatML\_Master-Import.json
But it generates gibberish only. Sometimes it runs halfway ok but there will still be grammatical errors, half words and sometimes loops (doesn't get a stop token), often it's just entire word salad.

Now wtf am I doing wrong? How do I run EXL2 or 3 locally?

Screenshot is default bot's response to "Hello".

https://preview.redd.it/k6q7nbylcauh1.jpg?width=1159&format=pjpg&auto…

2 0 2 10/8 19:30 10/9 01:36 UTC
scorecomments2 sightings
first seen 2026-10-08 19:30 UTClast seen 2026-10-09 01:36 UTCscore then 1score now 2gained +1sightings 2
open on reddit ↗ 💬 9