7900 XTX — two "low-thinking" Qwen 3.8 27B quants (Swift + ThinkingCap) vs the regular quant

First, do they actually produce less tokens? Yes. Total tokens per benchmark run (4 scenarios): base quant \~66k, ThinkingCap \~49k (−26%), Swift \~45k (−33%). So the "less thinking" is real — and Swift cuts the most. Then the cost: and this is where it got interesting. The two quants don't trade off the same way: Decode: base \~48 t/s. ThinkingCap barely changes (\~43). Swift drops hard (\~32, −33%). Prefill / TTFT — the opposite of what I expected: Swift is the fastest (\~614 t/s, TTFT \~1s), base in between (\~530 t/s, \~1.2s), ThinkingCap the slowest (\~100 t/s, TTFT 6–11s). * Quality holds: \~84–87 on my eval, same band as base. Full run data: ThinkingCap finishes \~23% faster than base. The token savings win even with the slow prefill. Swift is break-even because of the slower decode speed. |*Quant*|*Prefill*|*Decode*|*Quality*|*Runtime*| |:-|:-|:-|:-|:-| |Base (unsloth)|\~530 t/s|\~48 t/s|\~85|\~1407s| |ThinkingCap|\~100 t/s|\~43 t/s|\~85|\~1087s| |Swift|\~614 t/s|\~32 t/s|\~86|\~1400s| Caveat: 2 runs per quant only, so single-run variance will move these. Prefill speed of ThinkingCap is oddly low. Need to do some more tests on that. Side-by-side (thinking xhigh, Q4\_K\_M, all 7900 XTX) of 3 of the runs: https://llm-bench.io/compare/runs?runs=cmufxkpkb00bj01o05h9vsu5o,cmufwold900ay01o0ae1frktu,cmufyiq3p00by01o0vwbpilew