Swift-1.5-Qwen3.8-27b-oQ8e-mtp on Apple M5 Max — 34.8 tok/s — llm-bench.io
I ran Swift-1.5-Qwen3.8-27b-oQ8e-mtp through the llm-bench.io a few times today: oMLX on M5 Max 64 GB, thinking on at xhigh, 262k context window.
The big difference between Swift 1.5 and the base Qwen3.8 27B is how much it writes. Per full run (agent workflow, code generation, research, role play) Swift averages 51k generated tokens and Qwen3.8 averages 77k.
| Scenario|Swift 1.5|Qwen 3.8 27B|
|:-|:-|:-|
|Code generation|28.6k|40.1k|
|Research|11.1k|21.2k|
|Agent workflow|7.5k|11.9k|
|Role play|4.0k|3.8k|
The full benchmark run duration: avg. 24 min for Swift 1.5, avg. 38 min for base Qwen 3.8 27B
Everything else is about equal:
- generation speed: 34.6 vs 32.8 tok/s
- prompt processing: around 400 tok/s for both
- quality score (the site's LLM judge): 85.8 vs 85.0. My four runs range from 84.3 to 87.2, so well within the expected variance of the llm judge. I'd call it a tie.
Still 3/4 of what Swift generates is reasoning, it just does less than Qwen 3.8 27B. The output is still very usable. I'll for sure give it a try to be my daily driver for a few day.
Runs Swift 1.5:
- https://llm-bench.io/benchmarks/cmumgubnk004o01o00kqgaqcv
- https://llm-bench.io/benchmarks/cmumilzxq004v01o0husl2vcx
- https://llm-bench.io/benchmarks/cmumjhwz6005201o0w2irsf21
- https://llm-bench.io/benchmarks/cmumkfc4x005901o0nt9a726k
Runs Qwen 3.8 27B: