Dual Radeon improved speeds using Vulkan
I've been struggling to keep my Radeon Instinct MI50 GPU cool. I'm looking for budget friendly solutions. While running multiple GPUs it doesn't usually get too hot. I was also getting lower benchmarks using standard Vulkan 'llama 7B Q4\_0' model benchmark, but it wasn't caused by thermal throttling. Time to optimize. MI50 with Radeon VII firmware 16GB Vram My previous post I tested several model using same GPUs. I made some changes. I moved the MI50 16gb into the primary PCIe 16x slot and moved the RX 7900 GRE 16gb into a slower PCIe 4x slot. Overall system inference performance increased. I used Google Gemini to helped my optimize my llama-bench settings and it taught be about: RADV_PERFTEST=nogttspill is an AMD Linux driver flag used when running llama.cpp with the Vulkan backend. It forces the RADV (Mesa Vulkan) driver to prioritize keeping all model allocations inside dedicated video memory (VRAM) rather than spilling over into system RAM (GTT/Graphics Translation Table). \[1, 2, 3\] I saw llama 7B Q4\_0 score jump back to where is it should be. So I tested a few other models. GGML_VK_VISIBLE_DEVICES=0,1 RADV_PERFTEST=nogttspill time /llama-b11053/llama-bench -fa on -ngl 99 -m /llama-2-7b.Q4_0.gguf Previous benchmarks: https://www.reddit.com/r/LocalLLM/s/PT6Bd5nUUE see end for comparison The list has been sorted by pp512 improvement in descending order (highest gain to highest loss). |Model Name|Improvement (pp512)|Improvement (tg128)| |:-|:-|:-| |Laguna-XS-2.1-APEX-I-Balanced.gguf|\+402.14%|\-0.20%| |gemma-4-31B-it-Q6\_K.gguf|\+397.84%|\+9.24%| |Muse-Glimmer-30B-UD-Q6\_K\_XL.gguf|\+386.58%|\+48.63%| |llama-2-7b.Q4\_0.gguf|\+176.57%|\+20.29%| |Qwen3.8-27B-Q6\_K.gguf|\+48.13%|\+0.07%| |medgemma-27b-it-UD-Q6\_K\_XL.gguf|\+37.89%|\-0.52%| |Qwen3.6-35B-A3B-UD-Q5\_K\_XL.gguf|\+1.17%|\+0.49%| |Gemma4-26B-A4B-QAT-Uncensored-HauhauCS-Balanced-Q4\_K\_M.gguf|\+0.68%|\+4.16%| |NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Q5\_K\_M.gguf|\+0.28%|\-4.28%| |granite-4.2-30b-Q6\_K\_L.gguf|\-0.12%|0.00%| |GLM-4.7-Flash-Uncen-Hrt-NEO-CODE-MAX-imt-D\_AU-Q6\_K.gguf|\-0.63%|\-7.83%| |Qwen3-Coder-30B-A3B-Instruct-UD-Q6\_K\_XL.gguf|\-0.69%|\-0.86%| |Accio-Lab\_occamy-1.0-Q5\_K\_S.gguf|\-0.81%|\+0.17%| These are the models tested in the same order as tables below: GGUF Model List (in order): 1. Qwen3.6-35B-A3B-UD-Q5\_K\_XL.gguf 2. Accio-Lab\_occamy-1.0-Q5\_K\_S.gguf 3. Laguna-XS-2.1-APEX-I-Balanced.gguf 4. NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Q5\_K\_M.gguf 5. gemma-4-31B-it-Q6\_K.gguf 6. Qwen3-Coder-30B-A3B-Instruct-UD-Q6\_K\_XL.gguf 7. GLM-4.7-Flash-Uncen-Hrt-NEO-CODE-MAX-imt-D\_AU-Q6\_K.gguf 8. granite-4.2-30b-Q6\_K\_L.gguf 9. Muse-Glimmer-30B-UD-Q6\_K\_XL.gguf 10. Qwen3.8-27B-Q6\_K.gguf 11. medgemma-27b-it-UD-Q6\_K\_XL.gguf 12. Gemma4-26B-A4B-QAT-Uncensored-HauhauCS-Balanced-Q4\_K\_M.gguf 13. llama-2-7b.Q4\_0.gguf Supporting data sorted order (by Params descending, then Size descending). All models running dual Radeon GPU, Vulkan backend, and flash attention on. # Table 1: RADV_PERFTEST=nogttspill is being used |model|size|params|pp512|tg128| |:-|:-|:-|:-|:-| |qwen35moe 35B.A3B Q5\_K - Medium|24.76 GiB|34.66 B|1252.09 ± 11.84|53.54 ± 0.51| |qwen35moe 35B.A3B Q5\_K - Small|23.40 GiB|34.66 B|1245.11 ± 11.76|53.42 ± 0.28| |laguna 30B.A3B Q5\_K - Medium|22.64 GiB|33.44 B|1027.42 ± 11.32|65.72 ± 0.51| |nemotron\_h\_moe 31B.A3.5B Q5\_K - Medium|25.10 GiB|32.91 B|1120.31 ± 12.85|63.82 ± 1.52| |gemma4 31B Q6\_K|23.46 GiB|30.70 B|200.33 ± 0.20|15.61 ± 0.05| |qwen3moe 30B.A3B Q6\_K|24.53 GiB|30.53 B|1176.48 ± 36.62|66.04 ± 0.25| |deepseek2 30B.A3B Q6\_K|23.26 GiB|29.94 B|914.86 ± 8.75|40.02 ± 0.12| |granite 30B Q6\_K|23.01 GiB|29.28 B|92.09 ± 0.30|17.41 ± 0.03| |muse-glimmer 30B Q6\_K|24.45 GiB|27.85 B|279.04 ± 0.20|17.42 ± 0.03| |qwen35 27B Q6\_K|21.30 GiB|27.32 B|235.12 ± 1.05|14.42 ± 0.03| |gemma3 27B Q6\_K|22.09 GiB|27.01 B|247.18 ± 0.54|17.30 ± 0.13| |gemma4 26B.A4B Q4\_K - Medium|15.63 GiB|25.23 B|1316.62 ± 24.18|55.56 ± 0.16| |llama 7B Q4\_0|3.56 GiB|6.74 B|1349.35 ± 15.66|74.62 ± 0.37| # Table 2: RADV_PERFTEST=nogttspill is NOT being used |model|size|params|pp512|tg128| |:-|:-|:-|:-|:-| |qwen35moe 35B.A3B Q5\_K - Medium|24.76 GiB|34.66 B|1237.55 ± 25.29|53.28 ± 0.24| |qwen35moe 35B.A3B Q5\_K - Small|23.40 GiB|34.66 B|1255.24 ± 6.03|53.33 ± 0.48| |laguna 30B.A3B Q5\_K - Medium|22.64 GiB|33.44 B|204.55 ± 1.66|65.85 ± 0.29| |nemotron\_h\_moe 31B.A3.5B Q5\_K - Medium|25.10 GiB|32.91 B|1117.17 ± 10.11|66.68 ± 0.24| |gemma4 31B Q6\_K|23.46 GiB|30.70 B|40.22 ± 0.12|14.29 ± 0.03| |qwen3moe 30B.A3B Q6\_K|24.53 GiB|30.53 B|1184.70 ± 27.98|66.61 ± 0.27| |deepseek2 30B.A3B Q6\_K|23.26 GiB|29.94 B|920.67 ± 5.82|43.42 ± 0.09| |granite 30B Q6\_K|23.01 GiB|29.28 B|92.20 ± 0.21|17.41 ± 0.04| |muse-glimmer 30B Q6\_K|24.45 GiB|27.85 B|57.15 ± 0.87|11.69 ± 0.09| |qwen35 27B Q6\_K|21.30 GiB|27.32 B|158.73 ± 0.57|14.41 ± 0.41| |gemma3 27B Q6\_K|22.09 GiB|27.01 B|179.26 ± 0.44|17.39 ± 0.02| |gemma4 26B.A4B Q4\_K - Medium|15.63 GiB|25.23 B|1307.73 ± 23.51|53.34 ± 0.22| |llama 7B Q4\_0|3.56 GiB|6.74 B|487.92 ± 37.05|62.03 ± 0.60| Swapping PCIe locations for the Radeon Instinct MI50 and Radeon RX 7900 GRE and using RADV_PERFTEST=nogttspill flag resulted in improvements over my first baseline benchmarks. Note: Only models appearing in both datasets are listed. The table is sorted by Parameters in descending order. |Model Name|Improvement (pp512)|Improvement (tg128)| |:-|:-|:-| |qwen35moe 35B.A3B Q5\_K - Medium|\+221.14%|\+33.85%| |qwen35moe 35B.A3B Q5\_K - Small|\+215.17%|\+1.25%| |laguna 30B.A3B Q5\_K - Medium|\+537.77%|\+17.25%| |nemotron\_h\_moe 31B.A3.5B Q5\_K - Medium|\+267.75%|\-3.90%| |gemma4 31B Q6\_K|\+17.00%|\+23.60%| |qwen3moe 30B.A3B Q6\_K|\+211.32%|\-0.97%| |deepseek2 30B.A3B Q6\_K|\+191.00%|\-14.54%| |muse-glimmer 30B Q6\_K|\+12.49%|\+0.17%| |qwen35 27B Q6\_K|\+17.00%|\-6.79%| |gemma3 27B Q6\_K|\+14.83%|\+1.71%| |gemma4 26B.A4B Q4\_K - Medium|\+158.79%|\+7.28%| Looks like MoE models benefit the most. Muse-glimmer 30B Q6\_K didn't real see much improvement but it seems to be the most optimized dense model. MI50 continues to impress. I purchased them used at $150 each. I can now run models 30B to 35B using Q6 quant with long content and decent speed.