Files
lsf-bench/results.md
2025-10-09 22:45:46 +00:00

258 lines
9.8 KiB
Markdown

| Rank | Model | Weighted % | Total Count | Sum Total |
|---:|---|---:|---:|---:|
| 1 | gpt-oss:20b | 89.1% | 49 | 733 |
| 2 | hf.co/BasedBase/Qwen3-Coder-30B-A3B-Instruct-480B-Distill-V2:Q4_K_M | 88.2% | 49 | 726 |
| 3 | hf.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_S | 85.9% | 49 | 707 |
| 4 | hf.co/mradermacher/aquif-3.5-8B-Think-GGUF:Q6_K | 82.5% | 49 | 679 |
| 5 | hf.co/mradermacher/aquif-3.5-7B-GGUF:Q8_0 | 79.2% | 49 | 652 |
| 6 | hf.co/unsloth/Qwen3-4B-Instruct-2507-GGUF:Q8_0 | 78.9% | 49 | 649 |
| 7 | hf.co/unsloth/Qwen2.5-Coder-7B-Instruct-GGUF:Q8_0 | 76.7% | 49 | 631 |
| 8 | hf.co/unsloth/Qwen3-4B-Instruct-2507-GGUF:F16 | 75.0% | 49 | 617 |
| 9 | hf.co/TorpedoSoftware/Luau-Devstral-24B-Instruct-v0.1:Q4_K_M | 73.8% | 49 | 607 |
| 10 | hf.co/bartowski/NousResearch_Hermes-4-14B-GGUF:Q4_K_M | 73.6% | 49 | 606 |
| 11 | hf.co/mradermacher/aquif-3-moe-17b-a2.8b-thinking-GGUF:Q4_K_M | 71.9% | 49 | 592 |
| 12 | hf.co/unsloth/gemma-3n-E4B-it-GGUF:Q8_0 | 66.0% | 49 | 543 |
| 13 | hf.co/unsloth/Phi-4-mini-reasoning-GGUF:Q8_0 | 65.5% | 49 | 539 |
| 14 | hf.co/mradermacher/aquif-3.5-3B-GGUF:F16 | 60.6% | 49 | 499 |
| 15 | hf.co/unsloth/gemma-3-12b-it-GGUF:Q4_K_M | 56.3% | 49 | 463 |
| 16 | hf.co/unsloth/gemma-3n-E2B-it-GGUF:Q8_0 | 49.9% | 49 | 411 |
| 17 | hf.co/unsloth/Qwen3-0.6B-GGUF:BF16 | 45.8% | 49 | 377 |
| 18 | hf.co/unsloth/gemma-3-4b-it-GGUF:Q8_0 | 44.2% | 49 | 364 |
| 19 | hf.co/bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0 | 42.2% | 49 | 347 |
| 20 | hf.co/unsloth/gemma-3-1b-it-GGUF:BF16 | 35.0% | 49 | 288 |
| 21 | hf.co/mradermacher/Gemma-3-1B-Roblox-Luau-GGUF:F16 | 19.7% | 49 | 162 |
---
## Per-Category Stats
### 1) gpt-oss:20b
- **Aggregated:** 89.1% - **Count:** 49 - **Sum Total:** 733
| Category | % | Count | Total | Max |
|---|---:|---:|---:|---:|
| syntax | 100.0% | 8 | 95 | 95 |
| types | 97.5% | 11 | 193 | 198 |
| compatibility | 95.2% | 8 | 100 | 105 |
| advanced | 91.3% | 8 | 105 | 115 |
| internals | 86.0% | 8 | 185 | 215 |
| runtimes | 57.9% | 6 | 55 | 95 |
### 2) hf.co/BasedBase/Qwen3-Coder-30B-A3B-Instruct-480B-Distill-V2:Q4_K_M
- **Aggregated:** 88.2% - **Count:** 49 - **Sum Total:** 726
| Category | % | Count | Total | Max |
|---|---:|---:|---:|---:|
| syntax | 100.0% | 8 | 95 | 95 |
| types | 95.5% | 11 | 189 | 198 |
| advanced | 93.0% | 8 | 107 | 115 |
| internals | 88.4% | 8 | 190 | 215 |
| runtimes | 78.9% | 6 | 75 | 95 |
| compatibility | 66.7% | 8 | 70 | 105 |
### 3) hf.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_S
- **Aggregated:** 85.9% - **Count:** 49 - **Sum Total:** 707
| Category | % | Count | Total | Max |
|---|---:|---:|---:|---:|
| syntax | 100.0% | 8 | 95 | 95 |
| types | 90.4% | 11 | 179 | 198 |
| compatibility | 88.6% | 8 | 93 | 105 |
| advanced | 87.0% | 8 | 100 | 115 |
| internals | 83.7% | 8 | 180 | 215 |
| runtimes | 63.2% | 6 | 60 | 95 |
### 4) hf.co/mradermacher/aquif-3.5-8B-Think-GGUF:Q6_K
- **Aggregated:** 82.5% - **Count:** 49 - **Sum Total:** 679
| Category | % | Count | Total | Max |
|---|---:|---:|---:|---:|
| advanced | 91.3% | 8 | 105 | 115 |
| internals | 88.4% | 8 | 190 | 215 |
| types | 83.3% | 11 | 165 | 198 |
| syntax | 80.0% | 8 | 76 | 95 |
| compatibility | 79.0% | 8 | 83 | 105 |
| runtimes | 63.2% | 6 | 60 | 95 |
### 5) hf.co/mradermacher/aquif-3.5-7B-GGUF:Q8_0
- **Aggregated:** 79.2% - **Count:** 49 - **Sum Total:** 652
| Category | % | Count | Total | Max |
|---|---:|---:|---:|---:|
| internals | 85.1% | 8 | 183 | 215 |
| types | 83.8% | 11 | 166 | 198 |
| advanced | 82.6% | 8 | 95 | 115 |
| syntax | 82.1% | 8 | 78 | 95 |
| compatibility | 76.2% | 8 | 80 | 105 |
| runtimes | 52.6% | 6 | 50 | 95 |
### 6) hf.co/unsloth/Qwen3-4B-Instruct-2507-GGUF:Q8_0
- **Aggregated:** 78.9% - **Count:** 49 - **Sum Total:** 649
| Category | % | Count | Total | Max |
|---|---:|---:|---:|---:|
| syntax | 89.5% | 8 | 85 | 95 |
| internals | 87.4% | 8 | 188 | 215 |
| advanced | 82.6% | 8 | 95 | 115 |
| types | 78.8% | 11 | 156 | 198 |
| compatibility | 71.4% | 8 | 75 | 105 |
| runtimes | 52.6% | 6 | 50 | 95 |
### 7) hf.co/unsloth/Qwen2.5-Coder-7B-Instruct-GGUF:Q8_0
- **Aggregated:** 76.7% - **Count:** 49 - **Sum Total:** 631
| Category | % | Count | Total | Max |
|---|---:|---:|---:|---:|
| syntax | 94.7% | 8 | 90 | 95 |
| advanced | 93.9% | 8 | 108 | 115 |
| types | 84.3% | 11 | 167 | 198 |
| compatibility | 71.4% | 8 | 75 | 105 |
| internals | 70.2% | 8 | 151 | 215 |
| runtimes | 42.1% | 6 | 40 | 95 |
### 8) hf.co/unsloth/Qwen3-4B-Instruct-2507-GGUF:F16
- **Aggregated:** 75.0% - **Count:** 49 - **Sum Total:** 617
| Category | % | Count | Total | Max |
|---|---:|---:|---:|---:|
| syntax | 94.7% | 8 | 90 | 95 |
| advanced | 82.6% | 8 | 95 | 115 |
| internals | 79.1% | 8 | 170 | 215 |
| compatibility | 71.4% | 8 | 75 | 105 |
| types | 69.2% | 11 | 137 | 198 |
| runtimes | 52.6% | 6 | 50 | 95 |
### 9) hf.co/TorpedoSoftware/Luau-Devstral-24B-Instruct-v0.1:Q4_K_M
- **Aggregated:** 73.8% - **Count:** 49 - **Sum Total:** 607
| Category | % | Count | Total | Max |
|---|---:|---:|---:|---:|
| compatibility | 83.8% | 8 | 88 | 105 |
| advanced | 78.3% | 8 | 90 | 115 |
| types | 77.8% | 11 | 154 | 198 |
| runtimes | 74.7% | 6 | 71 | 95 |
| syntax | 74.7% | 8 | 71 | 95 |
| internals | 61.9% | 8 | 133 | 215 |
### 10) hf.co/bartowski/NousResearch_Hermes-4-14B-GGUF:Q4_K_M
- **Aggregated:** 73.6% - **Count:** 49 - **Sum Total:** 606
| Category | % | Count | Total | Max |
|---|---:|---:|---:|---:|
| syntax | 82.1% | 8 | 78 | 95 |
| types | 80.8% | 11 | 160 | 198 |
| advanced | 78.3% | 8 | 90 | 115 |
| internals | 75.8% | 8 | 163 | 215 |
| runtimes | 57.9% | 6 | 55 | 95 |
| compatibility | 57.1% | 8 | 60 | 105 |
### 11) hf.co/mradermacher/aquif-3-moe-17b-a2.8b-thinking-GGUF:Q4_K_M
- **Aggregated:** 71.9% - **Count:** 49 - **Sum Total:** 592
| Category | % | Count | Total | Max |
|---|---:|---:|---:|---:|
| compatibility | 81.0% | 8 | 85 | 105 |
| internals | 75.3% | 8 | 162 | 215 |
| syntax | 74.7% | 8 | 71 | 95 |
| advanced | 73.9% | 8 | 85 | 115 |
| types | 68.7% | 11 | 136 | 198 |
| runtimes | 55.8% | 6 | 53 | 95 |
### 12) hf.co/unsloth/gemma-3n-E4B-it-GGUF:Q8_0
- **Aggregated:** 66.0% - **Count:** 49 - **Sum Total:** 543
| Category | % | Count | Total | Max |
|---|---:|---:|---:|---:|
| syntax | 76.8% | 8 | 73 | 95 |
| advanced | 73.0% | 8 | 84 | 115 |
| internals | 73.0% | 8 | 157 | 215 |
| types | 59.6% | 11 | 118 | 198 |
| runtimes | 55.8% | 6 | 53 | 95 |
| compatibility | 55.2% | 8 | 58 | 105 |
### 13) hf.co/unsloth/Phi-4-mini-reasoning-GGUF:Q8_0
- **Aggregated:** 65.5% - **Count:** 49 - **Sum Total:** 539
| Category | % | Count | Total | Max |
|---|---:|---:|---:|---:|
| syntax | 76.8% | 8 | 73 | 95 |
| internals | 76.7% | 8 | 165 | 215 |
| types | 70.7% | 11 | 140 | 198 |
| runtimes | 55.8% | 6 | 53 | 95 |
| advanced | 50.4% | 8 | 58 | 115 |
| compatibility | 47.6% | 8 | 50 | 105 |
### 14) hf.co/mradermacher/aquif-3.5-3B-GGUF:F16
- **Aggregated:** 60.6% - **Count:** 49 - **Sum Total:** 499
| Category | % | Count | Total | Max |
|---|---:|---:|---:|---:|
| advanced | 85.2% | 8 | 98 | 115 |
| compatibility | 60.0% | 8 | 63 | 105 |
| syntax | 63.2% | 8 | 60 | 95 |
| types | 63.1% | 11 | 125 | 198 |
| runtimes | 50.5% | 6 | 48 | 95 |
| internals | 48.8% | 8 | 105 | 215 |
### 15) hf.co/unsloth/gemma-3-12b-it-GGUF:Q4_K_M
- **Aggregated:** 56.3% - **Count:** 49 - **Sum Total:** 463
| Category | % | Count | Total | Max |
|---|---:|---:|---:|---:|
| syntax | 88.4% | 8 | 84 | 95 |
| types | 61.1% | 11 | 121 | 198 |
| internals | 60.5% | 8 | 130 | 215 |
| advanced | 53.0% | 8 | 61 | 115 |
| compatibility | 40.0% | 8 | 42 | 105 |
| runtimes | 26.3% | 6 | 25 | 95 |
### 16) hf.co/unsloth/gemma-3n-E2B-it-GGUF:Q8_0
- **Aggregated:** 49.9% - **Count:** 49 - **Sum Total:** 411
| Category | % | Count | Total | Max |
|---|---:|---:|---:|---:|
| syntax | 62.1% | 8 | 59 | 95 |
| internals | 54.4% | 8 | 117 | 215 |
| advanced | 48.7% | 8 | 56 | 115 |
| types | 47.5% | 11 | 94 | 198 |
| compatibility | 44.8% | 8 | 47 | 105 |
| runtimes | 40.0% | 6 | 38 | 95 |
### 17) hf.co/unsloth/Qwen3-0.6B-GGUF:BF16
- **Aggregated:** 45.8% - **Count:** 49 - **Sum Total:** 377
| Category | % | Count | Total | Max |
|---|---:|---:|---:|---:|
| advanced | 60.9% | 8 | 70 | 115 |
| internals | 49.8% | 8 | 107 | 215 |
| compatibility | 45.7% | 8 | 48 | 105 |
| types | 42.4% | 11 | 84 | 198 |
| runtimes | 36.8% | 6 | 35 | 95 |
| syntax | 34.7% | 8 | 33 | 95 |
### 18) hf.co/unsloth/gemma-3-4b-it-GGUF:Q8_0
- **Aggregated:** 44.2% - **Count:** 49 - **Sum Total:** 364
| Category | % | Count | Total | Max |
|---|---:|---:|---:|---:|
| syntax | 58.9% | 8 | 56 | 95 |
| advanced | 53.0% | 8 | 61 | 115 |
| types | 47.5% | 11 | 94 | 198 |
| internals | 44.2% | 8 | 95 | 215 |
| compatibility | 31.4% | 8 | 33 | 105 |
| runtimes | 26.3% | 6 | 25 | 95 |
### 19) hf.co/bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0
- **Aggregated:** 42.2% - **Count:** 49 - **Sum Total:** 347
| Category | % | Count | Total | Max |
|---|---:|---:|---:|---:|
| advanced | 65.2% | 8 | 75 | 115 |
| syntax | 46.3% | 8 | 44 | 95 |
| internals | 40.5% | 8 | 87 | 215 |
| runtimes | 37.9% | 6 | 36 | 95 |
| types | 35. suspension% | 11 | 70 | 198 |
| compatibility | 33.3% | 8 | 35 | 105 |
### 20) hf.co/unsloth/gemma-3-1b-it-GGUF:BF16
- **Aggregated:** 35.0% - **Count:** 49 - **Sum Total:** 288
| Category | % | Count | Total | Max |
|---|---:|---:|---:|---:|
| advanced | 43.5% | 8 | 50 | 115 |
| runtimes | 38.9% | 6 | 37 | 95 |
| internals | 37.2% | 8 | 80 | 215 |
| types | 33.3% | 11 | 66 | 198 |
| syntax | 31.6% | 8 | 30 | 95 |
| compatibility | 23.8% | 8 | 25 | 105 |
### 21) hf.co/mradermacher/Gemma-3-1B-Roblox-Luau-GGUF:F16
- **Aggregated:** 19.7% - **Count:** 49 - **Sum Total:** 162
| Category | % | Count | Total | Max |
|---|---:|---:|---:|---:|
| advanced | 38.3% | 8 | 44 | 115 |
| syntax | 26.3% | 8 | 25 | 95 |
| runtimes | 21.1% | 6 | 20 | 95 |
| compatibility | 19.0% | 8 | 20 | 105 |
| types | 15.7% | 11 | 31 | 198 |
| internals | 10.2% | 8 | 22 | 215 |