LLM Inference Benchmark
NB480 · Windows 10
CPU: Intel64 Family 6 Model 154 Stepping 4, GenuineIntel (12 cores) · RAM: 16 GB
GPU: no NVIDIA GPU detected
CPU: Intel64 Family 6 Model 154 Stepping 4, GenuineIntel (12 cores) · RAM: 16 GB
GPU: no NVIDIA GPU detected
Per-concurrency detail (latency percentiles)
Efficiency = actual total throughput / perfect scaling (concurrency x single-request speed). 100% = ideal scaling; dropping = losses; flat = saturation.
| Model | Conc. | Gen tok/s | Total tok/s | Efficiency | TTFR p50 ms | TTFR p95 ms | TTFR p99 ms |
|---|---|---|---|---|---|---|---|
| qwen2.5:0.5b | 1 | 0.0 | 0.0 | 100% | 348 | 355 | 356 |
| qwen2.5:0.5b | 2 | 0.0 | 0.0 | 100% | 436 | 494 | 498 |
| qwen2.5:0.5b | 4 | 0.0 | 0.0 | 100% | 546 | 675 | 707 |
| qwen2.5-coder:7b | 1 | 0.0 | 0.0 | 100% | 780 | 797 | 799 |
| qwen2.5-coder:7b | 2 | 0.0 | 0.0 | 100% | 913 | 1153 | 1160 |
| qwen2.5-coder:7b | 4 | 0.0 | 0.0 | 100% | 1598 | 2488 | 2556 |
Token load
| Model | Prompt (tok) | Result (tok) | Tokenizer |
|---|---|---|---|
| qwen2.5:0.5b | 11 | 0 | tiktoken:cl100k_base |
| qwen2.5-coder:7b | 11 | 0 | tiktoken:cl100k_base |
Direct comparison (key metrics)
| Model | Prompt throughput tok/s | Time to first response ms |
|---|---|---|
| qwen2.5:0.5b | 4.45 | 445.61 |
| qwen2.5-coder:7b | 1.59 | 1108.88 |
Dashboard
