Compare across machines

Join key is the model family (weights), not the target id. vLLM is the same family as llama.cpp on Linux, extra column — not a different model. Laguna is not joined: Mac laguna is XS.2, Linux is official XS-2.1.

A cell is shown only when that machine's run used the shared profile: temperature 0, seed 42, one 16384-token slot, C as -std=gnu11. Runs that differ in stack, context, or sampling are omitted. The raw files are still in each results directory.

Not comparable across machines (kept on the per-machine reports): tok/s, RSS, perplexity, concurrency throughput, generated-code wall-clock. Quant and serving stack can change quality — that difference is the point of this page.

Coding total

model	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
aya-23 35b llama.cpp Metal · llama.cpp CUDA · llama.cpp Vulkan	66/126	61/126	73/126	73/126	64/126
qwen3.5-35b llama.cpp Metal · llama.cpp CUDA · llama.cpp Vulkan	96/126	105/126	96/126	96/126	95/126
qwen3.8-flash-next 125B llama.cpp Metal · llama.cpp CUDA · llama.cpp Vulkan	108/126	117/126	111/126	111/126	115/126
glm-4.7-flash llama.cpp Metal · llama.cpp CUDA · llama.cpp Vulkan	94/126	94/126	96/126	96/126	103/126
mistral-small-4 119B llama.cpp Metal · llama.cpp CUDA · llama.cpp Vulkan	91/126	87/126	83/126	83/126	82/126
qwen3-coder-next 80B llama.cpp Metal · llama.cpp CUDA · llama.cpp Vulkan	99/126	99/126	108/126	108/126	105/126
qwen3.5-27b llama.cpp Metal · llama.cpp CUDA · llama.cpp Vulkan	87/126	93/126	96/126	96/126	93/126
qwen3.6-35b llama.cpp Metal · llama.cpp CUDA · llama.cpp Vulkan	96/126	96/126	102/126	102/126	94/126
qwen3-coder-30b llama.cpp Metal · llama.cpp CUDA · llama.cpp Vulkan	86/126	93/126	90/126	90/126	87/126
seed-oss 36b llama.cpp Metal · llama.cpp CUDA · llama.cpp Vulkan	79/126	82/126	81/126	81/126	86/126
laguna-s 2.1 117B llama.cpp Metal · llama.cpp CUDA · llama.cpp Vulkan	101/126	108/126	108/126	108/126	107/126
gpt-oss-120b llama.cpp Metal · llama.cpp CUDA · llama.cpp Vulkan	106/126	111/126	107/126	107/126	109/126
k2-horizon 36B-A4B llama.cpp Metal · llama.cpp CUDA · llama.cpp Vulkan	86/126	91/126	88/126	88/126	91/126
devstral-2 24b llama.cpp Metal · llama.cpp CUDA · llama.cpp Vulkan	94/126	96/126	98/126	98/126	96/126
laguna-xs-2.1 llama.cpp CUDA · llama.cpp Vulkan	—	105/126	101/126	102/126	102/126
gemma-4-26b llama.cpp Metal · llama.cpp CUDA · llama.cpp Vulkan	102/126	105/126	102/126	102/126	105/126
nemotron-3-super 120B llama.cpp Metal · llama.cpp CUDA · llama.cpp Vulkan	102/126	105/126	105/126	105/126	104/126
qwen3.5-122b-a10b llama.cpp Metal · llama.cpp CUDA · llama.cpp Vulkan	90/126	93/126	93/126	93/126	93/126
qwen3.8-27b llama.cpp Metal · llama.cpp CUDA · llama.cpp Vulkan	102/126	102/126	102/126	102/126	105/126
gpt-oss-20b llama.cpp Metal · llama.cpp CUDA · llama.cpp Vulkan	104/126	102/126	102/126	102/126	104/126
qwen3.6-27b llama.cpp Metal · llama.cpp CUDA · llama.cpp Vulkan	99/126	99/126	99/126	99/126	100/126

By suite

aya-23 35b

suite	M5 Max · llama.cpp Metal	RTX PRO 6000 · llama.cpp CUDA	GB10 · llama.cpp CUDA	GB10-2 · llama.cpp CUDA	Radeon 8060S · llama.cpp Vulkan
C	27/48	25/48	37/48	37/48	27/48
Python	21/24	21/24	21/24	21/24	21/24
Bash	15/24	12/24	12/24	12/24	12/24
C (hard)	0/9	0/9	0/9	0/9	0/9
Python (hard)	3/9	3/9	3/9	3/9	3/9
Bash (hard)	0/9	0/9	0/9	0/9	0/9
Research	0/3	0/3	0/3	0/3	1/3

qwen3.5-35b

suite	M5 Max · llama.cpp Metal	RTX PRO 6000 · llama.cpp CUDA	GB10 · llama.cpp CUDA	GB10-2 · llama.cpp CUDA	Radeon 8060S · llama.cpp Vulkan
C	36/48	42/48	39/48	39/48	36/48
Python	24/24	24/24	21/24	21/24	24/24
Bash	15/24	18/24	18/24	18/24	16/24
C (hard)	6/9	6/9	3/9	3/9	5/9
Python (hard)	9/9	9/9	9/9	9/9	8/9
Bash (hard)	3/9	3/9	3/9	3/9	3/9
Research	3/3	3/3	3/3	3/3	3/3

qwen3.8-flash-next 125B

suite	M5 Max · llama.cpp Metal	RTX PRO 6000 · llama.cpp CUDA	GB10 · llama.cpp CUDA	GB10-2 · llama.cpp CUDA	Radeon 8060S · llama.cpp Vulkan
C	48/48	48/48	45/48	45/48	46/48
Python	24/24	24/24	24/24	24/24	24/24
Bash	15/24	18/24	18/24	18/24	20/24
C (hard)	6/9	9/9	6/9	6/9	8/9
Python (hard)	9/9	9/9	9/9	9/9	8/9
Bash (hard)	3/9	6/9	6/9	6/9	6/9
Research	3/3	3/3	3/3	3/3	3/3

glm-4.7-flash

suite	M5 Max · llama.cpp Metal	RTX PRO 6000 · llama.cpp CUDA	GB10 · llama.cpp CUDA	GB10-2 · llama.cpp CUDA	Radeon 8060S · llama.cpp Vulkan
C	42/48	45/48	45/48	45/48	45/48
Python	24/24	24/24	24/24	24/24	24/24
Bash	15/24	15/24	18/24	18/24	15/24
C (hard)	3/9	3/9	1/9	1/9	6/9
Python (hard)	7/9	3/9	3/9	3/9	7/9
Bash (hard)	0/9	1/9	3/9	3/9	3/9
Research	3/3	3/3	2/3	2/3	3/3

mistral-small-4 119B

suite	M5 Max · llama.cpp Metal	RTX PRO 6000 · llama.cpp CUDA	GB10 · llama.cpp CUDA	GB10-2 · llama.cpp CUDA	Radeon 8060S · llama.cpp Vulkan
C	38/48	35/48	33/48	33/48	37/48
Python	24/24	22/24	22/24	22/24	21/24
Bash	15/24	15/24	14/24	14/24	12/24
C (hard)	7/9	7/9	5/9	5/9	6/9
Python (hard)	3/9	3/9	3/9	3/9	3/9
Bash (hard)	3/9	3/9	3/9	3/9	3/9
Research	1/3	2/3	3/3	3/3	0/3

qwen3-coder-next 80B

suite	M5 Max · llama.cpp Metal	RTX PRO 6000 · llama.cpp CUDA	GB10 · llama.cpp CUDA	GB10-2 · llama.cpp CUDA	Radeon 8060S · llama.cpp Vulkan
C	42/48	39/48	45/48	45/48	42/48
Python	21/24	21/24	21/24	21/24	21/24
Bash	15/24	18/24	21/24	21/24	20/24
C (hard)	6/9	6/9	6/9	6/9	6/9
Python (hard)	6/9	6/9	6/9	6/9	8/9
Bash (hard)	6/9	6/9	6/9	6/9	5/9
Research	3/3	3/3	3/3	3/3	3/3

qwen3.5-27b

suite	M5 Max · llama.cpp Metal	RTX PRO 6000 · llama.cpp CUDA	GB10 · llama.cpp CUDA	GB10-2 · llama.cpp CUDA	Radeon 8060S · llama.cpp Vulkan
C	33/48	30/48	36/48	36/48	33/48
Python	24/24	24/24	24/24	24/24	24/24
Bash	12/24	15/24	15/24	15/24	15/24
C (hard)	0/9	6/9	3/9	3/9	3/9
Python (hard)	9/9	9/9	9/9	9/9	9/9
Bash (hard)	6/9	6/9	6/9	6/9	6/9
Research	3/3	3/3	3/3	3/3	3/3

qwen3.6-35b

suite	M5 Max · llama.cpp Metal	RTX PRO 6000 · llama.cpp CUDA	GB10 · llama.cpp CUDA	GB10-2 · llama.cpp CUDA	Radeon 8060S · llama.cpp Vulkan
C	42/48	39/48	42/48	42/48	39/48
Python	21/24	21/24	21/24	21/24	21/24
Bash	15/24	18/24	18/24	18/24	18/24
C (hard)	3/9	6/9	9/9	9/9	6/9
Python (hard)	9/9	9/9	6/9	6/9	6/9
Bash (hard)	3/9	3/9	3/9	3/9	3/9
Research	3/3	0/3	3/3	3/3	1/3

qwen3-coder-30b

suite	M5 Max · llama.cpp Metal	RTX PRO 6000 · llama.cpp CUDA	GB10 · llama.cpp CUDA	GB10-2 · llama.cpp CUDA	Radeon 8060S · llama.cpp Vulkan
C	36/48	36/48	39/48	39/48	36/48
Python	24/24	24/24	24/24	24/24	24/24
Bash	9/24	12/24	12/24	12/24	14/24
C (hard)	6/9	6/9	6/9	6/9	6/9
Python (hard)	6/9	9/9	6/9	6/9	5/9
Bash (hard)	2/9	3/9	0/9	0/9	0/9
Research	3/3	3/3	3/3	3/3	2/3

seed-oss 36b

suite	M5 Max · llama.cpp Metal	RTX PRO 6000 · llama.cpp CUDA	GB10 · llama.cpp CUDA	GB10-2 · llama.cpp CUDA	Radeon 8060S · llama.cpp Vulkan
C	32/48	29/48	30/48	30/48	34/48
Python	24/24	24/24	24/24	24/24	24/24
Bash	11/24	17/24	20/24	20/24	17/24
C (hard)	3/9	3/9	3/9	3/9	2/9
Python (hard)	3/9	4/9	3/9	3/9	4/9
Bash (hard)	3/9	2/9	0/9	0/9	2/9
Research	3/3	3/3	1/3	1/3	3/3

laguna-s 2.1 117B

suite	M5 Max · llama.cpp Metal	RTX PRO 6000 · llama.cpp CUDA	GB10 · llama.cpp CUDA	GB10-2 · llama.cpp CUDA	Radeon 8060S · llama.cpp Vulkan
C	47/48	48/48	48/48	48/48	48/48
Python	24/24	24/24	24/24	24/24	24/24
Bash	15/24	18/24	18/24	18/24	18/24
C (hard)	6/9	6/9	6/9	6/9	6/9
Python (hard)	9/9	9/9	9/9	9/9	9/9
Bash (hard)	0/9	3/9	3/9	3/9	2/9
Research	0/3	0/3	0/3	0/3	0/3

gpt-oss-120b

suite	M5 Max · llama.cpp Metal	RTX PRO 6000 · llama.cpp CUDA	GB10 · llama.cpp CUDA	GB10-2 · llama.cpp CUDA	Radeon 8060S · llama.cpp Vulkan
C	42/48	45/48	40/48	40/48	42/48
Python	24/24	24/24	24/24	24/24	24/24
Bash	12/24	18/24	16/24	16/24	19/24
C (hard)	9/9	8/9	9/9	9/9	7/9
Python (hard)	9/9	9/9	9/9	9/9	9/9
Bash (hard)	8/9	4/9	6/9	6/9	6/9
Research	2/3	3/3	3/3	3/3	2/3

k2-horizon 36B-A4B

suite	M5 Max · llama.cpp Metal	RTX PRO 6000 · llama.cpp CUDA	GB10 · llama.cpp CUDA	GB10-2 · llama.cpp CUDA	Radeon 8060S · llama.cpp Vulkan
C	36/48	37/48	36/48	36/48	36/48
Python	24/24	21/24	23/24	23/24	21/24
Bash	10/24	15/24	12/24	12/24	16/24
C (hard)	6/9	4/9	6/9	6/9	6/9
Python (hard)	6/9	6/9	4/9	4/9	6/9
Bash (hard)	3/9	5/9	4/9	4/9	3/9
Research	1/3	3/3	3/3	3/3	3/3

devstral-2 24b

suite	M5 Max · llama.cpp Metal	RTX PRO 6000 · llama.cpp CUDA	GB10 · llama.cpp CUDA	GB10-2 · llama.cpp CUDA	Radeon 8060S · llama.cpp Vulkan
C	36/48	36/48	35/48	35/48	36/48
Python	24/24	24/24	24/24	24/24	24/24
Bash	15/24	18/24	18/24	18/24	18/24
C (hard)	6/9	3/9	6/9	6/9	3/9
Python (hard)	6/9	6/9	6/9	6/9	6/9
Bash (hard)	4/9	6/9	6/9	6/9	6/9
Research	3/3	3/3	3/3	3/3	3/3

laguna-xs-2.1

suite	RTX PRO 6000 · llama.cpp CUDA	GB10 · llama.cpp CUDA	GB10-2 · llama.cpp CUDA	Radeon 8060S · llama.cpp Vulkan
C	38/48	38/48	38/48	34/48
Python	24/24	24/24	24/24	24/24
Bash	21/24	18/24	18/24	21/24
C (hard)	6/9	5/9	5/9	6/9
Python (hard)	9/9	9/9	9/9	9/9
Bash (hard)	5/9	5/9	5/9	6/9
Research	2/3	2/3	3/3	2/3

gemma-4-26b

suite	M5 Max · llama.cpp Metal	RTX PRO 6000 · llama.cpp CUDA	GB10 · llama.cpp CUDA	GB10-2 · llama.cpp CUDA	Radeon 8060S · llama.cpp Vulkan
C	42/48	42/48	42/48	42/48	43/48
Python	24/24	24/24	24/24	24/24	24/24
Bash	12/24	15/24	15/24	15/24	15/24
C (hard)	6/9	9/9	6/9	6/9	7/9
Python (hard)	9/9	9/9	9/9	9/9	9/9
Bash (hard)	6/9	3/9	3/9	3/9	4/9
Research	3/3	3/3	3/3	3/3	3/3

nemotron-3-super 120B

suite	M5 Max · llama.cpp Metal	RTX PRO 6000 · llama.cpp CUDA	GB10 · llama.cpp CUDA	GB10-2 · llama.cpp CUDA	Radeon 8060S · llama.cpp Vulkan
C	45/48	45/48	48/48	48/48	45/48
Python	24/24	24/24	24/24	24/24	24/24
Bash	15/24	18/24	15/24	15/24	15/24
C (hard)	3/9	3/9	6/9	6/9	5/9
Python (hard)	9/9	9/9	9/9	9/9	9/9
Bash (hard)	3/9	3/9	3/9	3/9	3/9
Research	3/3	3/3	0/3	0/3	3/3

qwen3.5-122b-a10b

suite	M5 Max · llama.cpp Metal	RTX PRO 6000 · llama.cpp CUDA	GB10 · llama.cpp CUDA	GB10-2 · llama.cpp CUDA	Radeon 8060S · llama.cpp Vulkan
C	33/48	36/48	33/48	33/48	36/48
Python	24/24	24/24	24/24	24/24	24/24
Bash	12/24	18/24	15/24	15/24	15/24
C (hard)	3/9	3/9	6/9	6/9	6/9
Python (hard)	9/9	9/9	9/9	9/9	9/9
Bash (hard)	6/9	3/9	3/9	3/9	3/9
Research	3/3	0/3	3/3	3/3	0/3

qwen3.8-27b

suite	M5 Max · llama.cpp Metal	RTX PRO 6000 · llama.cpp CUDA	GB10 · llama.cpp CUDA	GB10-2 · llama.cpp CUDA	Radeon 8060S · llama.cpp Vulkan
C	45/48	42/48	42/48	42/48	45/48
Python	24/24	24/24	24/24	24/24	24/24
Bash	12/24	15/24	15/24	15/24	15/24
C (hard)	6/9	6/9	6/9	6/9	6/9
Python (hard)	9/9	9/9	9/9	9/9	9/9
Bash (hard)	3/9	3/9	3/9	3/9	3/9
Research	3/3	3/3	3/3	3/3	3/3

gpt-oss-20b

suite	M5 Max · llama.cpp Metal	RTX PRO 6000 · llama.cpp CUDA	GB10 · llama.cpp CUDA	GB10-2 · llama.cpp CUDA	Radeon 8060S · llama.cpp Vulkan
C	47/48	44/48	46/48	46/48	45/48
Python	24/24	23/24	24/24	24/24	24/24
Bash	11/24	18/24	15/24	15/24	16/24
C (hard)	6/9	6/9	4/9	4/9	6/9
Python (hard)	9/9	7/9	7/9	7/9	8/9
Bash (hard)	6/9	4/9	6/9	6/9	5/9
Research	1/3	0/3	0/3	0/3	0/3

qwen3.6-27b

suite	M5 Max · llama.cpp Metal	RTX PRO 6000 · llama.cpp CUDA	GB10 · llama.cpp CUDA	GB10-2 · llama.cpp CUDA	Radeon 8060S · llama.cpp Vulkan
C	36/48	36/48	33/48	33/48	36/48
Python	24/24	24/24	24/24	24/24	24/24
Bash	18/24	21/24	21/24	21/24	21/24
C (hard)	6/9	6/9	6/9	6/9	7/9
Python (hard)	9/9	9/9	9/9	9/9	9/9
Bash (hard)	6/9	3/9	3/9	3/9	3/9
Research	0/3	0/3	3/3	3/3	0/3

One machine only

Pinned: everything the harness controls, held identical

The main tables above are each machine's ordinary sweep, and those sweeps are not identical: two of three trials run at temperature 0.7, the Mac usually serves MLX where Linux serves llama.cpp, and C is compiled with the box's own libc under -std=c11 (glibc hides strdup; Apple's libc does not). A pinned run (scripts/run-pinned.sh) removes every one of those: the same GGUF, the same llama.cpp fork build, one slot × 16384 ctx (Aya-23 clamps to 8192), temperature 0 with seed 42 on all three trials, thinking off, and -std=gnu11. What is left is the GPU backend and the box's toolchain, listed per model below.

gpt-oss-20b

suite	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S	identical
C	47/48	44/48	46/48	46/48	45/48	11/16
Python	24/24	23/24	24/24	24/24	24/24	7/8
Bash	11/24	18/24	15/24	15/24	16/24	3/8
C (hard)	6/9	6/9	4/9	4/9	6/9	1/3
Python (hard)	9/9	7/9	7/9	7/9	8/9	2/3
Bash (hard)	6/9	4/9	6/9	6/9	5/9	0/3
Research	1/3	0/3	0/3	0/3	0/3	0/1
pinned total	104/126	102/126	102/126	102/126	104/126	24/42
original run (temp 0.7 retries, c11)	104/126	102/126	102/126	102/126	104/126

Tasks that still differ

suite	task	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
C	reverse_string	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✗✗`
C	binary_search	`✓✓✓`	`✓✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`
C	popcount	`✓✓✓`	`✓✓✓`	`✓✗✗`	`✓✗✗`	`✓✓✓`
C	cmp_desc	`✓✓✓`	`✓✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✗`
C	atoi_strict	`✗✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Python	is_valid_ipv4	`✓✓✓`	`✗✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Bash	count_matches	`✓✓✓`	`✗✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✗`
Bash	largest_file	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Bash	retry3	`✗✗✗`	`✗✓✓`	`✗✗✗`	`✗✗✗`	`✓✗✓`
Bash	rotate	`✗✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Bash	json_get	`✗✗✗`	`✗✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`
C (hard)	glob_match	`✓✓✓`	`✓✓✓`	`✓✗✗`	`✓✗✗`	`✓✓✓`
C (hard)	csv_field	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
Python (hard)	json_diff	`✓✓✓`	`✓✗✗`	`✓✗✗`	`✓✗✗`	`✗✓✓`
Bash (hard)	find_dupes	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
Bash (hard)	top_errors	`✗✗✗`	`✓✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Bash (hard)	backup_rotate	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✗✓`
Research	rhel10_nfs	`✓✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`

Trial-to-trial flips at temperature 0

Same prompt, same seed, three requests — and a different answer. llama.cpp batches 4 slots and the floating-point reduction order is not fixed, so a near-tie between two tokens can land either way. Where all three trials fail, the extracted code is byte-identical across trials and across machines.

What was held fixed and what was not

Rows shaded amber differ between machines. The GPU backend row is the one the harness cannot equalise: Metal, CUDA and Vulkan kernels round differently.

harness	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
llama.cpp build	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101
slots × ctx	1 × 16384	1 × 16384	1 × 16384	1 × 16384	1 × 16384
chat template	8d326e0f4f4e	8d326e0f4f4e	8d326e0f4f4e	8d326e0f4f4e	8d326e0f4f4e
sampling	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42
C standard	gnu11	gnu11	gnu11	gnu11	gnu11
GPU backend	Metal	CUDA	CUDA	CUDA	Vulkan
OS	Darwin 25.6.0 arm64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64
cc	Apple clang version 21.0.0 (clang-2100.1.1.101)	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)
bash	5.3.20(1)-release (aarch64-apple-darwin25.6.0)	5.2.26(1)-release (x86_64-redhat-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.26(1)-release (x86_64-redhat-linux-gnu)
Python	3.14.7	3.12.14	3.12.3	3.12.3	3.12.14

gpt-oss-120b

suite	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S	identical
C	42/48	45/48	40/48	40/48	42/48	14/16
Python	24/24	24/24	24/24	24/24	24/24	8/8
Bash	12/24	18/24	16/24	16/24	19/24	3/8
C (hard)	9/9	8/9	9/9	9/9	7/9	1/3
Python (hard)	9/9	9/9	9/9	9/9	9/9	3/3
Bash (hard)	8/9	4/9	6/9	6/9	6/9	1/3
Research	2/3	3/3	3/3	3/3	2/3	0/1
pinned total	106/126	111/126	107/126	107/126	109/126	30/42
original run (temp 0.7 retries, c11)	106/126	111/126	107/126	107/126	109/126

Tasks that still differ

suite	task	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
C	popcount	`✗✗✗`	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`
C	ring	`✓✓✓`	`✓✓✓`	`✓✗✗`	`✓✗✗`	`✓✓✓`
Bash	count_matches	`✓✓✓`	`✗✗✗`	`✗✓✓`	`✗✓✓`	`✗✓✓`
Bash	largest_file	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Bash	extract_urls	`✗✗✗`	`✓✗✗`	`✓✗✗`	`✓✗✗`	`✗✓✓`
Bash	csv_col	`✗✗✗`	`✓✓✓`	`✓✗✗`	`✓✗✗`	`✓✓✓`
Bash	json_get	`✗✗✗`	`✗✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`
C (hard)	glob_match	`✓✓✓`	`✓✗✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
C (hard)	csv_field	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✗✓✗`
Bash (hard)	find_dupes	`✗✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
Bash (hard)	top_errors	`✓✓✓`	`✓✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Research	rhel10_nfs	`✗✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✗`

Trial-to-trial flips at temperature 0

What was held fixed and what was not

Rows shaded amber differ between machines. The GPU backend row is the one the harness cannot equalise: Metal, CUDA and Vulkan kernels round differently.

harness	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
llama.cpp build	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101
slots × ctx	1 × 16384	1 × 16384	1 × 16384	1 × 16384	1 × 16384
chat template	8d326e0f4f4e	8d326e0f4f4e	8d326e0f4f4e	8d326e0f4f4e	8d326e0f4f4e
sampling	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42
C standard	gnu11	gnu11	gnu11	gnu11	gnu11
GPU backend	Metal	CUDA	CUDA	CUDA	Vulkan
OS	Darwin 25.6.0 arm64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64
cc	Apple clang version 21.0.0 (clang-2100.1.1.101)	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)
bash	5.3.20(1)-release (aarch64-apple-darwin25.6.0)	5.2.26(1)-release (x86_64-redhat-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.26(1)-release (x86_64-redhat-linux-gnu)
Python	3.14.7	3.12.14	3.12.3	3.12.3	3.12.14

gemma-4-26b

suite	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S	identical
C	42/48	42/48	42/48	42/48	43/48	15/16
Python	24/24	24/24	24/24	24/24	24/24	8/8
Bash	12/24	15/24	15/24	15/24	15/24	7/8
C (hard)	6/9	9/9	6/9	6/9	7/9	2/3
Python (hard)	9/9	9/9	9/9	9/9	9/9	3/3
Bash (hard)	6/9	3/9	3/9	3/9	4/9	2/3
Research	3/3	3/3	3/3	3/3	3/3	1/1
pinned total	102/126	105/126	102/126	102/126	105/126	38/42
original run (temp 0.7 retries, c11)	102/126	105/126	102/126	102/126	105/126

Tasks that still differ

suite	task	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
C	cmp_desc	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✓✗✗`
Bash	largest_file	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
C (hard)	csv_field	`✗✗✗`	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✓`
Bash (hard)	top_errors	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✓✗✗`

Trial-to-trial flips at temperature 0

What was held fixed and what was not

Rows shaded amber differ between machines. The GPU backend row is the one the harness cannot equalise: Metal, CUDA and Vulkan kernels round differently.

harness	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
llama.cpp build	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101
slots × ctx	1 × 16384	1 × 16384	1 × 16384	1 × 16384	1 × 16384
chat template	83b040a055e4	83b040a055e4	83b040a055e4	83b040a055e4	83b040a055e4
sampling	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42
C standard	gnu11	gnu11	gnu11	gnu11	gnu11
GPU backend	Metal	CUDA	CUDA	CUDA	Vulkan
OS	Darwin 25.6.0 arm64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64
cc	Apple clang version 21.0.0 (clang-2100.1.1.101)	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)
bash	5.3.20(1)-release (aarch64-apple-darwin25.6.0)	5.2.26(1)-release (x86_64-redhat-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.26(1)-release (x86_64-redhat-linux-gnu)
Python	3.14.7	3.12.14	3.12.3	3.12.3	3.12.14

qwen3-coder-next 80B

suite	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S	identical
C	42/48	39/48	45/48	45/48	42/48	14/16
Python	21/24	21/24	21/24	21/24	21/24	8/8
Bash	15/24	18/24	21/24	21/24	20/24	5/8
C (hard)	6/9	6/9	6/9	6/9	6/9	3/3
Python (hard)	6/9	6/9	6/9	6/9	8/9	2/3
Bash (hard)	6/9	6/9	6/9	6/9	5/9	1/3
Research	3/3	3/3	3/3	3/3	3/3	1/1
pinned total	99/126	99/126	108/126	108/126	105/126	34/42
original run (temp 0.7 retries, c11)	99/126	99/126	108/126	108/126	105/126

Tasks that still differ

suite	task	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
C	itoa	`✓✓✓`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`
C	atoi_strict	`✗✗✗`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✗✗✗`
Bash	count_matches	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Bash	largest_file	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Bash	csv_col	`✓✓✓`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✗✓✓`
Python (hard)	json_diff	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✓✓`
Bash (hard)	find_dupes	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
Bash (hard)	top_errors	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✗✓✓`

Trial-to-trial flips at temperature 0

What was held fixed and what was not

Rows shaded amber differ between machines. The GPU backend row is the one the harness cannot equalise: Metal, CUDA and Vulkan kernels round differently.

harness	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
llama.cpp build	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101
slots × ctx	1 × 16384	1 × 16384	1 × 16384	1 × 16384	1 × 16384
chat template	27f30d7e06cc	27f30d7e06cc	27f30d7e06cc	27f30d7e06cc	27f30d7e06cc
sampling	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42
C standard	gnu11	gnu11	gnu11	gnu11	gnu11
GPU backend	Metal	CUDA	CUDA	CUDA	Vulkan
OS	Darwin 25.6.0 arm64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64
cc	Apple clang version 21.0.0 (clang-2100.1.1.101)	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)
bash	5.3.20(1)-release (aarch64-apple-darwin25.6.0)	5.2.26(1)-release (x86_64-redhat-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.26(1)-release (x86_64-redhat-linux-gnu)
Python	3.14.7	3.12.14	3.12.3	3.12.3	3.12.14

devstral-2 24b

suite	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S	identical
C	36/48	36/48	35/48	35/48	36/48	12/16
Python	24/24	24/24	24/24	24/24	24/24	8/8
Bash	15/24	18/24	18/24	18/24	18/24	7/8
C (hard)	6/9	3/9	6/9	6/9	3/9	2/3
Python (hard)	6/9	6/9	6/9	6/9	6/9	3/3
Bash (hard)	4/9	6/9	6/9	6/9	6/9	2/3
Research	3/3	3/3	3/3	3/3	3/3	1/1
pinned total	94/126	96/126	98/126	98/126	96/126	35/42
original run (temp 0.7 retries, c11)	94/126	96/126	98/126	98/126	96/126

Tasks that still differ

suite	task	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
C	count_words	`✗✗✗`	`✓✓✓`	`✓✗✗`	`✓✗✗`	`✗✗✗`
C	itoa	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
C	trim	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
C	atoi_strict	`✓✓✓`	`✗✗✗`	`✓✗✗`	`✓✗✗`	`✓✓✓`
Bash	largest_file	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
C (hard)	rpn_eval	`✓✓✓`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✗✗✗`
Bash (hard)	backup_rotate	`✓✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`

Trial-to-trial flips at temperature 0

What was held fixed and what was not

Rows shaded amber differ between machines. The GPU backend row is the one the harness cannot equalise: Metal, CUDA and Vulkan kernels round differently.

harness	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
llama.cpp build	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101
slots × ctx	1 × 16384	1 × 16384	1 × 16384	1 × 16384	1 × 16384
chat template	47802e4b5241	47802e4b5241	47802e4b5241	47802e4b5241	47802e4b5241
sampling	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42
C standard	gnu11	gnu11	gnu11	gnu11	gnu11
GPU backend	Metal	CUDA	CUDA	CUDA	Vulkan
OS	Darwin 25.6.0 arm64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64
cc	Apple clang version 21.0.0 (clang-2100.1.1.101)	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)
bash	5.3.20(1)-release (aarch64-apple-darwin25.6.0)	5.2.26(1)-release (x86_64-redhat-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.26(1)-release (x86_64-redhat-linux-gnu)
Python	3.14.7	3.12.14	3.12.3	3.12.3	3.12.14

mistral-small-4 119B

suite	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S	identical
C	38/48	35/48	33/48	33/48	37/48	13/16
Python	24/24	22/24	22/24	22/24	21/24	7/8
Bash	15/24	15/24	14/24	14/24	12/24	6/8
C (hard)	7/9	7/9	5/9	5/9	6/9	1/3
Python (hard)	3/9	3/9	3/9	3/9	3/9	3/3
Bash (hard)	3/9	3/9	3/9	3/9	3/9	3/3
Research	1/3	2/3	3/3	3/3	0/3	0/1
pinned total	91/126	87/126	83/126	83/126	82/126	33/42
original run (temp 0.7 retries, c11)	91/126	87/126	83/126	83/126	82/126

Tasks that still differ

suite	task	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
C	parse_csv_ints	`✗✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✓✓✓`
C	bswap32	`✓✓✓`	`✓✓✓`	`✗✓✓`	`✗✓✓`	`✓✓✓`
C	trim	`✓✓✓`	`✗✓✓`	`✓✗✗`	`✓✗✗`	`✓✗✗`
Python	is_valid_ipv4	`✓✓✓`	`✓✗✗`	`✓✗✗`	`✓✗✗`	`✗✗✗`
Bash	retry3	`✓✓✓`	`✓✓✓`	`✗✓✓`	`✗✓✓`	`✓✓✓`
Bash	rotate	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✗✗✗`
C (hard)	glob_match	`✓✓✓`	`✓✓✓`	`✗✓✓`	`✗✓✓`	`✓✓✓`
C (hard)	csv_field	`✓✗✗`	`✓✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
Research	rhel10_nfs	`✓✗✗`	`✗✓✓`	`✓✓✓`	`✓✓✓`	`✗✗✗`

Trial-to-trial flips at temperature 0

What was held fixed and what was not

Rows shaded amber differ between machines. The GPU backend row is the one the harness cannot equalise: Metal, CUDA and Vulkan kernels round differently.

harness	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
llama.cpp build	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101
slots × ctx	1 × 16384	1 × 16384	1 × 16384	1 × 16384	1 × 16384
chat template	c922853c2727	c922853c2727	c922853c2727	c922853c2727	c922853c2727
sampling	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42
C standard	gnu11	gnu11	gnu11	gnu11	gnu11
GPU backend	Metal	CUDA	CUDA	CUDA	Vulkan
OS	Darwin 25.6.0 arm64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64
cc	Apple clang version 21.0.0 (clang-2100.1.1.101)	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)
bash	5.3.20(1)-release (aarch64-apple-darwin25.6.0)	5.2.26(1)-release (x86_64-redhat-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.26(1)-release (x86_64-redhat-linux-gnu)
Python	3.14.7	3.12.14	3.12.3	3.12.3	3.12.14

qwen3.8-27b

suite	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S	identical
C	45/48	42/48	42/48	42/48	45/48	15/16
Python	24/24	24/24	24/24	24/24	24/24	8/8
Bash	12/24	15/24	15/24	15/24	15/24	7/8
C (hard)	6/9	6/9	6/9	6/9	6/9	3/3
Python (hard)	9/9	9/9	9/9	9/9	9/9	3/3
Bash (hard)	3/9	3/9	3/9	3/9	3/9	3/3
Research	3/3	3/3	3/3	3/3	3/3	1/1
pinned total	102/126	102/126	102/126	102/126	105/126	40/42
original run (temp 0.7 retries, c11)	102/126	102/126	102/126	102/126	105/126

Tasks that still differ

suite	task	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
C	itoa	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✓✓✓`
Bash	largest_file	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`

Trial-to-trial flips at temperature 0

What was held fixed and what was not

Rows shaded amber differ between machines. The GPU backend row is the one the harness cannot equalise: Metal, CUDA and Vulkan kernels round differently.

harness	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
llama.cpp build	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101
slots × ctx	1 × 16384	1 × 16384	1 × 16384	1 × 16384	1 × 16384
chat template	08a763ee5e33	08a763ee5e33	08a763ee5e33	08a763ee5e33	08a763ee5e33
sampling	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42
C standard	gnu11	gnu11	gnu11	gnu11	gnu11
GPU backend	Metal	CUDA	CUDA	CUDA	Vulkan
OS	Darwin 25.6.0 arm64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64
cc	Apple clang version 21.0.0 (clang-2100.1.1.101)	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)
bash	5.3.20(1)-release (aarch64-apple-darwin25.6.0)	5.2.26(1)-release (x86_64-redhat-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.26(1)-release (x86_64-redhat-linux-gnu)
Python	3.14.7	3.12.14	3.12.3	3.12.3	3.12.14

qwen3.6-35b

suite	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S	identical
C	42/48	39/48	42/48	42/48	39/48	13/16
Python	21/24	21/24	21/24	21/24	21/24	6/8
Bash	15/24	18/24	18/24	18/24	18/24	7/8
C (hard)	3/9	6/9	9/9	9/9	6/9	1/3
Python (hard)	9/9	9/9	6/9	6/9	6/9	2/3
Bash (hard)	3/9	3/9	3/9	3/9	3/9	3/3
Research	3/3	0/3	3/3	3/3	1/3	0/1
pinned total	96/126	96/126	102/126	102/126	94/126	32/42
original run (temp 0.7 retries, c11)	96/126	96/126	102/126	102/126	94/126

Tasks that still differ

suite	task	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
C	reverse_string	`✓✓✓`	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✓✓✓`
C	itoa	`✓✓✓`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✗✗✗`
C	atoi_strict	`✗✗✗`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✗✗✗`
Python	merge_intervals	`✓✓✓`	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✓✓✓`
Python	csv_column_sum	`✗✗✗`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✗✗✗`
Bash	largest_file	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
C (hard)	glob_match	`✗✗✗`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✗✗✗`
C (hard)	csv_field	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Python (hard)	json_diff	`✓✓✓`	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`
Research	rhel10_nfs	`✓✓✓`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✗✗`

Trial-to-trial flips at temperature 0

What was held fixed and what was not

Rows shaded amber differ between machines. The GPU backend row is the one the harness cannot equalise: Metal, CUDA and Vulkan kernels round differently.

harness	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
llama.cpp build	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101
slots × ctx	1 × 16384	1 × 16384	1 × 16384	1 × 16384	1 × 16384
chat template	78c9fe670a12	78c9fe670a12	78c9fe670a12	78c9fe670a12	78c9fe670a12
sampling	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42
C standard	gnu11	gnu11	gnu11	gnu11	gnu11
GPU backend	Metal	CUDA	CUDA	CUDA	Vulkan
OS	Darwin 25.6.0 arm64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64
cc	Apple clang version 21.0.0 (clang-2100.1.1.101)	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)
bash	5.3.20(1)-release (aarch64-apple-darwin25.6.0)	5.2.26(1)-release (x86_64-redhat-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.26(1)-release (x86_64-redhat-linux-gnu)
Python	3.14.7	3.12.14	3.12.3	3.12.3	3.12.14

qwen3.5-35b

suite	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S	identical
C	36/48	42/48	39/48	39/48	36/48	12/16
Python	24/24	24/24	21/24	21/24	24/24	7/8
Bash	15/24	18/24	18/24	18/24	16/24	6/8
C (hard)	6/9	6/9	3/9	3/9	5/9	1/3
Python (hard)	9/9	9/9	9/9	9/9	8/9	2/3
Bash (hard)	3/9	3/9	3/9	3/9	3/9	3/3
Research	3/3	3/3	3/3	3/3	3/3	1/1
pinned total	96/126	105/126	96/126	96/126	95/126	32/42
original run (temp 0.7 retries, c11)	96/126	105/126	96/126	96/126	95/126

Tasks that still differ

suite	task	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
C	count_words	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✗✗✗`
C	itoa	`✓✓✓`	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`
C	parse_csv_ints	`✗✗✗`	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✓✓✓`
C	atoi_strict	`✗✗✗`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✗✗✗`
Python	csv_column_sum	`✓✓✓`	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✓✓✓`
Bash	largest_file	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Bash	json_get	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✗✗`
C (hard)	rpn_eval	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✗`
C (hard)	csv_field	`✓✓✓`	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✓✓✓`
Python (hard)	json_diff	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✗✓✓`

Trial-to-trial flips at temperature 0

What was held fixed and what was not

Rows shaded amber differ between machines. The GPU backend row is the one the harness cannot equalise: Metal, CUDA and Vulkan kernels round differently.

harness	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
llama.cpp build	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101
slots × ctx	1 × 16384	1 × 16384	1 × 16384	1 × 16384	1 × 16384
chat template	cb80bd67175c	cb80bd67175c	cb80bd67175c	cb80bd67175c	cb80bd67175c
sampling	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42
C standard	gnu11	gnu11	gnu11	gnu11	gnu11
GPU backend	Metal	CUDA	CUDA	CUDA	Vulkan
OS	Darwin 25.6.0 arm64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64
cc	Apple clang version 21.0.0 (clang-2100.1.1.101)	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)
bash	5.3.20(1)-release (aarch64-apple-darwin25.6.0)	5.2.26(1)-release (x86_64-redhat-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.26(1)-release (x86_64-redhat-linux-gnu)
Python	3.14.7	3.12.14	3.12.3	3.12.3	3.12.14

qwen3.6-27b

suite	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S	identical
C	36/48	36/48	33/48	33/48	36/48	15/16
Python	24/24	24/24	24/24	24/24	24/24	8/8
Bash	18/24	21/24	21/24	21/24	21/24	7/8
C (hard)	6/9	6/9	6/9	6/9	7/9	2/3
Python (hard)	9/9	9/9	9/9	9/9	9/9	3/3
Bash (hard)	6/9	3/9	3/9	3/9	3/9	2/3
Research	0/3	0/3	3/3	3/3	0/3	0/1
pinned total	99/126	99/126	99/126	99/126	100/126	37/42
original run (temp 0.7 retries, c11)	99/126	99/126	99/126	99/126	100/126

Tasks that still differ

suite	task	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
C	itoa	`✓✓✓`	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✓✓✓`
Bash	largest_file	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
C (hard)	csv_field	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✓✗`
Bash (hard)	find_dupes	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
Research	rhel10_nfs	`✗✗✗`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✗✗✗`

Trial-to-trial flips at temperature 0

What was held fixed and what was not

Rows shaded amber differ between machines. The GPU backend row is the one the harness cannot equalise: Metal, CUDA and Vulkan kernels round differently.

harness	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
llama.cpp build	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101
slots × ctx	1 × 16384	1 × 16384	1 × 16384	1 × 16384	1 × 16384
chat template	78c9fe670a12	78c9fe670a12	78c9fe670a12	78c9fe670a12	78c9fe670a12
sampling	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42
C standard	gnu11	gnu11	gnu11	gnu11	gnu11
GPU backend	Metal	CUDA	CUDA	CUDA	Vulkan
OS	Darwin 25.6.0 arm64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64
cc	Apple clang version 21.0.0 (clang-2100.1.1.101)	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)
bash	5.3.20(1)-release (aarch64-apple-darwin25.6.0)	5.2.26(1)-release (x86_64-redhat-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.26(1)-release (x86_64-redhat-linux-gnu)
Python	3.14.7	3.12.14	3.12.3	3.12.3	3.12.14

qwen3-coder-30b

suite	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S	identical
C	36/48	36/48	39/48	39/48	36/48	15/16
Python	24/24	24/24	24/24	24/24	24/24	8/8
Bash	9/24	12/24	12/24	12/24	14/24	6/8
C (hard)	6/9	6/9	6/9	6/9	6/9	2/3
Python (hard)	6/9	9/9	6/9	6/9	5/9	1/3
Bash (hard)	2/9	3/9	0/9	0/9	0/9	2/3
Research	3/3	3/3	3/3	3/3	2/3	0/1
pinned total	86/126	93/126	90/126	90/126	87/126	34/42
original run (temp 0.7 retries, c11)	86/126	93/126	90/126	90/126	87/126

Tasks that still differ

suite	task	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
C	fizzbuzz	`✗✗✗`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✗✗✗`
Bash	largest_file	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Bash	rotate	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✓✗✓`
C (hard)	csv_field	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
Python (hard)	topological_sort	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✗✓`
Python (hard)	json_diff	`✗✗✗`	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`
Bash (hard)	backup_rotate	`✗✓✓`	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`
Research	rhel10_nfs	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✗✓✓`

Trial-to-trial flips at temperature 0

What was held fixed and what was not

Rows shaded amber differ between machines. The GPU backend row is the one the harness cannot equalise: Metal, CUDA and Vulkan kernels round differently.

harness	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
llama.cpp build	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101
slots × ctx	1 × 16384	1 × 16384	1 × 16384	1 × 16384	1 × 16384
chat template	0c431480681c	0c431480681c	0c431480681c	0c431480681c	0c431480681c
sampling	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42
C standard	gnu11	gnu11	gnu11	gnu11	gnu11
GPU backend	Metal	CUDA	CUDA	CUDA	Vulkan
OS	Darwin 25.6.0 arm64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64
cc	Apple clang version 21.0.0 (clang-2100.1.1.101)	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)
bash	5.3.20(1)-release (aarch64-apple-darwin25.6.0)	5.2.26(1)-release (x86_64-redhat-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.26(1)-release (x86_64-redhat-linux-gnu)
Python	3.14.7	3.12.14	3.12.3	3.12.3	3.12.14

aya-23 35b

suite	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S	identical
C	27/48	25/48	37/48	37/48	27/48	9/16
Python	21/24	21/24	21/24	21/24	21/24	8/8
Bash	15/24	12/24	12/24	12/24	12/24	7/8
C (hard)	0/9	0/9	0/9	0/9	0/9	1/3
Python (hard)	3/9	3/9	3/9	3/9	3/9	3/3
Bash (hard)	0/9	0/9	0/9	0/9	0/9	3/3
Research	0/3	0/3	0/3	0/3	1/3	0/1
pinned total	66/126	61/126	73/126	73/126	64/126	31/42
original run (temp 0.7 retries, c11)	66/126	61/126	73/126	73/126	64/126

Tasks that still differ

suite	task	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
C	reverse_string	`✗✗✗`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✗✗✗`
C	max_subarray	`✗✗✗`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✗✗✗`
C	itoa	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
C	rot13	`✗✗✗`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✗✗✗`
C	parse_csv_ints	`✗✗✗`	`✗✗✗`	`✗✓✓`	`✗✓✓`	`✗✗✗`
C	trim	`✓✓✓`	`✓✗✗`	`✗✓✓`	`✗✓✓`	`✓✓✓`
C	atoi_strict	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
Bash	retry3	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
C (hard)	glob_match	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
C (hard)	csv_field	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
Research	rhel10_nfs	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✓✗✗`

Trial-to-trial flips at temperature 0

What was held fixed and what was not

Rows shaded amber differ between machines. The GPU backend row is the one the harness cannot equalise: Metal, CUDA and Vulkan kernels round differently.

harness	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
llama.cpp build	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101
slots × ctx	1 × 8192	1 × 8192	1 × 8192	1 × 8192	1 × 8192
chat template	899134038234	899134038234	899134038234	899134038234	899134038234
sampling	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42
C standard	gnu11	gnu11	gnu11	gnu11	gnu11
GPU backend	Metal	CUDA	CUDA	CUDA	Vulkan
OS	Darwin 25.6.0 arm64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64
cc	Apple clang version 21.0.0 (clang-2100.1.1.101)	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)
bash	5.3.20(1)-release (aarch64-apple-darwin25.6.0)	5.2.26(1)-release (x86_64-redhat-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.26(1)-release (x86_64-redhat-linux-gnu)
Python	3.14.7	3.12.14	3.12.3	3.12.3	3.12.14

glm-4.7-flash

suite	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S	identical
C	42/48	45/48	45/48	45/48	45/48	15/16
Python	24/24	24/24	24/24	24/24	24/24	8/8
Bash	15/24	15/24	18/24	18/24	15/24	6/8
C (hard)	3/9	3/9	1/9	1/9	6/9	1/3
Python (hard)	7/9	3/9	3/9	3/9	7/9	1/3
Bash (hard)	0/9	1/9	3/9	3/9	3/9	1/3
Research	3/3	3/3	2/3	2/3	3/3	0/1
pinned total	94/126	94/126	96/126	96/126	103/126	32/42
original run (temp 0.7 retries, c11)	94/126	94/126	96/126	96/126	103/126

Tasks that still differ

suite	task	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
C	fizzbuzz	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Bash	largest_file	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Bash	retry3	`✓✓✓`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✗✗✗`
C (hard)	glob_match	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✓✓✓`
C (hard)	rpn_eval	`✓✓✓`	`✓✓✓`	`✓✗✗`	`✓✗✗`	`✓✓✓`
Python (hard)	topological_sort	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✓✓✓`
Python (hard)	json_diff	`✓✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✓✗✗`
Bash (hard)	top_errors	`✗✗✗`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✗✗✗`
Bash (hard)	backup_rotate	`✗✗✗`	`✓✗✗`	`✗✗✗`	`✗✗✗`	`✓✓✓`
Research	rhel10_nfs	`✓✓✓`	`✓✓✓`	`✗✓✓`	`✗✓✓`	`✓✓✓`

Trial-to-trial flips at temperature 0

What was held fixed and what was not

Rows shaded amber differ between machines. The GPU backend row is the one the harness cannot equalise: Metal, CUDA and Vulkan kernels round differently.

harness	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
llama.cpp build	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101
slots × ctx	1 × 16384	1 × 16384	1 × 16384	1 × 16384	1 × 16384
chat template	257e6e854b7e	257e6e854b7e	257e6e854b7e	257e6e854b7e	257e6e854b7e
sampling	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42
C standard	gnu11	gnu11	gnu11	gnu11	gnu11
GPU backend	Metal	CUDA	CUDA	CUDA	Vulkan
OS	Darwin 25.6.0 arm64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64
cc	Apple clang version 21.0.0 (clang-2100.1.1.101)	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)
bash	5.3.20(1)-release (aarch64-apple-darwin25.6.0)	5.2.26(1)-release (x86_64-redhat-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.26(1)-release (x86_64-redhat-linux-gnu)
Python	3.14.7	3.12.14	3.12.3	3.12.3	3.12.14

laguna-xs.2

suite	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S	identical
C	39/48	38/48	38/48	38/48	34/48	11/16
Python	24/24	24/24	24/24	24/24	24/24	8/8
Bash	16/24	21/24	18/24	18/24	21/24	5/8
C (hard)	6/9	6/9	5/9	5/9	6/9	2/3
Python (hard)	9/9	9/9	9/9	9/9	9/9	3/3
Bash (hard)	4/9	5/9	5/9	5/9	6/9	2/3
Research	2/3	2/3	2/3	3/3	2/3	0/1
pinned total	100/126	105/126	101/126	102/126	102/126	31/42
original run (temp 0.7 retries, c11)	100/126	105/126	101/126	102/126	102/126

Tasks that still differ

suite	task	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
C	is_prime	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
C	itoa	`✗✗✗`	`✗✓✓`	`✗✗✗`	`✗✗✗`	`✓✗✗`
C	popcount	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✗✗✓`
C	bswap32	`✗✗✗`	`✗✗✗`	`✗✓✓`	`✗✓✓`	`✗✗✗`
C	trim	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✗✓`
Bash	count_matches	`✓✓✓`	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✓✓✓`
Bash	largest_file	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Bash	csv_col	`✓✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
C (hard)	glob_match	`✓✓✓`	`✓✓✓`	`✗✓✓`	`✗✓✓`	`✓✓✓`
Bash (hard)	backup_rotate	`✓✗✗`	`✗✓✓`	`✗✓✓`	`✗✓✓`	`✓✓✓`
Research	rhel10_nfs	`✗✓✓`	`✗✓✓`	`✗✓✓`	`✓✓✓`	`✓✓✗`

Trial-to-trial flips at temperature 0

What was held fixed and what was not

Rows shaded amber differ between machines. The GPU backend row is the one the harness cannot equalise: Metal, CUDA and Vulkan kernels round differently.

harness	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
llama.cpp build	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101
slots × ctx	1 × 16384	1 × 16384	1 × 16384	1 × 16384	1 × 16384
chat template	8cf80871164b	8cf80871164b	8cf80871164b	8cf80871164b	8cf80871164b
sampling	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42
C standard	gnu11	gnu11	gnu11	gnu11	gnu11
GPU backend	Metal	CUDA	CUDA	CUDA	Vulkan
OS	Darwin 25.6.0 arm64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64
cc	Apple clang version 21.0.0 (clang-2100.1.1.101)	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)
bash	5.3.20(1)-release (aarch64-apple-darwin25.6.0)	5.2.26(1)-release (x86_64-redhat-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.26(1)-release (x86_64-redhat-linux-gnu)
Python	3.14.7	3.12.14	3.12.3	3.12.3	3.12.14

qwen3.8-flash-next 125B

suite	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S	identical
C	48/48	48/48	45/48	45/48	46/48	15/16
Python	24/24	24/24	24/24	24/24	24/24	8/8
Bash	15/24	18/24	18/24	18/24	20/24	6/8
C (hard)	6/9	9/9	6/9	6/9	8/9	2/3
Python (hard)	9/9	9/9	9/9	9/9	8/9	2/3
Bash (hard)	3/9	6/9	6/9	6/9	6/9	2/3
Research	3/3	3/3	3/3	3/3	3/3	1/1
pinned total	108/126	117/126	111/126	111/126	115/126	36/42
original run (temp 0.7 retries, c11)	108/126	117/126	111/126	111/126	115/126

Tasks that still differ

suite	task	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
C	parse_csv_ints	`✓✓✓`	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✓✗✗`
Bash	largest_file	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Bash	rotate	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✓✓`
C (hard)	csv_field	`✗✗✗`	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✓✓✗`
Python (hard)	json_diff	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✗✓`
Bash (hard)	top_errors	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`

Trial-to-trial flips at temperature 0

What was held fixed and what was not

Rows shaded amber differ between machines. The GPU backend row is the one the harness cannot equalise: Metal, CUDA and Vulkan kernels round differently.

harness	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
llama.cpp build	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101
slots × ctx	1 × 16384	1 × 16384	1 × 16384	1 × 16384	1 × 16384
chat template	a7e79f8fe37f	a7e79f8fe37f	a7e79f8fe37f	a7e79f8fe37f	a7e79f8fe37f
sampling	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42
C standard	gnu11	gnu11	gnu11	gnu11	gnu11
GPU backend	Metal	CUDA	CUDA	CUDA	Vulkan
OS	Darwin 25.6.0 arm64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64
cc	Apple clang version 21.0.0 (clang-2100.1.1.101)	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)
bash	5.3.20(1)-release (aarch64-apple-darwin25.6.0)	5.2.26(1)-release (x86_64-redhat-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.26(1)-release (x86_64-redhat-linux-gnu)
Python	3.14.7	3.12.14	3.12.3	3.12.3	3.12.14

k2-horizon 36B-A4B

suite	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S	identical
C	36/48	37/48	36/48	36/48	36/48	13/16
Python	24/24	21/24	23/24	23/24	21/24	7/8
Bash	10/24	15/24	12/24	12/24	16/24	4/8
C (hard)	6/9	4/9	6/9	6/9	6/9	1/3
Python (hard)	6/9	6/9	4/9	4/9	6/9	2/3
Bash (hard)	3/9	5/9	4/9	4/9	3/9	2/3
Research	1/3	3/3	3/3	3/3	3/3	0/1
pinned total	86/126	91/126	88/126	88/126	91/126	29/42
original run (temp 0.7 retries, c11)	86/126	91/126	88/126	88/126	91/126

Tasks that still differ

suite	task	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
C	itoa	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
C	trim	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
C	atoi_strict	`✗✗✗`	`✓✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
Python	csv_column_sum	`✓✓✓`	`✗✗✗`	`✗✓✓`	`✗✓✓`	`✗✗✗`
Bash	largest_file	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Bash	retry3	`✗✗✗`	`✗✓✓`	`✗✗✗`	`✗✗✗`	`✗✓✗`
Bash	rotate	`✓✗✗`	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✓✓✓`
Bash	csv_col	`✓✓✓`	`✓✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`
C (hard)	glob_match	`✓✓✓`	`✓✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`
C (hard)	csv_field	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
Python (hard)	topological_sort	`✓✓✓`	`✓✓✓`	`✓✗✗`	`✓✗✗`	`✓✓✓`
Bash (hard)	backup_rotate	`✗✗✗`	`✗✓✓`	`✓✗✗`	`✓✗✗`	`✗✗✗`
Research	rhel10_nfs	`✓✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`

Trial-to-trial flips at temperature 0

What was held fixed and what was not

Rows shaded amber differ between machines. The GPU backend row is the one the harness cannot equalise: Metal, CUDA and Vulkan kernels round differently.

harness	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
llama.cpp build	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101
slots × ctx	1 × 16384	1 × 16384	1 × 16384	1 × 16384	1 × 16384
chat template	31fb55ec90b2	31fb55ec90b2	31fb55ec90b2	31fb55ec90b2	31fb55ec90b2
sampling	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42
C standard	gnu11	gnu11	gnu11	gnu11	gnu11
GPU backend	Metal	CUDA	CUDA	CUDA	Vulkan
OS	Darwin 25.6.0 arm64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64
cc	Apple clang version 21.0.0 (clang-2100.1.1.101)	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)
bash	5.3.20(1)-release (aarch64-apple-darwin25.6.0)	5.2.26(1)-release (x86_64-redhat-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.26(1)-release (x86_64-redhat-linux-gnu)
Python	3.14.7	3.12.14	3.12.3	3.12.3	3.12.14

qwen3.5-122b-a10b

suite	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S	identical
C	33/48	36/48	33/48	33/48	36/48	14/16
Python	24/24	24/24	24/24	24/24	24/24	8/8
Bash	12/24	18/24	15/24	15/24	15/24	6/8
C (hard)	3/9	3/9	6/9	6/9	6/9	2/3
Python (hard)	9/9	9/9	9/9	9/9	9/9	3/3
Bash (hard)	6/9	3/9	3/9	3/9	3/9	2/3
Research	3/3	0/3	3/3	3/3	0/3	0/1
pinned total	90/126	93/126	93/126	93/126	93/126	35/42
original run (temp 0.7 retries, c11)	90/126	93/126	93/126	93/126	93/126

Tasks that still differ

suite	task	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
C	parse_csv_ints	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
C	cmp_desc	`✓✓✓`	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✓✓✓`
Bash	largest_file	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Bash	retry3	`✗✗✗`	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`
C (hard)	csv_field	`✗✗✗`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Bash (hard)	find_dupes	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
Research	rhel10_nfs	`✓✓✓`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✗✗✗`

Trial-to-trial flips at temperature 0

What was held fixed and what was not

Rows shaded amber differ between machines. The GPU backend row is the one the harness cannot equalise: Metal, CUDA and Vulkan kernels round differently.

harness	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
llama.cpp build	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101
slots × ctx	1 × 16384	1 × 16384	1 × 16384	1 × 16384	1 × 16384
chat template	cb80bd67175c	cb80bd67175c	cb80bd67175c	cb80bd67175c	cb80bd67175c
sampling	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42
C standard	gnu11	gnu11	gnu11	gnu11	gnu11
GPU backend	Metal	CUDA	CUDA	CUDA	Vulkan
OS	Darwin 25.6.0 arm64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64
cc	Apple clang version 21.0.0 (clang-2100.1.1.101)	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)
bash	5.3.20(1)-release (aarch64-apple-darwin25.6.0)	5.2.26(1)-release (x86_64-redhat-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.26(1)-release (x86_64-redhat-linux-gnu)
Python	3.14.7	3.12.14	3.12.3	3.12.3	3.12.14

qwen3.5-27b

suite	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S	identical
C	33/48	30/48	36/48	36/48	33/48	14/16
Python	24/24	24/24	24/24	24/24	24/24	8/8
Bash	12/24	15/24	15/24	15/24	15/24	7/8
C (hard)	0/9	6/9	3/9	3/9	3/9	0/3
Python (hard)	9/9	9/9	9/9	9/9	9/9	3/3
Bash (hard)	6/9	6/9	6/9	6/9	6/9	1/3
Research	3/3	3/3	3/3	3/3	3/3	1/1
pinned total	87/126	93/126	96/126	96/126	93/126	34/42
original run (temp 0.7 retries, c11)	87/126	93/126	96/126	96/126	93/126

Tasks that still differ

suite	task	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
C	itoa	`✗✗✗`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`
C	trim	`✓✓✓`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✗✗✗`
Bash	largest_file	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
C (hard)	glob_match	`✗✗✗`	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`
C (hard)	rpn_eval	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
C (hard)	csv_field	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
Bash (hard)	find_dupes	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
Bash (hard)	top_errors	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`

Trial-to-trial flips at temperature 0

What was held fixed and what was not

Rows shaded amber differ between machines. The GPU backend row is the one the harness cannot equalise: Metal, CUDA and Vulkan kernels round differently.

harness	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
llama.cpp build	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101
slots × ctx	1 × 16384	1 × 16384	1 × 16384	1 × 16384	1 × 16384
chat template	cb80bd67175c	cb80bd67175c	cb80bd67175c	cb80bd67175c	cb80bd67175c
sampling	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42
C standard	gnu11	gnu11	gnu11	gnu11	gnu11
GPU backend	Metal	CUDA	CUDA	CUDA	Vulkan
OS	Darwin 25.6.0 arm64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64
cc	Apple clang version 21.0.0 (clang-2100.1.1.101)	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)
bash	5.3.20(1)-release (aarch64-apple-darwin25.6.0)	5.2.26(1)-release (x86_64-redhat-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.26(1)-release (x86_64-redhat-linux-gnu)
Python	3.14.7	3.12.14	3.12.3	3.12.3	3.12.14

laguna-s 2.1 117B

suite	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S	identical
C	47/48	48/48	48/48	48/48	48/48	15/16
Python	24/24	24/24	24/24	24/24	24/24	8/8
Bash	15/24	18/24	18/24	18/24	18/24	7/8
C (hard)	6/9	6/9	6/9	6/9	6/9	3/3
Python (hard)	9/9	9/9	9/9	9/9	9/9	3/3
Bash (hard)	0/9	3/9	3/9	3/9	2/9	2/3
Research	0/3	0/3	0/3	0/3	0/3	1/1
pinned total	101/126	108/126	108/126	108/126	107/126	39/42
original run (temp 0.7 retries, c11)	101/126	108/126	108/126	108/126	107/126

Tasks that still differ

suite	task	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
C	count_words	`✗✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Bash	largest_file	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Bash (hard)	top_errors	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✗✓`

Trial-to-trial flips at temperature 0

What was held fixed and what was not

Rows shaded amber differ between machines. The GPU backend row is the one the harness cannot equalise: Metal, CUDA and Vulkan kernels round differently.

harness	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
llama.cpp build	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101
slots × ctx	1 × 16384	1 × 16384	1 × 16384	1 × 16384	1 × 16384
chat template	f0ae8663a876	f0ae8663a876	f0ae8663a876	f0ae8663a876	f0ae8663a876
sampling	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42
C standard	gnu11	gnu11	gnu11	gnu11	gnu11
GPU backend	Metal	CUDA	CUDA	CUDA	Vulkan
OS	Darwin 25.6.0 arm64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64
cc	Apple clang version 21.0.0 (clang-2100.1.1.101)	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)
bash	5.3.20(1)-release (aarch64-apple-darwin25.6.0)	5.2.26(1)-release (x86_64-redhat-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.26(1)-release (x86_64-redhat-linux-gnu)
Python	3.14.7	3.12.14	3.12.3	3.12.3	3.12.14

nemotron-3-super 120B

suite	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S	identical
C	45/48	45/48	48/48	48/48	45/48	15/16
Python	24/24	24/24	24/24	24/24	24/24	8/8
Bash	15/24	18/24	15/24	15/24	15/24	6/8
C (hard)	3/9	3/9	6/9	6/9	5/9	2/3
Python (hard)	9/9	9/9	9/9	9/9	9/9	3/3
Bash (hard)	3/9	3/9	3/9	3/9	3/9	3/3
Research	3/3	3/3	0/3	0/3	3/3	0/1
pinned total	102/126	105/126	105/126	105/126	104/126	37/42
original run (temp 0.7 retries, c11)	102/126	105/126	105/126	105/126	104/126

Tasks that still differ

suite	task	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
C	atoi_strict	`✗✗✗`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✗✗✗`
Bash	largest_file	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Bash	csv_col	`✓✓✓`	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`
C (hard)	csv_field	`✗✗✗`	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✗`
Research	rhel10_nfs	`✓✓✓`	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✓✓✓`

Trial-to-trial flips at temperature 0

What was held fixed and what was not

Rows shaded amber differ between machines. The GPU backend row is the one the harness cannot equalise: Metal, CUDA and Vulkan kernels round differently.

harness	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
llama.cpp build	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101	b10671-35999d101
slots × ctx	1 × 16384	1 × 16384	1 × 16384	1 × 16384	1 × 16384
chat template	4ea9cddb5490	4ea9cddb5490	4ea9cddb5490	4ea9cddb5490	4ea9cddb5490
sampling	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42	temp 0 all trials, seed 42
C standard	gnu11	gnu11	gnu11	gnu11	gnu11
GPU backend	Metal	CUDA	CUDA	CUDA	Vulkan
OS	Darwin 25.6.0 arm64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.17.0-1029-nvidia aarch64	Linux 6.12.0-211.54.1.el10_2.x86_64 x86_64
cc	Apple clang version 21.0.0 (clang-2100.1.1.101)	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0	cc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4)
bash	5.3.20(1)-release (aarch64-apple-darwin25.6.0)	5.2.26(1)-release (x86_64-redhat-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.21(1)-release (aarch64-unknown-linux-gnu)	5.2.26(1)-release (x86_64-redhat-linux-gnu)
Python	3.14.7	3.12.14	3.12.3	3.12.3	3.12.14

seed-oss 36b

suite	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S	identical
C	32/48	29/48	30/48	30/48	34/48	8/16
Python	24/24	24/24	24/24	24/24	24/24	8/8
Bash	11/24	17/24	20/24	20/24	17/24	3/8
C (hard)	3/9	3/9	3/9	3/9	2/9	2/3
Python (hard)	3/9	4/9	3/9	3/9	4/9	2/3
Bash (hard)	3/9	2/9	0/9	0/9	2/9	1/3
Research	3/3	3/3	1/3	1/3	3/3	0/1
pinned total	79/126	82/126	81/126	81/126	86/126	24/42
original run (temp 0.7 retries, c11)	79/126	82/126	81/126	81/126	86/126

Tasks that still differ

suite	task	M5 Max	RTX PRO 6000	GB10	GB10-2	Radeon 8060S
C	count_words	`✗✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
C	itoa	`✗✗✗`	`✓✗✗`	`✗✗✗`	`✗✗✗`	`✓✓✓`
C	popcount	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✗✓`
C	rot13	`✓✓✓`	`✗✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
C	parse_csv_ints	`✗✗✗`	`✓✗✗`	`✓✗✗`	`✓✗✗`	`✗✓✓`
C	trim	`✗✗✗`	`✓✗✗`	`✗✓✓`	`✗✓✓`	`✗✗✓`
C	cmp_desc	`✓✓✓`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✓✗✓`
C	atoi_strict	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
Bash	count_matches	`✗✗✗`	`✗✓✓`	`✗✓✓`	`✗✓✓`	`✗✓✓`
Bash	largest_file	`✗✗✗`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Bash	csv_col	`✗✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✗✗`
Bash	sum_stdin	`✓✓✓`	`✗✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`
Bash	json_get	`✗✗✗`	`✓✗✗`	`✓✓✓`	`✓✓✓`	`✗✓✓`
C (hard)	rpn_eval	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✓`	`✓✓✗`
Python (hard)	topological_sort	`✗✗✗`	`✓✗✗`	`✗✗✗`	`✗✗✗`	`✓✗✗`
Bash (hard)	top_errors	`✗✗✗`	`✓✗✗`	`✗✗✗`	`✗✗✗`	`✗✗✗`
Bash (hard)	backup_rotate	`✓✓✓`	`✓✗✗`	`✗✗✗`	`✗✗✗`	`✓✗✓`
Research	rhel10_nfs	`✓✓✓`	`✓✓✓`	`✓✗✗`	`✓✗✗`	`✓✓✓`

Trial-to-trial flips at temperature 0

What was held fixed and what was not

Rows shaded amber differ between machines. The GPU backend row is the one the harness cannot equalise: Metal, CUDA and Vulkan kernels round differently.

Same model, same tests, different machines

Coding total

By suite

aya-23 35b

qwen3.5-35b

qwen3.8-flash-next 125B

glm-4.7-flash

mistral-small-4 119B

qwen3-coder-next 80B

qwen3.5-27b

qwen3.6-35b

qwen3-coder-30b

seed-oss 36b

laguna-s 2.1 117B

gpt-oss-120b

k2-horizon 36B-A4B

devstral-2 24b

laguna-xs-2.1

gemma-4-26b

nemotron-3-super 120B

qwen3.5-122b-a10b

qwen3.8-27b

gpt-oss-20b

qwen3.6-27b

One machine only

Pinned: everything the harness controls, held identical

gpt-oss-20b

Tasks that still differ

Trial-to-trial flips at temperature 0

What was held fixed and what was not

gpt-oss-120b

Tasks that still differ

Trial-to-trial flips at temperature 0

What was held fixed and what was not

gemma-4-26b

Tasks that still differ

Trial-to-trial flips at temperature 0

What was held fixed and what was not

qwen3-coder-next 80B

Tasks that still differ

Trial-to-trial flips at temperature 0

What was held fixed and what was not

devstral-2 24b

Tasks that still differ

Trial-to-trial flips at temperature 0

What was held fixed and what was not

mistral-small-4 119B

Tasks that still differ

Trial-to-trial flips at temperature 0

What was held fixed and what was not

qwen3.8-27b

Tasks that still differ

Trial-to-trial flips at temperature 0

What was held fixed and what was not

qwen3.6-35b

Tasks that still differ

Trial-to-trial flips at temperature 0

What was held fixed and what was not

qwen3.5-35b

Tasks that still differ

Trial-to-trial flips at temperature 0

What was held fixed and what was not

qwen3.6-27b

Tasks that still differ

Trial-to-trial flips at temperature 0

What was held fixed and what was not

qwen3-coder-30b

Tasks that still differ

Trial-to-trial flips at temperature 0

What was held fixed and what was not

aya-23 35b

Tasks that still differ

Trial-to-trial flips at temperature 0

What was held fixed and what was not

glm-4.7-flash

Tasks that still differ

Trial-to-trial flips at temperature 0

What was held fixed and what was not

laguna-xs.2

Tasks that still differ