Open the research drawer — sources, exact benchmark conditions, fit decisions & estimates
Research cutoff: 10 October 2026. Live vendor specifications, official model cards and identifiable community benchmarks were checked. No hardware was benchmarked during this research. “M” is reported measurement, not independent replication by this report. The single-page PDF is the concise brief; this drawer is its audit trail.
Hardware pricing · approximate USD, 10 October 2026
Prices cover the installed memory and GPU configurations in the main table. “System” includes the computer; “card” excludes a host, PSU, cooling and chassis. Tax, shipping, monitors, support and extra storage are excluded. Paired-device prices are simple multiples, excluding interconnect accessories. Retail asking prices are not completed-sale averages. Estimates are rounded budget allowances rather than checkout quotes.
- Apple laptops: approximately $2,499 for M5 32 GB / 1 TB and $3,699 for M5 Pro 20-core GPU / 64 GB / 1 TB from Apple’s US store and its 64 GB configuration. M5 Max 40-core GPU / 128 GB: allow roughly $6,000; an exact current configured checkout price was not confirmed. Screen size and SSD change the total.
- Ryzen AI Max+ 395 / 128 GB: $2,000–4,000 system allowance spans mini-PC and workstation builds. Engadget reports Ryzen AI Halo at $3,999 and DGX Spark at $4,699. The lower end is an indicative mini-PC budget, not a Halo quote. Two Sparks: about $9,400.
- Discrete GPUs: RTX 4090 $2,000–2,500 is a low-confidence used/remaining-stock budget, not a verified current offer. R9700 approximately $1,700 from US price tracking. RTX 5090 allowance $6,400–7,400 includes a $7,350 asking price on VideoCardPrices; its $1,999 launch MSRP is not the current street-price basis.
- RTX PRO 6000 Blackwell: budget $16,000–20,000 per Workstation card ($32,000–40,000 for two). Thunder Compute cites $16,000 on NVIDIA’s marketplace; VideoCardPrices lists $17,986.96. These are card-only prices. The measured single-card Flash-Next setup (B9) keeps its 51B n-gram table in host RAM and needs about 100 GB of spare system memory, so budget a well-equipped host on top of the card.
- M5 Ultra Studio: roughly $12,000 for 256 GB / 80-core GPU; $15,000–18,000 for the announced 512 GB tier. These are low-confidence budget estimates informed by reported configuration pricing, not verified Apple checkout quotes. The 512 GB configuration is expected late October. The MacStories performance review does not confirm these prices.
- DGX Station: approximately $99,000–188,500 system budget from Pi3g’s vendor pricing updated 8 October, net of VAT. Actual OEM configuration, region and support contract affect the quote.
One common model score
The shortlist now uses the Artificial Analysis Intelligence Index v4.3.2 throughout. It weights agents 30% (AA-Briefcase v1.1 15%, GDPval-AA v2.1 10%, AutomationBench-AA 5%), coding 20% (Terminal-Bench 4.0 10%, SciCode 10%), scientific reasoning 20% and general capability 30%. This is a broad independent composite, not a pure coding-agent score or a percentage success rate. No ad hoc average of vendor benchmarks is used.
Verified scores: GLM-5.3 Max 45; GLM-5.3-Flash 42; DeepSeek V4.1 Flash Max 39; Qwen3.8-27B Xhigh 34; Qwen3.8-Flash-Next reasoning 40; DeepSeek V4 Flash 0731 Max 34; Qwen3.6-35B-A3B reasoning 18. Use the stated reasoning configurations: this is a shared evaluation methodology, not an equal-compute comparison. Scores do not establish quality retention in any local quantization. The 0731 text model is distinct from its vision variant.
The AA Coding Agent Index v1.5 is more coding-specific (DeepSWE v1.1, Terminal-Bench 4.0 and SWE-Atlas-QnA), but complete public results for all seven local models could not be verified. Vendor-specific results below remain background evidence and are no longer mixed in the visible comparison.
Hardware sources and precision normalization
Arithmetic units: Mac rows select the numerically higher dense INT8 GPU estimate instead of the FP16 estimate; other GPU rows show floating-point throughput. Precision is labelled in every row, and a higher INT8 number does not imply faster floating-point inference. Figures are dense hardware peaks or estimates (E), with no credit for skipped zero operations or structured sparsity; different precisions are not directly comparable. Halo shows a mixed-operation sum of the separate GPU and NPU ratings: 60 FP16 TFLOPS + 50 INT8 TOPS = 110 trillion operations/s. These engines can work concurrently, but simultaneous peak rates are not verified and share power and memory. This aggregate is neither 110 TFLOPS nor an INT8-only rating, and the cited llama.cpp decode run uses the GPU, not the NPU. CPU compute is excluded. Apple Neural Engine compute is omitted because no verified dense rating for these configurations is available. Apple INT8 TOPS are GPU matrix-unit estimates, not Neural Engine ratings: adding them to FP16 matrix TFLOPS would double-count the same accelerator. A GPU + Neural Engine aggregate would require an independently sourced dense Neural Engine rate and concurrent execution support; Apple describes using both in Metal upscaling, but that supplies no numeric dense peak or evidence of combined throughput in these LLM runs. Two-device rows sum the same floating-point metric across devices; this does not predict inference scaling. Quantized weight storage does not establish execution precision.
Apple compute: GPU Neural Accelerators, not the Neural Engine. Every M5-family GPU core contains a matrix unit that Apple calls a Neural Accelerator. This is the hardware that LLM runtimes such as MLX use through Metal; the 16-core Neural Engine (32-core on Ultra) is a separate, smaller block that most local-LLM runtimes do not use. Apple publishes no FLOPS figure and claims only “over 4×” the peak GPU AI compute of the matching M4 chip (M5, M5 Pro and M5 Max). Microbenchmarks on the A19, whose GPU cores are the same generation, measured about 1,024 FP16 FLOPs and about 1,900 INT8 operations per GPU core per clock (FP32 and INT32 accumulation respectively); BF16 was not exposed and FP8/INT4 were not measured. Scaling by core count and an assumed clock (about 1.55 GHz for M5, about 1.75 GHz for Pro/Max/Ultra) gives approximately: M5 10-core 16 TFLOPS FP16 / 30 TOPS INT8; M5 Pro 20-core 36 / 67; M5 Max 40-core 70 / 130 (matching that author’s own M5 Max extrapolation); M5 Ultra 80-core 140 / 260. The table selects the higher INT8 estimate; it does not sum the two modes or establish the precision used by the decode benchmark. The M5 figure is consistent with Apple’s 4× claim over a roughly 4 TFLOPS M4 GPU. These are dense FP16/INT8 estimates, not FP4: NVIDIA’s headline FP4 numbers are not like-for-like. The only direct M5 Max matmul measurement found, about 19.9 TFLOPS FP16, is described by its author as directional and sits well below the derived peak, so treat the table values as ceilings. Why the Macs still decode well: single-user decode streams the active weights once per token, so it is limited by memory bandwidth (614 GB/s on M5 Max, 1.2 TB/s on M5 Ultra) and by fitting the model in unified memory, not by peak compute. Compute shows up in prefill: M5 Max prefills Flash-Next at about 1,100 tok/s (B1) against about 12,000 tok/s on an RTX PRO 6000 (B9), which is the practical cost of lower matrix throughput.
- H1 · Apple laptops. Apple technical specifications: M5 153 GB/s / 32 GB max, M5 Pro 307 / 64, full M5 Max 614 / 128. The 32-core M5 Max has only 460 GB/s. Apple’s GPU developer talk confirms neural accelerators and INT4/INT8 tensor APIs, but supplies no comparable absolute dense FP4/FP8 GPU peak. An unpublished value is not zero performance.
- H2 · AMD unified-memory systems. Ryzen AI Halo specifications; Max+ 395 specifications. 128 GB, 256-bit LPDDR5X-8000 = 256 GB/s. OEM settings affect GPU allocation; faster-memory variants can differ. The Halo platform specifications publish up to 60 FP16 TFLOPS and 50 NPU TOPS (INT8). The AMD Halo architecture whitepaper describes feeding the GPU, CPU and NPU simultaneously. We sum only the GPU and NPU ratings, 60 + 50 = 110 trillion mixed operations/s; simultaneous peaks have not been demonstrated here. The previous nominal FP32 estimate has been replaced by the published FP16 rate. The NPU is not used in the cited llama.cpp decode measurement. Dense INT8 matrix throughput is supported by the XDNA2 GEMM architecture study; no sparsity multiplier is applied to either component.
- H3 · RTX 4090. NVIDIA product specifications and Ada announcement: 24 GB, 1,008 GB/s, ~1,321 sparse FP8 TFLOPS, divided by two for ~661 dense. Quantized 4-bit weights can run without native FP4 arithmetic.
- H4 · R9700. AMD specifications (32 GB / 640 GB/s), Sapphire precision brochure, p. 1: explicitly “FP8 Matrix Dense — 383 TFLOPS”.
- H5 · RTX 5090. Product page; NVIDIA Blackwell architecture whitepaper: dense / sparse FP4 1,676 / 3,352 TFLOPS. Dense FP8 is half dense FP4 (838). 32 GB, 1,792 GB/s.
- H6 · RTX PRO 6000. NVIDIA PRO architecture whitepaper, pp. 45–47: Workstation FP4 2,015.2 / 4,030.4 TFLOPS and FP8 1,007.6 / 2,015.2 (dense / sparse). 96 GB ECC, 1,792 GB/s. The Workstation datasheet specifies PCIe 5.0 ×16. Its per-direction encoded-data ceiling is 32 GT/s × 16 lanes × 128/130 ÷ 8 = 63.0 GB/s before packet overhead. The dual-card row assumes full-width slots: this is a link ceiling, not measured GPU-to-GPU throughput; host routing, lane allocation, peer access and staging can reduce it. The 300 W Max-Q has different compute; the two-card row sums full-power Workstation-class peaks and does not imply linear inference scaling.
- H7 · Spark. NVIDIA specs and PNY’s explicit footnote, “Theoretical FP4 TOPS performance using sparsity feature.” 1,000 sparse TFLOPS / 2 = 500 dense FP4; FP8 is half-rate, 250. This is a theoretical normalization, not measured GEMM. The NVIDIA two-Spark guide specifies a 200 Gb/s direct link: 200 ÷ 8 = 25 GB/s per direction before protocol overhead. This is separate from each node’s memory bandwidth. Two Sparks require sharding and communication; 256 GB is not one bus at 546 GB/s.
- H8 · Mac Studio. Apple specs list 96/256/512 GB M5 Ultra, 1.2 TB/s. 21 September review explicitly says 512 GB is due late October. All direct M5 Ultra measurements cited here used 256 GB. M5 Max Studio has the same stated 614 GB/s maximum as the full Max laptop; thermals and runtime can change speed.
- H9 · DGX Station. Current NVIDIA datasheet, p. 3: 252 GB HBM3e at 7.1 TB/s plus 496 GB LPDDR5X at 396 GB/s; 900 GB/s NVLink-C2C, 1,600 W system / 20 A circuit. FP4 reads “20 | 15³ PFLOPS”; footnote 3 is “Without sparsity.” Thus 15 PFLOPS is the explicitly specified dense FP4, not 10 derived from a generic halving rule. FP8’s 10 PFLOPS is sparse; dense 5 PFLOPS is derived. This shipping/OEM spec supersedes the earlier 784 GB / 8 TB/s announcement. Optional RTX graphics is not counted.
Measured-speed ledger
These are heterogeneous, externally reported system results, not a chip leaderboard. MTP / speculative decoding is retained where used by the cited setup. A configured maximum context is not the actual prompt length.
| Source | Exact useful measurement | Conditions / limits |
|---|
| B1 · MTPLX | M5 Max 128 GB, Flash-Next Optimized Speed: 63.5 decode tok/s, 1,094 prefill tok/s, TTFT 60.1 s. | MTPLX 2.12.0, 65,502-token prompt, 512 generated, thinking off, MTP; fans at maximum. Pack: 115.1 GB download incl. 32 GB SSD n-gram table; ~83 GB resident, catalog peak ~87 GiB. Developer-reported, not this report’s replication. Other logged results: 79.3 tok/s at 9K; 50.3 at 200K. Older oMLX 0.6.3 result: 28.5 tok/s at 1K on M5 Max 40c/128 GB, oQ4e-mtp; illustrates runtime dependence. |
| B2 · Strix Halo | Qwen3.8-27B ROCmFP4_FAST: 30.56–36.04 tok/s with near-greedy MTP; ~14 without speculation. | Ryzen AI Max+395 128 GB. The 36.04 peak uses a 512-token structured/code prompt; 4K results are 30.56–32.24. Source documents a Vulkan MTP regression in v1.7.0. The range qualifies under the upper-bound rule, not as a sustained 35 tok/s guarantee. “ROCmFP4” is a storage/kernel layout, not evidence of native RDNA 3.5 FP4 tensor units. |
| B3 · MacStories | RTX 5090 Qwen27 Q4_K_M: 59 tok/s at 6,091 prompt tokens, no MTP. M5 Ultra 256 GB: GLM-5.3-Flash 41 tok/s short prose/code, 28 at 61,434-token retrieval; Flash-Next 108 tok/s prose, 143 code. | RTX: Windows, LM Studio CUDA 12, full GPU offload. Mac: oMLX 0.7.0.dev2 / macOS 27, pipenetwork GLM mixed-4_8bit and Jundot Flash-Next-oQ4e-mtp. Short rates are native runtime medians of three passes. GLM rate includes reasoning tokens (low effort). Flash-Next fixed-512 output at 4K/16K/64K/128K/256K: 90.7/87.8/83.8/60.6/74.7 tok/s, one run each. Do not read 108 as a full-context guarantee. GLM ran out of memory at 256K in the 256 GB system. |
| B4 · Kubesimplify | 2× RTX PRO 6000, Flash-Next FP8: 81.45 tok/s single stream, vLLM TP2. | 1,024 synthetic input / 512 output tokens, max context 32,768; checkpoint 172.8 GiB. Four GPUs were slower without MTP (64.61), showing communication costs. TP2 ran with VLLM_PLE_CPU_OFFLOAD=1. A single PRO 6000 NVFP4 load fit (74.75 GiB weights plus non-torch, 11.76 GiB KV) but the API server never came up, so this source has no single-card speed; B9 supplies one. Separate Spark UD-IQ1_S GGUF: 67.55 GiB, tg128 34.54 ±0.18 tok/s; not the NVFP4 Spark setup in the main table. |
| B5 · Madeye | 1× Spark, Flash-Next NVIDIA NVFP4: 36.83 prose / 45.75 code tok/s. Prefill 1,606.5 at 8,215 tokens; 1,824.95 at 32,791. | 10 September; vLLM nightly 8a728663, TP1, MTP3, FP8 KV, staged disk PLE (FP8 table 47.68 GiB), maximum 262,144 context. Decode: median of three 384-token generations after warmup; exact short prompt size not specified in the extracted report. Excludes TTFT. This is an offloaded-table fit, not all weights resident in 128 GB. |
| B6 · Dual Spark | DeepSeek V4 Flash original checkpoint, vLLM TP2: ~41 tok/s short, 40 at 8K/100K, 38 at 200K, 32 at 500K. | Official checkpoint described by tester as “FP8”; official model uses mixed FP4/FP8. 200 Gb/s direct QSFP56 RoCE. 100K prefill ~1,900 tok/s, TTFT ~52 s. The ~350 tok/s headline elsewhere is 32-stream aggregate and is deliberately not used here. This predates the 0731 update; the speed is not evidence for V4.1. |
| B7 · Halo Flash-Next | Ryzen AI Max+ 395 128 GB, Flash-Next UD-IQ4_XS: 31.7 tok/s new code and 47.1 file rewrite at 8K; 25.4–28.6 at 24K; 16.8 at 8K without speculation. | 29 August headline post: ROCm 7.1, llama.cpp qwen4exp with native MTP, GPU TOP_K fix and --spec-type draft-mtp,ngram-mod, greedy, single stream. Replications in the same thread: 38 tok/s on Windows/HIP at 262K (85–100% acceptance, 74.0 GB in a 96 GB carve, 26.8 GiB PLE table in host mmap); Vulkan with a Q5_K trunk 29–61 tok/s across prose, new code and rewrite at short–32K. One commenter saw 25–36 tok/s on identical rows across two days. Community results, not this report’s replication. |
| B8 · AMD day-0 | Radeon AI PRO R9700, Qwen3.8-27B: up to 51.8 tok/s. Ryzen AI Max+ 395: up to 24.5. | Vendor-reported, preliminary. Windows, llama.cpp Vulkan, MTP=2 on R9700 and MTP=4 on Max+ 395, average token generation over three or more runs. Quantization and prompt length not stated. Confirms that native-MTP drafting works on the R9700 Vulkan path. The Halo figure is below B2 and the Halo row now uses Flash-Next (B7). |
| B9 · PRO 6000 Flash-Next | 1× RTX PRO 6000, Flash-Next NVFP4: 81–88 tok/s single session without MTP; 120–173 tok/s with MTP in real workloads. | 28 August forum post, vLLM (version not stated). The 51B n-gram table sits in system RAM and needs about 100 GB spare. MTP cut maximum context from 268K to 200K. Prefill ~12,000 tok/s at 100K. 341 tok/s is a 6-client aggregate and is not used. Context Studios lists 207 tok/s for an SGLang recipe on the same card; its conditions are not documented and it is not used. |
| B10 · RTX 4090 | Qwen3.8-27B Q4_K_S (16.68 GiB): 46.16 tok/s at 4K, 44.32 at 16K, 38.41 at 64K. | Hardware Corner llama.cpp test as tabulated by OpenClaw DC: build 153d324bc, Flash Attention, no MTP. The same table gives RTX 3090 40.31 / 38.77 / 33.95. ComputingForGeeks reports more than 100 tok/s on a 4090 with UD-Q4_K_XL plus the built-in MTP head; not used for the headline. |
| B11 · 2× Spark GLM-Flash | GLM-5.3-Flash NVFP4, vLLM TP2, one request: 14.60 tok/s without speculation; MTP k=3/4/5 26.47/24.10/22.63; DFlash2 k=7 25.57. Earlier checkpoint: 30.84 with DFlash2. | Classmethod, 1–2 September, 262K context, greedy, 256 output tokens, median of 3. NVIDIA forum: 21.45–21.8 with MTP-4; the thread title’s “60 tok/s peak” and a second-hand 46.9 are not supported in the posts. These early-stack results are superseded by B16. |
| B12 · M5 oMLX | MacBook Pro M5 10-core GPU 32 GB, Qwen3.6-35B-A3B 4-bit: 59.5 tok/s at 1K, 56.9 at 4K; second run 52.9 / 51.0. | Community submissions, oMLX v0.3.6, macOS 26.4.1 / 26.3.1, 24 April. Peak memory 19.2–20.3 GB. Different macOS builds explain part of the spread. |
| B13 · 4× PRO GLM-Flash | 4× RTX PRO 6000, GLM-5.3-Flash community NVFP4, SGLang TP4: 63.7 tok/s one request; 82.8 with EAGLE speculation. | NUMA-local four-GPU engine, short single request. About 45 GiB weights per GPU, so this NVFP4 checkpoint does not fit two cards. Used only as an order-of-magnitude anchor for the dual-PRO IQ4_XS estimate. |
| B14 · 2× Spark Flash-Next | Flash-Next NVFP4, TP2 across two Sparks: 51.9 tok/s median (runs 56.6 / 51.7 / 51.9) vs 41.7 on one Spark in the same harness. SGLang recipe: ~41–42 tok/s over a 30-minute soak. | vLLM, MTP=3, thinking off, concurrency 1, 40-prompt harness, ConnectX-7 RoCE (~185 Gb/s, ~0.3% used during decode), 12 September; per-prompt code 61.8 / English prose 37.9. The one-node run used staged PLE and the two-node run did not, so the gain is not fully isolated. SGLang: NEXTN MTP steps 3, accept length ~2.3, 200G direct DAC, 262K context, 27 August. Community results. |
| B15 · Station GLM-Flash | DGX Station GB300, GLM-5.3-Flash NVFP4 in HBM: ~137 tok/s without speculation; ~305–470 with DFlash2 (7-token blocks). Separate report: 206 tok/s with MTP, 3 speculative tokens. | 1 September community post: vLLM glm53-flash image, TP1, local-inference-lab NVFP4 checkpoint, FP8 KV, 1M context configured, temperature 0, thinking off, draft acceptance 54–67%; code prompts at the top of the range. The 206 report (27 August, r/LocalLLaMA via mirror) used temperature 1.0 and gave no prompt details. Reference for the Station fallback; not the headline pairing. |
| B16 · 2× Spark GLM-Flash | EXL3 4-bit, TP2, DFlash2: 46.58 tok/s prose, 62.61 coding, 99.44 structured (n=10 each, 5 October); 17.9–18.2 without speculation. NVFP4 recipe: 46.9 code (warm, 74.1% acceptance, 28 August); nvidia checkpoint 51.9 code, 28.4 prose, 14.0 without speculation (24 September). | vLLM fork (0.1.dev20051+g487ecf187 plus overlays), ConnectX-7 RoCE 112–224 Gb/s, temperature 0, thinking off, 1M context configured, fp8 KV. EXL3: DFlash2 block 8, adaptive K; raw prose/coding median emitted tokens per step 3.1655/4.866; ~0.33 acceptance on prose in the NVFP4 repo. The NVFP4-KV variant is marked partial (long prompts crash rank 0); the fp8-KV TP2 setup is marked proven. Drafter licence CC BY-NC-ND 4.0. |
| B17 · oMLX DFlash2 | Mac Studio M5 Ultra 256 GB, GLM-5.3-Flash oQ4e (169 GiB resident): 38.8 → 62.4 tok/s at a 200-token prompt, 38.7 → 47.3 at 1K, 32.9 → 44.0 at 4K (baseline → DFlash2 adaptive). | Merged 30 September. Greedy, 256 new tokens, engine path. 59–77% acceptance, 2.4–4.3 committed tokens per verify cycle. Reviewer on M3 Ultra: 31.2/27.2/26.9 → 39.2/36.8/36.3. Its full 1.22–1.61× gain range is the drafting factor used for the GLM-5.3 estimates. |
| B18 · GLM-5.3 drafter | GLM-5.3 at concurrency 1: 113 tok/s autoregressive; MTP 1.83–2.63×; DFlash2 2.16–3.39× (MT-Bench to MATH-500). Acceptance length 4.19–6.02 with DFlash2. | Four GB300 GPUs, TP4, SGLang, temperature 1.0, block size 8, 128 samples. A different hardware/runtime setup; its gains are not transferable bounds for split-memory or Apple systems. Licence CC BY-NC-ND 4.0. |
| B19 · 2× PRO Server Edition | GLM-5.3-Flash, TP2, adaptive EAGLE: 164.8 tok/s median decode; 153.5 aggregate wall-clock tok/s at concurrency 1, 105.7 ms median TTFT. Code workload, forced 256 output tokens, n=3. | SGLang v0.4.3 SM120; ormandj/GLM-5.3-Flash-W4A16-NVFP4-K32-Experts-FP8-WO, revision ee0989a944b0e213589191d7fca63af825a0741e. Adaptive EAGLE 5/1/6, FP8 KV. Target 81.44 GB + draft 3.20 GB per rank; tight memory. Configured context 524,288, shared pool 500,000; prefill tested through 64K, not proof of a full 512K request fit. Different artifact, backend and Server Edition hardware from the Workstation IQ4_XS forecast. Experiment journal. |
How the estimates were made
Decode scope: one active request, warm model, short prompts; the remaining analytical scenarios target roughly 2–8K. Four rows remain estimates: M5 Pro, dual PRO, M5 Ultra 512 GB and Station. M5 Pro spans fresh coding through favorable repetitive editing. Measured rows retain their stated contexts and drafting settings: M5 Max, Spark, Halo, R9700, dual Spark and M5 Ultra 256 GB include MTP or DFlash2 drafting; RTX 4090, RTX 5090 and the single-PRO headline do not. The separate memory comparison budgets 128K context. The M5 Ultra 512 GB and Station estimates assume DFlash2 drafting; the dual-PRO estimate adds no speculative multiplier. Ranges are planning scenarios, not statistical confidence intervals.
- M5 Pro with drafting: Qwen3.8-27B MTPLX Optimized Speed (dynamic Q4, 20.4 GB weights), native MTP head, tuned draft depth 1–3, single stream, approximately 2–8K prompt, thinking off or medium reasoning. MTPLX reports 52.6 tok/s on M5 Max at 4,061 prompt tokens / 512 output; its model card reports 58.7 tok/s on a fresh medium-reasoning coding task, official sampling temperature 1.0 / top-p .95 / top-k 20. Both already include MTP; no extra speculative multiplier is applied.
- Cross-laptop extrapolation: take an M5 Max MTPLX reference at 614 GB/s, multiply by the laptop’s bandwidth ratio and a scenario factor of 0.8–1.0 for verification, runtime and thermal differences:
speed ≈ reference × (bandwidth / 614) × 0.8…1.0. On M5 Pro (307 GB/s) the fresh-coding references 52.6–58.7 give 21.0–29.4 and the 87.6 repetitive-editing reference gives 35.0–43.8; the headline 21–44 spans both (see M5 Pro). On base M5 (153 GB/s) Qwen3.8-27B scales to only 10.5–14.6 fresh and ~22 editing, so the base M5 row uses the measured small MoE instead. Verification compute need not scale with bandwidth, so real results can fall outside these ranges.
- Draft acceptance and fit: the model-card coding run reports acceptance by depth of .95 / .88 / .80 and a 50–53 ms verifier round on M5 Max. The estimates assume similarly favorable coding, not guaranteed acceptance on arbitrary tasks. MTPLX uses the model’s own head, not a second full draft model; its catalog reports roughly 25 GiB serving peak for this pack, so 32 GB needs a light application load and short context. Tune depth on the target Mac. Long xhigh reasoning was only 35.1–37.3 tok/s on M5 Max in the same card and is outside this faster planning scenario.
- M5 Ultra 512 GB → GLM-5.3, 18–40 tok/s with drafting, conditional estimate. The 512 GB model has the same chip and 1.2 TB/s as the 256 GB model; its extra capacity is what lets it run GLM-5.3 (365.3 GB IQ4_XS, or a ~420 GB oQ4-class MLX build). Without drafting: ~40B active parameters × ~0.60 bytes ≈ 24 GB per token; 1,200 GB/s × 30–50% effective utilization / 24 ≈ 15–25 tok/s, which fails the gate. Cross-check: GLM-5.2 oQ4 on a 512 GB M3 Ultra measured 14.5 tok/s, or ~21 scaled to 1.2 TB/s. With drafting, apply the full B17 gain range (1.22× at 1K to 1.61× at 200 tokens) as a transfer assumption: 18.3–40.2, displayed as 18–40. A DFlash2 drafter for GLM-5.3 exists (B18), and oMLX has merged DFlash2 for GLM-5.3-Flash, but no full GLM-5.3 run on a Mac was found. The configuration ships late October. Low confidence.
- DGX Station / GLM-5.3 IQ4_XS: 19–40 tok/s with drafting, conditional estimate. Start with 365.3 GB weights and 252 GB HBM; reserve 24–40 GB HBM for runtime, cache and verification buffers. Approximately 212–228 GB weights stay in HBM and 137–153 GB in CPU memory. Assume balanced expert placement makes the CPU-served share of active-weight traffic about 38–42%, rather than treating all 748 GB as HBM.
- Station bandwidth calculation: use the same approximate 24 GB active-weight traffic per token as the Ultra estimate. At CPU fraction
x, effective bandwidth fraction u, and non-weight overhead h, t ≈ 24×(1−x)/(7100×u) + 24×x/(396×u) + h seconds/token. With x=.38–.42, u=.50–.70 and h=.004–.008 s, this gives 15.9–25.1 tok/s without drafting. In this assumed split, CPU reads account for about 90% of modeled weight-read time, excluding other work. Resident weight fractions do not establish active traffic fractions. The 900 GB/s NVLink-C2C link does not eliminate the slower 396 GB/s CPU-memory supply; bandwidths are not added.
- Station drafting assumption: B17 reports gains of 1.22× at 1K, 1.34× at 4K and 1.61× at 200 tokens. Transferring the full 1.22–1.61× range gives 19.5–40.3 tok/s, displayed as 19–40. This is a sensitivity scenario, not a conservative bound: acceptance, expert reuse, drafter work and backend support differ for full GLM-5.3. Drafter weights, cache and verification workspace require an additional, currently unquantified memory allowance. Without working drafting, the 16–25 baseline fails the gate and GLM-5.3-Flash in HBM (B15) is the fallback.
- Station alternatives: MiMo-V2.6-Pro scores 46, above GLM-5.3, and is MIT-licensed, but its 1.0T / 42B-active MXFP4 checkpoint is ~566 GB. On Station 338–354 GB of it would sit in CPU memory (resident fractions do not establish actual traffic fractions). Assuming ~23 GB active-weight traffic per token and CPU traffic share x=.60–.63, the same u, h and transferred drafting gain give 15–29 tok/s; this scenario fails the gate, not a measured upper bound on the model. No measured local result was found. GLM-5.3-Flash on Station fits entirely in HBM and is measured at ~137 tok/s without speculation, 206 with MTP and 305–470 with DFlash2 (B15): the fast alternative if score 42 is enough.
Capacity-first pairings and projected speeds
Consistent selection rule: choose the highest AA-scoring model in the expanded local shortlist whose Q4-class or higher-precision weights fit with short-context serving headroom and whose displayed decode upper bound reaches 35 tok/s. Cloud ceiling models are excluded from hardware matching. SSD lookup-table offload is allowed; bulk expert weights remain in accelerator-accessible memory (Station uses its coherent CPU tier). Extremely compressed Q1/Q2 fits are excluded. Projected allocations, runtime support and drafting remain conditional.
- Two Sparks → GLM-5.3-Flash, score 42: the model fits two Sparks (82.4 GiB per node for EXL3 4-bit; the 242 GiB NVFP4 checkpoint also fits). The first published runs were slow: 14.6 tok/s without speculation and 21–31 with MTP or early DFlash2 (B11). Later stacks with the DFlash2 drafter reach the gate (B16): 28.4 prose and 46.9–51.9 code with NVFP4, and 46.6 prose, 62.6 coding and 99.4 structured output with EXL3 4-bit (n=10 each). The EXL3 configuration alone is displayed as 47–63, prose to coding; NVFP4 is a separate 28–52 alternative; structured output is excluded as unrepresentative. The DFlash2 drafter is licensed CC BY-NC-ND 4.0 (non-commercial). Flash-Next (40), split across both nodes with TP2, remains the fallback at 42–52 tok/s (B14).
- Two RTX PRO 6000s → GLM-5.3-Flash, score 42: select the B19 expert-NVFP4/FP8-WO checkpoint with SGLang TP2 and adaptive EAGLE. Measured on two Server Edition GPUs: 164.8 tok/s median single-stream decode on code prompts. The Workstation headline rounds this to 165 tok/s as a transfer estimate (E), assuming comparable full-power hardware, host PCIe topology and the pinned runtime. No matched Workstation run has been verified. Each rank loaded 81.44 GB target + 3.20 GB draft weights; cache and recurrent-state headroom is tight. This replaces the slower IQ4_XS layer-split forecast as the preferred setup.
- Halo 128 GB → Flash-Next, score 40: B7 measures 31.7–47.1 tok/s at 8K with UD-IQ4_XS, MTP and n-gram drafting on ROCm. The upper bound qualifies; prose and new-code workloads can stay below 35, and same-build runs varied 25–36 tok/s across days. The Flash-Next trunk (~60 GB) stays in GPU memory and the ~27 GiB PLE table in host memory, all within 128 GB. Qwen3.8-27B (34) is the fallback: ROCmFP4_FAST with near-greedy MTP reports 30.56–36.04 tok/s (B2), while AMD’s own result is 24.5 (B8).
- Single PRO 6000 → Flash-Next, score 40: headline measured (B9) at 120–173 tok/s with MTP; baseline 81–88 without MTP, on vLLM with the n-gram table in host RAM rather than on SSD. The weights fit: Kubesimplify’s load used 74.75 GiB (B4). Budget a host with roughly 100 GB of free system memory.
RTX PRO: measured anchors and speed forecasts
Verified anchor: Kubesimplify reports 81.45 tok/s on two RTX PRO 6000s with Flash-Next FP8, tensor parallelism 2, one request, 1,024 input / 512 output tokens, no MTP. Four cards achieved only 64.61 tok/s. This is direct evidence against scaling decode from summed peak bandwidth alone. Single-card FP8 did not fit. The single-card NVFP4 measurement is recorded in B9. B19 now measures two Server Edition cards at 164.8 tok/s with a different quantization and adaptive EAGLE. The preferred Workstation setup now uses the B19 recipe, with 165 tok/s explicitly projected from the Server measurement rather than a matched Workstation benchmark.
- Dual PRO / GLM-5.3-Flash IQ4_XS: best ~55 tok/s; 41–70 scenario range. Assume 10.8–13 GB/token, u=.50–.65 and h=5–10 ms including communication and non-weight work. Because the GGUF path splits layers in sequence, each token streams through one card’s 1,792 GB/s at a time: central
12/(1792×.60)+.007 = .01816 s gives 55.1 tok/s; endpoints are 40.8–70.1. This is a legacy no-drafter comparison, no longer the headline. If a tensor-parallel IQ4 backend overlaps reads across both cards (3,584 GB/s), the same assumptions give 58.0–103.8 (central 79.5); that is an alternative no-drafter scenario, not the headline. Cross-check: four PRO cards with SGLang TP4 and NVFP4 measured 63.7 tok/s (B13). PCIe topology or slow IQ4 kernels can push results below this range.
- Drafting and Spark cross-check: Spark’s 36.83–45.75 tok/s already includes MTP3, so multiplying it by 6.6 would mix memory scaling with verification and acceptance effects. The measured single-PRO speed with MTP (120–173, B9) is ~3.3–3.8× the Spark MTP3 result, not 6.6×: this unmatched comparison cannot isolate a fixed latency floor. The preferred dual-PRO estimate transfers B19’s complete drafting-enabled recipe; no generic speculative multiplier is applied. B13’s +30% EAGLE result on four cards is not used to scale it. These are best-effort engineering forecasts, not matched benchmark results or guaranteed bounds.
Confidence: the 165 tok/s Workstation projection remains unverified. It assumes the Server Edition B19 recipe transfers without a throughput penalty; topology, clocks, cooling, software and workload can change results. A matched Workstation run is needed. B19 is a short code-output test, while B9 spans unspecified real workloads; these are best documented setups, not a controlled cross-device comparison. Drafting can reduce context capacity and slow inference when acceptance is low.
Lighter models that pass the speed gate
Base M5 → Qwen3.6-35B-A3B: AA score 18, with downloadable weights and an existing Apple runtime pack. This is a ~3B-active MoE, not the older 27B dense model. No verified Qwen3.8 replacement in this small-active MoE class was found. Measured (B12): 51.0–59.5 tok/s at 1K–4K with 4-bit MLX on oMLX 0.3.6, peak 19.2–20.3 GB, displayed as 51–60. Even the favorable Qwen3.8-27B editing reference below scales to only ~22 tok/s on base M5, so the MoE remains the useful-speed choice.
M5 Pro → Qwen3.8-27B: use same-generation evidence rather than declaring 3.6 faster from unmatched tests. The 3.8 dynamic-Q4 pack reports 52.6–58.7 tok/s for fresh coding/short-prompt work on M5 Max, and 87.6 for rewriting a file it just wrote. Scale each by 307/614 and .8–1.0: ~21–29 for fresh coding, ~35–44 for repetitive editing. The headline 21–44 spans those workloads and meets the user's upper-bound rule, not a general 35 tok/s floor. It is an estimate on M5 Pro; the 87.6 source is a favorable MTPLX 2.10.0 result, not a matched cross-device measurement. The 3.6/3.8 configs share core dimensions, but quantization payload, learned draft acceptance and kernels still affect speed. The older 3.6-27B recommendation is removed. Independent cross-check: a 4-bit Qwen3.8-27B MTPLX pack reports a 2.33× MTP multiplier verified on M5 Pro (best depth 3, temperature 0.6; no absolute speeds published). Applied to the ~17–18 tok/s that oMLX community runs report for plain Qwen3.5-27B on M5 Pro, that gives roughly 40–42 tok/s, consistent with the upper half of the range. No direct M5 Pro measurement of this pack was found, so the row stays an estimate. For a faster lower-score option, oMLX reports 98–102 tok/s for the Qwen3.6-35B-A3B MTPLX pack on M5 Pro.
R9700 → Qwen3.8-27B Q4-class + MTP: AMD’s day-0 results (B8) report up to 51.8 tok/s on one R9700 with llama.cpp Vulkan and MTP=2. This is vendor-reported and preliminary, with quantization and prompt length unstated, but it settles the earlier condition that native-MTP drafting must work on this card. The previous 30–51 estimate (24–32 bandwidth baseline × assumed 1.25–1.6× drafting) is retired. Without drafting the 24–32 baseline would fail the gate; Qwen3.6-35B-A3B Q4 remains the no-drafting fallback.
Fallback memory budget: Qwen3.6-35B-A3B Q4 weights are budgeted at ~22 GB (quantization-dependent planning value). Its config has 10 full-attention layers with 2 KV heads of dimension 256: 128K 16-bit K/V is 2.68 GB; allow one MTP attention layer and FP32 recurrent state →~3 GB. Total ~25 GB plus 4–8 GB runtime, with no SSD tables. This is a memory example, not a 128K fit guarantee on base M5 after adding OS memory.
Release dates: each model row links to its release source. Dates identify the displayed model/version, not a third-party quantization. GLM-5.3 uses the 18 August API release-note entry; its earlier announcement and later open-weight availability are different events. DeepSeek 0731 uses its 31 July checkpoint release, not the original April preview.
Cloud reference trajectory: four Claude Opus releases on the same Intelligence Index v4.3.2 show how fast the frontier moved during 2026: Opus 4.6 (max, 5 February) 32, which AA marks as an estimate pending independent evaluation; Opus 4.8 (Max, 28 May) 42; Opus 5 (Max, 24 July) 51; and Opus 5.5 (Max, default fallback, 22 September) 58. Opus 4.7 is omitted for space. The best locally runnable model in this shortlist, GLM-5.3 (45), scores above Opus 4.8 and below Opus 5; GLM-5.3-Flash (42) matches Opus 4.8, and Flash-Next (40) is close to it. All four are proprietary references with no downloadable weights or disclosed serving memory. They show the trajectory, not hardware recommendations. GPT-6 Astra (53) was dropped to keep one consistent frontier series.
128K single-user memory budget
Common basis: 131,072 total cached tokens (prompt plus generated output), one active request, no extra prefix-cache copies or concurrent sessions. All displayed memory is decimal GB (10⁹ bytes), including converted GiB sources. Resident memory means weights plus persistent attention/recurrent/MTP state; SSD lookup tables are listed separately. Runtime allowances are planning reserves for scratch buffers, graph/workspace and allocator overhead, not measured peaks. Add roughly 6–10 GB for the OS and applications on unified-memory machines. Chunked prefill is assumed; unchunked long-prompt processing can require much more memory.
- Qwen3.8-27B: official config has 16 full-attention layers, 4 KV heads and 256-dimensional heads. With 16-bit K/V:
131072 × 16 × 2 × 4 × 256 × 2 = 8.59 GB. Allow one full-attention MTP layer (~0.54 GB) plus 48 FP32 recurrent states (~0.15 GB) → ~9.3 GB. With 20.4 GB dynamic-Q4 weights, resident total is ~30 GB; add 4–8 GB runtime. Thus the earlier 32 GB laptop fit is for short context, not this 128K budget. Cache quantization can reduce demand if the chosen runtime supports it.
- Flash-Next: official config has 12 full-attention layers, 2 KV heads, dimension 256, plus one MTP attention layer. Their 16-bit KV is ~3.49 GB. Add ~0.10 GB sparse-indexer keys (4× compression) and ~0.11 GB FP32 recurrent state → ~3.7 GB, rounded to 4. The selected MTPLX pack keeps ~83 GB weights resident and ~32 GB lookup tables on SSD: ~87 GB resident before 4–8 GB runtime. Different NVFP4 or all-resident packs have different weight budgets.
- GLM-5.3: official config has 78 layers, MLA latent rank 512 and rotary key dimension 64. With one MTP layer and compressed 16-bit latent cache:
131072 × 79 × (512+64) × 2 ≈ 11.93 GB. Budget ~0.74 GB for 22 sparse-indexer key sets → ~12.7 GB, rounded to 13. IQ4_XS weights 365.3 GB + cache ≈378 GB; runtime reserve 8–16 GB. This assumes a backend that stores compressed MLA, not expanded per-head K/V.
- GLM-5.3-Flash: official config has 11 sparse-attention layers with latent rank 512 and no rotary-key component, plus 34 recurrent layers. Budget one extra MLA layer for MTP: ~1.61 GB latent KV, ~0.10 GB 4×-pooled indexer keys and ~0.14 GB FP32 recurrent state → ~1.85 GB, rounded to 2. Depending on Q4 artifact, 156.82–199.71 GB weights + cache ≈159–202 GB; runtime reserve 8–16 GB. Backend distinction matters: expanded 64-head K/V instead of compressed MLA can consume ~95 GB for the 11 attention layers alone (~103 GB including one MTP layer), making the 128K fit much larger. The table is the compressed-cache scenario, not a claim that every oMLX/GGUF backend uses it.
- DeepSeek V4 Flash 0731: 0731 config uses 21 attention layers compressed 4× and 20 compressed 128×, 512-dimensional shared KV, 128-dimensional index keys and a 128-token local window. Following the reference implementation’s BF16 cache layout, main compressed KV plus index keys and local windows total ~0.91 GB at 128K; reserve ~1 GB including small compressor state. About 155.1 GB Q4 weights + cache ≈156 GB; add 8–16 GB runtime. Three 0731 draft layers add small local windows, not three full context copies.
- DeepSeek V4.1 Flash: DwarfStar Q4 main weights 294.15 GiB =315.84 GB; SSD Engram tables 188.83 GiB =202.75 GB. The model card’s packed FP4 global cache is 890 bytes/token, or 0.117 GB at 128K, plus local state. A BF16 reference-layout calculation from the shared KV/index source layers and implementation gives ~0.56 GB including local windows. Budget ~0.2–0.6 GB persistent cache; displayed resident total rounds up to ~317 GB, plus 8–16 GB runtime. These cache formats are alternative backend scenarios, not verified DwarfStar allocations. Optional vision weights add ~0.97 GB.
35 tok/s selection and ordering
Eligibility requires a measured value or range upper bound ≥35 tok/s, not a midpoint ≥35 and not a guaranteed floor. Choose the highest-scoring eligible model, then order devices by AA score and decode midpoint ascending. Forecasts are labelled E and remain conditional rather than demonstrated eligibility. The speed gate uses each row’s stated measurement/scenario; the separate 128K memory budget does not imply that speed at 128K. Halo (32–47), M5 Ultra 512 GB (18–40) and Station (19–40) qualify despite lower endpoints below 35. The two GLM-5.3 rows qualify only with an assumed DFlash2 drafter; without it both fall to 15–25 and the next pairing is GLM-5.3-Flash. MiMo-V2.6-Pro (46) fails on Station (15–29). Extra capacity buys headroom, not automatically more decode speed. Scores describe evaluated model settings, not verified local quantization quality.
Parameter counts and architecture
B = billion parameters; active = per-token backbone work, not resident memory. Dense Qwen27 uses essentially its whole language backbone; MoE models select experts, but all resident experts still consume memory. GLM-5.3: 753B / 40B active; GLM Flash: 320B / 18B; DeepSeek V4 Flash: 284B / 13B; Qwen3.8-27B: 27B dense; Qwen3.6-35B-A3B: 35B / 3B. Counts follow vendor conventions and do not always separately itemize vision or draft heads. Flash-Next’s ~180B sum is 125B backbone +51B lookup embeddings +4B MTP, with 6B active backbone. DeepSeek V4.1’s ~763B full checkpoint includes a 552B backbone and 196B Engram tables plus auxiliary components; 16B active is decode (prefill: 8B). Sparse table lookups and draft verification are additional work, so these active counts alone are not FLOP or bandwidth budgets. The cloud models’ parameter counts and Dense/MoE architecture are undisclosed.
Model storage, fit and selection notes
- Model size is not serving memory. GB = 10⁹ bytes; GiB = 2³⁰ bytes. Vendor installed capacities are retained as advertised. Quantization sources use their displayed units. Plan for runtime allocations, KV cache, vision, MTP and the operating system; native 1M context does not promise a 1M deployment on each device.
- Qwen3.8-27B. Official model card; concrete 16.0/20.4 GB Q4 and 29.4 GB Q8 packs. Its 61.7 SWE-bench Pro used Qwen’s corrected tasks, Claude Code, temperature 1 / top-p .95 and 256K context. It also reports 73.0 Terminal-Bench 2.1 (Terminus). A 24 GB GPU can fit compact Q4 at modest context; the high-quality dynamic Q4 plus long KV may not. Base M5 recommendation assumes the 32 GB configuration, not 16 GB.
- Flash-Next. Official card: 125B backbone + 51B n-gram embeddings + 4B MTP (~180B total); 6B active backbone. Do not size this as only 125B. Q4_K_XL shards total ~111.4 GB, separate MTP Q8 4.14 GB, vision ~0.9 GB. 128 GB all-resident is marginal; the selected M5 Max and Spark recipes explicitly offload lookup tables. MTPLX uses dynamic Q4 with sparse-attention projections at Q8. GGUF, NVFP4 and oQ4e are not interchangeable.
- 96 GB alternatives. Flash-Next UD-IQ1_S is ~67.55 GiB (effective 3.28 bits/weight), so it fits a PRO 6000 with serving headroom; its low-bit experts have unverified agentic quality here. The Spark measurement for this quant is not a PRO 6000 measurement. A PRO 6000 DeepSeek V4 Flash ds4.c result reports 46.9 tok/s, using more compressed weights than the dual-Spark reference. The main map instead projects a higher-precision Flash-Next fit with SSD lookup-table staging; Qwen27 Q8 remains the simpler high-fidelity fallback.
- DeepSeek V4 Flash. Official original card; 0731 Q4_K_XL shards ~155.1 GB. The 54.4 DeepSWE figure comes from the Qwen Flash-Next comparison explicitly labeling 0731. Original-release benchmarks (e.g. SWE-bench Verified 79.0 / Pro 52.6 and Terminal-Bench 2.0 56.9) are different versions/protocols and not substituted for newer scores.
- GLM-5.3-Flash. Official card and quantization guide: IQ4_XS 156.82 GB, Q4_K_XL 199.71 GB; the measured Mac uses a separate mixed-4/8 MLX artifact. 84.3 Terminal-Bench 2.1 uses Claude Code 2.1.207, while 63.4 DeepSWE uses mini-SWE at 400K / six hours. 128 GB can squeeze lower-bit variants, but this guide prefers useful precision and headroom. Token-level agreement percentages are not task success rates.
- DeepSeek V4.1 Flash (‡). Official card: 552B backbone + 196B Engram (HF total ~763B including other components), 8B prefill / 16B decode active; vision; 1M context; 890 bytes/token global KV claim. DwarfStar-specific quant card: Q4 482.98 GiB file = 294.15 GiB main weights + 188.83 GiB native FP8 Engram. Lookup rows stream from SSD, not all tables into RAM. Main Q4 weights ~316 GB plus buffers fit the 512 GB tier. Q2 main 151.77 GiB can fit 256 GB, but has greater compression; neither inherits the full model’s scores. Source checkpoint df42c109f1defefcbfcedbe7d905718a12266e40. Runtime support is branch-specific; no matched speed is verified here. 74.2 DeepSWE uses mini-SWE / effort 100 / 1M; 90.6 Terminal-Bench 2.1 is DeepSeek Harness Minimal. This is not a controlled comparison to GLM’s 400K harness.
- GLM-5.3. Official card; actual quantization table. IQ4_XS 365.3 GB leaves much more headroom on 512 GB than Q4_K_XL 467.3 GB. ~753B total / 40B active from AA (some guides use 744B backbone). GLM-5.3 on Station fits the total coherent capacity, not the 252 GB HBM alone. GLM-5.3-Flash Q4 is the smaller all-HBM alternative, conditional on runtime allocations.
Broader capability and alternative hardware
Artificial Analysis’s current open-weight index ranks MiMo-V2.6-Pro (46), GLM-5.3 (45), Kimi K3 (44), then GLM-5.3-Flash (42) in broad intelligence. Considered but not shortlisted: MiMo-V2.6-Pro (1.0T / 42B active, ~566 GB MXFP4) fits only Station and fails the gate there (see Station alternatives); Kimi K3 (2.8T / 104B active) fits none of these devices at Q4; MiMo-V2.6-Flash (38; 309B / 15B active; MIT; ~166–178 GB at 4-bit, 59.4 tok/s reported on M3 Ultra 256 GB) scores below both GLM-5.3-Flash and Flash-Next, so it wins no row. Scores shown are v4.3.2 (not the earlier index that gave Qwen27 ~52). AA’s hosted-provider tokens/sec is deliberately not used as local hardware speed. Qwen3.8-Max and proprietary frontier models are not assumed to have downloadable weights. “Open weight” does not mean every license is identical: Flash-Next uses Qwen Community terms; GLM-5.3 has its own license; Flash variants from DeepSeek and GLM-5.3-Flash list MIT.
Tenstorrent QuietBox 2: four Blackhole chips on two p300c cards; 128 GB accelerator GDDR6 (32/chip), 1,024 GB/s per card / 2,048 GB/s aggregate; separate 256 GB host DDR5. The guide’s supported models include Qwen3-32B, Gemma 4 31B and Llama 70B; gpt-oss-120B and Qwen3.6-27B are experimental. A comparable verified dense FP4/FP8 figure and current target-model speeds were not established here. It is a specialist-stack alternative, not a drop-in CUDA/MLX equivalent. Rack GPUs (B200/B300/MI350) require server infrastructure and are outside this desk/laptop brief; DGX Station is the enterprise upper bound.
Devices not yet in the map: the Ryzen AI Max+ PRO 495 with 192 GB LPDDR5X-8533 (273 GB/s, up to 160 GB usable by the GPU, $6,399–9,249 across vendors, first systems ship November 2026) has no independent LLM measurements yet. Its 160 GB GPU limit is too tight for GLM-5.3-Flash IQ4_XS (157 GB) plus cache, so Flash-Next at Halo/Spark-class speed is the likely pairing. The used RTX 3090 (24 GB, 936 GB/s) runs Qwen3.8-27B at 40.3 tok/s at 4K without MTP and 66.4 with MTP at 8K. The Intel Arc Pro B70 (32 GB, 608 GB/s, $949 MSRP) measures only ~23 tok/s on 4-bit Qwen3.8-27B, so it would pair with Qwen3.6-35B-A3B.