Part 12 · The full sweep

Three backends, two axes, and the number hiding behind the yardstick.

One 30‑billion‑parameter model. One R9700. Every runtime, both halves of inference — and a prefill headline of 8,326 tok/s.

AMD Field AI · Pradeep Nallimelli · July 2026 · R9700 (gfx1201, RDNA4) · Qwen3‑30B‑A3B MoE

The experiment

One model, three ~4‑bit backends, two axes.

Same base weights (Qwen3‑30B‑A3B), same card, same -fa 1, concurrency 1. We measured prefill and decode across three runtimes.

Runtime 1

llama.cpp · HIP

ROCm rocBLAS, UD‑Q4_K_XL GGUF (~4.5 bpw). The compute‑bound baseline.

Runtime 2

llama.cpp · Vulkan

RADV KHR_coopmat matrix cores, same GGUF. The bandwidth play.

Runtime 3

vLLM · AWQ

AWQ‑4bit, paged / chunked prefill. The datacenter scheduler.

Two axes because inference has two halves: prefill (read the prompt, compute‑bound) and decode (write tokens, bandwidth‑bound). No single backend wins both.

Axis 1 · Prefill

Prefill scales with context — vLLM peaks at 8,326 tok/s.

vLLM AWQHIPVulkan

vLLM's paged prefill climbs with context (8,326 @ pp4096 ≈ 1.9× HIP, 2.5× Vulkan). HIP is a hump — peaks 4,635 @ pp2048, then declines. Long prompts are where the R9700 shines.

Axis 2 · Decode

Single‑stream decode flips the winner — Vulkan at 167.6 tok/s.

Decode is memory‑bandwidth bound. Vulkan's coopmat path hits 167.6 tok/s — 1.8× HIP, 3.4× vLLM, and past the old series baseline of 108.

vLLM's low decode (49) is a concurrency=1 artifact. Its strength is batched throughput, not single‑stream latency. Judge it on prefill, not this slide.

The verdict

Three champions, no overall winner — a routing rule.

Long‑context prefillvLLM

AWQ paged prefill scales up with context. 8,326 @ pp4096. Pick it for RAG, long documents, big system prompts.

Single‑stream decodeVulkan

RADV coopmat, 167.6 tok/s. Pick it for one interactive user who wants the fastest tokens‑per‑second.

Balanced / unknownHIP

Second in both axes, never last. The safe default when the workload mix isn't known yet.

Routing rule a customer can use tomorrow: prefill‑heavy → vLLM · latency‑sensitive decode → Vulkan · mixed → HIP. Same card, same weights, pick by workload.

The knob

Micro‑batch `ub=2048` is a universal resonant point.

Backend	pp2048 @ ub512	pp2048 @ ub2048	Gain
HIP	3,391	4,635	+36.7%
Vulkan	3,019	3,675	+21.7%

The -ub sweet spot reproduces on both HIP and Vulkan — it's a property of this MoE's working set on RDNA4, not a HIP quirk. And pp512 can never show it: -ub is pinned by prompt length.

The revelation

The old "3,491 record" was measured where the knob is switched off.

For eleven parts we benchmarked at pp512 — the one prompt length where micro‑batch tuning is mathematically disabled.

Old yardstick · pp5123,491

"Resisted all optimization." It couldn't — the winning knob was off.

Move to pp2048 · HIP4,635

+32.8% with zero new code — just the right axis.

Right backend · pp40968,326

vLLM paged prefill — the number that was there all along.

Measuring in the right place is optimization work. It's the unglamorous half — and the half honest enough to publish.

Intellectual honesty

What we can — and cannot — claim.

Can claim

The first documented, reproducible, apples‑to‑apples three‑way characterization (HIP vs Vulkan/coopmat vs vLLM AWQ) of prefill and decode for this 30B MoE on R9700 / gfx1201. Plus a real method finding — pp512 hides a 2.5× gap; ub=2048 is universal — and a routing rule.

Should not claim

An absolute throughput record. MI300X, H100/H200, even a 5090 exceed 8,326. It's an excellent number for a 32 GB workstation card — not a global maximum. And no new kernel was invented; the discipline is the contribution.

"I didn't do anything" — not quite. Realizing the baseline was measured where the knob is disabled is the work.

From measurement to engine

Now beat your own number — an R9700‑native prefill engine.

Today's sweep is the harness a purpose‑built engine proves itself against — the R9700 answer to a q36‑class CUDA prefill engine.

Path A · prefer first

MXFP4 → INT4 (AWQ)

Route experts through the wvSplitK INT4 MoE GEMM — vindicated on 30B (75 µs, no hang). Prefill first.

Path B

MXFP4 → FP8 (E4M3)

Upcast to RDNA4 FP8 WMMA matrix cores — the other native path to tensor throughput.

New kernel

gated‑DeltaNet scan

HIP chunked‑scan for the hybrid linear‑attention trunk — where long‑prefill work pays off.

Measure first, then beat it — every kernel improvement lands against a number that's already honest, reproducible, and on the record.

Part 12 · fin

8,326 was always there. We finally moved the yardstick to see it.

Three backends. Two axes. One honest number — and a runway to beat it.

Data · /home/amd/pradeepn/local_agent/knowledge/r9700_sweep/ · Lab notebook Ch. 11
Series · deepindeepsea.github.io/r9700-vllm-blogs · © 2026 AMD Field AI

Three backends, two axes, and the number hiding behind the yardstick.

One model, three ~4‑bit backends, two axes.

llama.cpp · HIP

llama.cpp · Vulkan

vLLM · AWQ

Prefill scales with context — vLLM peaks at 8,326 tok/s.

Single‑stream decode flips the winner — Vulkan at 167.6 tok/s.

Three champions, no overall winner — a routing rule.

Micro‑batch ub=2048 is a universal resonant point.

The old "3,491 record" was measured where the knob is switched off.

What we can — and cannot — claim.

Now beat your own number — an R9700‑native prefill engine.

MXFP4 → INT4 (AWQ)

MXFP4 → FP8 (E4M3)

gated‑DeltaNet scan

8,326 was always there. We finally moved the yardstick to see it.

Micro‑batch `ub=2048` is a universal resonant point.