AMD's reference architecture for agentic AI in the datacenter is eight stages: Gateway, Assemble context, Plan + Route, RAG, Reasoning, Tool exec, Verify, Stream. Every stage gets its own service in a rack. This part shows the same eight stages running as named processes on one Radeon AI PRO R9700 plus one Genoa-class EPYC. The shape of the reference is identical. Only the scale unit changes.
AMD's datacenter agentic AI reference is normally drawn as a row of services across many racks. On this Genoa + R9700 box, each stage is a named process — or a small pool of them. The mapping is one-to-one with the rack version. The unit of scale is what shrinks.
| Stage | Datacenter version | This box |
|---|---|---|
| 1. Gateway | LB + auth + rate limit | teams_claude_bridge · telegram_bot · open-webui (3 frontends) |
| 2. Assemble context | Vector store, BM25, doc fetch | ingest_email · ingest_teams · ingest_net · kb_ingest |
| 3. Plan + Route | Router service | brain_loop.sh worker pool |
| 4. RAG | Retrieval cluster | vLLM prefix cache (live 45% → 65% as KB warmed) |
| 5. Reasoning | LLM cluster | vLLM Qwen3-30B-A3B-AWQ on R9700 |
| 6. Tool exec | Tool registry + workers | meeting_loop · klue_loop subagents |
| 7. Verify | Critic / judge service | verify_worker pool |
| 8. Stream response | SSE / WS edge | vLLM SSE → frontends |
Part 9 captured a 4-day window at 49,288 requests. Two days later the same container has crossed 79k — load grew about 23% across an extra two days as the panel of agents kept firing. The error and abort counters are still zero. Prefix cache is warming, not cold-starting.
vLLM logs print a one-line scheduler snapshot every iteration. Captured live during one observation hour, the same R9700 that does 48 tok/s for a single agent rolls up to several hundred aggregate decode tok/s as the queue depth grows. The peak Running counter caught 36 concurrent requests in flight at one moment. KV cache use at that peak was 15.6%.
| Concurrent reqs | Aggregate decode tok/s | KV cache |
|---|---|---|
| 1 | 48 | 0.3% |
| 3 | 241 | 4.2% |
| 8 | 192–211 | 6.3% |
| 10 | 171 | 7.0% |
| 12 | 128–147 (+ 1131 prefill spike) | 6.8% |
| 36 (peak Running) | caught between batches | 15.6% |
The reference workflow lives on a Genoa-class EPYC because the orchestration is constant. Nineteen distinct host processes — three frontends, four ingestors, three cognitive loops, three dashboards, plus vLLM — share the socket. Steady-state CPU is sub-percent. p95 is 13%. p99 is 14.2%. Max is 28.2%. That is the signature of an orchestration tier doing real work in short bursts, not an oversized CPU sitting idle.
Two rocm-smi samples taken inside the same observation hour tell the story. Mid-quiet: 3% util, 21 W draw, sclk clock-gated down to 41 MHz. Mid-burst: 100% util, 275 W, full clocks. VRAM stays 92% resident the whole time — the model never reloads. The card is exactly the right size: it can rest, and it can sprint, and it does both inside the same minute.
| State | GPU % | Power | sclk | VRAM | Edge / Junction / Mem |
|---|---|---|---|---|---|
| Quiet (no in-flight requests) | 3% | 21 W | 41 MHz | 92% | 49 · 52 · 58 °C |
| Burst (peak Running ~12) | 100% | 275 W | full | 92% | 49 · 52 · 58 °C |
The 2.5-day telemetry window covers a normal weekday cycle. Only a single quiet window appears, 04:00–05:00 UTC, when overnight maintenance windows drain the ingest queues. Every other hour the box is doing real work. The heaviest single hour was 00:00–01:00 UTC at load1 p95 ~21.5 — the speech-preprocessing batch firing alongside an ingest sweep.
The Radeon AI PRO R9700 sits in the mainstream of the AI PRO stack — PCIe, GDDR6, no HBM. Yet 32 GB is enough to keep Qwen3-30B-A3B-AWQ resident with a 32k context and serve three frontends, nineteen background processes, and tens of thousands of requests per day with 84% of KV cache still free at peak. Headroom is not a small word here.
Parts 1 through 10 measured the box from the outside — throughput counters, scheduler snapshots, KV cache numbers, power and thermal samples, hourly load curves. They proved the box does the work. They did not open the lid on where the cycles go.
| Open question | Tool stack that will answer it | Coming in |
|---|---|---|
| Where is the CPU actually spending cycles? | perf · uProf · PerfXpert | Part 11 |
| Where is the GPU spending kernel time? | rocprof · rocprofv3 · omniperf | Part 12 |
| What is the per-stage latency budget across the 8-stage loop? | end-to-end timing instrumentation | Part 13 |
Ten parts in. One R9700, one Genoa EPYC, an eight-stage datacenter blueprint mapped end-to-end onto a single node. Same box, same workload, sharper lens from here on out.