hardware.co · AI hardware for on-prem LLM deploymentsVendor-neutral · Quote onlyBuilt to spec
hardware.co · November 18, 2026

EPYC vs Xeon for GPU Servers, PCIe Lanes and RAM Sizing

How AMD EPYC and Intel Xeon compare as host CPUs for GPU servers, with guidance on PCIe lane budgets, system RAM sizing, NUMA and core counts.

By Samuel Edwards

An open GPU server chassis showing CPU sockets, DDR5 DIMMs and PCIe risers feeding multiple GPUs

EPYC leads in PCIe lanes; Xeon leads in peak memory speed and multi-socket scaling. For most on-prem LLM builds, the host CPU is a container for lanes, memory channels and DRAM capacity, not the engine doing the work. Pick the family that fits your GPU count and I/O topology, then size cores last.

This piece puts specific numbers on the trade: lanes per socket, DDR5 channels and speeds, NUMA layout on the chiplet and tile architectures, and the core counts NVIDIA's own reference architectures ask for. Prices are quote only, so the goal is a spec that matches the workload rather than the biggest SKU the configurator will let you add.

The decision shows up on almost every quote: a 48- to 64-core host CPU paired with four or eight GPUs, with a second socket added by default. Half of those defaults are wrong for an inference box.

PCIe Lanes Are the First Constraint

An LLM server's host CPU exists to feed the GPUs, the NICs and the NVMe. Each PCIe Gen 5 GPU link is x16 at roughly 64 GB/s per direction. Four GPUs consume 64 lanes before you plug in networking. Add two 400 GbE NICs at x16 each and a few U.2 NVMe drives at x4, and lane budgets get tight fast.

A single-socket AMD EPYC 9005 'Turin' processor exposes 128 lanes of PCIe Gen 5 plus a handful of Gen 3 lanes for board management, and a two-socket EPYC system can present up to 160 PCIe Gen 5 lanes to the chassis. Intel's flagship Xeon 6980P brings 96 PCIe 5.0 lanes per socket. For single-socket 4-GPU workstations, that difference is the headline: EPYC fits four x16 GPUs, two NICs, and a boot pair without a switch; Xeon's 96 lanes force compromises or a PCIe switch that adds latency.

Lane count is not the whole story. Topology matters: how many lanes come off each root complex, whether GPUs share a switch, and how peer-to-peer traffic routes. Those details decide whether your 4-GPU server behaves like one fabric or four isolated pairs. We cover the switch and root-complex details in PCIe lane bottlenecks and keep the focus here on the CPU side.

PCIe lanes per socket, Intel Xeon vs AMD EPYC generations
PCIe lanes per socket, Intel Xeon vs AMD EPYC generationsXeon E5 v3/v4: 40 lanes; Xeon Scalable (Skylake): 48 lanes; Xeon Scalable (Cascade Lake): 48 lanes; Xeon Sapphire Rapids: 80 lanes; EPYC Naples: 128 lanes; EPYC Rome / Milan: 128 lanes; EPYC Genoa: 128 lanes0 lanes40 lanesXeon E5 v3/v448 lanesXeon Scalable (Skylake)48 lanesXeon Scalable (Cascade Lake)80 lanesXeon Sapphire Rapids128 lanesEPYC Naples128 lanesEPYC Rome / Milan128 lanesEPYC Genoa
EPYC has held 128 lanes per socket since Rome; Xeon climbed from 40 to 80 across the same window. Source: Serve The Home, 2024

Memory Channels, Capacity and the 2x Rule

Both flagships land at 12 DDR5 channels per socket. Intel's 6980P supports MRDIMM at up to 8800 MT/s, where EPYC 9005 tops out at DDR5-6400. For LLM inference, the GPUs hold the model, so host memory bandwidth rarely limits token throughput. Capacity and channel count matter more than peak MT/s.

Puget Systems' long-standing recommendation is to install at least twice the aggregate GPU VRAM as system memory. A 4-GPU H200 node with 141 GB per GPU carries 564 GB of VRAM; the rule points to 1.1 TB of DRAM, which sits comfortably inside a 12-channel socket populated with 96 GB RDIMMs. An 8-GPU B200 node puts aggregate VRAM at 1.5 TB; the rule asks for 3 TB of DRAM.

The NVIDIA DGX B200 pairs two Intel Xeon Platinum 8570 processors with 2 TB across 32 DIMMs, upgradable to 4 TB. That configuration sits under the strict 2x rule and leans on fast host I/O rather than oversized DRAM. For custom builds that hold multiple models in RAM for warm swap, or that run large retrieval indexes on the host, treat 2x as a floor, not a ceiling.

EPYC 9005 vs Xeon 6 flagship, per socket
MetricAMD EPYC 9005Intel Xeon 6 (6900P / 6700E)
PCIe Gen 5 lanes / socket12896
DDR5 memory channels / socket1212
Max P-cores / socket128 (Zen 5)128
Max dense cores / socket192 (Zen 5c)288 (E-core)
MRDIMM supportNoYes (up to 8800 MT/s)
Max sockets per server2Up to 8 (6700P)
Flagship-to-flagship: EPYC leads on lanes and max cores; Xeon leads on MRDIMM speed and multi-socket scaling. Source: Things Embedded, 2025
A server CPU socket populated with a processor and flanked by twelve filled DDR5 DIMM slots

NUMA Layout and the IOD vs Tile Approach

AMD's chiplet design routes all I/O through a single I/O die, so a single-socket EPYC behaves as one NUMA node by default, with NPS modes available if you want to carve it up. Every PCIe slot sees the same memory latency. For a 4-GPU server with two NICs, that uniformity cuts a class of pinning bugs out of the picture.

Intel's Granite Rapids-AP splits compute and I/O across tiles connected by EMIB bridges. Sub-NUMA Clustering and the tile layout mean that on dual-socket Xeon, not every PCIe root is equidistant from every memory controller. GPU-to-GPU and GPU-to-NIC paths that cross the socket boundary pay a latency and bandwidth tax that shows up under load.

Two practical consequences. First, a single-socket EPYC host is usually the simpler topology for a 4-GPU or 8-GPU PCIe server. You keep one NUMA domain, you keep one PCIe fabric, and vLLM or TensorRT-LLM tensor-parallel traffic sees a flat view. Second, the second socket is a real buy decision, not a default. Add it when you need the memory capacity, the I/O for a second GPU tray, or a workload that genuinely scales across sockets. For single-model inference serving, one socket is usually enough.

Core Count Is Usually Oversized

NVIDIA's HGX reference architecture recommends a minimum of 32 physical CPU cores per socket, with 56 recommended, for 8-GPU H100, H200 and B200 nodes. A 4-GPU server scales down from there. A 64-core SKU is plenty of headroom for data loaders, pre- and post-processing, the vLLM scheduler and the gRPC front door.

Buying the 128-core flagship for an inference host is almost always wrong. The extra cores add TDP, cost and often a second DRAM population pass without adding tokens per second. AMD launched EPYC 'Turin' 9005 on 10 October 2024 with SKUs that top out at 192 cores and 500 W; Intel's 6980P ships at the same 500 W TDP. The thermal envelope of a 4-GPU chassis has limits, and the host CPU is competing with the GPUs for power and airflow.

Where high core counts earn their slot: multi-tenant inference with heavy pre-processing, document AI pipelines that run OCR and parsing before the LLM, and training or fine-tuning runs with CPU-bound data augmentation. Our breakdown of 4-GPU server cost drivers shows where CPU SKU choice lands against GPUs, memory and NVMe in a realistic build.

Xeon and EPYC shipping generations, 2012-2023
Xeon and EPYC shipping generations, 2012-2023Xeon Sandy Bridge (Gen 3, 40 lanes): 2,012; Xeon Skylake / Cascade Lake (Gen 3, 48 lanes): 2,015; Xeon Sapphire Rapids (Gen 5): 2,023; EPYC Naples (7001): 2,017; EPYC Rome (7002, Gen 4, 128 lanes): 2,019; EPYC Milan (7003, Gen 4, 128 lanes): 2,021; EPYC Genoa (9004, Gen 5, 128 lanes): 2,02220122016201920232026Xeon Sandy Bridge (Ge…2012–2015Xeon Skylake / Cascad…2015–2023Xeon Sapphire Rapids…2023–2026EPYC Naples (7001)2017–2019EPYC Rome (7002, Gen…2019–2021EPYC Milan (7003, Gen…2021–2022EPYC Genoa (9004, Gen…2022–2026
Each bar is a generation's shipping life, from launch to successor launch — overlaps show when both families' current flagships coexisted on different PCIe generations. Source: Serve The Home, 2024

How to Spec the Host CPU for an LLM Workload

Start with GPU count and interconnect. For PCIe-attached 4- or 8-GPU builds, EPYC's 128 lanes per socket avoid a PCIe switch and keep the topology flat. For HGX SXM nodes where the GPUs talk over NVLink and the host CPU mainly serves data, both families work and the 2x DRAM rule drives the DIMM choice more than the brand.

Then size memory by aggregate VRAM: 2x as a floor, 3x to 4x if you hold multiple models or large indexes in host DRAM. Match core count to the heaviest pre- and post-processing step, not to the GPU count. For most inference boxes, 48 to 64 cores per socket is enough; reserve 96 cores and above for training or mixed pipelines.

Our AI hardware estimator turns model size, concurrency and context length into a target VRAM and DRAM footprint, which then pins down the CPU and lane budget. For the full menu of chassis sizes that pair with these CPU choices, the cstmAI hardware range covers 1-GPU workstations through multi-node clusters.

EPYC Turin flagship vs Genoa flagship, percent change per spec
EPYC Turin flagship vs Genoa flagship, percent change per specMax cores / socket (192 vs 96): 100%; DDR5 memory speed (6400 vs 4800 MT/s): 33%; Max TDP (500 W vs 400 W): -25%Max cores / socket (1…100%DDR5 memory speed (64…33%Max TDP (500 W vs 400…-25%
Turin flagship (EPYC 9965) vs Genoa flagship (EPYC 9654): gains shown positive, the extra power budget shown negative as a cost. Source: KW Servers, 2025

Where Each Family Earns the Socket

Choose EPYC when the server is PCIe-GPU heavy and you want 128 lanes per socket, a single NUMA domain and the shortest path from each GPU to every NIC. Choose Xeon when the configuration is an HGX SXM node where NVIDIA's reference designs already specify the Platinum SKU, or when MRDIMM bandwidth genuinely matters for a host-bound workload you have measured. Add the second socket only when memory capacity, PCIe root count or socket-scalable workloads force it, and size cores to the pre-processing and serving stack rather than to the GPU count.

Get a quote

Spec your system.

Tell us the models you want to run, how many people will use them and where the hardware should live. An engineer replies with a first configuration and the questions that decide the quote.

Form CSTM-Q · Quote only