Three years ago a 4-GPU AI server was essentially a general-purpose rack chassis with four PCIe accelerators, a pair of Xeons, and whatever 10 or 25 gigabit NICs the vendor stocked. You sized it like a database box, added a little extra power, and moved on. Today the same four-slot footprint carries an HGX baseboard with NVSwitch-fused GPUs, 400 or 800 gigabit host NICs, 3 kW power shelves, and a cooling plan that may include rear-door heat exchangers or direct-to-chip loops. The dollar weight of the quote has shifted. GPUs still dominate it, but the lines underneath, including interconnect, networking, power, and cooling, decide whether the box ever runs at the throughput you paid for.
GPUs Still Carry Most of the Quote
Four accelerators remain the headline line item and account for the bulk of a 4-GPU AI server cost. The practical options for an inference-focused box today are HGX H100, HGX H200, or HGX B200, plus PCIe variants of each. HBM capacity is what moves a quote most between SKUs. Going from H100 at 80 GB to the H200 at 141 GB of HBM3e changes which models fit on a single node in full precision with a healthy KV cache. A 70B-parameter model that previously needed two H100 nodes can fit on one H200 node with room for concurrency, which collapses hardware count and site power in a single refresh. The B200 pushes capacity and bandwidth further at a higher per-GPU cost and higher TDP. Our GPU inference comparison walks through the price-per-token tradeoffs, and the sizing estimator turns concurrency and model mix into a GPU count. Skip the top shelf when 80 GB covers your model; pay for larger HBM when context length and user count force it.
NVLink and NVSwitch Are Not Optional on HGX
On an HGX baseboard, the four GPUs are fused by NVSwitch so any GPU can reach any other at full NVLink bandwidth. That fabric is part of the baseboard and comes bundled; you do not configure it out and you do not substitute it. The PCIe variant of the same GPU costs less, but you lose the all-to-all fabric and fall back on PCIe 5.0 bandwidth of about 128 GB/s per x16 slot between cards. For a single large model split across the four GPUs under tensor parallelism in vLLM or TensorRT-LLM, that bandwidth difference drives real differences in tokens per second. For many smaller models served independently, PCIe-only is fine and saves budget. A dual-root motherboard with the PCIe switching and host CPU lanes to match the accelerators adds cost that only pays back when you push the GPUs hard. The workstation versus rackmount comparison on this site walks through where each configuration earns the money.
Networking Is the Second-Largest Line
A single H200 can generate tokens faster than a legacy 25 GbE top-of-rack can hand them off to an LLM gateway and back. Modern 4-GPU quotes assume 200 or 400 gigabit per host NIC, often one NIC per GPU for tensor-parallel deployments, plus a separate management and storage network. The ceiling moved again recently.
For most 4-GPU single-node inference today, 800 Gb/s host NICs are overkill. What you need instead is clarity about NIC count, cable type (DAC versus active optical versus transceivers), and the per-port cost of your top-of-rack switch. Each of those is a line item buyers routinely miss when they compare quotes that look otherwise identical. Storage networking is often the larger surprise: NVMe-oF or RDMA-backed object stores for model weights and KV cache offload add their own HBAs and switch ports, and each one has to be sized for peak concurrent load rather than average.
Power Delivery Changes the Rack
A 4-GPU HGX chassis typically ships with four 3 kW Titanium-grade power supplies in a 2+2 redundant configuration. That is 12 kW of installed capacity for a single 4U chassis before counting the rest of the rack. Pair the server with storage, switching, and head nodes and a single rack can push past the 15 to 20 kW circuits most colocation sites were built around. Three follow-on line items usually appear: higher-amperage PDUs (often 60A three-phase rather than 30A single-phase), the whip and breaker work on the facility side, and sometimes a UPS upgrade to carry the new draw. If your site runs 208V and the rack lands at 25 kW, you need either higher-voltage distribution or a second power whip. Our deeper walkthrough of power, cooling and noise covers the physical build; the point here is that this cost shows up on the facility invoice, not the server quote.
Cooling Decides the Rack Count
Air cooling still works at 4-GPU densities if the room can handle it. The breaker is rack-level heat rejection. The practical ceiling for air sits around 40 kW per rack, and dense AI configurations routinely blow past that. Rear-door heat exchangers are the lowest-disruption path: they bolt onto the back of the rack and transfer most of the heat into a facility water loop. Direct-to-chip cold plates handle higher densities but ask for site changes you cannot reverse in a hurry. ASHRAE's 5th Edition thermal guidelines now define liquid-cooling classes by facility supply temperature, running from W17 through W+, which matters because your chilled-water setpoint drives both chiller cost and the GPU's thermal margin. The CDU and secondary loop are rarely in the server vendor's quote. Budget them as a separate facility line and plan the coolant distribution in parallel with the server order, not after.
Service, Warranty and Integration Round Out the Bill
Three to five year hardware warranties, 24x7 parts-on-site, and professional installation are quoted separately from the hardware itself. For a box this expensive, spare-GPU coverage and a next-business-day on-site SLA earn their place in the quote. A single failed accelerator can idle a tensor-parallel deployment until the replacement arrives and the HGX baseboard is re-fused. Software integration is its own cost line: installing and tuning vLLM or TensorRT-LLM, provisioning orchestration, wiring in identity (SAML/OIDC), hooking storage, and validating throughput against a real workload. The vendor contract questions worth asking shape whether those lines get itemized or buried under a single "services" number. On the configuration side, the four-size comparison shows how line items shift between a 4-GPU server and the neighbouring builds you may also be sizing against.
What Moves Next Year's Budget
The variable most likely to move a 2027 quote is the memory tier. HBM supply has kept per-GB pricing firm, and the next architecture generations lean harder on both capacity and interconnect width. For workloads bound by KV cache or long context, a 4-GPU box built on the newer memory tier can replace two older ones and flip the capex math in a single refresh cycle. Networking will follow a similar pattern as 800 Gb/s host NICs move from early-access into standard SKUs, narrowing the delta between training-grade and inference-grade network builds. Buyers who wrote 400 Gb/s into this year's quotes should ask for upgrade paths that do not require re-cabling the room. The 4-GPU AI server budget that reads cleanly next year is the one that reserves headroom now in memory, in switch ports, and in watts.