NVIDIA H100 (SXM)
The GPU inside most 2023-2024-era AI training/inference servers. Sold as part of an OEM server (e.g. NVIDIA DGX), not standalone.
80 GB
GPU Memory vs. tier max
700 W
Power vs. tier max
$32,500
Price vs. tier max
| The dedicated memory built into the graphics card. An AI model's entire set of "weights" (its learned parameters) must fit into this memory before the card can run it. If a model needs more VRAM than one card has, you need multiple cards working together, or a card with more memory. | 80 GB |
|---|---|
| Thermal Design Power — roughly how many watts the card draws under full load. Higher-performance cards need more electricity and generate more heat, which affects your electricity bill and cooling needs. | 700 W |
| Approximate cost to acquire the hardware. For consumer cards this is usually a fixed retail price. For datacenter-class hardware, list prices are rarely published — buyers negotiate directly with NVIDIA or its partners, so we show a typical range or say "contact for pricing" rather than inventing a number. |
$32,500
No official retail price; press-reported range $25,000-$40,000/GPU (or ~$2-$11/hr as cloud rental)
|
Why this matters
A card or server needs at least as much
The dedicated memory built into the graphics card. An AI model's entire set of "weights" (its learned parameters) must fit into this memory before the card can run it. If a model needs more VRAM than one card has, you need multiple cards working together, or a card with more memory.
as an AI model requires,
or the model simply won't load. See Get a Recommendation
to see which models this hardware can run, or
add it to a side-by-side comparison.