Published Updated
NVIDIA H200 vs H100: Which GPU for AI?
- NVIDIA
- H200
- H100
- Comparison
H200 vs H100: The Guide
The NVIDIA H200 keeps the Hopper architecture of the H100 with more memory, in HBM3e. At GPU Cloud, the catalog offers the H100 in these versions: H100 PCIe, H100 NVLink and H100 SXM; the H200 SXM is available from $49.56/h (8 GPUs).
Specifications
| Specification | H100 PCIe | H100 SXM | H200 SXM |
|---|---|---|---|
| Architecture | Hopper | Hopper | Hopper |
| Memory per GPU | 80 GB HBM2e | 80 GB HBM3 | 141 GB HBM3e |
| GPU interconnect | PCIe Gen5 | NVLink 4.0 (900 GB/s) | NVLink 4.0 (900 GB/s) |
| GPUs per server | 1, 2, 4 or 8 | 8 | 8 |
| Price | from $3.90/h (1 GPU) | from $39.75/h (8 GPUs) | from $49.56/h (8 GPUs) |
Prices in Canadian dollars, taxes extra.
When to choose the H200?
- The model weights and KV cache do not fit in 80 GB per GPU: with 141 GB HBM3e per GPU, the model is split across fewer GPUs.
- Inference on long contexts, where the KV cache grows with the context length.
When the H100 is enough
- The model fits in 80 GB per GPU, for inference as well as fine-tuning.
- You want to start small: the H100 PCIe is rented in servers of 1, 2, 4 or 8 GPUs, hourly and with no commitment.
- Multi-GPU training: the H100 SXM links its GPUs through NVLink 4.0 (900 GB/s).
For several months: a dedicated H100 or H200
If the GPU runs every day for years, the long-term reservation rents a dedicated H100 or H200 server in Montreal, yours alone, for 3 years, paid in advance at a locked price. The configurator shows its price at once.
To go further, read the H100 SXM and H100 PCIe comparison.
Indicative prices, reviewed every hour according to the market and the exchange rate.