GPU guide
Rent an 8-GPU server: 8x H100 or 8x A100 NVLink, by the hour
Updated on
Short answer
GPU Cloud rents whole 8-GPU NVIDIA servers by the hour in Montreal, from CA$5.35/h (8 × RTX A6000) to CA$34.27/h (8 × H100 SXM), taxes extra. The NVLink versions (8 × A100 NVLink at CA$14.99/h, 8 × H100 NVLink at CA$27.84/h and 8 × H100 SXM at CA$34.27/h) link their GPUs with a fast link for distributed training.
Every 8-GPU server
| GPU | Link between GPUs | Server GPU memory | Per hour | 24 hours |
|---|---|---|---|---|
| 8 × RTX A6000 | PCIe Gen4 | 384 GB | $5.35 | $128.40 |
| 8 × L40 | PCIe Gen4 | 384 GB | $10.71 | $257.04 |
| 8 × A100 PCIe | PCIe Gen4 | 640 GB | $14.46 | $347.04 |
| 8 × A100 NVLink | PCIe Gen4 + NVLink (600 GB/s) | 640 GB | $14.99 | $359.76 |
| 8 × RTX PRO 6000 | PCIe Gen5 | 768 GB | $19.81 | $475.44 |
| 8 × H100 PCIe | PCIe Gen5 | 640 GB | $26.77 | $642.48 |
| 8 × H100 NVLink | PCIe Gen5 + NVLink (600 GB/s) | 640 GB | $27.84 | $668.16 |
| 8 × H100 SXM | NVLink 4.0 (900 GB/s) | 640 GB | $34.27 | $822.48 |
Price of the whole server, on demand, in Canadian dollars, taxes extra, read from the catalog when the page is built. Link: specs published by NVIDIA.
| GPU | GPUs per server | Server GPU memory | On demand, per hour | Availability |
|---|---|---|---|---|
| NVIDIA RTX A6000 | 8 | 384 GB | $5.35 | |
| NVIDIA L40 | 8 | 384 GB | $10.71 | |
| NVIDIA A100 PCIe | 8 | 640 GB | $14.46 | |
| NVIDIA A100 NVLink | 8 | 640 GB | $14.99 | |
| NVIDIA RTX PRO 6000 | 8 | 768 GB | $19.81 | |
| NVIDIA H100 PCIe | 8 | 640 GB | $26.77 | |
| NVIDIA H100 NVLink | 8 | 640 GB | $27.84 | |
| NVIDIA H100 SXM | 8 | 640 GB | $34.27 |
Prices in Canadian dollars, taxes extra, read from the catalog when the page is built. Availability is read live; a machine out of stock shows the closest equivalent machine in stock.
NVLink or PCIe: when the link between GPUs matters
- NVLink matters when the 8 GPUs work on the same model and exchange data at every step: distributed training (FSDP, DeepSpeed), tensor parallelism, inference of a model split over several GPUs.
- PCIe is enough when each GPU does its own work: several trainings side by side, a hyperparameter search, rendering or independent batch inference.
- For independent jobs, an 8 × RTX A6000 server at CA$5.35/h costs far less than an NVLink machine.
What fits in 640 GB of GPU memory
- A 70 billion parameter model in 16-bit: 140 GB of weights, with plenty of room for the KV cache and concurrent requests.
- A 405 billion parameter model in 8-bit: 405 GB of weights, served on the 8 GPUs together.
- Full fine-tuning of a 70 billion parameter model needs about 1,120 GB (weights, gradients and optimizer states), more than one server. LoRA and QLoRA make it fit: see the fine-tuning guide.
Availability and reservation
8-GPU servers are few. The table shows their availability live: when a machine is out of stock, the closest equivalent machine in stock is shown with its price.
If you need one machine in particular, reserve it with a saved card and a maximum hourly price: the first reservation in line is launched automatically when the machine is back in stock, and you get an e-mail.
Cost examples
- 8 × RTX A6000: $128.40 for 24 hours, $898.80 for a full week (168 hours), or $3,737.60 for a month paid in advance.
- 8 × A100 NVLink: $359.76 for 24 hours, $2,518.32 for a full week (168 hours), or $10,453.60 for a month paid in advance.
- 8 × H100 SXM: $822.48 for 24 hours, $5,757.36 for a full week (168 hours), or $23,871.00 for a month paid in advance.
Formula: hourly rate of the whole server × number of hours, rounded to the cent, taxes extra.
Good to know before you launch
- Billing is hourly, at the server’s hourly rate, from prepaid credit. There is no per-minute billing, and the first hour is charged when you deploy.
- A stopped server is still billed at the full hourly rate. To pay less, hibernate it ($0.02 per hour (about $14.60 per month), disk kept) or delete it. A Spot machine cannot be hibernated: delete it when you are done.
- The minimum top-up is CA$25.00, paid by credit card.
- Servers and their disks stay in Canada (region canada-montreal, in Montreal). Account information (identity, billing, e-mails) may be processed outside Quebec, as the privacy policy explains.
Frequently asked questions
How much does an 8x H100 server cost per hour?
Depending on the version, from CA$26.77/h to CA$34.27/h for the whole server, taxes extra.
Can I rent 8 A100s with NVLink?
Yes, 8 × A100 NVLink rents at CA$14.99/h, with 640 GB of GPU memory in total.
Is the price per GPU or per server?
Per server: the hourly rate shown covers the 8 GPUs, the CPUs, the memory and the disk of the machine.
What if the 8-GPU server I want is out of stock?
Deploy the equivalent machine offered first, or reserve the server you want: the reservation is launched automatically when it is back in stock.
Do I need NVLink?
Only if the GPUs work together on the same model, as in distributed training. For independent jobs, a PCIe server costs less.
Need GPUs? We’ve got you.
Launch an NVIDIA GPU server by the hour in Canada, paid in Canadian dollars by credit card, or reserve a sold-out GPU: it launches as soon as one frees up.
Other guides
- Your AI says you need a GPU? Run your Python script on a GPU in minutes
- CUDA out of memory: causes, fixes, and when to move to a GPU with more VRAM
- torch.cuda.is_available() returns False: causes and fixes (Mac, PC without NVIDIA, Docker)
- How much does a GPU cost per hour in Canada, in Canadian dollars?
- OpenAI API too expensive? Host your LLM on a GPU by the hour, in Canada
- Rent a GPU by the hour in Canada: prices, steps and billing
- GPU cloud in Montreal, Canada: NVIDIA GPU servers hosted in the country
- Rent an H100 in Canada: PCIe, NVLink or SXM, by the hour
- Which GPU to fine-tune an LLM? Memory, machine and cost
- Cheap GPU cloud: how to pay as little as possible for a GPU
- Ollama on a cloud GPU: run an LLM online on your own server
- GPU for Stable Diffusion and ComfyUI: generate images on a rented server
- RunPod alternative in Canada: GPUs by the hour, in Canadian dollars
- GPU for university research in Quebec: rent a GPU by the hour
- Health data and AI: GPUs hosted in Canada