GPU guide
Cheap GPU cloud: how to pay as little as possible for a GPU
Updated on
Short answer
The cheapest GPU at GPU Cloud is an NVIDIA RTX A6000 (48 GB) at CA$0.67/h, or CA$0.54/h on Spot, taxes extra. To pay as little as possible, take the smallest memory that fits your model, and hibernate or delete the server as soon as you are done.
The strongest lever: the smallest memory that fits
A server’s price goes up with its GPU memory and the generation of its card. A model that fits in 48 GB does not need an 80 GB card: pick the server that fits, not a bigger one.
| Model | Precision | Memory to plan for | Cheapest server that fits | On demand / h | Spot / h |
|---|---|---|---|---|---|
| 8 billion parameters | 16-bit | 20 GB | 1 × RTX A6000 | $0.67 | $0.54 |
| 8 billion parameters | 4-bit | 5 GB | 1 × RTX A6000 | $0.67 | $0.54 |
| 13 billion parameters | 16-bit | 32 GB | 1 × RTX A6000 | $0.67 | $0.54 |
| 13 billion parameters | 4-bit | 8 GB | 1 × RTX A6000 | $0.67 | $0.54 |
| 34 billion parameters | 16-bit | 82 GB | 2 × RTX A6000 | $1.34 | $1.07 |
| 34 billion parameters | 4-bit | 21 GB | 1 × RTX A6000 | $0.67 | $0.54 |
| 70 billion parameters | 16-bit | 168 GB | 4 × RTX A6000 | $2.68 | $2.14 |
| 70 billion parameters | 4-bit | 42 GB | 1 × RTX A6000 | $0.67 | $0.54 |
Memory to plan for: model weights × 1.2 for the KV cache and the CUDA context, an estimate for contexts of common length. A long context or many concurrent requests need more. Prices in Canadian dollars, taxes extra.
Loading the model in 4-bit cuts the memory of the weights by about four, and often allows a much cheaper server. The CUDA out of memory guide details the memory of the weights by precision.
These servers, with their availability
| GPU | GPUs per server | Server GPU memory | On demand, per hour | Availability |
|---|---|---|---|---|
| NVIDIA RTX A6000 | 1 | 48 GB | $0.67 | |
| NVIDIA RTX A6000 | 2 | 96 GB | $1.34 | |
| NVIDIA RTX A6000 | 4 | 192 GB | $2.68 |
Prices in Canadian dollars, taxes extra, read from the catalog when the page is built. Availability is read live; a machine out of stock shows the closest equivalent machine in stock.
The other ways to pay less
- Spot: about 19% less ($0.54 instead of $0.67 per hour on an RTX A6000), for a job that can take an interruption without notice and resumes from a checkpoint.
- Hibernate instead of stopping: a stopped server is still billed at the full rate, a hibernated server costs $0.02 per hour (about $14.60 per month) and keeps its disk.
- Delete as soon as the job is done: fetch your results, then delete the server; nothing is billed anymore.
- The monthly term only for a server that runs all the time: on an RTX A6000, $474.50 instead of $489.10 for 730 hours, that is $14.60 less.
- Let your AI assistant rent for you with a spending-capped MCP key: its spending is limited by the key’s cap (see the Developers page).
Compare GPU prices without mistakes
- The currency: a price in US dollars gets converted, and your bank may add fees. Here, everything is in Canadian dollars.
- Taxes: check whether the price includes them. Here, they are added depending on the billing province.
- What is billed when stopped: at GPU Cloud, a stopped server is still billed; only hibernation and deletion stop the GPU charges.
- The whole server: compare machines with the same number of GPUs and the same memory, and look at the server price, not only the price of one GPU.
Every price, from 1 to 8 GPUs, is on the GPU pricing page for Canada.
Good to know before you launch
- Billing is hourly, at the server’s hourly rate, from prepaid credit. There is no per-minute billing, and the first hour is charged when you deploy.
- A stopped server is still billed at the full hourly rate. To pay less, hibernate it ($0.02 per hour (about $14.60 per month), disk kept) or delete it. A Spot machine cannot be hibernated: delete it when you are done.
- The minimum top-up is CA$25.00, paid by credit card.
- Servers and their disks stay in Canada (region canada-montreal, in Montreal). Account information (identity, billing, e-mails) may be processed outside Quebec, as the privacy policy explains.
Frequently asked questions
What is the cheapest GPU?
An NVIDIA RTX A6000 with 48 GB, at CA$0.67/h on demand and CA$0.54/h on Spot, taxes extra.
Is Spot worth it?
Yes for a job that resumes from a checkpoint (training, batch processing). No for a service that must stay online: a Spot machine can be interrupted without notice.
How do I pay nothing when I am not using the GPU?
Delete the server. To keep your disk, hibernate it ($0.02 per hour (about $14.60 per month)): stopping the server is not enough, it is still billed at the full rate.
Is the monthly term cheaper?
For a server that runs all month, yes: $474.50 instead of $489.10 on an RTX A6000. For use in periods, hourly with hibernation costs less.
Is a cheaper GPU slower?
Often, but a model that fits in the memory of a cheaper card can run fast enough for your use there. Try one hour on the cheapest server that fits before going bigger.
Need GPUs? We’ve got you.
Launch an NVIDIA GPU server by the hour in Canada, paid in Canadian dollars by credit card, or reserve a GPU for a given date.
Other guides
- Your AI says you need a GPU? Run your Python script on a GPU in minutes
- CUDA out of memory: causes, fixes, and when to move to a GPU with more VRAM
- torch.cuda.is_available() returns False: causes and fixes (Mac, PC without NVIDIA, Docker)
- How much does a GPU cost per hour in Canada, in Canadian dollars?
- OpenAI API too expensive? Host your LLM on a GPU by the hour, in Canada
- Rent a GPU by the hour in Canada: prices, steps and billing
- GPU cloud in Montreal, Canada: NVIDIA GPU servers hosted in the country
- Rent an H100 in Canada: PCIe, NVLink or SXM, by the hour
- Which GPU to fine-tune an LLM? Memory, machine and cost