How to Deploy an LLM on a Cloud GPU Server

Deploy an LLM Step by Step

You can serve a large language model with vLLM in two ways on GPU Cloud: with the one-click template, or by installing it yourself over SSH.

Step 1: Choose the GPU

GPU memory sets the size of the model. In 16-bit precision, each billion parameters takes about 2 GB, plus the KV cache.

• 7 or 8 billion parameter model (about 16 GB of weights): a 48 GB GPU leaves room for the context, for instance the RTX A6000, from $0.79/h (1 GPU), or the L40, from $1.57/h (1 GPU).
• 70 billion parameter model (about 140 GB of weights in 16 bits): a server with 4 GPUs of 80 GB, such as the A100 PCIe (80 GB HBM2e) or the H100 PCIe (80 GB HBM2e), with vLLM tensor parallelism.

Step 2, option A: the one-click vLLM template

Pick the OpenAI compatible vLLM server template when you create the server. The installation starts on its own; the console then shows an HTTPS address and an API key generated for your server.

Step 2, option B: manual installation

1. In the console, create a server with the Ubuntu image (NVIDIA driver, CUDA and Docker) and add your SSH key.
2. When the server is ready, the console shows its IP address.
3. Log in, install vLLM and start the model:

# Log in as the ubuntu user
ssh ubuntu@IP_ADDRESS

# Isolated Python environment for vLLM
sudo apt-get update && sudo apt-get install -y python3-venv
python3 -m venv ~/vllm
source ~/vllm/bin/activate
pip install vllm

# Start the model (OpenAI compatible API on port 8000)
vllm serve Qwen/Qwen2.5-7B-Instruct --api-key YOUR_SECRET_KEY

Step 3: Test the API

From the server, check that the model answers:

curl http://localhost:8000/v1/models -H "Authorization: Bearer YOUR_SECRET_KEY"

Before opening port 8000 to the Internet, keep the --api-key option and restrict access to your addresses. The one-click template puts the API behind a key-protected HTTPS access, with no setup on your side.