Dedicated GPU servers

A dedicated server with a graphics card is a physical machine with one or more cards installed, and all of that hardware works for you alone. There is no hypervisor between your model and the GPU, no neighbours share the PCIe bus, the disks or the uplink with you, and the BIOS, driver and operating system settings stay in your hands. The table above lists such configurations, from a single workstation card to builds with several accelerators. If your job is short and you want hourly billing, compare the cloud plans in the main GPU servers section.

When a dedicated GPU server pays off over a cloud GPU

• The card is busy almost around the clock. An hourly cloud GPU is great while you switch it on for an evening. When a service answers users without a break, a monthly dedicated GPU server rental usually costs less than the same load billed by the hour.
• Training runs for weeks. A long training or fine-tuning run does not like interruptions. On your own machine it will not be stopped because the provider ran short of capacity, and checkpoints sit on a local disk instead of paid external storage.
• Data must not leave the machine. Medical scans, customer conversations, internal company documents: on a dedicated server the disks are not shared with other customers, and it is easier to show an auditor where the data physically lives and who can reach it.
• Licences are tied to the hardware. Some professional software for rendering, CAD and simulation is licensed per machine or per physical card. A permanent server with the same GPUs saves you from reactivating after every virtual machine restart.
• You need several cards with a known topology. For distributed training it matters exactly how the cards are linked. In the cloud the interconnect is often hidden, while for a dedicated server you can ask in advance and verify it after delivery.

If you do not need GPU computing and simply want a powerful physical machine for databases or virtualization, the general catalog of dedicated servers is a better place to look: more configurations and lower prices.

Card, video memory and the link between cards

The first thing to read in a plan line is the card model and its memory size. The RTX A4000 and A5000 workstation cards are built for long rack-mounted workloads and run quieter, the RTX 3090 and 4090 give a lot of compute for the money, and the A100 and H100 are needed when a model does not fit into 24 GB or training uses large batches. Older Tesla K80 and P40 cards show up in budget configurations: they have plenty of memory, but they do not support recent CUDA versions or modern number formats, so they suit experiments rather than production. Also watch out for low-end cards such as NVS or entry-level Quadro: in servers they are often installed only for display output and are almost useless for neural networks.

A "2×" or "8×" in the name is the number of cards, not the memory of a single card. Two 24 GB cards are not the same as one 48 GB card: the model has to be split, and the speed of exchange between the cards starts to decide the outcome. Cards joined by an NVLink bridge exchange data several times faster than over the PCIe bus, and in HGX-class multi-GPU platforms all accelerators are connected to each other directly. Ask the host how many PCIe lanes each card gets and whether they all hang off one processor: if half of the cards are attached through the second socket, traffic between them is slower.

Processor, disks, power and network

• CPU and RAM. The processor prepares the data, decodes images and tokenizes text. Plan several physical cores per card and at least as much RAM as the total video memory, otherwise the accelerators wait for data.
• NVMe for datasets. A large dataset is read from disk on every epoch. Local NVMe noticeably shortens training compared with SATA disks and network volumes.
• Power and cooling. One powerful card draws hundreds of watts, and an eight-accelerator server draws as much as a small rack. Not every data centre can supply that much power to a single unit, so multi-card servers come in small batches and often show up in the table as "out of stock".
• Port and traffic. Model weights and datasets weigh hundreds of gigabytes. A gigabit port without a tight traffic cap spares you the wait on the first download and on regular checkpoint uploads.
• Setup fee and minimum term. GPU servers more often than regular ones come with a setup fee and a minimum rental term from one month to a year. Count the cost over the whole period, not just the monthly line.
• Delivery time. Servers in stock are handed over within hours; configurations with scarce cards are built to order, and the wait stretches to days or weeks.

What to check before you pay

1. Image with the driver. Ask whether the host installs a system with the NVIDIA driver and CUDA or hands over a clean OS. Match the driver version against what your framework and containers require.
2. Remote console. KVM or IPMI lets you reinstall the system and fix the boot after a failed driver update without waiting for support. On a GPU server this matters even more: a driver failure sometimes leaves the machine without network.
3. Replacement of a failed card. Graphics cards under constant load fail more often than disks. Find out how quickly a card is replaced, whether spares are kept on site and whether the rental is extended for the downtime.
4. Data location. Check the data centre country and storage terms if the server will process personal data or trade secrets. For customers in the EU this decides whether GDPR rules on transfers outside the Union apply; location also affects how fast datasets download.
5. Test after delivery. In the first hours run a real job at full load and watch card temperatures and clocks: overheating and throttling show up right away, and a complaint is easier to make at the start of the term.

If you are still choosing between server classes, start with the guide to choosing a VPS. And for a small local model without strict speed requirements, a regular server without a graphics card from the VPS for Ollama selection is sometimes enough.