GPU server: which card for which job
You choose a server GPU by its video memory first and its speed second. If the model or scene does not fit in the card's memory, the server will not do, however fast it computes. Below: which jobs run on which cards, how server cards differ from gaming ones, and what a month of each costs according to the accsly catalog.
The short answer. For small models, speech recognition and video processing a 16–24 GB card is enough: T4, L4, RTX 4090. Language models with tens of billions of parameters need a 48–96 GB card: L40S or RTX PRO 6000. Large models are trained on A100 and H100, usually several of them in one server.
Three numbers that decide
- Video memory (VRAM). It holds the neural network's weights, the data and the intermediate results. Run out of it and the job either fails with an error or part of the model moves to the server's ordinary memory and runs many times slower. How much memory a language model needs is covered in a separate article, your own language model on a server.
- Memory bandwidth. How many gigabytes per second the card reads from its memory. For language models this is the main speed figure: while generating an answer, every new word requires reading all the model's weights. NVIDIA states plainly that this stage is limited by memory speed, not by computation.
- Links between cards. If a model does not fit in one card, it is split across several. Cards with NVLink exchange data faster than over the PCIe bus. On the H100 SXM that is 900 GB/s, while the L40S has no NVLink at all.
The cards most often offered for rent
I took the GPU plans from the accsly catalog on 13 November 2026 and kept only those with one whole card in the server. The price is the market median per month; in brackets is the range covering the middle half of the plans. The catalog stores prices in roubles, so the dollar figures are converted and rounded. Specifications come from the cards' pages on NVIDIA's site.
| Card | Memory | Bandwidth | Price per month | Plans / hosts |
|---|---|---|---|---|
| RTX A4000 | 16 GB | 448 GB/s | $373 (301–514) | 11 / 8 |
| RTX 3090 | 24 GB | 936 GB/s | $421 (390–501) | 12 / 7 |
| L4 | 24 GB | 300 GB/s | $532 (448–853) | 20 / 10 |
| T4 | 16 GB | 320 GB/s | $549 (400–825) | 29 / 9 |
| RTX 4090 | 24 GB | 1,008 GB/s | $559 (480–649) | 15 / 9 |
| RTX 5090 | 32 GB | 1,792 GB/s | $699 (647–823) | 11 / 7 |
| L40S | 48 GB | 864 GB/s | $929 (679–1,170) | 11 / 9 |
| A100 | 40 or 80 GB | 1,935–2,039 GB/s | $1,099 (1,066–1,224) | 9 / 8 |
| RTX PRO 6000 | 96 GB | 1,597 GB/s | $1,499 (1,137–2,547) | 29 / 11 |
| H100 | 80 GB | 3,350 GB/s (SXM) | $2,400 (2,019–2,974) | 11 / 10 |
Two things stand out. The old T4 costs almost as much as the L4, although the L4 is newer and has more memory. And the gaming RTX 3090 and 4090 are cheaper than server cards with the same memory, while their memory is three times faster than the L4's.
Which card for which job
Speech recognition, small models, video
Speech recognition models, classifiers, semantic search and compressed language models with 7–8 billion parameters take a few gigabytes. 16–24 GB is enough for them. The L4 draws 72 W and takes one low-profile slot, so hosts fit several into a server. The T4 is in the same class: 16 GB and 70 W, but a generation older. Both have hardware video encoding. Selections: servers with L4, servers with T4.
Image generation and language models up to 30 billion parameters
Here both memory and speed matter. The RTX 4090 and 3090 give 24 GB and about a terabyte per second, the RTX 5090 32 GB and 1.8 TB/s. For a single user or a small team this is the cheapest way to get a fast card. The limits of gaming cards: the RTX 4090 and 5090 have no NVLink to pair them, and no gaming card can be split into isolated parts. In the catalog, 12 of the 15 RTX 4090 plans are dedicated servers, so the whole machine is yours.
70-billion-parameter language models, rendering
A 70-billion-parameter model compressed to 4 bits weighs about 35 GB, more than any gaming card holds. You need a card with 48 GB or more. The L40S gives 48 GB with error-correcting memory (ECC), and NVIDIA positions it for inference, image generation and rendering. The RTX PRO 6000 Blackwell has 96 GB and 1.6 TB/s; a 70-billion model in 8-bit format fits on it. Selections: servers with L40S, servers with RTX PRO 6000.
Training and large models
Training needs several times more memory than running a finished model and almost always runs on several cards at once. This is the job for the A100 (40 or 80 GB, up to 2 TB/s) and the H100 (80 GB, 3.35 TB/s on the SXM version) with NVLink. The H100 costs twice as much as the A100, but its memory is one and a half times faster. Selections: servers with A100, servers with H100.
Remote workstations and graphics
For virtual desktops, CAD and 3D software, the professional RTX A4000, RTX 4000 Ada and A16 fit. The A16 carries four GPUs with 16 GB each on one board and is designed for many users with a light graphics load.
A whole card or part of one
Server cards can be shared between customers. The A100 splits into seven isolated parts, the RTX PRO 6000 into four. A host may sell such a part as a "server with A100", though you get 10 GB of memory, not 80. In the catalog such plans occur with the A16, A40 and L4: they have 2, 4 or 8 GB of video memory. Before ordering, check the amount of video memory in the plan description. It should match the card's full memory if you need the whole card.
Cloud, VPS or dedicated server
In the catalog half of the GPU plans are dedicated servers, half are cloud and VPS. A cloud server can be created for a few hours and deleted, which suits experiments and one-off jobs. How to work out what is cheaper is covered in the article hourly GPU rental. A dedicated server is cheaper if the card is busy around the clock: the whole machine is yours and no neighbours share the processor or disk with you. Selection: dedicated servers with GPU.
If you need the card only for a small model and response speed is not critical, do without one. Models with 7–8 billion parameters also run on a processor if there is enough RAM. Selection: VPS for neural networks on CPU.
What to check before ordering
- The amount of video memory — a whole card or part of one.
- RAM and disk. The model is first downloaded to disk and read into the server's memory. The median for one-card plans is 64 GB of RAM. For large models take NVMe from 200 GB.
- Driver and CUDA version. Find out whether the host installs the driver and which version. Recent frameworks need a recent CUDA.
- The number of cards and the link between them, if the model does not fit in one.
- One-off fees. 24 plans in the catalog, almost all of them dedicated servers, charge a setup fee.