All products 
Renting a server with NVIDIA L4: what fits in 24 GB and where the card beats its neighbours
The NVIDIA L4 is an Ada Lovelace card built for servers that care about power draw: 72 W, a low-profile single-slot passive design, powered straight from its PCIe 4.0 x16 slot. Where a chassis takes one big card, it can take several L4s. That defines its role: people rent a GPU server with L4 not to train large models but for round-the-clock inference, speech recognition and video processing, where the hourly price and the number of parallel streams matter most. It is the successor of the T4 and noticeably faster at the same jobs.
What fits in 24 GB of video memory
The L4 has 24 GB of GDDR6. For language models, a rule of thumb: FP16 needs about 2 GB per billion parameters, 8-bit about 1 GB, 4-bit 0.5–0.6 GB, plus the KV cache, which grows with context length and the number of concurrent requests.
- 7–8B parameters in FP16 fit, but with a long context and several users at once the headroom runs out fast.
- 13B in 8-bit fits with room for context.
- Around 30B in 4-bit is the limit for one card, with a short context.
- Whisper, embeddings, rerankers are the L4's core work: the models are small, and one card can host several such services.
- Image generation with Stable Diffusion or Flux runs at moderate speed; for a heavy stream of images a stronger card is more comfortable.
Fine-tuning works as LoRA or QLoRA for models up to 7–8B, but full training is not what this card is for. FP8 support (compute capability 8.9) helps inference: in FP8 a model takes half the memory of FP16 and usually loses less quality than 4-bit formats, provided your inference engine supports FP8.
Where the L4 hits its limit and what to compare it with
The main limit is memory bandwidth, about 300 GB/s. Token generation speed in a language model depends on it first of all, so a chatbot on an L4 answers more slowly than on a card with the same capacity but faster memory.
- A10: also 24 GB, but about 600 GB/s at 150 W. Usually faster for chat with live users; for video and small models the extra cost is not always justified.
- L40S: 48 GB and far more compute. Needed when a model does not fit in 24 GB or a large model gets many concurrent requests.
Video is where the L4 shines. Its hardware NVENC and NVDEC support AV1, and one card transcodes many streams at once: live-stream transcoding, clip cutting, camera footage analysis with object detection on the same card.
A slice, one card or several
The cheapest rows in the table are not whole cards but vGPU slices with 2 or 4 GB of video memory. The L4 has no MIG; it is shared through vGPU profiles. A slice suits video, remote desktops and light recognition models, but not language models: even a small LLM with a usable context will not fit in 2–4 GB. The other plans give you whole cards, one, two or four.
The L4 has no NVLink either, so cards talk only over PCIe. A server with L4 ×2 or ×4 therefore pays off not for one big model split into parts but for several independent workers: each card runs its own copy of the model and a load balancer spreads the requests. If you need a single 40–70B model, one card with more memory is the better choice.
Drivers, prices and billing
The L4 needs an NVIDIA driver from the 525 branch or newer and CUDA 11.8+; FP8 needs CUDA 12 and recent builds of PyTorch, vLLM or TensorRT-LLM. In a ready-made image check the nvidia-smi output: card model, memory (24 GB or the slice size) and driver version. On vGPU slices the guest driver has to match the host side, so it usually comes from the host's image rather than from the NVIDIA website.
Renting a server with L4 in the catalogue costs roughly from $60 a month for a slice to about $5,000 for a dedicated machine with four cards, with the middle of the range around $870. Most plans bill by the hour, which pays off for tests, batch video jobs and services needed only during working hours. If the card is busy around the clock, compare the hourly price with the monthly one; see the page on hourly billing. Plans are spread across several regions: for personal data of EU residents pick an EU data centre to keep GDPR compliance simple, and for live inference choose a location close to your users to keep latency low. Most hosts accept international bank cards.