All products 
NVIDIA T4: the budget card for inference and video, what it handles and where it stops
The NVIDIA T4 came out in 2018 on the Turing architecture and is still the most affordable data-centre card for AI work. It has 16 GB of GDDR6 memory, draws only 70 W, fits a low-profile single slot and is passively cooled: the server fans push air through it, so data centres can put it into almost any machine. That is why renting a server with a T4 usually costs much less than any plan with a newer card. The table above lists cloud VPS plans with a T4 and dedicated servers, from a single card up to builds with four.
What fits into 16 GB
A rule of thumb: in FP16 a model takes about 2 GB per billion parameters, in 8-bit about 1 GB, in 4-bit 0.5–0.6 GB. On top of that you need headroom for the KV cache, which grows with context length and the number of concurrent requests.
- Language models. A 7–8B model runs in 8-bit or 4-bit quantisation, a 3–4B model in full FP16. That is enough for a support chatbot, ticket classification or answers over a document base. Anything above 13B will not fit on one card with a usable context.
- Speech recognition. Whisper and similar models are one of the best jobs for a T4: transcribing calls and podcasts runs many times faster than real time.
- Embeddings, OCR and computer vision. Vectorising documents for search, reading text from scans, object detection and image classification fit into memory with plenty to spare.
- Image generation. Stable Diffusion runs, but each image takes noticeably longer than on RTX-class cards; for large models such as Flux, 16 GB is too little without heavy compression.
- Training. Small models and LoRA or QLoRA fine-tuning up to 7B are possible, but slow. The T4 is not a card for serious training.
What the T4 cannot do and what to check in the image
The T4 tensor cores handle FP16, INT8 and INT4, but not BF16 or FP8. Many recent models are published in BF16, and on a T4 you have to run them in FP16, which sometimes takes extra configuration. The card has compute capability 7.5, while some newer inference engines and attention kernels, FlashAttention 2 for example, require 8.0 or higher, meaning Ampere or later. Before renting, check that your stack supports Turing.
Drivers are not a problem: current CUDA 12.x still supports Turing and needs a driver from the 525 branch or newer. In the image, check the driver version, the CUDA version and whether the NVIDIA Container Toolkit is installed if you work with Docker.
A separate advantage is the hardware NVENC and NVDEC codecs. A GPU server with a T4 transcodes several video streams to H.264 and HEVC at once while barely loading the CPU. Turing has no AV1 encoding.
One card, a share of a card or several
Some cloud plans give you not a whole T4 but a share of it through vGPU: the virtual machine gets part of the video memory, for example 4 or 8 GB, and part of the compute. That is enough for embeddings or a small speech model, but for a 7B language model take a full card. The T4 has no MIG, the technology that splits A100 and H100 cards into isolated parts.
Plans marked "×2" and "×4" mean two or four separate 16 GB cards. The T4 has no NVLink, the cards talk over PCIe 3.0, so splitting one model across them is slow. A multi-card server pays off when each card runs its own copy of the model or its own job queue: that way it is easier to serve many parallel requests.
Prices, billing and what to pick instead
In the catalogue, T4 plans cost roughly $210 to $4,000 a month; the low end is a cloud VPS with one card, the high end is dedicated servers with four cards, lots of RAM and a many-core CPU. Most plans are billed by the hour. If you need the card a few hours a day for batch transcription or overnight processing, hourly billing is cheaper; for a service running around the clock, compare the monthly plan or a dedicated server. The section on hourly billing explains how to calculate it. Most plans accept card payment; for EU customers, the data centre location also decides GDPR questions and latency to your users.
If you hit the limits of the T4, look at the neighbouring cards:
- NVIDIA L4 is the direct successor on the Ada architecture: 24 GB, 72 W, BF16 and FP8 support, AV1 encoding and noticeably higher speed. It costs more but removes almost all T4 limitations.
- NVIDIA A2 also has 16 GB and draws even less power, with the Ampere architecture and BF16, but it has less compute than the T4. It suits light workloads where compatibility with newer engines matters.
If you are not sure yet which card you need, start with the overview of GPU servers. A T4 for rent is a sensible first step: it is a cheap way to find out how many resources your model really needs before paying for higher-end cards.