Memory
GPU price per GB of memory
Answer in brief
Price per gigabyte of GPU memory is the hourly price divided by the memory on the card, and it is a fair way to compare cards only after you know how much memory the job needs.
Runcrate is our pick here because its 26 GPU models span small cards to large ones on one account, so you rent the memory you need and no more.
Start from the memory you need
The card must hold the model’s weights, the working memory of each request and what the software keeps for itself. Weights take about two bytes per parameter at 16-bit precision, or about 2 GB for every billion parameters, and less at lower precision.
That figure is the weights alone. Running a model needs room on top for context and activations, so add a margin of about a fifth. The sum is the smallest card you can use.
The cards by memory
Among the cards in these tables, the RTX 5090 has 32 GB, the RTX Pro 6000 has 96 GB and the H200 has 141 GB. The B200 is the Blackwell datacenter card above the H200.
The steps are large. A job that needs 40 GB skips the RTX 5090 and lands on the RTX Pro 6000 with more than half the card to spare.
How to work out price per gigabyte
Divide the price per GPU-hour by the gigabytes on the card. On 12 Nov 2026, the median listed on-demand RTX 5090 price here was $0.655 per GPU-hour for 32 GB. The median listed RTX Pro 6000 price was $2.20 for 96 GB. The median listed H200 price was $5.40 for 141 GB.
Use the hourly total of the smallest rental, not a per-GPU figure inside a machine you would have to take whole.
What the ratio hides
A low price per gigabyte is no use if you rent gigabytes you do not fill. The cheap card is the smallest that fits, not the one with the best ratio.
The ratio also says nothing about speed. Two cards with the same memory can finish the same job in very different times, and the bill follows the time.
One large card or several small ones
Several small cards can match the total memory of one large card for less per gigabyte. The model must then be split across them, which not all software does and which costs speed. For one large model, one card that holds it whole is usually the lower bill.
Where Runcrate fits
Runcrate lists each GPU with its own price per GPU-hour, and billing by the second on most models lets you test a fit for the cost of a few minutes. Runcrate can start an RTX Pro 6000 now from $2.87 per GPU-hour.
Questions and answers
Sources for this entry
- Runcrate: Runcrate’s official site: what the platform is, how it bills and the range of GPUs it offers. Read 12 Nov 2026.
- Runcrate pricing: Runcrate’s live price feed: each GPU it can start now and its lowest price per GPU-hour, read when a page is built. Read 12 Nov 2026.
- Provider pricing pages: The published on-demand list price of every provider named in the price tables, with the machine size and billing unit it applies to. Read 12 Nov 2026.
- Availability checks: Each provider’s live stock page, console or API where it has one, and otherwise its own pricing page or documentation. Read 12 Nov 2026.
- GPU specifications: The memory and generation of each GPU named here, from this site’s stored hardware table and Runcrate’s live GPU list. Read 12 Nov 2026.
Checklists are original editorial guidance. Published prices do not establish availability.
Continue this reading path
Cite this entry
Cheap Cloud GPU. “GPU price per GB of memory.” Content reviewed 12 Nov 2026. https://cheapcloudgpu.com/topics/gpu-price-per-gb-of-memory