Spot vs On-Demand GPU Pricing: A100 and H100 Rates Compared
The difference between spot and on-demand GPU pricing is the single largest price gap in cloud compute — larger than the gap between providers and larger than the gap between GPU generations. This page puts the two side by side for the same GPU at the same provider, adds the reserved and negotiated-contract tiers so you can see the full ladder, and works through when the spot discount is real money and when interruptions eat it.
What the two prices are
On-demand is the public list price: a guaranteed instance, billed per second or per hour, that you can keep for as long as you pay. Spot (Google calls it Spot VM, formerly preemptible; marketplaces call it "interruptible") is the same hardware sold from the provider's unused capacity. Yes — spot rates are low precisely because the capacity would otherwise sit idle earning nothing, and the provider reserves the right to reclaim it for a full-price customer with seconds to minutes of notice. You are paying less in exchange for accepting that risk; the hardware and performance are identical.
Spot vs on-demand, same GPU, same provider
Illustrative on-demand and spot rates per GPU-hour. Spot prices float with demand; the figures are representative of the range seen during review, not a live quote.
| Provider | GPU | On-demand | Spot | Saving | Termination notice | Typical spot availability |
|---|---|---|---|---|---|---|
| AWS EC2 | A100 40GB (p4d, per GPU) | $3.22 | $0.97 | −70% | 2 minutes | Moderate — pools drain fast in us-east-1 |
| Google Cloud | A100 40GB (a2) | $2.77 | $0.83 | −70% | 30 seconds | Good in secondary regions |
| Azure | A100 80GB (ND A100 v4, per GPU) | $3.67 | $1.47 | −60% | 30 seconds | Moderate |
| Lambda Labs | A100 40GB | $2.40 | $1.20 | −50% | 5 minutes | Good |
| Vast.ai | A100 40GB | $1.79 | $0.51 | −71% | None guaranteed (outbid = gone) | Variable by host |
| RunPod | A100 80GB | $1.89 | $0.76 | −60% | 10 seconds | Good |
Three patterns are visible. Hyperscalers have the highest on-demand rate and the deepest percentage discount, so their spot price lands close to a specialist cloud's on-demand price. Marketplaces have the lowest absolute spot price but the weakest interruption guarantee. And the spot-to-on-demand ratio is remarkably stable at 30–40% of list, whichever provider you look at — that ratio is a better planning number than any single quote.
The full ladder: spot, on-demand, reserved and contract for the H100
Spot and on-demand are two rungs of a longer ladder. Using the H100 80GB, where the price index has the most listings:
| Tier | H100 $/GPU-hr | vs specialist on-demand | Commitment | Guarantee |
|---|---|---|---|---|
| Hyperscaler on-demand | $3.99 – 4.99 | +35% to +68% | None | Full |
| Specialist-cloud on-demand (index median) | $2.97 | — | None | Full |
| Marketplace on-demand (index low) | $2.29 | −23% | None | Full, host-dependent |
| 1-year reserved / committed use | ≈ $2.00 – 2.40 | −20% to −35% | 12 months | Full |
| 3-year reserved | ≈ $1.60 – 2.00 | −35% to −45% | 36 months | Full |
| Spot / interruptible | ≈ $1.00 – 1.80 | −40% to −65% | None | None — reclaimable |
| Negotiated multi-year cluster contract | Below public rates | Not published | Multi-year, minimum spend | Full, with priority on scarce SKUs |
Two consequences. The spot-versus-contract spread is narrower than the spot-versus-on-demand spread: a three-year reservation gets most of the way to spot pricing with none of the interruption risk, which is why large inference fleets run reserved, not spot. And on a given day spot can briefly cost more than a reservation when a region is short of capacity — spot is a floating price, the other rungs are fixed.
When spot is actually cheaper
The headline discount is not what you save. Every interruption costs you the work since the last checkpoint plus the time to reprovision and reload, so the effective spot price is:
Worked example on an A100 at $0.97 spot versus $3.22 on-demand (the AWS row). If you checkpoint every 15 minutes, lose on average half an interval (7.5 min) plus 5 minutes to recover, and get interrupted once every four hours, the wasted fraction is 0.25 × 12.5/60 ≈ 5%. Effective spot cost is $1.02/hr — still 68% below on-demand. Push interruptions to once an hour with 30-minute checkpoints and the waste rises to ~33%; effective cost $1.29/hr, still a 60% saving. Spot loses only when either the job cannot checkpoint at all, or an SLA makes any interruption unacceptable.
The break-even rule of thumb: spot wins for any restartable job as long as (minutes lost per interruption × interruptions per hour) < 60 × (1 − spot/on-demand). At a typical 30–40% spot ratio, that leaves room for a great deal of interruption before spot stops paying.
Which price to use for which workload
| Workload | Best tier | Why |
|---|---|---|
| Pre-training, fine-tuning, hyperparameter sweeps | Spot (with checkpoints) | Restartable, throughput-bound, large GPU-hour count — the discount compounds |
| Batch inference, data preprocessing, evaluation | Spot | Embarrassingly parallel; a lost worker costs one shard |
| Development, notebooks, short experiments | On-demand | Interruptions cost human time, which is worth more than the saving |
| Production inference endpoint | Reserved / committed for the baseline, on-demand for peaks | Latency SLA; spot variance is unacceptable |
| Hundreds of GPUs for a year or more | Negotiated contract | Guaranteed access to scarce SKUs at below-public pricing |
Related reading
- Spot GPU instances guide — the operational playbook: checkpointing, termination handlers, recovery scripts, provider strategies.
- Spot vs on-demand vs reserved pricing models — the decision framework for all five pricing models.
- GPU rental price index — low / median / high on-demand rates per accelerator.
- GPU pricing by region — how the same on-demand rate moves between the US and Europe.
- GPU cost calculator — toggle the spot option to see the effect on a specific job.