The same H100 can cost $2/hr or $4/hr depending on where you rent it. Spot instances are 50-70% cheaper but can disappear mid-training. Here's what things actually cost:
GPU prices are all over the place. Some platforms charge 10x others for the same hardware. Here's what ML teams are actually using and what it really costs.
Price drops, new platforms, and what's worth your compute budget. Weekly.
Join 900+ ML engineers · Unsubscribe anytime
✓ You're in. See you Thursday.
The same H100 can cost $2/hr or $4/hr depending on where you rent it. Spot instances are 50-70% cheaper but can disappear mid-training. Here's what things actually cost:
Spot instances are 50-70% cheaper but can be terminated anytime. Use them for fault-tolerant training with checkpointing. On-demand for anything you can't restart.
The serverless GPU platform ML Twitter loves. Write Python, decorate with @modal.gpu("A100"), it runs in the cloud. Cold starts ~1-2 seconds. $30/month free credits.
If you need raw GPU hours at the best price, Lambda is hard to beat. H100s at $2.49/hr are the cheapest we've found for reliable availability. They also sell physical hardware.
H100: $2.49/hr A100: $1.29/hr 8xH100: $19.92/hrBest for: serious training runs needing hours of uninterrupted compute.
Spot instance marketplace. A100s often under $1/hr. Can be preempted, availability varies. Good for batch jobs where you can checkpoint and resume.
Not just a model hub anymore. Inference endpoints, AutoTrain for no-code fine-tuning, Spaces for demos. If you're working with open models, you're probably here anyway.
500K+ models AutoTrain: no code Inference: $0.06/hr+Run and fine-tune models via API. Great for deploying open models without managing infra.
Try it →Fast inference for open models. Often the cheapest way to run Llama, Mixtral, etc.
Try it →Peer-to-peer GPU rental. Cheapest option, reliability varies. Good for experiments.
Try it →Experimenting? Modal free tier or RunPod spot. Vast.ai for absolute cheapest.
Fine-tuning? Modal or RunPod. Hugging Face AutoTrain for no-code.
Serious training? Lambda Labs for best H100 prices.
Production inference? Replicate or Together AI APIs. Modal for more control.
Enterprise? AWS/GCP if already there. More expensive but ecosystem benefits.
More from BuiltForAI