Is an AI Data Center a Good Business?
Everyone is talking about the AI gold rush. Hyperscalers are spending tens of billions on GPUs, “neoclouds” like CoreWeave and Lambda have become household names in tech, and GPU rental marketplaces are full of people renting out compute by the hour.
So a natural question pops up: can a regular person — or a small company — get a slice of that pie? Not by building the next ChatGPT, but by doing the most “picks and shovels” thing possible: buying GPUs and selling compute time.
Let’s run the numbers.
The premise: a small AI compute operator
Let’s imagine we want to become an AI compute operator. We’re not Microsoft. We don’t have a gigawatt campus or a team of 200 SREs. We have limited resources — enough capital (own or borrowed) to buy one rack of hardware and place it in a colocation facility.
The business model is simple:
- Buy GPU servers.
- Host them in a data center (colocation — we rent space, power and cooling).
- Sell GPU-hours — either on a marketplace (Vast.ai, RunPod, Akash), directly to customers, or via a longer reserved contract.
- Profit = what customers pay − what it costs us to keep the lights on − what we paid for the hardware.
The hardware of choice: NVIDIA H100. It’s no longer the newest chip (Blackwell B200/B300 is here), but it’s the workhorse of the industry, it’s widely available, prices have dropped from the 2023 madness, and there’s a deep rental market for it.
The rack
A standard building block in this world is an HGX H100 server: one box with 8× H100 SXM GPUs connected with NVLink. Our rack will hold four of them — 32 GPUs total.
Why only four? Power. Each 8-GPU server pulls around 5.6 kW for the GPUs alone (8 × 700 W) and realistically you need to budget about 8 kW per server once CPUs, memory, NICs and fans are counted. Four servers ≈ 32 kW per rack. A classic enterprise rack is 5–10 kW, so we already need a high-density, AI-ready colocation facility.
Part 1: What it costs to buy
| Item | Cost |
|---|---|
| HGX 8× H100 SXM server (new) | ~$285,000 each (range $250–320k) |
| 4 servers | ~$1,140,000 |
| Networking (InfiniBand switches, cables, optics), storage | ~$80,000 |
| Total investment | ~$1,220,000 |
Most of that server price is the GPUs themselves. A single H100 SXM costs roughly $35–40k new; refurbished cards go for $18–22k. In a typical HGX quote, about $200–240k of the ~$285k is GPU silicon. The rest is the chassis, CPUs, RAM, NVLink, networking and integration.
So: about 1.2 million dollars for 32 GPUs. That’s our “limited resources” scenario. Already not exactly pocket money.
Part 2: What it costs to run
Owning GPUs isn’t a one-time cost. Every month the rack needs space, power, cooling, connectivity and someone to fix it when something breaks.
| Monthly cost item | Estimate |
|---|---|
| Colocation, ~32 kW at $150–200 per kW/month | ~$5,600 |
| Electricity (~$0.10/kWh, PUE 1.4) | ~$2,300 |
| Support contracts, spare parts, bandwidth, remote hands | ~$7,000 |
| Total operating cost |
A few notes:
- Colocation is priced per kW, not per rack. For AI, power is the scarce resource. Primary North American markets had vacancy of around 1.4% at the end of 2025 — finding a facility that will take a 30+ kW rack is a real challenge.
- PUE (Power Usage Effectiveness) of 1.4 means that for every 1 kW our servers use, the facility spends another 0.4 kW on cooling and overhead — and we pay for it.
- The “support” line is the one people forget. GPUs fail. NVLink boards fail. Optics fail. Without a support contract and a spare or two, one dead server means 25% of your revenue is gone until it’s fixed.
What’s not in the table: our own time, sales and marketing, insurance, accounting, and — importantly — interest if the hardware is financed. We’ll come back to that.
Part 3: What we can earn
Now the fun part. How much will someone pay to rent an H100?
Prices vary a lot depending on who’s selling:
| Seller type | Price per GPU-hour (H100, 2026) |
|---|---|
| Hyperscalers on-demand (AWS, Google Cloud, Azure, Oracle) | $6.90 – $11.00 |
| Neoclouds on-demand (Lambda, Nebius, CoreWeave, Crusoe) | $3.30 – $4.00 |
| Industry median, on-demand | ~$3.39 |
| Marketplaces (Vast.ai, RunPod, Akash) | $1.30 – $3.00 |
| 1-year reserved contracts | ~$2.10 – $2.70 |
| 3-year reserved contracts | ~$1.30 – $1.80 |
Here’s the catch: we are not AWS. Customers pay hyperscaler prices for the ecosystem, compliance, integration and brand. A small operator with one rack competes on price, which puts us in the marketplace / reserved-contract tier: realistically $1.80 – $2.70 per GPU-hour. Marketplaces also take a cut of what you earn.
A word about price trends
H100 rental prices have been on a rollercoaster:
- 2023 launch: ~$4.70/hr, spiking above $8 during the shortage.
- 2024–2025: a steep slide. AWS cut its H100 on-demand price by 44% in mid-2025. By mid-2026 the on-demand median was roughly 58% below early-2024 levels.
- Late 2025 – 2026: a rebound. One-year contract prices rose about 40% between October 2025 and March 2026, as demand for inference kept growing.
The long-term direction, though, is clear: every new generation (Blackwell now, Rubin next) offers more performance per dollar, and older GPUs get cheaper to rent. Whatever price we get today, we should expect less in year 3.
Utilization: the number that decides everything
A GPU that sits idle earns nothing but still costs money. Utilization — the percentage of hours the GPUs are actually rented — matters more than anything else in this business.
Our rack has 32 GPUs × 730 hours = 23,360 GPU-hours per month available to sell.
Part 4: The math — when do we get our money back?
Let’s look at three scenarios.
| Scenario | Price/GPU-hr | Utilization | Monthly revenue | Monthly profit (after opex) | Payback | Result after 3 years |
|---|---|---|---|---|---|---|
| Pessimistic — marketplace, patchy demand | $1.80 | 60% | ~$25,200 | ~$10,200 | ~10 years | −$850,000 |
| Base — mix of marketplace & direct | $2.30 | 75% | ~$40,300 | ~$25,300 | ~4 years | −$310,000 |
| Optimistic — reserved contract, near-full | $2.70 | 90% | ~$56,800 | ~$41,800 | ~2.4 years | +$280,000 |
(Investment: $1.22M. Opex: $15k/month. No financing costs, no resale value.)
Let that sink in:
- In the pessimistic case, the rack never pays for itself. The hardware will be obsolete long before the 10-year mark.
- In the base case — which is pretty reasonable for a small player — we’re still $310k under water after three years.
- Only in the optimistic case, with a long contract and the GPUs busy almost all the time, do we make a decent return.
Independent analyses come to a similar conclusion: owning H100s beats renting them only above ~70% sustained utilization over three years. At 70% the effective cost of an owned H100 is about $1.56 per GPU-hour; at 40% it jumps to $2.43 — more than you’d get renting it out.
“But I read it pays off in 13–18 months!”
You probably did. Those numbers usually compare the price of the GPU card alone with a $3+ hourly rate at 100% utilization. That’s not a business, that’s a spreadsheet fantasy. Add the server around the GPU, the networking, colocation, power, support, realistic prices and realistic utilization — and the payback stretches to 2.5–4+ years.
Things that make it worse
- Interest. Financing $1.2M at 7–10% adds roughly $7–10k per month in interest — which wipes out most of the base-case profit.
- Depreciation and obsolescence. Used H100 card prices are already ~85% below their 2023 peak in some secondary markets. Each new GPU generation pushes rental rates of the old one down.
- Downtime. One failed server = 25% of the rack’s revenue gone.
- Your time. Somebody has to find customers, handle support tickets and drive to the data center.
Things that make it better
- Resale value. If H100s still sell for something in year 3–4, that money can flip the base case into a small profit.
- Locking in a contract before buying. If you sign a 1–3 year deal at $2.30+ with a solid customer, you’ve removed the biggest risk — utilization.
- Buying refurbished. Cheaper hardware shortens payback, at the cost of shorter remaining lifetime and warranty risk.
- Cheap power. Electricity and colocation prices vary a lot between regions.
So, is an AI data center a good business?
For the big players — yes, it can be. They buy at scale, get better hardware prices, have cheap power deals, sign multi-year contracts with AI labs, and can afford to wait.
For a small operator with one rack — it’s a risky bet. The economics only work if you:
- Have a customer before you buy the hardware,
- Keep the GPUs busy 85%+ of the time,
- Get $2.30+ per GPU-hour on a contract long enough to recover the investment,
- Don’t pay much in interest.
Without that, you’re basically making a leveraged bet that GPU rental prices will stay high for the next 3–4 years — against an industry that releases a faster chip every year.
The “picks and shovels” idea is good. But in this gold rush, the shovels cost $1.2 million, lose value every month, and the shop next door is NVIDIA’s newest generation.
If you need GPUs, rent them. If you want to sell GPUs, sell the contract first — and buy the hardware second.
All prices are from public sources as of autumn 2026 and change quickly. This article is a back-of-the-envelope analysis, not financial advice.
