Calculators › Inference $/M tokens

Inference $/M tokens

Compare rented and owned inference capacity in $/M tokens on the same throughput and productive-use assumptions, with the price-decline context you should underwrite against.

ScenarioNVIDIA GB300 NVL72 · 56 active / 72 installed GPUs · 1 racks · 0.14 MW IT · $9M programguide defaults (as of 2026-09)Edit your project assumptions →

Inference cost per million tokens

Rented node cost$/hrDefault = an 8-GPU H100 node at $19.20/hr (~$2.40/GPU-hr) at ~2,800 tok/s, a short-context illustration. The guide's 32k-context × 64-concurrency profile needs a 16-GPU replica. Price the same replica you measured throughput on.Owned node calendar cost$/hrThroughputtok/sGuide assumption; replace with a steady-state measurement from the serving profileOwned-use utilization%

$1.21 / M tokens owned · $1.42 rented

Owned capacity at 70% productive utilization$1.21/M tokens

Rented capacity at the same 70% productive utilization$1.42/M tokens

Compare ownership and rental on the same throughput and productive-use denominator. Owned node cost is the calendar-hour allocation of capex, energy, and opex; rented node cost is the capacity held for that hour. Prices enterprises paid for tokens fell ~$10 → ~$2.50/M in a year (Ramp spend data, Mar 2025), so underwrite inference revenue with a price-decline curve → Ch 1.8.

Worked decision — stated assumptions

All ledger inputs are assumed teaching values chosen to exercise this chapter’s method; they are not market quotes, OEM performance claims or operating records. Arithmetic uses the full entered precision.

Assumed inputs
Input	Value
Rented node cost	20 $/hr
Owned node calendar cost	10 $/hr
Throughput	1000 tok/s
Owned-use utilization	50 %

Illustrative serving case: assume one tested node produces 1,000 useful output tokens/s within the declared latency target and runs productively 50% of each paid hour. Useful output is 1,000 × 3,600 × 0.50 = 1.8 million tokens/hour. Assumed rental of $20/hour costs $11.1 per million output tokens; an assumed owned calendar cost of $10/hour costs $5.56 per million.

Against an illustrative $6 per million output-token cost allowance, ownership passes and rented capacity fails. At 40% productive use the owned cost rises to $6.94 and also fails. Keep the model, precision, prompt/output mix, batching, latency target and cost boundary identical; prompt tokens and cached tokens do not silently enter this output denominator.

The owned case crosses the $6 allowance at 46.3% productive utilization: 10 / (1,000 × 3,600 × 6 / 1,000,000).

Guide derivation. Method and handoff: Chapter 1.8 — Business Models, Economics & ROI.

These are transparent screening estimates, not final designs, financial advice, or project approvals. Replace every default with current project data and have the responsible project authorities validate the result. You can save, share a permalink, or export to CSV; see the full calculator suite.