Accelerator Selection, TCO & Procurement Strategy

The preceding chapters covered the silicon itself: NVIDIA's annual cadence (Chapter 7.2), AMD's open challenge (Chapter 7.3), the hyperscaler XPUs and custom ASICs (Chapter 7.4, Chapter 7.5), the HBM and packaging constraints that gate all of it (Chapter 7.6, Chapter 7.7), and the software ecosystems that lock or liberate the choice (Chapter 7.9). Selection pulls those threads into a single decision: which complete accelerator configurations qualify, how many are required, who can deliver them, and on what schedule? This is where a spec sheet becomes a testable power, memory and delivery obligation.

The sequence matters because each step gates the next. Naming the governing metric comes first, since the wrong denominator silently picks the wrong chip. The eligibility calculation then turns measured per-node useful output into a fleet count and power requirement. Whether you optimize tokens-per-dollar or tokens-per-megawatt turns on a prior question — power-limited vs capex-limited — that re-ranks the same shortlist. Chapter 1.8 compares buy, rent and build cash flows from the qualified service; the heterogeneous fleet is an engineering choice with a carrying cost; and supply allocation, plus the RFP that makes cross-vendor bids honest, closes it out. The pricing and economic-life calculation lives in Chapter 1.8; here we supply its eligible equipment and service operands. The per-generation perf/watt and cost-per-token trajectory is consolidated in Chapter 16.2; here we use today's snapshot to decide.

Most selection mistakes trace to comparing chips on the wrong axis. A GPU that wins on raw FLOPS can lose on delivered tokens; a chip that wins on $/GPU-hour can lose on $/M-tokens; a part that wins on $/M-tokens at the bench can lose on tokens/MW once the grid binds. The service objective and simultaneous power, capital and schedule constraints determine the useful denominator, so name it before you shortlist.

Raw FLOPS and HBM capacity are inputs, not metrics — they tell you what the silicon could do, not what it will do against your workload at your batch size and precision. The realized-MFU gap between paper FLOPS and delivered throughput is workload- and release-specific; interpret each dated MI300X comparison at its named workload, precision and release. Shortlist on what a chip delivers, not on what its data sheet promises.

$/GPU-hour fits when you are buying or selling raw compute supply — a neocloud renter buying raw allocation; a training decision instead needs cost and elapsed time to the same validated model quality. $/M-tokens fits when you sell a product to an end user, because tokens are what the customer buys; it is now the governing metric for most operators, while Deloitte’s November 2025 forecast puts inference at ~2/3 of AI compute in 2026, not the mix of every operator. tokens/MW — tokens per second per megawatt, which is what perf/watt means here — takes over the moment power is the binding constraint, and in 2026 that is true for a growing share of operators. Utilization and tokens-per-GPU-second link the three, and a part that wins on one can lose on another. A credible pro-forma states which denominator it optimizes and why. → metric definitions in Chapter 0.3; the financial denominators in Chapter 1.8.

Chapter 1.8 converts qualified cost and service inputs into the governing economic metric. Selection supplies both sides: the complete installed configuration and the useful output it can deliver within the acceptance limits.

The numerator is more than the chip. Start with the installed configuration: SemiAnalysis’s October 3, 2024 eight-GPU H100 chassis example is $270k before, $256.4k after host/DPU optimization, before rack/cluster fabric, storage and software, and the GPU is only part of it — host CPUs, HBM (capacity/bandwidth obligations, see Chapter 7.6), NICs, the NVSwitch/scale-up fabric, the chassis, and integration. That node is the labelled historical anchor, not the 2026 unit of purchase: new frontier capacity is bought as an NVL72-class rack — GB300 NVL72 in 2026 volume — whose SKU already contains the NVSwitch trays, the busbar, and the liquid loop (Chapter 7.2), so build the numerator per rack and divide down to $/GPU before you compare. A second, genuinely air-cooled tier persists alongside it — 8-GPU HGX B300 nodes and RTX PRO-class enterprise servers — and it prices like the node example, not like the rack. Then identify recurring power and cooling, networking, facility/lease, staff, and software/support cash; keep the server and network acquisition prices with the upfront inputs. Chapter 1.8 separates that cash timing from the book-depreciation schedule. Carry the actual node/rack acquisition price, shared-system costs, paid capacity and net disposal assumption into Chapter 1.8; do not derive another hourly ownership model from the sticker price here.

The denominator is useful output under Chapter 14.1’s accounting boundary, and “useful” carries the weight. For training it is delivered model-FLOPS-utilization against the job, not nameplate FLOPS — a fabric that starves the all-reduce (see Chapter 8.5) collapses the denominator no matter how fast the chip. For inference it is tokens-per-GPU-second at your SLO, your precision, and your batch efficiency, which is where FP4/NVFP4 quantization (Chapter 7.10) and serving discipline (Chapter 10.11) move the number several-fold. Carry the earning-life scenarios — the contested 2–3-year bear case versus published 4–6-year useful-life estimates — separately from the 5–6-year book policies into Chapter 1.8. A shorter earning period can change the hardware shortlist’s matched cash ranking; a longer book schedule alone cannot create earning time or disposal receipts.

Chapter 1.6 frames procurement archetypes; Chapter 1.8 owns their matched-service NPV comparison. Here qualify the accelerator, installed price, service availability and contract rights before carrying those inputs into that calculation. Utilization, economic earning life and net resale proceeds belong in its cash flows; book depreciation is a separate expense schedule.

Rental eligibility requires the provider’s available capacity, supported workload and cancellation rights to fit the job’s timing. A short or spiky workload alone does not establish those rights. Historical H100 pricing in Chapter 1.8 separates Spheron spot, Lambda configurations and AWS/Azure on-demand observations; a posted price does not establish capacity or cancellation rights. Carry the qualified installed price, calendar-billing terms, cancellation rights, delivery date and net disposal assumption into Chapter 1.8’s matched-service cash flows. Include host, fabric, storage, facility allocation and operating support on both sides. A missed delivery date requires bridge capacity and fewer owned earning periods; an unqualified exit right retains the rental bills. Chapter 2.5 then tests whether the chosen commitment supports its debt obligations. Cash-breakeven utilization cannot select ownership versus rental.

Composition is the next choice: a single-vendor fleet, or a heterogeneous mix of GPUs, hyperscaler XPUs, and custom ASICs. This trades software velocity against cost-per-token leverage, and the second stack carries a quantifiable cost.

Single-vendor (in practice, NVIDIA + CUDA) buys software velocity, the broadest library and kernel coverage, and operational simplicity — one stack to staff, one toolchain to qualify, one supplier relationship to manage. The cost is allocation pain and price exposure: you inherit the supply queue and pay the merchant premium. Heterogeneous fleets chase cost-per-token leverage and supply resilience. The economics that justify the second stack are real: a matched AMD quote and qualified inference run can beat an NVIDIA offer after acquisition and support terms are compared on the same boundary; custom ASICs are eating the inference workload precisely on a tokens/$/W basis. The structural pull is strong — ASIC-based AI servers are projected at ~27.8% of shipments in 2026, a server-shipment forecast rather than a fleet workload allocation (TrendForce / industry synthesis, 2026), because stable-architecture inference is exactly where fixed-function silicon's efficiency beats general-purpose flexibility.

Match silicon to sub-workload, and price the carrying cost of the second stack honestly. Custom ASICs and TPUs can win on qualified, high-volume workloads where the architecture is frozen and cost-per-token dominates — but you surrender portability and inherit a provider-controlled toolchain (XLA/JAX or Neuron, see Chapter 7.9). Merchant GPUs can reward changing workloads — frontier training, research, anything that needs CUDA's velocity and kernel coverage — because reprogrammable flexibility is worth the premium when the model architecture is still moving. The carrying cost of heterogeneity is a second software stack: engineers, qualification, CI, kernel ports, and the realized-MFU gap on the less-supported part. Only carry it if the cost-per-token saving on the workload you actually run exceeds that engineering tax. → the GPU-vs-ASIC framing in Chapter 7.5; lock-in quantification in Chapter 7.9.

Selection assumes you can get the silicon. In 2026 you frequently cannot — not on your timeline — and a procurement strategy that ignores allocation is guesswork. The binding constraint upstream of every accelerator is HBM and advanced packaging: the 2026 HBM output its suppliers disclosed already committed, HBM4 reaching volume only in H2 2026, and CoWoS-class packaging is the most-cited binding constraint on AI compute through 2030 (see Chapter 7.6, Chapter 7.7). Your purchase order does not set the arrival date; the supplier's HBM and CoWoS allocation does.

Three concrete consequences follow for procurement. First, lead time is a selection criterion, not a footnote: a part that is 15% cheaper per token but arrives two generations into your ramp may strand a power slot you fought years to energize (see Chapter 3.2). Second, allocation favors anchor commitments — take-or-pay volume, multi-generation agreements, and (increasingly) vendor-equity entanglements move you up the queue, which is why allocation strategy is inseparable from the financing structure (Chapter 2.5) and the end-to-end equipment supply chain (Chapter 2.3). Third, a second-source qualification is a hedge with option value: carrying a qualified AMD or ASIC path is not only a cost-per-token play, it is insurance against a single-supplier allocation squeeze. The heterogeneous fleet and the allocation hedge are the same decision viewed from two angles.

The RFP makes or breaks accelerator selection, because it is the only instrument that renders heterogeneous bids comparable. The common, expensive failure: vendors bid on the spec sheet, you compare paper FLOPS, and the realized-MFU gap turns the cheapest bid into the most expensive cluster. A defensible RFP forecloses that by fixing the comparison on delivered goodput, not nameplate performance.

• Specify the metric, not the part. State the workload, the precision, the SLO (TTFT/TPOT for inference; job-completion-time for training), and demand bids in your governing denominator — $/M-tokens at the SLO, or $/effective-PFLOP-hour at delivered MFU — so a cheap, slow part cannot win on sticker price.
• Mandate a benchmark on your workload. Require a measured run on a representative model at your batch and sequence length, not a vendor-supplied number on a favorable case. The MI300X-vs-H100 benchmarking literature exists precisely because measured diverges from spec; bake the measurement into the bid.
• Score on tokens/MW when power-limited. If your interconnection is capped, the bid that wins on $/M-tokens but loses on perf/watt cannot be energized at scale — require a tokens/MW line and weight it to the binding constraint.
• Bind the delivery schedule. Demand a contractually-committed, HBM/CoWoS-allocated delivery curve with remedies for slip — because an unbacked lead time is the single most common reason a 'won' procurement misses its ramp.
• Set a goodput acceptance bar. Tie acceptance and payment milestones to a sustained, measured goodput threshold (see Chapter 7.14), not to power-on — so a cluster that benches well but cannot hold MFU through burn-in is the vendor's problem, not yours.

Selection resolves into an ordered procedure, each step gating the next. (1) Name the denominator — $/GPU-hr, $/M-tokens, or tokens/MW — from what you sell and what binds you. (2) Determine the regime — power-limited or capex-limited — because it picks between tokens-per-dollar and tokens-per-megawatt. (3) Build the TCO model with an explicit, dual depreciation assumption (economic and book life), since that one term re-ranks the shortlist. (4) Decide buy vs rent vs build at your real utilization and resale liquidity, treating the answer as a crossover that moves, not a fixed policy. (5) Compose the fleet — single-vendor for velocity, heterogeneous for cost-per-token and supply resilience — pricing the carrying cost of the second stack against the workload's stability. (6) Execute the RFP on realized goodput with a bound, allocated delivery schedule. The output is not a chip; it is a purchase commitment, a power budget, and a depreciation schedule that the rest of Part 7 and Part 1 have to live with.

For each technically supported offer, divide required useful output by per-node output and round upward to whole nodes. Multiply that count by rack-input power; reject any memory or power failure. Arrival is the latest predecessor release plus transit. The calculation below shows how a system-release slip can remove the sole eligible offer before any financial comparison.

Carry B’s node count, installed/running power, operating window, release tuple, delivery milestones and support obligations to 1.8, with acquisition, facility enabling works, staff, energy, migration and residual quotations. Do not award a purchase from this screen: the current result is HOLD until synthetic performance and dates are replaced by acceptance evidence and contractual commitments.

Method: NVIDIA CUDA best-practices measurement method. Chapter 1.8 owns the next handoff.