Advanced Packaging & the Integration Substrate

"How fast is this chip?" used to be answered at the transistor: a smaller node, more transistors, a faster clock. That era is functionally over for AI silicon. A single reticle field — the largest area a lithography scanner can pattern in one shot — is about 858 mm² (roughly 26 × 33 mm), and the largest useful monolithic die is already pressed against that limit. An H100-class GPU is essentially a full-reticle die. You cannot make the logic meaningfully bigger by printing it; you have hit the reticle wall. Everything that has happened to AI accelerator performance since is, at the physical level, a packaging story: how to stitch multiple reticle-sized fields together, how to bolt a dozen HBM stacks beside the logic, and how to wire it all with enough bandwidth that the assembled system behaves like one chip. The package is the integration substrate, and the firm that controls the largest, highest-yielding package controls how much compute the world can ship.

What follows is the engineering of that substrate: the 2.5D/3D packaging taxonomy and why it is the most-cited binding constraint on AI compute through 2030; the CoWoS-S/R/L families and SoIC / hybrid bonding, and the reticle-stitching and per-package-area ceilings each imposes; the interposer fork (silicon vs RDL vs glass) and its reach/cost/yield triad; the way HBM-stack-count-per-package falls directly out of interposer area — the engineering driver behind the allocation view of Chapter 7.6; chiplet disaggregation and UCIe and the economics of every die-to-die crossing; and the thermal, warpage, and yield consequences of building packages the size of a coaster. Procurement and allocation of CoWoS capacity stay in Chapter 7.6 and Chapter 2.3; here the concern is the physics.

The constraint moved upstream the same way power did in the facility. The industry assumed the gate on accelerator supply was wafer-out logic — how many GPUs a leading-edge fab could print. It is not. The gate is advanced packaging capacity, specifically TSMC’s CoWoS lines, which through 2026 are fully booked across both the CoWoS-S and CoWoS-L families. Reported allocation for the year runs to roughly a million wafers — NVIDIA alone ~595,000, about 60%, with Broadcom ~15% and AMD ~11% (Tom’s Hardware, May 2026) — while monthly TSMC capacity climbs from ~75–80k wafers toward ~120–140k by year-end, plus ~50–60k at OSAT partners (TrendForce, June 2026). TSMC discloses only relative growth, so every absolute figure is an analyst estimate, and none of them is this buyer’s monthly allocation. A logic wafer that cannot be packaged is not an accelerator: a TSMC/CoWoS design can hold N3/N5 logic allocation yet ship late if its package slot or qualified HBM lot is missing, and a total wafer count cannot be converted into delivered GPUs without package area, yield, test and product mix.

Choosing a more aggressive package — more reticles stitched together, more HBM stacks — buys performance, but it consumes more interposer area per accelerator (a 5.5×-reticle part eats far more interposer area, and yield risk, than a 2× part) and can put the program onto a newer process with less production-yield evidence. A conservative package preserves yield and supply but caps memory bandwidth and compute area. No axis is free: the package is where the GOODPUT ambition of the silicon team collides with the DENSITY-RAMP reality of the packaging line.

Advanced packaging splits cleanly into two regimes, and the fork between them is the first decision an architecture team makes. 2.5D places multiple dies side-by-side on a shared horizontal carrier — an interposer — that routes thousands of fine wires between them. This is how logic meets HBM today: the GPU die and its HBM stacks sit beside each other on a silicon or RDL interposer; a glass-core package substrate is a different layer, with HBM's wide thousand-plus-bit bus fanned out across that carrier. CoWoS is the canonical 2.5D flow. 3D stacks active dies vertically, face-to-face, bonded directly so that a die sits on top of another die rather than beside it. TSMC's SoIC and the hybrid-bonding processes underneath it are the 3D flow; AMD's MI300 stacks compute chiplets on base dies this way, and the HBM stack itself is a 3D structure internally.

The consequence of the fork is geometric. 2.5D buys you area — you spread components across a large flat carrier, and the limit is how large an interposer you can yield. 3D buys you proximity and density — wires between stacked dies are microns long instead of millimeters, slashing interconnect energy and latency, but you inherit a severe thermal problem (with a top-mounted cold plate, a buried lower die must send heat through overlying material toward the spreader and plate) and a yield-multiplication problem (a bad die anywhere in the stack can scrap the whole assembly). Some 2026 accelerators are hybrids: a 2.5D interposer carrying 3D-stacked logic and 3D-stacked HBM. The package is two integration technologies at once.

CoWoS-S uses one big slab of TSV-bearing silicon as the interposer — offering dense silicon routing, but a large silicon interposer is expensive and its own reticle-stitching limits cap how big it can get. CoWoS-R drops the silicon for an organic RDL carrier — with different routing, warpage and assembly constraints; price, yield and enabled stack count require the selected flow. CoWoS-L is the 2026 frontier: it abandons the single monolithic silicon interposer for an RDL carrier with small local silicon interconnect bridges embedded only where dense die-to-die routing is needed — letting the package span far more than a single reticle (3.5× in production since 2024, 5.5× certified and ramping in 2026, and a ~9.5× node in development for ~2027 supporting 12+ HBM4-generation stacks) while spending silicon only where it earns its cost. The consequence of choosing -L is that you inherit a new family of yield-loss mechanisms — bridge-to-RDL alignment, stitching seams, larger-package warpage — in exchange for a larger qualified routing area; compare usable routes and package yield, not the family suffix.

Where 2.5D spreads outward, SoIC (System-on-Integrated-Chips) stacks upward using hybrid bonding — a bumpless, copper-to-copper, dielectric-to-dielectric direct bond that eliminates the solder microbumps of older 3D stacking. The payoff is interconnect density: classic microbump pitch is ~40–50 µm; hybrid bonding takes the bond pitch into the single-digit-micron range and, on the roadmap, sub-micron. That is one-to-two orders of magnitude more vertical connections per unit area, which is what lets two dies behave electrically as if they were one — picojoule-per-bit interconnect, full-bandwidth die-to-die, latency measured in fractions of a clock. AMD's MI300-class parts stack compute chiplets on a base die this way; HBM's internal joining process is a separate question, and it has not settled in hybrid bonding's favour: SK hynix announced 16-layer HBM3E in 2024 and shipped 12-layer, 48 GB HBM4E samples in June 2026, both on Advanced MR-MUF. Treat stack height and joining process as independent attributes, per supplier and per product.

The 3D ceiling is thermal, and it is a hard one. When you put logic on top of logic, one die is always buried — and with the cold plate above the stack it is the lower die whose heat must traverse the upper die to escape, while power density at the bond interface can exceed what any external cooling can extract without throttling. Draw the cooler, the die order and the heat path for the actual assembly before naming the bottleneck: AMD puts second-generation 3D V-Cache below the CPU cores precisely to keep the hotter logic facing the cooler. This is why 3D-on-logic is selective — you stack the parts that benefit most from proximity (cache, certain compute tiles) and keep the hottest, highest-power logic where it can see the cold plate directly. The decision to go 3D is a decision to make the cooling problem of Part 5 harder in exchange for interconnect you cannot get any other way. The forward pointer is explicit: large-package hotspot and heat-flux behavior is engineered in Chapter 5.1 (the density wall) and removed by the cold plate in Chapter 5.4.

Underneath the CoWoS branding sits the real material decision: what is the carrier made of? Separate the interposer or redistribution layer from the package substrate core before comparing reach, cost and yield. Silicon interposers route the finest lines and carry TSVs for power and signal straight through — dense routing for HBM's wide bus — but a large silicon interposer is costly, is itself reticle-limited (you cannot pattern one bigger than a scanner field without stitching), and its CTE mismatch against the organic substrate drives warpage as it grows. RDL (redistribution-layer) carriers can be made large, but their routes are coarser than local silicon interconnect: CoWoS-L uses silicon bridges where the RDL carrier alone cannot supply the required density. Cost and warpage depend on that complete stack-up, rather than following automatically from the carrier material. Glass is the emerging third option: glass-core package substrates and glass interposers are separate structures; proposals for each target the dimensional stability and flatness of silicon with potential panel-scale manufacturing, with excellent high-frequency loss characteristics — but the ecosystem is immature, with handling, via-formation, and crack-propagation risks still being industrialized through 2026.

This is the link between this chapter and the memory chapter, and it is close to an identity: HBM stacks per package is a function of interposer area, not of how much HBM you can buy. Each HBM stack occupies a fixed footprint beside the logic die and must be reached by a thousand-plus-wire bus fanned across the interposer. Read the platform sizes as envelopes, not as a law of stack count: TSMC's ~3.3–3.5×-reticle platforms accommodate eight HBM stacks, and the ~5.5× platform that CoWoS-L begins delivering in volume in 2026 accommodates up to twelve; 16+ stacks need the next reticle-multiple node still ramping. Interposer size follows the whole floorplan — logic-die area, I/O, and routing as well as stacks — not the HBM generation alone. So when Chapter 7.6 describes HBM as the top-3 BOM line and the binding allocation gate, the engineering reason a given accelerator carries the stack count it does lives here: the package area the program chose set the stack count, which set the memory capacity and bandwidth, which set where it lands on the memory roadmap.

Memory ambition and packaging ambition are therefore one decision. A team that commits to an 8-stack, ~288 GB HBM4 accelerator (Rubin-class, ~2.75 TB/s/stack — ~22 TB/s per package) has by that act committed to a large-reticle CoWoS-L package, its yield curve, and its slice of the scarcest packaging capacity on earth — though NVIDIA has not disclosed Rubin's exact reticle multiple, and eight stacks alone do not require the full 5.5×. You cannot buy your way to more bandwidth without buying your way to more interposer area, and interposer area is exactly the thing in shortest supply. The procurement reflex — "order more HBM" — is necessary but not sufficient; without the CoWoS slot to mount it on, the HBM is inventory, not bandwidth. That allocation logic lives in Chapter 2.3; the engineering driver lives here.

Once the reticle wall makes one big die impossible, the question becomes how to build a big system from small dies — and that is the chiplet decision. Disaggregation splits a would-be monolithic SoC into multiple dies ("chiplets") that are packaged together: compute tiles, I/O dies, cache dies, each potentially on a different process node tuned to its job. The upside is real and quantifiable. At a fixed defect density, smaller die area improves individual-die yield — defect density punishes large dies brutally, so four small dies can improve known-good-die supply, but assembly yield and testing decide the cost of a usable package. You reuse a chiplet across many products. You put I/O on a cheap mature node and spend leading-edge wafers only on the logic that needs them. AMD productized this years ago; it is now the default architecture for the largest accelerators.

The cost of disaggregation is paid at every die-to-die crossing, and this is where UCIe (Universal Chiplet Interconnect Express) enters. A monolithic SoC pays wire, buffering and switching energy on-die. A chiplet system must drive and receive across a physical die boundary, with serialization and encoding determined by its interface — spending area on PHYs, power on the link (picojoules per bit that add up across terabytes per second), and latency on the crossing. UCIe standardizes that interface to support interoperability within an implemented, qualified profile: UCIe-S targets standard 2D packages, UCIe-A targets advanced 2.5D packages, and UCIe-3D covers 3D integration; 64 GT/s arrived with UCIe 3.0, while the 3D interface was introduced in UCIe 2.0. UCIe's published catalogue lists 3.0 as the current revision; voltage, bump pitch, lanes, protocol, test access and thermal/mechanical compatibility still need agreement. The strategic opportunity is an open chiplet market: a buyer can, in principle, assemble an accelerator from best-of-breed dies rather than one vendor's monolith — the packaging-era analog of the open-system disaggregation Part 8 describes for the network.

Transmitted rate = 1.0×10¹² B/s × 8 bit/B × 1.10 = 8.8 Tb/s. Combined endpoint power A = rate × 1.0 pJ/bit = 8.8 W; B at 3.0 pJ/bit needs about 26 W and exceeds 20 W. Buried-die rise = 600 W × 0.040 K/W = 24 K, within 30 K. A’s payload threshold is 20 W/(8×1.10×1.0 pJ/bit), about 2.3 TB/s; 2.3 itself is above that exact bound. Independently, the thermal crossover is 30 K/600 W = 0.050 K/W; at 0.060 K/W the rise is 36 K and fails.

Qualify A’s actual bump pitch, voltage, lane width, protocol, test access and assembly process. A catalogue UCIe version alone cannot make chiplets interchangeable. Request the thermal and assembly-yield evidence before reserving its HBM placement and packaging capacity.

Method: UCIe Consortium specification catalogue. Chapter 7.14 owns the next handoff.

Large packages punish you three ways, and each is a downstream cost of the area you bought to defeat the reticle wall. Warpage is the first: a large silicon interposer on an organic substrate has a coefficient-of-thermal-expansion mismatch, and as the assembly heats and cools it bows. Past a certain span the bow threatens the solder joints to the board and the bond integrity across the package — which is a major reason CoWoS-L's RDL-plus-bridge construction exists (the organic carrier is more compliant) and why glass cores are attractive (dimensionally stable and flat). Warpage is not a yield footnote; it is a first-order limit on how large a package you can build at all.

Yield is the second, and large packages multiply it sharply. A 12-HBM, multi-reticle, stitched package is the product of many independent yields (each die, each HBM stack, each bond, each stitch seam), and a defect anywhere late in the flow scraps an assembly carrying enormous accumulated value (a dozen HBM stacks and multiple known-good logic dies). This is why known-good-die testing before assembly is non-negotiable and why the largest packages carry the largest scrap-cost exposure: you are gambling the most expensive components on the last and least-reversible step. Thermal is the third: a large, dense package concentrates hundreds of watts into a small footprint with internal hotspots over the 3D-stacked regions, producing heat fluxes that air cannot remove and that even direct-to-chip liquid must be engineered for. The package's heat-flux map is the boundary condition the cold plate inherits — the explicit hand-off to Chapter 5.1 and Chapter 5.4.

Walk the chain forward and the package sits at the center of it. The reticle-multiple you choose sets the interposer area; interposer area sets the HBM-stack count, which sets memory capacity and bandwidth (→ Chapter 7.6); the reticle-multiple also sets which CoWoS family and OSAT can build it, which sets yield and lead time (→ Chapter 2.3); the package's power and heat-flux map sets the cooling boundary condition (→ Chapter 5.1, Chapter 5.4); and the on-package power-delivery and di/dt behavior that a big, dense package demands is its own engineering problem (→ Chapter 7.12). The package is the substrate every other accelerator decision is mounted on. Get its reticle-multiple and family wrong and you have either an accelerator you cannot get capacity to build or one whose memory and compute ambition you under-shot — and unlike a board respin, a package family change is a multi-quarter, multi-million-dollar reset.