Scaling the Memory Wall: The Rise and Roadmap of HBM

The first portion of this report will explain HBM, the manufacturing process, dynamics between vendors, KVCache offload, disaggregated prefill decode, and wide / high-rank EP. The rest of the repor...