Rambus Highlights Tiered Memory Approach for AI Inference at South Korea SoC Event

Rambus announced two presentations for the D&R IP SoC South Korea event, including one focused on AI inference memory design. In that session, the company argues that inference workloads split into different stages with different bottlenecks, so a single memory approach is inefficient. Rambus says lower-cost GDDR or LPDDR can support prefill workloads, while HBM should be reserved for stages that depend more heavily on bandwidth and latency, with pooled memory used for KV offload. For RamTrend, the key takeaway is not the event itself but the continued push toward mixed memory hierarchies in AI systems, which could shape how future demand is distributed across premium and mainstream memory categories.