Micron and Liqid demonstrate CXL memory pooling for AI inference

Micron’s Great Abaco system, built with Liqid for the U.S. Department of Energy’s Pacific Northwest National Laboratory, combines multiple hosts with a shared CXL 2.0 DRAM pool. The setup is intended to reduce repeated work when GPUs process prompts that share prefixes by allowing servers to reuse key-value cache blocks held in the pool. Blocks & Files reports that the system can expose up to 160 TB of DDR5. In a Llama 3 70B test with highly shared prefixes, a 32 GB cache configuration produced 2.8 times the throughput and about 2.9 times faster time to first token than a same-sized DRAM baseline. The results are tied to this system and workload; they do not establish that CXL memory universally matches local GPU memory.