Lightbits introduces KV-cache engine spanning GPU memory, DRAM and NVMe

Lightbits Labs has introduced Inferra, a KV-cache orchestration engine for AI inference, according to StorageReview. The software is designed to virtualize memory resources across GPU memory, DRAM, and NVMe storage rather than keeping all cache data in the most expensive tier. The company says the approach can raise session density by up to sixteen times and support contexts reaching ten million tokens. Those are vendor claims and should be evaluated against workload, hardware, and latency requirements. Tiering KV-cache data is relevant to the memory market because it can change how AI operators balance scarce accelerator memory against system DRAM and flash storage. The product's commercial adoption and its effect on component demand remain to be established.