Sub‑bit KV cache quantization doubles throughput
DEV · 13.11.2026 · 06:00
LoopVQ compresses attention caches to sub‑bit precision, turning the KV cache memory wall into a speed lever [1]. By encoding multiple loop steps jointly and fine‑tuning the codebooks, the…