Back to Blog

TurboQuant on vision models

A series in 4 parts, published from to .

  1. I implemented Google's TurboQuant algorithm for KV cache compression and validated it on Molmo2 video inference on an RTX 4090 — 3.76x compression with near-identical output at 1.78x overhead.

    · explanationPart 1 of 4

  2. Google published TurboQuant at ICLR 2026 for text models.

    · explanationPart 2 of 4

  3. Build a container image with turboquant-vllm baked in, serve a vision-language model with 3.76x KV cache compression, and verify it works — in under five minutes.

    · how-toPart 3 of 4

  4. turboquant-vllm started as a Molmo2-only proof of concept. v1.3.0 validates seven model families — but getting there meant rewriting Triton kernels for non-standard head dimensions and teaching the cache about sliding window attention.

    · explanationPart 4 of 4

© 2026 Alberto Nieto. All rights reserved.