Back to Blog

vramfit

A series in 6 parts, published from to .

  1. I shrank a 93 GB model onto a 24 GB card by measuring which layers survive being crushed, instead of guessing.

    · explanationPart 1 of 6

  2. I claimed a smaller card gives you a different quantization recipe, not the same one squeezed, and then didn't prove it.

    · explanationPart 2 of 6

  3. Measure a model's per-layer quantization damage, solve for a recipe that fits your card, and check the result before you trust it.

    · how-toPart 3 of 6

  4. The smallest 2-bit-labeled build of Nemotron 3.5 Lightning is 17.54 GiB — it doesn't fit a 16 GiB card, and its label names 12 of 417 tensors.

    · explanationPart 4 of 6

  5. The official Q4_0 build of Gemma 4 31B is 16.44 GiB and fits a 24 GiB card with room to spare, so "it fits" was never the claim.

    · explanationPart 5 of 6

  6. Download the published fit24gib pack, install llama.cpp at the build the numbers were measured on, and serve it at 86,016 tokens of text context or 73,728 with an image aboard.

    · how-toPart 6 of 6

© 2026 Alberto Nieto. All rights reserved.