Model Compression Calculator

For a given starting model size, switching the Quantization Bits setting moves Compressed Size more than an equivalent change to Pruning % or Distillation Ratio, because it jumps between fixed byte-width multipliers -- FP32 at 4 bytes per weight down to INT4 at just 0.5 bytes, an 8x range -- rather than applying a smooth percentage adjustment. Pruning and distillation still matter, but neither alone can match the swing a single quantization-level change produces.

In this calculator, yes -- it applies pruning first (removing a percentage of the original size), then quantization (multiplying by a fixed byte-width ratio), then distillation (dividing by the distillation ratio), matching the order a real compression pipeline typically follows. Because pruning and quantization are both simple multiplicative factors, applying them in either order would actually give the same final size mathematically -- distillation dividing last is the step where sequencing genuinely changes the intermediate numbers shown.

Because fewer bits per weight means coarser rounding of each value, and the calculator's accuracy-impact lookup reflects that: it estimates about a 0.5% accuracy impact for INT8 versus about 2% for INT4, a real pattern in published quantization research where accuracy loss grows faster than linearly as bit width drops, since there are exponentially fewer representable values at each step down.

No -- it is a rough additive estimate built from independent per-technique rules of thumb (a fixed percent per 10% pruned, a fixed percent per quantization level, a fixed percent per distillation step), not a measurement of your specific model and dataset. Real compression techniques often interact in ways a simple sum cannot capture, so always validate a compressed model's actual accuracy on a held- out test set before deploying it.

A ratio of 1 means no knowledge distillation is applied at all -- the Compressed Size after pruning and quantization is left unchanged, since dividing by 1 has no effect. You would only raise this value if you are training a smaller "student" model to mimic a larger "teacher" model's behavior, which is a separate training process this calculator does not simulate, only account for by size.