Skip to content

Quantization

Quantization modules for model compression and acceleration.

Components

  • MindBNB - BitsAndBytes quantization implementation
  • SmoothQuant - SmoothQuant quantization method