|
|
--- |
|
|
title: DFloat11 - Lossless LLM Compression for Efficient GPU Inference |
|
|
emoji: ⚡ |
|
|
colorFrom: red |
|
|
colorTo: gray |
|
|
sdk: static |
|
|
pinned: false |
|
|
--- |
|
|
|
|
|
# ⚡️ DFloat11: Lossless LLM Compression for Efficient GPU Inference |
|
|
|
|
|
DFloat11 is a lossless compression framework that reduces the size of LLMs and Diffusion Models by approximately 30% while preserving bit-for-bit identical outputs to the original model. It enables efficient GPU inference on resource-constrained hardware without sacrificing accuracy. |
|
|
|
|
|
## 🚀 Key Features |
|
|
|
|
|
* **Lossless Compression**: Achieves \~30% model size reduction with outputs identical to the original BFloat16 models. |
|
|
* **GPU-Efficient**: All decompression is handled on-GPU, eliminating CPU overhead and host-device data transfers. |
|
|
* **Scalable Performance**: Decompression overhead remains constant per forward pass and is independent of batch size. |
|
|
* **Broad Model Support**: Compatible with various models, including Qwen3, Gemma3, Llama3, Phi4, Wan2.1, FLUX.1, and BAGEL. |
|
|
|
|
|
## 🛠 Installation |
|
|
|
|
|
Ensure you have a CUDA-compatible GPU and PyTorch installed. |
|
|
|
|
|
```bash |
|
|
# For CUDA 12 |
|
|
pip install -U dfloat11[cuda12] |
|
|
|
|
|
# For CUDA 11 |
|
|
pip install -U dfloat11[cuda11] |
|
|
``` |
|
|
|
|
|
## 🧪 Quick Start |
|
|
|
|
|
For example usage, refer to the [examples directory](https://github.com/LeanModels/DFloat11/tree/master/examples) in the GitHub repository. |
|
|
|
|
|
## 📄 Learn More |
|
|
|
|
|
* **Paper**: [70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float](https://arxiv.org/abs/2504.11651) |
|
|
* **GitHub Repository**: [LeanModels/DFloat11](https://github.com/LeanModels/DFloat11) |
|
|
* **Hugging Face Models**: [DFloat11 on Hugging Face](https://huggingface.co/DFloat11) |
|
|
|
