accelerate
bitsandbytes
torch
git+https://github.com/Muennighoff/transformers/tree/olmoe
einops
sentencepiece