DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
Paper
•
2401.06066
•
Published
•
56
MOE
-
-
Mixtral of Experts
Paper • 2401.04088 • Published • 160 -
Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM
Paper • 2401.02994 • Published • 52 -
LLM Augmented LLMs: Expanding Capabilities through Composition
Paper • 2401.02412 • Published • 39 -
Scaling Laws for Fine-Grained Mixture of Experts
Paper • 2402.07871 • Published • 14 -
OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models
Paper • 2402.01739 • Published • 29 -
MoE-LLaVA: Mixture of Experts for Large Vision-Language Models
Paper • 2401.15947 • Published • 54