[2604.10152] SpecMoE: A Fast and Efficient Mixture

2026 Subjects: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG) Cite as: arXiv:2604.10152 [cs.AI] (or arXiv:2604.10152v1 [cs.AI] for this version) https://doi.org/10.48550/arXiv.2604.10152 Focus to learn more arXiv-issued DOI via DataCite ， Eunyeong Cho，。

which is accepted for publication at the 63rd ACM/IEEE Design Automation Conference (DAC)。

they offer limited efficiency， particularly for large batch sizes. In this work， its high memory requirements and sub-optimal parameter efficiency pose significant challenges for efficient deployment. Although CPU-offloaded MoE inference systems have been proposed in the literature， while significantly reducing bandwidth requirements of both memory and interconnect on memory-constrained systems. Comments: This is an extended version of our work， [Submitted on 11 Apr 2026] Title: SpecMoE: A Fast and Efficient Mixture-of-Experts Inference via Self-Assisted Speculative Decoding Authors: Jehyeon Bang， we propose SpecMoE， a memory-efficient MoE inference system based on our self-assisted speculative decoding algorithm. SpecMoE demonstrates the effectiveness of applying speculative decoding to MoE inference without requiring additional model training or fine-tuning. Our system improves inference throughput by up to $4.30\times$。

Jinha Chung， Minsoo Rhu View a PDF of the paper titled SpecMoE: A Fast and Efficient Mixture-of-Experts Inference via Self-Assisted Speculative Decoding。

by Jehyeon Bang and 4 other authors View PDFHTML (experimental) Abstract: The Mixture-of-Experts (MoE) architecture has emerged as a promising approach to mitigate the rising computational costs of large language models (LLMs) by selectively activating parameters. However， Ranggi Hwang。