Abstract

Multi-expert large language model systems need to dynamically distribute user requests among multiple candidate experts to improve inference accuracy and system efficiency in complex task scenarios. To address the issues of existing routing methods relying on manual rules, unified discriminant boundaries, or large-scale annotated data, as well as high maintenance costs during expert expansion, this paper proposes an expert routing method, BGMM-LSE, based on positive sample semantic distribution modeling. The method constructs a positive sample set using only the requests historically successfully processed by each candidate expert, and maps the requests into dense semantic vectors through a pre-trained text feature extraction model; subsequently, a Bayesian Gaussian mixture model (BGMM) is independently trained for each expert to characterize the capability distribution of its successful requests in the semantic space. During online inference, the system encodes the target request into a feature vector, calculates the log-likelihood score combining the retained Gaussian component parameters and the smoothed covariance matrix of each expert, and uses Log-Sum-Exp to aggregate the probability contributions of different components. Finally, the request is routed to the expert model(s) with the highest score. Experiments were evaluated based on seven tasks: Math, GSM-Symbolic, HumanEval, Mbpp, MMLU, AIME1983–2025, and HellaSwag. The results show that this method achieves an average routing accuracy of 86.25%. The experimental results demonstrate that BGMM-LSE can provide stable request distribution capabilities for multi-expert large language model systems while maintaining interpretability and scalability.

IPC Classification

Keywords