АДАПТИВНЫЙ МАРШРУТИЗАТОР ВЫВОДА С УЧЕТОМ КЛЮЧЕВЫХ ПОКАЗАТЕЛЕЙ ЭФФЕКТИВНОСТИ ДЛЯ МОДЕЛЕЙ ЕСТЕСТВЕННОГО ЯЗЫКА

Abstract

This paper proposes a KPI-oriented cascade router for multi-class text classification that sequentially employs lightweight, medium-sized, and heavyweight models depending on the confidence level of the prediction. The relevance of this study stems from the need to jointly manage classification accuracy and computational costs in applied natural language processing (NLP) systems, where relying exclusively on the most accurate models is often inefficient in terms of response time and inference cost. The experiments were conducted on the WOS-11967 corpus, which contains 11,967 scientific texts. To ensure proper parameter tuning and independent evaluation, a two-stage protocol with separate VAL and FINAL subsets was employed. The lightweight, medium-sized, and heavyweight models differ in their text representation methods and computational complexity, while routing between them is based on confidence thresholds. For the lightweight and medium-sized models, confidence was defined as the difference between the two highest decision function values. The thresholds were selected on the VAL subset while accounting for the computational budget constraint governed by parameter α. The results demonstrate that the proposed cascade can accelerate inference compared with the heavyweight model, with a moderate reduction in accuracy, while maintaining a controlled trade-off between performance and computational cost. The practical significance of this work lies in the potential application of the proposed approach in systems where balancing accuracy, speed, and resource constraints is essential.