Haibo Shi

Wenjie Zhou, Zhenxin Ding, Xiaodong Zhang et al.

Pre-trained language models have become an integral component of question-answering systems, achieving remarkable performance. However, for practical deployment, it is crucial to perform knowledge distillation to maintain high performance while operating under computational constraints. In this paper, we address a key question: given the importance of unsupervised distillation for student model performance, how can knowledge from multiple teacher models be effectively ensemble during this stage without the guidance of labels? We propose a novel algorithm, GOVERN, to tackle this issue. GOVERN has demonstrated significant improvements in both offline and online experiments, enabling the student model to achieve results comparable to that of teacher ensembles. Our experiments show that GOVERN remarkably requires a mere 1\% of the ensemble method's inference budget to achieve 99.5\% of performance. The proposed algorithm has been successfully deployed in a real-world commercial question-answering system, demonstrating its real-world applicability.

Zhonghua Han, Quan Zhang, Haibo Shi et al.

In order to solve the limited buffer scheduling problems in flexible flow shops with setup times, this paper proposes an improved whale optimization algorithm (IWOA) as a global optimization algorithm. Firstly, this paper presents a mathematic programming model for limited buffer in flexible flow shops with setup times, and applies the IWOA algorithm as the global optimization algorithm. Based on the whale optimization algorithm (WOA), the improved algorithm uses Levy flight, opposition-based learning strategy and simulated annealing to expand the search range, enhance the ability for jumping out of local extremum, and improve the continuous evolution of the algorithm. To verify the improvement of the proposed algorithm on the optimization ability of the standard WOA algorithm, the IWOA algorithm is tested by verification examples of small-scale and large-scale flexible flow shop scheduling problems, and the imperialist competitive algorithm (ICA), bat algorithm (BA), and whale optimization algorithm (WOA) are used for comparision. Based on the instance data of bus manufacturer, simulation tests are made on the four algorithms under variouis of practical evalucation scenarios. The simulation results show that the IWOA algorithm can better solve this type of limited buffer scheduling problem in flexible flow shops with setup times compared with the state of the art algorithms.