Shanay Mehta

Ritesh Goru, Shanay Mehta, Prateek Jain

Fine-tuning Large Language Models (LLMs) on multi-turn reasoning datasets requires N (number of turns) separate forward passes per conversation due to reasoning token visibility constraints, as reasoning tokens for a turn are discarded in subsequent turns. We propose duplicating response tokens along with a custom attention mask to enable single-pass processing of entire conversations. We prove our method produces identical losses to the N-pass approach while reducing time complexity from $O\bigl(N^{3}\bigl)$ to $O\bigl(N^{2}\bigl)$ and maintaining the same memory complexity for a transformer based model. Our approach achieves significant training speedup while preserving accuracy. Our implementation is available online (https://github.com/devrev/One-Pass-to-Reason).

Shanay Mehta, Shlok Mehendale, Nicole Fernandes et al.

Detection of anomalous situations for complex mission-critical systems hold paramount importance when their service continuity needs to be ensured. A major challenge in detecting anomalies from the operational data arises due to the imbalanced class distribution problem since the anomalies are supposed to be rare events. This paper evaluates a diverse array of Machine Learning (ML)-based anomaly detection algorithms through a comprehensive benchmark study. The paper contributes significantly by conducting an unbiased comparison of various anomaly detection algorithms, spanning classical ML, including various tree-based approaches to Deep Learning (DL) and outlier detection methods. The inclusion of 104 publicly available enhances the diversity of the study, allowing a more realistic evaluation of algorithm performance and emphasizing the importance of adaptability to real-world scenarios. The paper evaluates the general notion of DL as a universal solution, showing that, while powerful, it is not always the best fit for every scenario. The findings reveal that recently proposed tree-based evolutionary algorithms match DL methods and sometimes outperform them in many instances of univariate data where the size of the data is small and number of anomalies are less than 10%. Specifically, tree-based approaches successfully detect singleton anomalies in datasets where DL falls short. To the best of the authors' knowledge, such a study on a large number of state-of-the-art algorithms using diverse data sets, with the objective of guiding researchers and practitioners in making informed algorithmic choices, has not been attempted earlier.