Alejandro Maté

Jorge García-Carrasco, Alejandro Maté, Juan Trujillo

Large Language Models (LLMs), characterized by being trained on broad amounts of data in a self-supervised manner, have shown impressive performance across a wide range of tasks. Indeed, their generative abilities have aroused interest on the application of LLMs across a wide range of contexts. However, neural networks in general, and LLMs in particular, are known to be vulnerable to adversarial attacks, where an imperceptible change to the input can mislead the output of the model. This is a serious concern that impedes the use of LLMs on high-stakes applications, such as healthcare, where a wrong prediction can imply serious consequences. Even though there are many efforts on making LLMs more robust to adversarial attacks, there are almost no works that study \emph{how} and \emph{where} these vulnerabilities that make LLMs prone to adversarial attacks happen. Motivated by these facts, we explore how to localize and understand vulnerabilities, and propose a method, based on Mechanistic Interpretability (MI) techniques, to guide this process. Specifically, this method enables us to detect vulnerabilities related to a concrete task by (i) obtaining the subset of the model that is responsible for that task, (ii) generating adversarial samples for that task, and (iii) using MI techniques together with the previous samples to discover and understand the possible vulnerabilities. We showcase our method on a pretrained GPT-2 Small model carrying out the task of predicting 3-letter acronyms to demonstrate its effectiveness on locating and understanding concrete vulnerabilities of the model.

Sandra García-Ponsoda, Alejandro Maté, Juan Trujillo

Background: EEG signals are commonly used in ADHD diagnosis, but they are often affected by noise and artifacts. Effective preprocessing and segmentation methods can significantly enhance the accuracy and reliability of ADHD classification. Methods: We applied filtering, ASR, and ICA preprocessing techniques to EEG data from children with ADHD and neurotypical controls. The EEG recordings were segmented, and features were extracted and selected based on statistical significance. Classification was performed using various EEG segments and channels with Machine Learning models (SVM, KNN, and XGBoost) to identify the most effective combinations for accurate ADHD diagnosis. Results: Our findings show that models trained on later EEG segments achieved significantly higher accuracy, indicating the potential role of cognitive fatigue in distinguishing ADHD. The highest classification accuracy (86.1%) was achieved using data from the P3, P4, and C3 channels, with key features such as Kurtosis, Katz fractal dimension, and power spectrums in the Delta, Theta, and Alpha bands contributing to the results. Conclusion: This study highlights the importance of preprocessing and segmentation in improving the reliability of ADHD diagnosis through EEG. The results suggest that further research on cognitive fatigue and segmentation could enhance diagnostic accuracy in ADHD patients.

Jorge García-Carrasco, Alejandro Maté, Juan Trujillo

Transformer-based language models are treated as black-boxes because of their large number of parameters and complex internal interactions, which is a serious safety concern. Mechanistic Interpretability (MI) intends to reverse-engineer neural network behaviors in terms of human-understandable components. In this work, we focus on understanding how GPT-2 Small performs the task of predicting three-letter acronyms. Previous works in the MI field have focused so far on tasks that predict a single token. To the best of our knowledge, this is the first work that tries to mechanistically understand a behavior involving the prediction of multiple consecutive tokens. We discover that the prediction is performed by a circuit composed of 8 attention heads (~5% of the total heads) which we classified in three groups according to their role. We also demonstrate that these heads concentrate the acronym prediction functionality. In addition, we mechanistically interpret the most relevant heads of the circuit and find out that they use positional information which is propagated via the causal mask mechanism. We expect this work to lay the foundation for understanding more complex behaviors involving multiple-token predictions.

Jorge García-Carrasco, Alejandro Maté, Juan Trujillo

Large Language Models (LLMs) have shown impressive performance across a wide range of tasks. However, the size of LLMs is steadily increasing, hindering their application on computationally constrained environments. On the other hand, despite their general capabilities, there are many situations where only one specific task is performed, rendering all other capabilities unnecessary and wasteful. This leads us to the following question: Is it possible to extract the minimal subset from an LLM that is able to perform a specific task in a faster, standalone manner? Recent works on Mechanistic Interpretability (MI) have shown that specific tasks are performed by a localized subset of components, or circuit. However, current techniques used to identify the circuit cannot be used to extract it for its standalone usage. In this work, we propose a novel approach to automatically extract the subset of the LLM that properly performs a targeted task requiring no additional training and a small amount of data samples. We evaluate our approach on different tasks and show that the resulting models are (i) considerably smaller, reducing the number of parameters up to 82.77% and (ii) more interpretable, as they focus on the circuit that is used to carry out the specific task, and can therefore be understood using MI techniques.