39.4LGApr 4
Understanding and inverse design of implicit bias in stochastic learning: a geometric perspectiveNicola Aladrah, Emanuele Ballarin, Matteo Biagetti et al.
A key challenge in machine learning is to explain how learning dynamics select among the many solutions that achieve identical loss values in overparameterized models - a phenomenon known as implicit bias. Controlling this bias provides a direct mechanism on learned representations, which are central to interpretability, robustness, and reasoning in modern AI systems. Yet, despite its importance, existing explanations remain largely ad hoc and lack a unifying mechanism. We develop a theoretical and constructive framework in which implicit bias emerges as a geometric correction induced by the interplay between gradient noise and continuous symmetries of the loss. We compute the induced bias across a range of architectures, predicting new behaviors and explaining known ones. The approach also enables inverse design: by engineering predictor - preserving parameterizations, it is possible to shape the bias, with sparsity and spectral sparsity emerging as canonical instances. Numerical experiments support the theory and validate the inverse - design framework in controlled settings.
LGMay 24, 2023
Frequency maps reveal the correlation between Adversarial Attacks and Implicit BiasLorenzo Basile, Nikos Karantzas, Alberto d'Onofrio et al.
Despite their impressive performance in classification tasks, neural networks are known to be vulnerable to adversarial attacks, subtle perturbations of the input data designed to deceive the model. In this work, we investigate the correlation between these perturbations and the implicit bias of neural networks trained with gradient-based algorithms. To this end, we analyse a representation of the network's implicit bias through the lens of the Fourier transform. Specifically, we identify unique fingerprints of implicit bias and adversarial attacks by calculating the minimal, essential frequencies needed for accurate classification of each image, as well as the frequencies that drive misclassification in its adversarially perturbed counterpart. This approach enables us to uncover and analyse the correlation between these essential frequencies, providing a precise map of how the network's biases align or contrast with the frequency components exploited by adversarial attacks. To this end, among other methods, we use a newly introduced technique capable of detecting nonlinear correlations between high-dimensional datasets. Our results provide empirical evidence that the network bias in Fourier space and the target frequencies of adversarial attacks are highly correlated and suggest new potential strategies for adversarial defence.
SIDec 31, 2019
Evidence of disorientation towards immunization on online social media after contrasting political communication on vaccines. Results from an analysis of Twitter data in ItalySamantha Ajovalasit, Veronica Dorgali, Angelo Mazza et al.
Background. In Italy, in recent years, vaccination coverage for key immunizations as MMR has been declining to worryingly low levels. In 2017, the Italian Gov't expanded the number of mandatory immunizations introducing penalties to unvaccinated children's families. During the 2018 general elections campaign, immunization policy entered the political debate with the Gov't in charge blaming oppositions for fuelling vaccine scepticism. A new Gov't established in 2018 temporarily relaxed penalties. Objectives and Methods. Using a sentiment analysis on tweets posted in Italian during 2018, we aimed to: (i) characterize the temporal flow of vaccines communication on Twitter (ii) evaluate the polarity of vaccination opinions and usefulness of Twitter data to estimate vaccination parameters, and (iii) investigate whether the contrasting announcements at the highest political level might have originated disorientation amongst the Italian public. Results. Vaccine-relevant tweeters interactions peaked in response to main political events. Out of retained tweets, 70.0% resulted favourable to vaccination, 16.5% unfavourable, and 13.6% undecided, respectively. The smoothed time series of polarity proportions exhibit frequent large changes in the favourable proportion, enhanced by an up and down trend synchronized with the switch between gov't suggesting evidence of disorientation among the public. Conclusion. The reported evidence of disorientation documents that critical immunization topics, should never be used for political consensus. This is especially true given the increasing role of online social media as information source, which might yield to social pressures eventually harmful for vaccine uptake, and is worsened by the lack of institutional presence on Twitter, calling for efforts to contrast misinformation and the ensuing spread of hesitancy.