Deep learning languages: a key fundamental shift from probabilities to weights?
This addresses a foundational issue in machine learning for researchers, but it is incremental as it questions an existing trend without presenting new experimental results.
The paper examines the shift from probabilistic to weighted representations in deep learning language models, highlighting the limitations of classical probabilistic approaches for protein sequence classification and the need for principled methods to learn non-probabilistic models.
Recent successes in language modeling, notably with deep learning methods, coincide with a shift from probabilistic to weighted representations. We raise here the question of the importance of this evolution, in the light of the practical limitations of a classical and simple probabilistic modeling approach for the classification of protein sequences and in relation to the need for principled methods to learn non-probabilistic models.