High--Dimensional Brain in a High-Dimensional World: Blessing of Dimensionality
It addresses the fundamental understanding of high-dimensional data for machine learning and neuroscience researchers, but is incremental as it reviews existing ideas.
The paper reviews the 'blessing of dimensionality,' highlighting that high-dimensional data often has simple geometric properties, contrasting with the 'curse of dimensionality' to show a tradeoff between complexity and simplicity in such spaces.
High-dimensional data and high-dimensional representations of reality are inherent features of modern Artificial Intelligence systems and applications of machine learning. The well-known phenomenon of the "curse of dimensionality" states: many problems become exponentially difficult in high dimensions. Recently, the other side of the coin, the "blessing of dimensionality", has attracted much attention. It turns out that generic high-dimensional datasets exhibit fairly simple geometric properties. Thus, there is a fundamental tradeoff between complexity and simplicity in high dimensional spaces. Here we present a brief explanatory review of recent ideas, results and hypotheses about the blessing of dimensionality and related simplifying effects relevant to machine learning and neuroscience.