(Im)possibility of Collective Intelligence
This work addresses the problem of enabling AI systems to learn across diverse environments for researchers and practitioners, but it is incremental as it builds on existing axiomatic frameworks to derive impossibility results.
The paper tackles the challenge of achieving collective intelligence across heterogeneous environments by proving that, under reasonable axioms, the only rational learning algorithm is empirical risk minimization (ERM) that learns from a single environment without sharing information. This result highlights a fundamental trade-off, making collective learning inherently hard in areas like out-of-distribution generalization and federated learning.
Modern applications of AI involve training and deploying machine learning models across heterogeneous and potentially massive environments. Emerging diversity of data not only brings about new possibilities to advance AI systems, but also restricts the extent to which information can be shared across environments due to pressing concerns such as privacy, security, and equity. Based on a novel characterization of learning algorithms as choice correspondences on a hypothesis space, this work provides a minimum requirement in terms of intuitive and reasonable axioms under which the only rational learning algorithm in heterogeneous environments is an empirical risk minimization (ERM) that unilaterally learns from a single environment without information sharing across environments. Our (im)possibility result underscores the fundamental trade-off that any algorithms will face in order to achieve Collective Intelligence (CI), i.e., the ability to learn across heterogeneous environments. Ultimately, collective learning in heterogeneous environments are inherently hard because, in critical areas of machine learning such as out-of-distribution generalization, federated/collaborative learning, algorithmic fairness, and multi-modal learning, it can be infeasible to make meaningful comparisons of model predictive performance across environments.