Shaping representations through communication: community size effect in artificial learning systems
This addresses representation learning in AI by showing how communication dynamics can shape more generalizable representations, though it is incremental as it builds on autoencoder frameworks.
The paper tackled the problem of how community size affects representation learning by framing it as a communication task, finding that larger communities reduce idiosyncrasies in learned codes and improve encoding of concept categories and correlation with human feature norms.
Motivated by theories of language and communication that explain why communities with large numbers of speakers have, on average, simpler languages with more regularity, we cast the representation learning problem in terms of learning to communicate. Our starting point sees the traditional autoencoder setup as a single encoder with a fixed decoder partner that must learn to communicate. Generalizing from there, we introduce community-based autoencoders in which multiple encoders and decoders collectively learn representations by being randomly paired up on successive training iterations. We find that increasing community sizes reduce idiosyncrasies in the learned codes, resulting in representations that better encode concept categories and correlate with human feature norms.