Compositionality Through Language Transmission, using Artificial Neural Networks
This work addresses the challenge of compositionality in neural networks for language processing, but it is incremental as it builds on existing ILM methods with limited gains.
The authors tackled the problem of achieving compositionality in artificial neural networks through language transmission using the Iterated Learning Model (ILM), resulting in a modest improvement in compositionality as measured by holdout accuracy and topologic similarity, with anti-correlation observed between these metrics and increased compositionality when using non-symbolic high-dimensional images as input.
We propose an architecture and process for using the Iterated Learning Model ("ILM") for artificial neural networks. We show that ILM does not lead to the same clear compositionality as observed using DCGs, but does lead to a modest improvement in compositionality, as measured by holdout accuracy and topologic similarity. We show that ILM can lead to an anti-correlation between holdout accuracy and topologic rho. We demonstrate that ILM can increase compositionality when using non-symbolic high-dimensional images as input.