Compressed Computation under $L^4$ Loss is likely Computation in Superposition
This work provides a toy model and mechanistic understanding of computation in superposition, a phenomenon relevant to understanding how neural networks can compute more functions than their width suggests, but it is incremental as it only studies a small-scale synthetic setup.
The paper studies computation in superposition, where a neural network computes more functions than it has neurons. Using a toy model with 50 neurons computing ReLU of 100 sparse inputs, they show that training under L4 loss elicits a solution computing all features in superposition, which they reverse-engineer to find that the network uses sparse binary codewords and a pseudoinverse decoder.
Neural networks are thought to represent concepts as directions in their activation space, and superposition lets them encode more concepts than they have dimensions. It is natural to ask whether they can also compute more functions than they have neurons, i.e., perform computation in superposition. In this regime many functions of sparse inputs are evaluated by a layer with fewer neurons than there are functions to compute. Representation in superposition is by now fairly well understood, but computation in superposition is not, and there are few toy models of it arising through training rather than being hand designed. As a toy model of computation in superposition we study the compressed-computation setup: a single-hidden-layer ReLU network with 50 neurons that must compute the ReLU of each of 100 sparse input features. We show that training it under an $L^4$ loss (the mean fourth power of the error), rather than the usual $L^2$, elicits a solution that appears to compute all features in superposition. We then reverse-engineer this solution. We find that the network assigns each feature a sparse binary codeword over neurons and decodes it with a pseudoinverse of the encoder. Given these codewords, a description with only three scalars recovers most of the network's performance, and we validate it by building equivalent networks from hand-designed codes.