Neural Fields as World Models
This work addresses the challenge of intuitive physics prediction in AI and neuroscience, offering a novel approach that could enhance robot learning and cognitive modeling, though it appears incremental by building on neural fields and world models.
The paper tackled the problem of predicting physical outcomes in world models by proposing isomorphic architectures that preserve sensory topology, using neural fields with motor-gated channels. The results showed that policies trained in imagination transferred to real physics at nearly twice the rate of latent-space alternatives, and local connectivity learned ballistic physics without teleporting.
How does the brain predict physical outcomes while acting in the world? Machine learning world models compress visual input into latent spaces, discarding the spatial structure that characterizes sensory cortex. We propose isomorphic world models: architectures preserving sensory topology so that physics prediction becomes geometric propagation rather than abstract state transition. We implement this using neural fields with motor-gated channels, where activity evolves through local lateral connectivity and motor commands multiplicatively modulate specific populations. Three experiments support this approach: (1) local connectivity is sufficient to learn ballistic physics, with predictions traversing intermediate locations rather than "teleporting"; (2) policies trained entirely in imagination transfer to real physics at nearly twice the rate of latent-space alternatives; and (3) motor-gated channels spontaneously develop body-selective encoding through visuomotor prediction alone. These findings suggest intuitive physics and body schema may share a common origin in spatially structured neural dynamics.