Lei M. Zhang

SIMA team, Adrian Bolton, Alexander Lerchner et al.

We introduce SIMA 2, a generalist embodied agent that understands and acts in a wide variety of 3D virtual worlds. Built upon a Gemini foundation model, SIMA 2 represents a significant step toward active, goal-directed interaction within an embodied environment. Unlike prior work (e.g., SIMA 1) limited to simple language commands, SIMA 2 acts as an interactive partner, capable of reasoning about high-level goals, conversing with the user, and handling complex instructions given through language and images. Across a diverse portfolio of games, SIMA 2 substantially closes the gap with human performance and demonstrates robust generalization to previously unseen environments, all while retaining the base model's core reasoning capabilities. Furthermore, we demonstrate a capacity for open-ended self-improvement: by leveraging Gemini to generate tasks and provide rewards, SIMA 2 can autonomously learn new skills from scratch in a new environment. This work validates a path toward creating versatile and continuously learning agents for both virtual and, eventually, physical worlds.

Andrew W. Senior, Thomas Edlich, Francisco J. H. Heras et al.

Fault-tolerant quantum computing will require error rates far below those achievable with physical qubits. Quantum error correction (QEC) bridges this gap, but depends on decoders being simultaneously fast, accurate, and scalable. This combination of requirements has not yet been met by a machine-learning decoder, nor by any decoder for promising resource-efficient codes such as the colour code. Here we introduce AlphaQubit 2, a neural-network decoder that achieves near-optimal logical error rates for both surface and colour codes at large scales under realistic noise. For the colour code, it is orders of magnitude faster than other high-accuracy decoders. For the surface code, we demonstrate real-time decoding faster than 1 microsecond per cycle up to distance 11 on current commercial accelerators with better accuracy than leading real-time decoders. These results support the practical application of a wider class of promising QEC codes, and establish a credible path towards high-accuracy, real-time neural decoding at the scales required for fault-tolerant quantum computation.

Cultural General Intelligence Team, Avishkar Bhoopchand, Bethanie Brownfield et al.

Cultural transmission is the domain-general social skill that allows agents to acquire and use information from each other in real-time with high fidelity and recall. In humans, it is the inheritance process that powers cumulative cultural evolution, expanding our skills, tools and knowledge across generations. We provide a method for generating zero-shot, high recall cultural transmission in artificially intelligent agents. Our agents succeed at real-time cultural transmission from humans in novel contexts without using any pre-collected human data. We identify a surprisingly simple set of ingredients sufficient for generating cultural transmission and develop an evaluation methodology for rigorously assessing it. This paves the way for cultural evolution as an algorithm for developing artificial general intelligence.

Rishabh Agarwal, Avi Singh, Lei M. Zhang et al. · mila

Large language models (LLMs) excel at few-shot in-context learning (ICL) -- learning from a few examples provided in context at inference, without any weight updates. Newly expanded context windows allow us to investigate ICL with hundreds or thousands of examples -- the many-shot regime. Going from few-shot to many-shot, we observe significant performance gains across a wide variety of generative and discriminative tasks. While promising, many-shot ICL can be bottlenecked by the available amount of human-generated examples. To mitigate this limitation, we explore two new settings: Reinforced and Unsupervised ICL. Reinforced ICL uses model-generated chain-of-thought rationales in place of human examples. Unsupervised ICL removes rationales from the prompt altogether, and prompts the model only with domain-specific questions. We find that both Reinforced and Unsupervised ICL can be quite effective in the many-shot regime, particularly on complex reasoning tasks. Finally, we demonstrate that, unlike few-shot learning, many-shot learning is effective at overriding pretraining biases, can learn high-dimensional functions with numerical inputs, and performs comparably to fine-tuning. We also find that inference cost increases linearly in the many-shot regime, and frontier LLMs benefit from many-shot ICL to varying degrees. Our analysis also reveals the limitations of next-token prediction loss as an indicator of downstream ICL performance.

Eser Aygün, Laurent Orseau, Ankit Anand et al.

Traditional automated theorem provers for first-order logic depend on speed-optimized search and many handcrafted heuristics that are designed to work best over a wide range of domains. Machine learning approaches in literature either depend on these traditional provers to bootstrap themselves or fall short on reaching comparable performance. In this paper, we propose a general incremental learning algorithm for training domain specific provers for first-order logic without equality, based only on a basic given-clause algorithm, but using a learned clause-scoring function. Clauses are represented as graphs and presented to transformer networks with spectral features. To address the sparsity and the initial lack of training data as well as the lack of a natural curriculum, we adapt hindsight experience replay to theorem proving, so as to be able to learn even when no proof can be found. We show that provers trained this way can match and sometimes surpass state-of-the-art traditional provers on the TPTP dataset in terms of both quantity and quality of the proofs.

Lei M. Zhang, Matthias Plappert, Wojciech Zaremba

We propose a method to predict the sim-to-real transfer performance of RL policies. Our transfer metric simplifies the selection of training setups (such as algorithm, hyperparameters, randomizations) and policies in simulation, without the need for extensive and time-consuming real-world rollouts. A probabilistic dynamics model is trained alongside the policy and evaluated on a fixed set of real-world trajectories to obtain the transfer metric. Experiments show that the transfer metric is highly correlated with policy performance in both simulated and real-world robotic environments for complex manipulation tasks. We further show that the transfer metric can predict the effect of training setups on policy transfer performance.

Mario Milicevic, Chen Feng, Lei M. Zhang et al.

The speed at which two remote parties can exchange secret keys over a fixed-length fiber-optic cable in continuous-variable quantum key distribution (CV-QKD) is currently limited by the computational complexity of post-processing algorithms for key reconciliation. Multi-edge low-density parity-check (LDPC) codes with low code rates and long block lengths were proposed for CV-QKD, in order to extend the maximum reconciliation distance between the two remote parties. Key reconciliation over multiple dimensions has been shown to further improve the error-correction performance of multi-edge LDPC codes in CV-QKD, thereby increasing both the secret key rate and distance. However, the computational complexity of LDPC decoding for long block lengths on the order of 10^6 bits remains a challenge. This work introduces a quasi-cyclic (QC) code construction for multi-edge LDPC codes that is highly suitable for hardware-accelerated decoding on a modern graphics processing unit (GPU). When combined with an 8-dimensional reconciliation scheme, the LDPC decoder achieves a raw decoding throughput of 1.72Mbit/s and an information throughput of 7.16Kbit/s using an NVIDIA GeForce GTX 1080 GPU at a maximum distance of 160km with a secret key rate of 4.10x10^{-7} bits/pulse for a rate 0.02 multi-edge code with block length of 10^6 bits when finite-size effects are considered. This work extends the previous maximum CV-QKD distance of 100km to 160km, while delivering between 1.07x and 8.03x higher decoded information throughput over the upper bound on the secret key rate for a lossy channel. The GPU-based QC-LDPC decoder achieves a 1.29x improvement in throughput over the best existing GPU decoder implementation for a rate 1/10 multi-edge LDPC code with block length of 2^{20} bits. These results show that LDPC decoding is no longer the computational bottleneck in long-distance CV-QKD.