Language models can generate molecules, materials, and protein binding sites directly in three dimensions as XYZ, CIF, and PDB files
This enables direct 3D exploration of chemical space for diverse structures, benefiting researchers in chemistry and materials science, though it is incremental as it applies an existing method to new data types.
The authors tackled the problem of generating 3D molecular structures like materials and protein binding sites, which require spatial atom positioning beyond graph-based representations, by training language models directly on chemical file formats (XYZ, CIF, PDB) without architecture changes, achieving performance comparable to state-of-the-art models.
Language models are powerful tools for molecular design. Currently, the dominant paradigm is to parse molecular graphs into linear string representations that can easily be trained on. This approach has been very successful, however, it is limited to chemical structures that can be completely represented by a graph -- like organic molecules -- while materials and biomolecular structures like protein binding sites require a more complete representation that includes the relative positioning of their atoms in space. In this work, we show how language models, without any architecture modifications, trained using next-token prediction -- can generate novel and valid structures in three dimensions from various substantially different distributions of chemical structures. In particular, we demonstrate that language models trained directly on sequences derived directly from chemical file formats like XYZ files, Crystallographic Information files (CIFs), or Protein Data Bank files (PDBs) can directly generate molecules, crystals, and protein binding sites in three dimensions. Furthermore, despite being trained on chemical file sequences -- language models still achieve performance comparable to state-of-the-art models that use graph and graph-derived string representations, as well as other domain-specific 3D generative models. In doing so, we demonstrate that it is not necessary to use simplified molecular representations to train chemical language models -- that they are powerful generative models capable of directly exploring chemical space in three dimensions for very different structures.