Generalist Equivariant Transformer Towards 3D Molecular Interaction Learning
This addresses the challenge of modeling diverse 3D molecular interactions in biology and drug discovery, which is incremental as it builds on existing equivariant and hierarchical methods.
The paper tackles the problem of learning 3D molecular interactions across different molecule types by proposing a universal representation as a geometric graph of sets and a Generalist Equivariant Transformer (GET) to capture domain-specific hierarchies and domain-agnostic physics, achieving effectiveness and generalization in experiments on proteins, small molecules, and RNA/DNAs.
Many processes in biology and drug discovery involve various 3D interactions between molecules, such as protein and protein, protein and small molecule, etc. Given that different molecules are usually represented in different granularity, existing methods usually encode each type of molecules independently with different models, leaving it defective to learn the various underlying interaction physics. In this paper, we first propose to universally represent an arbitrary 3D complex as a geometric graph of sets, shedding light on encoding all types of molecules with one model. We then propose a Generalist Equivariant Transformer (GET) to effectively capture both domain-specific hierarchies and domain-agnostic interaction physics. To be specific, GET consists of a bilevel attention module, a feed-forward module and a layer normalization module, where each module is E(3) equivariant and specialized for handling sets of variable sizes. Notably, in contrast to conventional pooling-based hierarchical models, our GET is able to retain fine-grained information of all levels. Extensive experiments on the interactions between proteins, small molecules and RNA/DNAs verify the effectiveness and generalization capability of our proposed method across different domains.