Iterative SE(3)-Transformers
This work addresses a specific technical issue in SE(3)-equivariant models for researchers in computational biology or 3D data processing, but it appears incremental as it builds on existing SE(3)-Transformer methods.
The paper tackles the challenge of applying SE(3)-equivariant attention models iteratively for 3D data manipulation, such as in protein structure prediction, by implementing an iterative SE(3)-Transformer and comparing it to a single-pass version on a toy problem.
When manipulating three-dimensional data, it is possible to ensure that rotational and translational symmetries are respected by applying so-called SE(3)-equivariant models. Protein structure prediction is a prominent example of a task which displays these symmetries. Recent work in this area has successfully made use of an SE(3)-equivariant model, applying an iterative SE(3)-equivariant attention mechanism. Motivated by this application, we implement an iterative version of the SE(3)-Transformer, an SE(3)-equivariant attention-based model for graph data. We address the additional complications which arise when applying the SE(3)-Transformer in an iterative fashion, compare the iterative and single-pass versions on a toy problem, and consider why an iterative model may be beneficial in some problem settings. We make the code for our implementation available to the community.