Universal Transforming Geometric Network
This work provides a more efficient and accurate model for protein structure prediction, which is crucial for computational biology and drug discovery, though it is incremental as it builds on existing transformer architectures.
The paper tackled the problem of protein structure prediction by addressing the limitations of the recurrent geometric network (RGN), such as long training times and unstable gradients, by proposing the Universal Transforming Geometric Network (UTGN). The result was a 1.7 Å improvement in free modeling and a 0.7 Å improvement in template-based modeling on the CASP12 competition compared to RGN.
The recurrent geometric network (RGN), the first end-to-end differentiable neural architecture for protein structure prediction, is a competitive alternative to existing models. However, the RGN's use of recurrent neural networks (RNNs) as internal representations results in long training time and unstable gradients. And because of its sequential nature, it is less effective at learning global dependencies among amino acids than existing transformer architectures. We propose the Universal Transforming Geometric Network (UTGN), an end-to-end differentiable model that uses the encoder portion of the Universal Transformer architecture as an alternative for internal representations. Our experiments show that compared to RGN, UTGN achieve a $1.7$ \si{\angstrom} improvement on the free modeling portion and a $0.7$ \si{\angstrom} improvement on the template based modeling of the CASP12 competition.