Learning to Exploit Invariances in Clinical Time-Series Data using Sequence Transformer Networks
This addresses the challenge of handling diverse invariances in clinical data for healthcare applications, representing an incremental improvement over existing methods.
The paper tackled the problem of exploiting various invariances in clinical time-series data by proposing Sequence Transformer Networks, an end-to-end trainable architecture that learns to identify and account for these invariances, achieving an AUROC of 0.851 compared to a baseline CNN's 0.838 for in-hospital mortality prediction.
Recently, researchers have started applying convolutional neural networks (CNNs) with one-dimensional convolutions to clinical tasks involving time-series data. This is due, in part, to their computational efficiency, relative to recurrent neural networks and their ability to efficiently exploit certain temporal invariances, (e.g., phase invariance). However, it is well-established that clinical data may exhibit many other types of invariances (e.g., scaling). While preprocessing techniques, (e.g., dynamic time warping) may successfully transform and align inputs, their use often requires one to identify the types of invariances in advance. In contrast, we propose the use of Sequence Transformer Networks, an end-to-end trainable architecture that learns to identify and account for invariances in clinical time-series data. Applied to the task of predicting in-hospital mortality, our proposed approach achieves an improvement in the area under the receiver operating characteristic curve (AUROC) relative to a baseline CNN (AUROC=0.851 vs. AUROC=0.838). Our results suggest that a variety of valuable invariances can be learned directly from the data.