Explicit and data-Efficient Encoding via Gradient Flow
This work addresses the need for precise and efficient encoding in data-scarce scenarios in the physical sciences, offering an incremental improvement over traditional autoencoders.
The paper tackles the problem of suboptimal representations in autoencoders by introducing a decoder-only method using gradient flow to directly encode data into latent space via ODEs, eliminating approximate encoder inversion and demonstrating explicit encoding and superior data efficiency, with a 2nd-order ODE variant for faster convergence and adaptive solver for robustness.
The autoencoder model typically uses an encoder to map data to a lower dimensional latent space and a decoder to reconstruct it. However, relying on an encoder for inversion can lead to suboptimal representations, particularly limiting in physical sciences where precision is key. We introduce a decoder-only method using gradient flow to directly encode data into the latent space, defined by ordinary differential equations (ODEs). This approach eliminates the need for approximate encoder inversion. We train the decoder via the adjoint method and show that costly integrals can be avoided with minimal accuracy loss. Additionally, we propose a $2^{nd}$ order ODE variant, approximating Nesterov's accelerated gradient descent for faster convergence. To handle stiff ODEs, we use an adaptive solver that prioritizes loss minimization, improving robustness. Compared to traditional autoencoders, our method demonstrates explicit encoding and superior data efficiency, which is crucial for data-scarce scenarios in the physical sciences. Furthermore, this work paves the way for integrating machine learning into scientific workflows, where precise and efficient encoding is critical. \footnote{The code for this work is available at \url{https://github.com/k-flouris/gfe}.}