Tree Echo State Autoencoders with Grammars
This work addresses the problem of efficient autoencoding for tree data in domains like computer programs or natural language, offering a faster and more data-efficient alternative to existing methods, though it is incremental in improving speed and error reduction.
The paper tackled the challenge of constructing functions with tree-formed output for tasks like optimization or time series prediction by proposing tree echo state autoencoders (TES-AE), which use a tree grammar and reservoir computing to achieve faster training and lower autoencoding error compared to a state-of-the-art deep learning approach when data and time are limited.
Tree data occurs in many forms, such as computer programs, chemical molecules, or natural language. Unfortunately, the non-vectorial and discrete nature of trees makes it challenging to construct functions with tree-formed output, complicating tasks such as optimization or time series prediction. Autoencoders address this challenge by mapping trees to a vectorial latent space, where tasks are easier to solve, and then mapping the solution back to a tree structure. However, existing autoencoding approaches for tree data fail to take the specific grammatical structure of tree domains into account and rely on deep learning, thus requiring large training datasets and long training times. In this paper, we propose tree echo state autoencoders (TES-AE), which are guided by a tree grammar and can be trained within seconds by virtue of reservoir computing. In our evaluation on three datasets, we demonstrate that our proposed approach is not only much faster than a state-of-the-art deep learning autoencoding approach (D-VAE) but also has less autoencoding error if little data and time is given.