Theoretically informed selection of latent activation in autoencoder based recommender systems
This provides a systematic method for hyperparameter selection in recommender systems, though it is incremental as it builds on existing autoencoder frameworks.
The paper tackled the challenge of designing autoencoder-based recommender systems by identifying three key mathematical properties for encoders to improve accuracy, and found that sigmoid-like activations are suitable while common ones like ReLU and tanh are not.
Autoencoders may lend themselves to the design of more accurate and computationally efficient recommender systems by distilling sparse high-dimensional data into dense lower-dimensional latent representations. However, designing these systems remains challenging due to the lack of theoretical guidance. This work addresses this by identifying three key mathematical properties that the encoder in an autoencoder should exhibit to improve recommendation accuracy: (1) dimensionality reduction, (2) preservation of similarity ordering in dot product comparisons, and (3) preservation of non-zero vectors. Through theoretical analysis, we demonstrate that common activation functions, such as ReLU and tanh, cannot fulfill these properties jointly within a generalizable framework. In contrast, sigmoid-like activations emerge as suitable choices for latent activations. This theoretically informed approach offers a more systematic method for hyperparameter selection, enhancing the efficiency of model design.