Generalizing Linear Autoencoder Recommenders with Decoupled Expected Quadratic Loss
This work provides an incremental improvement for researchers and practitioners using linear autoencoders in recommender systems by expanding the solvable hyperparameter space and improving performance.
This paper generalizes the objective function of Linear Autoencoder (LAE) recommenders, specifically the Emphasized Denoising Linear Autoencoder (EDLAE), to a Decoupled Expected Quadratic Loss (DEQL). This generalization allows for closed-form solutions in a broader hyperparameter range (b > 0), which were previously unavailable, leading to improved testing performance on benchmark datasets compared to the b=0 EDLAE baseline.
Linear autoencoders (LAEs) have gained increasing popularity in recommender systems due to their simplicity and strong empirical performance. Most LAE models, including the Emphasized Denoising Linear Autoencoder (EDLAE) introduced by (Steck, 2020), use quadratic loss during training. However, the original EDLAE only provides closed-form solutions for the hyperparameter choice $b = 0$, which limits its capacity. In this work, we generalize EDLAE objective into a Decoupled Expected Quadratic Loss (DEQL). We show that DEQL simplifies the process of deriving EDLAE solutions and reveals solutions in a broader hyperparameter range $b > 0$, which were not derived in Steck's original paper. Additionally, we propose an efficient algorithm based on Miller's matrix inverse theorem to ensure the computational tractability for the $b > 0$ case. Empirical results on benchmark datasets show that the $b > 0$ solutions provided by DEQL outperform the $b = 0$ EDLAE baseline, demonstrating that DEQL expands the solution space and enables the discovery of models with better testing performance.