To Share or Not to Share: Investigating Weight Sharing in Variational Graph Autoencoders
This addresses a design problem for researchers and practitioners using VGAEs, but it is incremental as it focuses on an understudied practice rather than a new breakthrough.
The paper tackled the unclear relevance of weight sharing in variational graph autoencoders by analyzing its implications and showing through experiments that its benefits consistently outweigh drawbacks, recommending it as an effective approach for optimization, regularization, and simplification without significant performance loss.
This paper investigates the understudied practice of weight sharing (WS) in variational graph autoencoders (VGAE). WS presents both benefits and drawbacks for VGAE model design and node embedding learning, leaving its overall relevance unclear and the question of whether it should be adopted unresolved. We rigorously analyze its implications and, through extensive experiments on a wide range of graphs and VGAE variants, demonstrate that the benefits of WS consistently outweigh its drawbacks. Based on our findings, we recommend WS as an effective approach to optimize, regularize, and simplify VGAE models without significant performance loss.