Low-rank approximations of hyperbolic embeddings
This work addresses the need for efficient hyperbolic embeddings in machine learning and NLP for modeling hierarchies, but it appears incremental as it builds on existing hyperbolic embedding methods by adding low-rank constraints.
The paper tackles the problem of learning hyperbolic embeddings for hierarchical tasks by proposing a low-rank factorization approach, resulting in computationally efficient algorithms with empirical efficacy.
The hyperbolic manifold is a smooth manifold of negative constant curvature. While the hyperbolic manifold is well-studied in the literature, it has gained interest in the machine learning and natural language processing communities lately due to its usefulness in modeling continuous hierarchies. Tasks with hierarchical structures are ubiquitous in those fields and there is a general interest to learning hyperbolic representations or embeddings of such tasks. Additionally, these embeddings of related tasks may also share a low-rank subspace. In this work, we propose to learn hyperbolic embeddings such that they also lie in a low-dimensional subspace. In particular, we consider the problem of learning a low-rank factorization of hyperbolic embeddings. We cast these problems as manifold optimization problems and propose computationally efficient algorithms. Empirical results illustrate the efficacy of the proposed approach.