Optimization of Retrieval Algorithms on Large Scale Knowledge Graphs
This work addresses a critical bottleneck in efficiently using graph databases like Neo4j for domains such as life sciences, though it appears incremental by building on existing optimization research.
The paper tackled the challenge of optimizing retrieval algorithms on large-scale knowledge graphs, achieving speedups of 44 to 3839 times faster than naive querying on a biomedical graph with over 71 million nodes and 850 million relationships.
Knowledge graphs have been shown to play an important role in recent knowledge mining and discovery, for example in the field of life sciences or bioinformatics. Although a lot of research has been done on the field of query optimization, query transformation and of course in storing and retrieving large scale knowledge graphs the field of algorithmic optimization is still a major challenge and a vital factor in using graph databases. Few researchers have addressed the problem of optimizing algorithms on large scale labeled property graphs. Here, we present two optimization approaches and compare them with a naive approach of directly querying the graph database. The aim of our work is to determine limiting factors of graph databases like Neo4j and we describe a novel solution to tackle these challenges. For this, we suggest a classification schema to differ between the complexity of a problem on a graph database. We evaluate our optimization approaches on a test system containing a knowledge graph derived biomedical publication data enriched with text mining data. This dense graph has more than 71M nodes and 850M relationships. The results are very encouraging and - depending on the problem - we were able to show a speedup of a factor between 44 and 3839.