Bigger Buffer k-d Trees on Multi-Many-Core Systems
This work addresses scalability issues for nearest neighbor search in data-intensive fields like astronomy, but it is incremental as it builds on existing buffer k-d tree methods.
The paper tackled the limitation of buffer k-d trees in handling massive datasets on single devices by modifying the data structure and workflow to support multiple devices, demonstrating its applicability in astronomy with large data volumes.
A buffer k-d tree is a k-d tree variant for massively-parallel nearest neighbor search. While providing valuable speed-ups on modern many-core devices in case both a large number of reference and query points are given, buffer k-d trees are limited by the amount of points that can fit on a single device. In this work, we show how to modify the original data structure and the associated workflow to make the overall approach capable of dealing with massive data sets. We further provide a simple yet efficient way of using multiple devices given in a single workstation. The applicability of the modified framework is demonstrated in the context of astronomy, a field that is faced with huge amounts of data.