Using the Open Science Data Federation for data distribution: Big Bear Solar Observatory use case
For scientific communities requiring large-scale data distribution, this work provides an operational infrastructure that enhances data accessibility and supports NSF sharing requirements.
The paper presents the Open Science Data Federation (OSDF), a global data distribution network built on StashCache, and demonstrates its use for distributing data from the Big Bear Solar Observatory (BBSO), enabling worldwide access and processing.
The growing demand for extensive data processing is now a standard in many scientific fields. Efficiently distributing data to processing sites and enabling seamless sharing has become crucial. The Open Science Data Federation (OSDF) builds on the success of the StashCache project to establish a global data distribution network. By expanding StashCache, OSDF integrates additional data origins and caches, enhancing accessibility and performance (20 origins and 30 caches), new access methods, and monitoring and accounting mechanisms. Additionally, the OSDF has become essential to the US national cyber-infrastructure landscape due to the sharing requirements of recent NSF solicitations. One use case for the OSDF is the data access to the Big Bear Solar Observatory (BBSO). Integrating the BBSO data into the OSDF provided standard and reliable data access. Moreover, the OSDF caches provide local data worldwide. Using the OSDF and the BBSO data, creating a pipeline to apply image processing techniques to all images from BBSO anywhere on the planet was possible.