Final Report for CHESS: Cloud, High-Performance Computing, and Edge for Science and Security
This work addresses the problem of inefficient distributed workflows for scientists and security applications, but it appears incremental as it builds on existing continuum platform methods.
The CHESS project tackled the challenge of automating the theory-experiment cycle by developing capabilities for distributed scientific workflows across cloud, HPC, and edge resources, achieving improvements in performance, energy, security, and reliability for scientific requirements.
Automating the theory-experiment cycle requires effective distributed workflows that utilize a computing continuum spanning lab instruments, edge sensors, computing resources at multiple facilities, data sets distributed across multiple information sources, and potentially cloud. Unfortunately, the obvious methods for constructing continuum platforms, orchestrating workflow tasks, and curating datasets over time fail to achieve scientific requirements for performance, energy, security, and reliability. Furthermore, achieving the best use of continuum resources depends upon the efficient composition and execution of workflow tasks, i.e., combinations of numerical solvers, data analytics, and machine learning. Pacific Northwest National Laboratory's LDRD "Cloud, High-Performance Computing (HPC), and Edge for Science and Security" (CHESS) has developed a set of interrelated capabilities for enabling distributed scientific workflows and curating datasets. This report describes the results and successes of CHESS from the perspective of open science.