algcomparison: Comparing the Performance of Graphical Structure Learning Algorithms with TETRAD
This tool addresses the need for standardized benchmarking of causal structure learning algorithms across platforms like TETRAD, MATLAB, Python, and R, but it is incremental as it builds on existing TETRAD capabilities.
The authors developed a tool for comparing the performance of graphical causal structure learning algorithms, implemented in the TETRAD suite, by simulating data with varying runs, sample sizes, and modalities to generate publishable reports.
In this report we describe a tool for comparing the performance of graphical causal structure learning algorithms implemented in the TETRAD freeware suite of causal analysis methods. Currently the tool is available as package in the TETRAD source code (written in Java). Simulations can be done varying the number of runs, sample sizes, and data modalities. Performance on this simulated data can then be compared for a number of algorithms, with parameters varied and with performance statistics as selected, producing a publishable report. The package presented here may also be used to compare structure learning methods across platforms and programming languages, i.e., to compare algorithms implemented in TETRAD with those implemented in MATLAB, Python, or R.