Olga Vitek

Charles Tapley Hoyt, Craig Bakker, Richard J. Callahan et al.

We present the $Y_0$ Python package, which implements causal identification algorithms that apply interventional, counterfactual, and transportability queries to data from (randomized) controlled trials, observational studies, or mixtures thereof. $Y_0$ focuses on the qualitative investigation of causation, helping researchers determine whether a causal relationship can be estimated from available data before attempting to estimate how strong that relationship is. Furthermore, $Y_0$ provides guidance on how to transform the causal query into a symbolic estimand that can be non-parametrically estimated from the available data. $Y_0$ provides a domain-specific language for representing causal queries and estimands as symbolic probabilistic expressions, tools for representing causal graphical models with unobserved confounders, such as acyclic directed mixed graphs (ADMGs), and implementations of numerous identification algorithms from the recent causal inference literature. The $Y_0$ source code can be found under the MIT License at https://github.com/y0-causal-inference/y0 and it can be installed with pip install y0.

Sara Mohammad Taheri, Omkar Terse, Eralp Dogu et al.

MSstatsQC [3] is an open-source software that provides longitudinal system suitability monitoring tools in the form of control charts for proteomic experiments. It includes simultaneous tools for the mean and dispersion of suitability metrics and presents alternative methods of monitoring through different tabs that are designed in the interface. This research focuses on investigating the usability of MSstatsQC software and the interpretability of the designed plots. In this study, we ask 4 test users, from the proteomics field, to complete a series of tasks and questionnaires. The tasks are designed to test the usability of the software in terms of importing data files, selecting appropriate metrics, guide set, and peptides, and finally creating decision rules (tasks 1 and 3 in appendix). The questionnaires ask about interpretability of the plots including control charts, box plots, heat maps, river plots, and radar plots (tasks 1 and 4 in appendix). The goal of the questions is to determine if the test users understand the plots and can interpret them. Results show limitations in usability and plot interpretability, especially in the data import section. We suggest the following modifications. I) providing conspicuous guides close to the window related to up-loading a datafile as well as providing error messages that pop-up when the data set has a wrong format II) providing plot descriptions, hints to interpret plots, plot titles and appropriate axis labels, and, III) Numbering tabs to show the flow of procedures in the software.

Jeremy Zucker, Kaushal Paneri, Sara Mohammad-Taheri et al. · oxford

Counterfactual inference is a useful tool for comparing outcomes of interventions on complex systems. It requires us to represent the system in form of a structural causal model, complete with a causal diagram, probabilistic assumptions on exogenous variables, and functional assignments. Specifying such models can be extremely difficult in practice. The process requires substantial domain expertise, and does not scale easily to large systems, multiple systems, or novel system modifications. At the same time, many application domains, such as molecular biology, are rich in structured causal knowledge that is qualitative in nature. This manuscript proposes a general approach for querying a causal biological knowledge graph, and converting the qualitative result into a quantitative structural causal model that can learn from data to answer the question. We demonstrate the feasibility, accuracy and versatility of this approach using two case studies in systems biology. The first demonstrates the appropriateness of the underlying assumptions and the accuracy of the results. The second demonstrates the versatility of the approach by querying a knowledge base for the molecular determinants of a severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)-induced cytokine storm, and performing counterfactual inference to estimate the causal effect of medical countermeasures for severely ill patients.

Emery D. Berger, Petr Maj, Olga Vitek et al.

Ray, Devanbu and Filkov issued a rebuttal of our TOPLAS paper "On the Impact of Programming Languages on Code Quality: A Reproduction Study". Our paper reproduced "A Large-Scale Study of Programming Languages and Code Quality in GitHub", which appeared at FSE 2014 and was subsequently republished as a CACM research highlight in 2017. This article is a rebuttal to that rebuttal.

Robert Osazuwa Ness, Kaushal Paneri, Olga Vitek

This manuscript contributes a general and practical framework for casting a Markov process model of a system at equilibrium as a structural causal model, and carrying out counterfactual inference. Markov processes mathematically describe the mechanisms in the system, and predict the system's equilibrium behavior upon intervention, but do not support counterfactual inference. In contrast, structural causal models support counterfactual inference, but do not identify the mechanisms. This manuscript leverages the benefits of both approaches. We define the structural causal models in terms of the parameters and the equilibrium dynamics of the Markov process models, and counterfactual inference flows from these settings. The proposed approach alleviates the identifiability drawback of the structural causal models, in that the counterfactual inference is consistent with the counterfactual trajectories simulated from the Markov process model. We showcase the benefits of this framework in case studies of complex biomolecular systems with nonlinear dynamics. We illustrate that, in presence of Markov process model misspecification, counterfactual inference leverages prior data, and therefore estimates the outcome of an intervention more accurately than a direct simulation.

Emery D. Berger, Celeste Hollenbeck, Petr Maj et al.

This paper is a reproduction of work by Ray et al. which claimed to have uncovered a statistically significant association between eleven programming languages and software defects in projects hosted on GitHub. First we conduct an experimental repetition, repetition is only partially successful, but it does validate one of the key claims of the original work about the association of ten programming languages with defects. Next, we conduct a complete, independent reanalysis of the data and statistical modeling steps of the original study. We uncover a number of flaws that undermine the conclusions of the original study as only four languages are found to have a statistically significant association with defects, and even for those the effect size is exceedingly small. We conclude with some additional sources of bias that should be investigated in follow up work and a few best practice recommendations for similar efforts.