Time After Time: Deep-Q Effect Estimation for Interventions on When and What to do
This addresses a key limitation in fields like healthcare and robotics where existing methods fail with irregular time, offering a solution for decision-making under such conditions, though it appears incremental as it builds on deep-Q methods.
The paper tackles the problem of estimating causal effects for policies that involve both what action to take and when to take it, particularly under irregular time intervals, by introducing a new deep-Q algorithm called Earliest Disagreement Q-Evaluation (EDQ), which provides accurate estimates as validated on survival time and tumor growth tasks.
Problems in fields such as healthcare, robotics, and finance requires reasoning about the value both of what decision or action to take and when to take it. The prevailing hope is that artificial intelligence will support such decisions by estimating the causal effect of policies such as how to treat patients or how to allocate resources over time. However, existing methods for estimating the effect of a policy struggle with \emph{irregular time}. They either discretize time, or disregard the effect of timing policies. We present a new deep-Q algorithm that estimates the effect of both when and what to do called Earliest Disagreement Q-Evaluation (EDQ). EDQ makes use of recursion for the Q-function that is compatible with flexible sequence models, such as transformers. EDQ provides accurate estimates under standard assumptions. We validate the approach through experiments on survival time and tumor growth tasks.