An Efficient Finite Difference Method for Parameter Sensitivities of Continuous Time Markov Chains
For researchers using continuous time Markov chain models (e.g., in biosciences, population processes, queuing networks), this method provides a more efficient sensitivity analysis tool.
The paper introduces a finite difference method for computing parameter sensitivities in continuous time Markov chains, achieving an order of magnitude lower variance through coupling. The method is easy to implement and often substantially reduces variance compared to other approaches.
We present an efficient finite difference method for the computation of parameter sensitivities that is applicable to a wide class of continuous time Markov chain models. The estimator for the method is constructed by coupling the perturbed and nominal processes in a natural manner, and the analysis proceeds by utilizing a martingale representation for the coupled processes. The variance of the resulting estimator is shown to be an order of magnitude lower due to the coupling. We conclude that the proposed method produces an estimator with a lower variance than other methods, including the use of Common Random Numbers, in most situations. Often the variance reduction is substantial. The method is no harder to implement than any standard continuous time Markov chain algorithm, such as "Gillespie's algorithm." The motivating class of models, and the source of our examples, are the stochastic chemical kinetic models commonly used in the biosciences, though other natural application areas include population processes and queuing networks.