Panagiotis Stinis

Dror Givon, Panagiotis Stinis, Jonathan Weare

We present a particle filter construction for a system that exhibits time-scale separation. The separation of time-scales allows two simplifications that we exploit: i) The use of the averaging principle for the dimensional reduction of the system needed to solve for each particle and ii) the factorization of the transition probability which allows the Rao-Blackwellization of the filtering step. Both simplifications can be implemented using the coarse projective integration framework. The resulting particle filter is faster and has smaller variance than the particle filter based on the original system. The method is tested on a multiscale stochastic differential equation and on a multiscale pure jump diffusion motivated by chemical reactions.

Panagiotis Stinis

We present numerical results for the solution of the 1D critical nonlinear Schrodinger with periodic boundary conditions and initial data that give rise to a finite time singularity. We construct, through the Mori-Zwanzig formalism, a reduced model which allows us to follow the solution after the formation of the singularity. The computed post-singularity solution exhibits the same characteristics as the post-singularity solutions constructed recently by Terence Tao.

Vasileios Maroulas, Panagiotis Stinis

We present a novel approach for improving particle filters for multi-target tracking. The suggested approach is based on drift homotopy for stochastic differential equations. Drift homotopy is used to design a Markov Chain Monte Carlo step which is appended to the particle filter and aims to bring the particle filter samples closer to the observations. Also, we present a simple Metropolis Monte Carlo algorithm for tackling the target-observation association problem. We have used the proposed approach on the problem of multi-target tracking for both linear and nonlinear observation models. The numerical results show that the suggested approach can improve significantly the performance of a particle filter.

Panagiotis Stinis

We present a collection of algorithms which utilize dimensional reduction to perform mesh refinement and study possibly singular solutions of time-dependent partial differential equations. The algorithms are inspired by constructions used in statistical mechanics to evaluate the properties of a system near a critical point. The first algorithm allows the accurate determination of the time of occurrence of a possible singularity. The second algorithm is an adaptive mesh refinement scheme which can be used to approach efficiently the possible singularity. Finally, the third algorithm uses the second algorithm until the available resolution is exhausted (as we approach the possible singularity) and then switches to a dimensionally reduced model which, when accurate, can follow faithfully the solution beyond the time of occurrence of the purported singularity. An accurate dimensionally reduced model should dissipate energy at the right rate. We construct two variants of each algorithm. The first variant assumes that we have actual knowledge of the reduced model. The second variant assumes that we know the form of the reduced model, i.e. the terms appearing in the reduced model, but not necessarily their coefficients. In this case, we also provide a way of determining the coefficients. We present numerical results for the Burgers equation with zero and nonzero viscosity to illustrate the use of the algorithms.

Panagiotis Stinis

We present a novel way of constructing reduced models for systems of ordinary differential equations. The reduced models we construct depend on coefficients which measure the importance of the different terms appearing in the model and need to be estimated. The proposed approach allows the estimation of these coefficients on the fly by enforcing the equality of integral quantities of the solution as computed from the original system and the reduced model. In particular, the approach combines the concepts of renormalization and effective field theory developed in the context of high energy physics and the Mori-Zwanzig formalism of irreversible statistical mechanics. It allows to construct stable reduced models of higher order than was previously possible. The method is applied to the problem of computing reduced models for ordinary differential equation systems resulting from Fourier expansions of singular (or near-singular) time-dependent partial differential equations. Results for the 1D Burgers and the 3D incompressible Euler equations are used to illustrate the construction. We also present, for the 1D Burgers and the 3D Euler equations, a simple and efficient recursive algorithm for calculating the higher order terms.

Panagiotis Stinis

In many time-dependent problems of practical interest the parameters entering the equations describing the evolution of the various quantities exhibit uncertainty. One way to address the problem of how this uncertainty impacts the solution is to expand the solution using polynomial chaos expansions and obtain a system of differential equations for the evolution of the expansion coefficients. We present an application of the Mori-Zwanzig formalism to the problem of constructing reduced models of such systems of differential equations. In particular, we construct reduced models for a subset of the polynomial chaos expansion coefficients that are needed for a full description of the uncertainty caused by the uncertain parameters. We also present a Markovian reformulation of the Mori-Zwanzig reduced equation which replaces the solution of the orthogonal dynamics equation with an infinite hierarchy of ordinary differential equations. The viscous Burgers equation with uncertain viscosity parameter is used to illustrate the construction. For this example we provide a way to estimate the necessary parameters that appear in the reduced model without having to solve the full system.

Vasileios Maroulas, Panagiotis Stinis

In recent work (arXiv:1006.3100v1), we have presented a novel approach for improving particle filters for multi-target tracking. The suggested approach was based on drift homotopy for stochastic differential equations. Drift homotopy was used to design a Markov Chain Monte Carlo step which is appended to the particle filter and aims to bring the particle filter samples closer to the observations. In the current work, we present an alternative way to append a Markov Chain Monte Carlo step to a particle filter to bring the particle filter samples closer to the observations. Both current and previous approaches stem from the general formulation of the filtering problem. We have used the currently proposed approach on the problem of multi-target tracking for both linear and nonlinear observation models. The numerical results show that the suggested approach can improve significantly the performance of a particle filter.

Panagiotis Stinis

In a recent preprint (arXiv:1211.4285v1) we addressed the problem of constructing reduced models for time-dependent systems described by differential equations which involve uncertain parameters. In the current work, we focus on the construction of reduced models for systems of differential equations when the initial condition is uncertain. While for both cases the reduced models are constructed through the Mori-Zwanzig formalism, the necessary estimation of the memory parameters is quite different. For the case of uncertain initial conditions we present an algorithm which allows to estimate on the fly the parameters appearing in the reduced model. The first part of the algorithm evolves the full system until the estimation of the parameters for the reduced model has converged. At the time instant that this happens, the algorithm switches to the evolution of only the reduced model with the estimated parameter values from the first part of the algorithm. The viscous Burgers equation with uncertain initial condition is used to illustrate the construction.

Panagiotis Stinis

We present an algorithm for the efficient sampling of conditional paths of stochastic differential equations (SDEs). While unconditional path sampling of SDEs is straightforward, albeit expensive for high dimensional systems of SDEs, conditional path sampling can be difficult even for low dimensional systems. This is because we need to produce sample paths of the SDE which respect both the dynamics of the SDE and the initial and endpoint conditions. The dynamics of a SDE are governed by the deterministic term (drift) and the stochastic term (noise). Instead of producing conditional paths directly from the original SDE, one can consider a sequence of SDEs with modified drifts. The modified drifts should be chosen so that it is easier to produce sample paths which satisfy the initial and endpoint conditions. Also, the sequence of modified drifts converges to the drift of the original SDE. We construct a simple Markov Chain Monte Carlo (MCMC) algorithm which samples, in sequence, conditional paths from the modified SDEs, by taking the last sampled path at each level of the sequence as an initial condition for the sampling at the next level in the sequence. The algorithm can be thought of as a stochastic analog of deterministic homotopy methods for solving nonlinear algebraic equations or as a SDE generalization of simulated annealing. The algorithm is particularly suited for filtering/smoothing applications. We show how it can be used to improve the performance of particle filters. Numerical results for filtering of a stochastic differential equation are included.

Panagiotis Stinis

We present a mesh refinement algorithm for detecting singularities of time-dependent partial differential equations. The main idea behind the algorithm is to treat the occurrence of singularities of time-dependent partial differential equations as phase transitions. We show how the mesh refinement algorithm can be used to calculate the blow-up rate as we approach the singularity. This calculation can be done in three different ways: i) the direct approach where one monitors the blowing-up quantity as it approaches the singularity and uses the data to calculate the blow-up rate ; ii) the "phase transition" approach (à la Wilson) where one treats the singularity as a fixed point of the renormalization flow equation and proceeds to compute the blow-up rate via an analysis in the vicinity of the fixed point and iii) the "scaling" approach (à la Widom-Kadanoff) where one postulates the existence of scaling laws for different quantities close to the singularity, computes the associated exponents and then uses them to estimate the blow-up rate. Our algorithm allows a unified presentation of these three approaches. The inviscid Burgers equation and the supercritical Schrodinger equation are used as instructive examples to illustrate the constructions.

Panagiotis Stinis

We present a reformulation of stochastic global optimization as a filtering problem. The motivation behind this reformulation comes from the fact that for many optimization problems we cannot evaluate exactly the objective function to be optimized. Similarly, we may not be able to evaluate exactly the functions involved in iterative optimization algorithms. For example, we may only have access to noisy measurements of the functions or statistical estimates provided through Monte Carlo sampling. This makes iterative optimization algorithms behave like stochastic maps. Naive global optimization amounts to evolving a collection of realizations of this stochastic map and picking the realization with the best properties. This motivates the use of filtering techniques to allow focusing on realizations that are more promising than others. In particular, we present a filtering reformulation of global optimization in terms of a special case of sequential importance sampling methods called particle filters. The increasing popularity of particle filters is based on the simplicity of their implementation and their flexibility. For parametric exponential density estimation problems, we utilize the flexibility of particle filters to construct a stochastic global optimization algorithm which converges to the optimal solution appreciably faster than naive global optimization. Several examples are provided to demonstrate the efficiency of the approach.

Ole H. Hald, Panagiotis Stinis

The "t-model" for dimensional reduction is applied to the estimation of the rate of decay of solutions of the Burgers equation and of the Euler equations in two and three space dimensions. The model was first derived in a statistical mechanics context, but here we analyze it purely as a numerical tool and prove its convergence. In the Burgers case the model captures the rate of decay exactly, as was already previously shown. For the Euler equations in two space dimensions, the model preserves energy as it should. In three dimensions, we find a power law decay in time and observe a temporal intermittency.

Panagiotis Stinis

In a recent paper \cite{CHSS06}, an infinitely long memory model (the t-model) for the Euler equations was presented and analyzed. The model can be derived by keeping the zeroth order term in a Taylor expansion of the memory integrand in the Mori-Zwanzig formalism. We present here a collection of models for the Euler equations which are based also on the Mori-Zwanzig formalism. The models arise from a Taylor expansion of a different operator, the orthogonal dynamics evolution operator, which appears in the memory integrand. The zero, first and second order models are constructed and simulated numerically. The form of the nonlinearity in the Euler equations, the special properties of the projection operator used and the general properties of any projection operator can be exploited to facilitate the recursive calculation of even higher order models. We use our models to compute the rate of energy decay for the Taylor-Green vortex problem. The results are in good agreement with the theoretical estimates. The energy decay appears to be organized in "waves" of activity, i.e. alternating periods of fast and slow decay. Our results corroborate the assumption in \cite{CHSS06}, that the modeling of the 3D Euler equations by a few low wavenumber modes should include a long memory.

Panagiotis Stinis, Alexandre J. Chorin

We show how to use numerical methods within the framework of successive scaling to analyse the microstructure of turbulence, in particular to find inertial range exponents and structure functions. The methods are first calibrated on the Burgers problem and are then applied to the 3D Euler equations. Known properties of low order structure functions appear with a relatively small computational outlay; however, more sensitive properties cannot yet be resolved with this approach well enough to settle ongoing controversies.

Panagiotis Stinis

We present a comparative study of two methods for the reduction of the dimensionality of a system of ordinary differential equations that exhibits time-scale separation. Both methods lead to a reduced system of stochastic differential equations. The novel feature of these methods is that they allow the use, in the reduced system, of higher order terms in the resolved variables. The first method, proposed by Majda, Timofeyev and Vanden-Eijnden, is based on an asymptotic strategy developed by Kurtz. The second method is a short-memory approximation of the Mori-Zwanzig projection formalism of irreversible statistical mechanics, as proposed by Chorin, Hald and Kupferman. We present conditions under which the reduced models arising from the two methods should have similar predictive ability. We apply the two methods to test cases that satisfy these conditions. The form of the reduced models and the numerical simulations show that the two methods have similar predictive ability as expected.

Alexandre J. Chorin, Panagiotis Stinis

Methods for the reduction of the complexity of computational problems are presented, as well as their connections to renormalization, scaling, and irreversible statistical mechanics. Several statistically stationary cases are analyzed; for time dependent problems averaging usually fails, and averaged equations must be augmented by appropriate memory and random forcing terms. Approximations are described and examples are given.

Panagiotis Stinis

We present an algorithm based on maximum likelihood for the estimation and renormalization (marginalization) of exponential densities. The moment-matching problem resulting from the maximization of the likelihood is solved as an optimization problem using the Levenberg-Marquardt algorithm. In the case of renormalization, the moments needed to set up the moment-matching problem are evaluated using Swendsen's renormalization method. We focus on the renormalization version of the algorithm, where we demonstrate its use by computing the critical temperature of the two-dimensional Ising model. Possible applications of the algorithm are discussed.