Egon Peršak

Egon Peršak, Miguel F. Anjos, Sebastian Lautz et al.

We propose a transformer architecture for time series forecasting with a focus on time series tokenisation and apply it to a real-world prediction problem from the pricing domain. Our architecture aims to learn effective representations at many scales across all available data simultaneously. The model contains a number of novel modules: a differentiated form of time series patching which employs multiple resolutions, a multiple-resolution module for time-varying known variables, a mixer-based module for capturing cross-series information, and a novel output head with favourable scaling to account for the increased number of tokens. We present an application of this model to a real world prediction problem faced by the markdown team at a very large retailer. On the experiments conducted our model outperforms in-house models and the selected existing deep learning architectures.

Egon Peršak, Miguel F. Anjos

Decision-focused learning is a promising development for contextual optimisation. It enables us to train prediction models that reflect the contextual sensitivity structure of the problem. However, there have been limited attempts to extend this paradigm to robust optimisation. We propose a double implicit layer model for training prediction models with respect to robust decision loss in uncertain convex quadratically constrained quadratic programs (QCQP). The first layer solves a deterministic version of the problem, the second layer evaluates the worst case realisation for an uncertainty set centred on the observation given the decisions obtained from the first layer. This enables us to learn model parameterisations that lead to robust decisions while only solving a simpler deterministic problem at test time. Additionally, instead of having to solve a robust counterpart we solve two smaller and potentially easier problems in training. The second layer (worst case problem) can be seen as a regularisation approach for predict-and-optimise by fitting to a neighbourhood of problems instead of just a point observation. We motivate relaxations of the worst-case problem in cases of uncertainty sets that would otherwise lead to trust region problems, and leverage various relaxations to deal with uncertain constraints. Both layers are typically strictly convex in this problem setting and thus have meaningful gradients almost everywhere. We demonstrate an application of this model on simulated experiments. The method is an effective regularisation tool for decision-focused learning for uncertain convex QCQPs.

Egon Peršak, Miguel F. Anjos

Most decision-focused learning work has focused on single stage problems whereas many real-world decision problems are more appropriately modelled using multistage optimisation. In multistage problems contextual information is revealed over time, decisions have to be taken sequentially, and decisions now have an intertemporal effect on future decisions. Decision-focused forecasting is a recurrent differentiable optimisation architecture that expresses a fully differentiable multistage optimisation approach. This architecture enables us to account for the intertemporal decision effects of forecasts. We show what gradient adjustments are made to account for the state-path caused by forecasting. We apply the model to multistage problems in energy storage arbitrage and portfolio optimisation and report that our model outperforms existing approaches.