LGNov 30, 2023
Tree-based Forecasting of Day-ahead Solar Power Generation from Granular Meteorological FeaturesNick Berlanger, Noah van Ophoven, Tim Verdonck et al.
Accurate forecasts for day-ahead photovoltaic (PV) power generation are crucial to support a high PV penetration rate in the local electricity grid and to assure stability in the grid. We use state-of-the-art tree-based machine learning methods to produce such forecasts and, unlike previous studies, we hereby account for (i) the effects various meteorological as well as astronomical features have on PV power production, and this (ii) at coarse as well as granular spatial locations. To this end, we use data from Belgium and forecast day-ahead PV power production at an hourly resolution. The insights from our study can assist utilities, decision-makers, and other stakeholders in optimizing grid operations, economic dispatch, and in facilitating the integration of distributed PV power into the electricity grid.
MLDec 30, 2024
Robust Matrix Completion for Discrete Rating-Scale Data: Coping with Fake Profiles in Recommender SystemsAurore Archimbaud, Andreas Alfons, Ines Wilms
Recommender systems are essential tools in the digital landscape for connecting users with content that more closely aligns with their preferences. Matrix completion is a widely used statistical framework for such systems, aiming to predict a user's preferences for items they have not yet rated by leveraging the observed ratings in a partially filled user-item rating matrix. Realistic applications of matrix completion in recommender systems must address several challenges that are too often neglected: (i) the discrete nature of rating-scale data, (ii) the presence of malicious users who manipulate the system to their advantage through the creation of fake profiles, and (iii) missing-not-at-random patterns, where users are more likely to rate items they expect to enjoy. Our goal in this paper is twofold. First, we propose a novel matrix completion method, robust discrete matrix completion (RDMC), designed specifically to handle the discrete nature of sparse rating-scale data and to remain reliable in the presence of adversarial manipulation. We evaluate RDMC through carefully designed experiments and realistic case studies. Our work therefore, secondly, offers a statistically-sound blueprint for future studies on how to evaluate matrix completion methods for recommender systems under realistic scenarios.
MEJan 29, 2021
Tree-based Node Aggregation in Sparse Graphical ModelsInes Wilms, Jacob Bien
High-dimensional graphical models are often estimated using regularization that is aimed at reducing the number of edges in a network. In this work, we show how even simpler networks can be produced by aggregating the nodes of the graphical model. We develop a new convex regularized method, called the tree-aggregated graphical lasso or tag-lasso, that estimates graphical models that are both edge-sparse and node-aggregated. The aggregation is performed in a data-driven fashion by leveraging side information in the form of a tree that encodes node similarity and facilitates the interpretation of the resulting aggregated nodes. We provide an efficient implementation of the tag-lasso by using the locally adaptive alternating direction method of multipliers and illustrate our proposal's practical advantages in simulation and in applications in finance and biology.
MLNov 9, 2017
Interpretable Vector AutoRegressions with Exogenous Time SeriesInes Wilms, Sumanta Basu, Jacob Bien et al.
The Vector AutoRegressive (VAR) model is fundamental to the study of multivariate time series. Although VAR models are intensively investigated by many researchers, practitioners often show more interest in analyzing VARX models that incorporate the impact of unmodeled exogenous variables (X) into the VAR. However, since the parameter space grows quadratically with the number of time series, estimation quickly becomes challenging. While several proposals have been made to sparsely estimate large VAR models, the estimation of large VARX models is under-explored. Moreover, typically these sparse proposals involve a lasso-type penalty and do not incorporate lag selection into the estimation procedure. As a consequence, the resulting models may be difficult to interpret. In this paper, we propose a lag-based hierarchically sparse estimator, called "HVARX", for large VARX models. We illustrate the usefulness of HVARX on a cross-category management marketing application. Our results show how it provides a highly interpretable model, and improves out-of-sample forecast accuracy compared to a lasso-type approach.
MEDec 17, 2014
High Dimensional Forecasting via Interpretable Vector AutoregressionWilliam B. Nicholson, Ines Wilms, Jacob Bien et al.
Vector autoregression (VAR) is a fundamental tool for modeling multivariate time series. However, as the number of component series is increased, the VAR model becomes overparameterized. Several authors have addressed this issue by incorporating regularized approaches, such as the lasso in VAR estimation. Traditional approaches address overparameterization by selecting a low lag order, based on the assumption of short range dependence, assuming that a universal lag order applies to all components. Such an approach constrains the relationship between the components and impedes forecast performance. The lasso-based approaches work much better in high-dimensional situations but do not incorporate the notion of lag order selection. We propose a new class of hierarchical lag structures (HLag) that embed the notion of lag selection into a convex regularizer. The key modeling tool is a group lasso with nested groups which guarantees that the sparsity pattern of lag coefficients honors the VAR's ordered structure. The HLag framework offers three structures, which allow for varying levels of flexibility. A simulation study demonstrates improved performance in forecasting and lag order selection over previous approaches, and a macroeconomic application further highlights forecasting improvements as well as HLag's convenient, interpretable output.