What If We Had Used a Different App? Reliable Counterfactual KPI Analysis in Wireless Systems
This work provides a method for network operators and designers to monitor and optimize wireless network operations by identifying suboptimal app selection strategies, addressing a key challenge in modern wireless architectures like O-RAN.
This paper tackles the problem of estimating Key Performance Indicators (KPIs) that would have been achieved if a different application (app) had been used in wireless systems, a challenge due to the inability to simultaneously observe actual and counterfactual KPIs. The authors propose a conformal-prediction-based counterfactual analysis method that provides reliable error bars for these estimated KPIs, even with covariate shift.
In modern wireless network architectures, such as Open Radio Access Network (O-RAN), the operation of the radio access network (RAN) is managed by applications, or apps for short, deployed at intelligent controllers. These apps are selected from a given catalog based on current contextual information. For instance, a scheduling app may be selected on the basis of current traffic and network conditions. Once an app is chosen and run, it is no longer possible to directly test the key performance indicators (KPIs) that would have been obtained with another app. In other words, we can never simultaneously observe both the actual KPI, obtained by the selected app, and the counterfactual KPI, which would have been attained with another app, for the same network condition, making individual-level counterfactual KPIs analysis particularly challenging. This what-if analysis, however, would be valuable to monitor and optimize the network operation, e.g., to identify suboptimal app selection strategies. This paper addresses the problem of estimating the values of KPIs that would have been obtained if a different app had been implemented by the RAN. To this end, we propose a conformal-prediction-based counterfactual analysis method for wireless systems that provides reliable error bars for the estimated KPIs, despite the inherent covariate shift between logged and test data. Experimental results for medium access control-layer apps and for physical-layer apps demonstrate the merits of the proposed method.