Logic-Based Explanations of Imbalance Price Forecasts using Boosted Trees

Explainability is one of the keys to foster the acceptance of Machine Learning (ML) models in safety-critical fields such as power systems. Given an input instance x and a complex ML model f, the driving features of the corresponding output are commonly derived using model-agnostic approaches such as SHAP. Although being generic, such approaches offer limited guarantees about the quality of the explanations they provide. In this paper, we opt for a logic-based approach to derive post-hoc explanations. Our approach provides formal guarantees about the explanations t that are generated for input instances x given an interval I containing f(x) and representing the admissible imprecision about f(x). Thus, our approach ensures that the prediction f(x’) on every instance x’ covered by t belongs to I as well. In our work, f is a boosted tree, which is accurate and associated with an equivalent logical representation. The forecasted variable is the imbalance price, which is an important market signal for trading strategies of energy traders. The outcomes --using data from the Belgian power system-- shed light on the input patterns that drive a high or low imbalance price prediction, while investigating whether such input patterns are intelligible for a human explainee.