Reducing Carbon Emissions at Scale: Interpretable and Efficient to Implement Reinforcement Learning via Policy Extraction

Abstract

Machine Learning is a promising avenue for combating climate change by developing control policies that can lead to greater efficiency, reducing cost and carbon footprint. However, while much literature has focused on how we might be able to learn and develop these policies, there are two key practical considerations that are often overlooked. Even if we had these algorithms that demonstrably lead to efficiency gains, how can we convince the public to adopt them, and how can complex policies be implemented in the real world, especially if they require continuous real time interaction between the algorithm and the system it is controlling? To address these two issues, we propose a single solution: policy extraction. We consider the problem of commercial building operation, one of the highest potential impact areas for Machine Learning. We generate a building HVAC control policy via Reinforcement Learning, and then extract that policy, going from a Neural Network to a human readable fixed setpoint schedule. The result is interpretable, and can be examined by a technician, making it easier to trust. It also has fewer requirements to implement, as it circumvents the need for continuous interaction and allows for the building to be controlled in a normal, albeit more efficient fashion.