Go Beyond Black-box Policies: Rethinking the Design of Learning Agent for Interpretable and Verifiable HVAC Control

Speaker: Dr. Wan Du
         Department of Computer Science and Engineering
         University of California


Title:  "Go Beyond Black-box Policies: Rethinking the Design of Learning Agent
         for Interpretable and Verifiable HVAC Control"

Date:    Thursday, 16 May 2024

Time:    2:00pm - 3:00pm

Venue:   Room 5504 (via lift 25/26), HKUST


Abstract:

Reinforcement Learning (RL) has been widely studied for improving the
energy efficiency efficiency of the Heating, Ventilation, and Air
Conditioning (HVAC) system in buildings. However, no RL-based control
agent has been used in real buildings. One of the reasons is that existing
methods rely mainly on a black-box neural network to learn a thermal
dynamics model or a control policy, lacking reliability guarantees and
posing risks to occupant health. In this talk, I will introduce the effort
my research group has made to develop safe HVAC control. The key idea is
to extract a decision tree from existing RL-based control policies. Since
decision trees are deterministic and interpretable, we can formally verify
the safety of a control agent before deployment. By distilling the
stochastic decisions of an RL-based controller into the deterministic
decisions of a decision tree, our control agent also improves energy
efficiency. Extensive experiments show that our method saves 68.4% more
energy and increases human comfort gain by 14.8% compared to the
state-of-the-art method.


******************
Biography:

Dr. Wan Du is an assistant professor in the Department of Computer Science
and Engineering at the University of California, Merced, USA. He received
his Ph.D. degree from the University of Lyon, France, and worked as a
research fellow at Nanyang Technological University (NTU), Singapore. His
current research interests include reinforcement learning for
cyber-physical systems, LoRa networking, and mobile computing. He received
the NSF CAREER Award 2023, the best paper runner-up award of IEEE
DCOSS-IoT 2022 and ACM BuildSys 2023 and 2020, and the best paper award of
ACM SenSys 2015. He is on the editorial board of the IEEE Internet of
Things Journal and has been a TPC member of conferences like SenSys, ATC,
and INFOCOM.