On Incorporating Forecast Information into MDPs

Apr 8 2022 - 1pm


Prof. Peter Glynn



In an increasing number of sequential decision-making applications, one has forecast information that can be used to enhance the decision process. In particular, in developing more efficient energy systems, one has the ability to use weather forecasts to improve the control of the associated system. In this talk, we will discuss ongoing work at Stanford that relates to the operation of Stanford’s district heating and cooling system, as well as a mathematical MDP formulation for linear control systems that takes into account forecast information in a principled way. By principled, we mean that the forecasts are required to be mathematically consistent with the underlying state dynamics, in the sense that the forecasts are required to satisfy a cerain martingale property. This work is joint with Jacques de Chalendar.




Peter W. Glynn is the Thomas Ford Professor in the Department of Management Science and Engineering (MS&E) at Stanford University, and also holds a courtesy appointment in the Department of Electrical Engineering. He was Director of Stanford's Institute for Computational and Mathematical Engineering from 2006 until 2010 and served as Chair of MS&E from 2011 through 2015. He is a Fellow of INFORMS and a Fellow of the Institute of Mathematical Statistics, and was an IMS Medallion Lecturer in 1995 and INFORMS Markov Lecturer in 2014. He was co-winner of the Outstanding Publication Awards from the INFORMS Simulation Society in 1993, 2008, and 2016, was a co-winner of the Best (Biannual) Publication Award from the INFORMS Applied Probability Society in 2009, was the co-winner of the John von Neumann Theory Prize from INFORMS in 2010, gave the INFORMS Philip McCord Morse Lecture in 2020, and received the INFORMS Simulation Society Lifetime Professional Achievement Award in 2021. In 2012, he was elected to the National Academy of Engineering. He was Founding Editor-in-Chief of Stochastic Systems and served as Editor-in-Chief of Journal of Applied Probability and Advances in Applied Probability from 2016 to 2018. His research interests lie in simulation, computational probability, queueing theory, statistical inference for stochastic processes, and stochastic modeling.