Articles & Issues
- Language
- English
- Conflict of Interest
- In relation to this article, we declare that there is no conflict of interest.
- Publication history
-
Received October 2, 2003
Accepted December 16, 2003
-
This is an Open-Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/bync/3.0) which permits
unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright © KIChE. All rights reserved.
All issues
Simulation-Based Learning of Cost-To-Go for Control of Nonlinear Processes
311 Ferst Dr. NW, School of Chemical and Biomolecular Engineering,, Georgia Institute of Technology, Atlanta, GA 30332-0100, USA
Korean Journal of Chemical Engineering, March 2004, 21(2), 338-344(7), 10.1007/BF02705417
Download PDF
Abstract
In this paper, we present a simulation-based dynamic programming method that learns the ‘cost-to-go’ function in an iterative manner. The method is intended to combat two important drawbacks of the conventional Model Predictive Control (MPC) formulation, which are the potentially exorbitant online computational requirement and the inability to consider the future interplay between uncertainty and estimation in the optimal control calculation. We use a nonlinear Van de Vusse reactor to investigate the efficacy of the proposed approach and identify further research issues.
Keywords
References
Astrom KJ, Helmersson A, Comp. Maths. Appl., 12A, 653 (1986)
Astrom KJ, Wittenmark B, "Adaptive Control," Addison-Wesley (1989)
Bellman RE, "Dynamic Programming," Princeton University Press, New Jersey (1957)
Bemporad A, Morari M, Automatica, 35(3), 407 (1999)
Bertsekas DP, "Dynamic Programming and Optimal Control," 2nd ed., Athena Scientific, Belmont, MA (2000)
Bertsekas DP, Tsitsiklis JN, "Neuro-Dynamic Programming," Athena Scientific, Belmont, MA (1996)
Chikkula Y, Lee JH, Ind. Eng. Chem. Res., 39(6), 2010 (2000)
Crites RH, Barto AG, "Improving Elevator Performance Using Reinforcement Learning," Advances in Neural Information Processing Systems 8, Touretzky, D.S., Mozer, M.C. and Haselmo, M.E., ed., MIT Press, Cambridge, MA, 1017 (1996)
Henson MA, Comput. Chem. Eng., 23(2), 187 (1998)
Howard RA, "Dynamic Programming and Markov Processes," MIT Press, Cambridge, MA (1960)
Kaisare NS, Lee JM, Lee JH, Int. J. Robust Nonlinear Control, 13, 347 (2002)
Lee JH, Cooley B, Chem. Process. Control, 5, 201 (1997)
Lee JH, Ricker NL, Ind. Eng. Chem. Res., 33(6), 1530 (1994)
Lee JH, Yu ZH, Automatica, 33(5), 763 (1997)
Lee JM, Lee JH, "Simulation-Based Dual Mode Controller for Nonlinear Processes," Proceedings of IFAC ADCHEM 2003, Accepted (2004)
Lee JM, Lee JH, "Neuro-Dynamic Programming Approach to Dual Control Problems," AIChE Annual Meeting, Reno, NV (2001)
Marbach P, Tsitsiklis JN, IEEE Trans. Autom. Control, 46(2), 191 (2001)
Mayne DQ, Rawlings JB, Rao CV, Scokaert POM, Automatica, 36(6), 789 (2000)
Meadows ES, Rawlings JB, "Model Predictive Control," Nonlinear Process Control, Henson, M.A. and Seborg, D.E., eds., Prentice Hall, New Jersey, 233 (1997)
Morari M, Lee JH, Comput. Chem. Eng., 23(4-5), 667 (1999)
Puterman ML, "Markov Decision Processes," Wiley, New York, NY (1994)
Sistu PB, Bequette BW, Chem. Eng. Sci., 50(6), 921 (1995)
Sutton RS, Barto AG, "Reinforcement Learning: An Introduction," MIT Press, Cambridge, MA (1998)
Tesauro GJ, Machine Learning, 8, 257 (1992)
VandeVusse JG, Chem. Eng. Sci., 19, 964 (1964)
Zhang W, Dietterich TG, "A Reinforcement Learning Approach to Job Shop Scheduling," Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, 1114 (1995)
Astrom KJ, Wittenmark B, "Adaptive Control," Addison-Wesley (1989)
Bellman RE, "Dynamic Programming," Princeton University Press, New Jersey (1957)
Bemporad A, Morari M, Automatica, 35(3), 407 (1999)
Bertsekas DP, "Dynamic Programming and Optimal Control," 2nd ed., Athena Scientific, Belmont, MA (2000)
Bertsekas DP, Tsitsiklis JN, "Neuro-Dynamic Programming," Athena Scientific, Belmont, MA (1996)
Chikkula Y, Lee JH, Ind. Eng. Chem. Res., 39(6), 2010 (2000)
Crites RH, Barto AG, "Improving Elevator Performance Using Reinforcement Learning," Advances in Neural Information Processing Systems 8, Touretzky, D.S., Mozer, M.C. and Haselmo, M.E., ed., MIT Press, Cambridge, MA, 1017 (1996)
Henson MA, Comput. Chem. Eng., 23(2), 187 (1998)
Howard RA, "Dynamic Programming and Markov Processes," MIT Press, Cambridge, MA (1960)
Kaisare NS, Lee JM, Lee JH, Int. J. Robust Nonlinear Control, 13, 347 (2002)
Lee JH, Cooley B, Chem. Process. Control, 5, 201 (1997)
Lee JH, Ricker NL, Ind. Eng. Chem. Res., 33(6), 1530 (1994)
Lee JH, Yu ZH, Automatica, 33(5), 763 (1997)
Lee JM, Lee JH, "Simulation-Based Dual Mode Controller for Nonlinear Processes," Proceedings of IFAC ADCHEM 2003, Accepted (2004)
Lee JM, Lee JH, "Neuro-Dynamic Programming Approach to Dual Control Problems," AIChE Annual Meeting, Reno, NV (2001)
Marbach P, Tsitsiklis JN, IEEE Trans. Autom. Control, 46(2), 191 (2001)
Mayne DQ, Rawlings JB, Rao CV, Scokaert POM, Automatica, 36(6), 789 (2000)
Meadows ES, Rawlings JB, "Model Predictive Control," Nonlinear Process Control, Henson, M.A. and Seborg, D.E., eds., Prentice Hall, New Jersey, 233 (1997)
Morari M, Lee JH, Comput. Chem. Eng., 23(4-5), 667 (1999)
Puterman ML, "Markov Decision Processes," Wiley, New York, NY (1994)
Sistu PB, Bequette BW, Chem. Eng. Sci., 50(6), 921 (1995)
Sutton RS, Barto AG, "Reinforcement Learning: An Introduction," MIT Press, Cambridge, MA (1998)
Tesauro GJ, Machine Learning, 8, 257 (1992)
VandeVusse JG, Chem. Eng. Sci., 19, 964 (1964)
Zhang W, Dietterich TG, "A Reinforcement Learning Approach to Job Shop Scheduling," Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, 1114 (1995)
