Invited Research Lecture by Professor Frank Lewis: Adaptive Tuning Structures for Optimal Process Control and Graphical Games

Date: 14 May 2013 - 14 May 2013

Venue: SIMTech Training Room 4, Tower Block, Level 3

This lecture is designed for participants to gain a better understanding of adaptive control structures, providing powerful techniques for online learning of effective controllers for unknown nonlinear systems. Optimal control design allows for savings in fuel consumption, reduced resource usage, increased actuator life, and minimum processing times. 
This event is supported by IEEE IES Singapore Chapter.  

This talk will discuss some new adaptive control structures for learning online the solutions to optimal control problems and multi-player differential games.  Applications in Process control and vehicle control are shown.  Techniques from reinforcement learning are used to design a new family of adaptive controllers based on actor-critic mechanisms that converge in real time to optimal control and game theoretic solutions.  A new sort of distributed game is defined, namely multi-agent graphical games, where the interactions between players are restricted by a distributed communication topology.

Optimal Control: Optimal feedback control design has been responsible for much of the successful performance of engineered systems in aerospace, industrial processes, vehicles, ships, robotics, and elsewhere since the 1960s.  Optimal design allows for savings in fuel consumption, reduced resource usage, increased actuator life, and minimum processing times.  However, optimal control design is performed offline by solving optimal design equations including the algebraic Riccati equation (ARE) and the Game ARE.  It is difficult to perform optimal designs for nonlinear systems since they rely on solutions to complicated Hamilton-Jacobi-Bellman (HJB).  Finally, optimal design generally requires that the full system dynamics models be known. System identification to determine the process dynamics can be expensive and may lead to inaccurate dynamics models.  Also, dynamics models may change over time.

Optimal Adaptive Control: Adaptive control has provided powerful techniques for online learning of effective controllers for unknown nonlinear systems.  It has especially been used in industrial process control for decades.  Adaptive control does not generally produce optimal control solutions.  Therefore, in this talk we discuss online adaptive algorithms for learning optimal control solutions for continuous-time linear and nonlinear systems.  This is a novel class of adaptive control algorithms that converge to optimal control solutions by online learning in real time.  It is a data-based approach that uses measurements of process inputs and outputs in real time. System dynamics models are not needed.  In the linear quadratic (LQ) case, the algorithms learn the solution to the ARE by adaptation along the system motion trajectories.  In the case of nonlinear systems with general performance measures, the algorithms learn the (approximate smooth local) solutions of HJB equations.  The algorithms are based on actor-critic reinforcement learning techniques. 

Multi-Player Differential Games: Multi-player control systems are important in disturbance rejection and control of multiple coordinated processes or vehicles.  New algorithms will be presented for solving online the non zero-sum multi-player games for continuous-time systems.  Applications to disturbance compensation will be shown.  We use an adaptive control structure motivated by reinforcement learning policy iteration.  The result is an adaptive control system with multiple tuned control loops that learns based on the interplay of agents in a game, to deliver true online gaming behavior. 

Graphical Games.  A new formulation for control of multi-agent cooperative systems is given.  A novel form of game among agents in a communication graph is formulated where each agent is allowed to interact only with its neighbors.  A new notion of Nash equilibrium is defined that is suitable for graphical games.  

About the Speaker
F.L. Lewis,Fellow IEEE,Fellow IFAC,Fellow U.K.Institute of Measurement & Control,PE Texas,U.K. Chartered Engineer,is Distinguished Scholar Professor, Distinguished Teaching Professor, and Moncrief-O'Donnell Chair at The University of Texas at Arlington Research Institute.  IEEE Control Systems SocietyDistinguished Lecturer.  He obtained the Bachelor's Degree in Physics/EE and the MSEE at Rice University,the MS in Aeronautical Engineering from Univ. W. Florida,and the Ph.D. at Ga. Tech.  He works in feedback control,reinforcement learning, intelligent systems,and distributed control systems.  He is author of 6 U.S. patents,250 journal papers,360 conference papers,15 books,44 chapters,and 11 journal special issues.  He received the Fulbright Research Award,NSF Research Initiation Grant,ASEETerman Award,Int. Neural Network Soc.Gabor Award2009,U.K. Inst Measurement & ControlHoneywell Field Engineering Medal2009.  Received IEEE Computational Intelligence SocietyNeural Networks Pioneer Award2012.  Distinguished Foreign Scholar, Nanjing Univ. Science & Technology. Project 111 Professor at Northeastern University, China. Received Outstanding Service Award from Dallas IEEE Section,selected as Engineer of the Year by Ft. Worth IEEE Section.  Listed in Ft. Worth Business Press Top 200 Leaders in Manufacturing. Received the 2010 IEEE Region 5 Outstanding Engineering Educator Award and the 2010 UTA Graduate Dean's Excellence in Doctoral Mentoring Award. Elected to UTA Academy of Distinguished Teachers 2012.  He served on the NAE Committee on Space Station in 1995.  Founding Member of the Board of Governors of the Mediterranean Control Association.  Helped win the IEEE Control Systems Society Best Chapter Award (as Founding Chairman of DFW Chapter),the National Sigma Xi Award for Outstanding Chapter (as President of UTA Chapter),and the US SBA Tibbets Award in 1996 (as Director of ARRI's SBIR Program).  

9.30am      Registration
10am         invited Research Lecture
11am         Networking & Refreshment
11.30am     End

Who Should Attend
Senior management, R&D Managers, Engineers, Professors and University students.

Contact Us
For technical enquiries: Mr Hu Jinwen, Scientist, MEC Group, Tel: 6793 8938; Email:

For General enquiries: Alice Koh, Tel: 6793 8249; Email: