This document provides an introduction to the textbook "Optimal Control Systems" by Desineni Subbaram Naidu. It discusses key topics in optimal control theory including the maximum principle proposed by Pontryagin and applications to diverse fields. The textbook aims to provide a simplified treatment of the subject for graduate students, incorporating MATLAB and SIMULINK. It is intended to cover all topics for a one-semester graduate course in control systems and optimization.
The document discusses different types of singular points in control systems:
1. A nodal point occurs when both eigenvalues are real and negative, causing all trajectories to converge to the origin in a stable manner.
2. A saddle point occurs when the eigenvalues are real and equal but opposite in sign, making the origin unstable with some trajectories converging and others diverging.
3. A focus point occurs when the eigenvalues are complex conjugates with negative real parts, causing the trajectories to spiral inward in a stable manner towards the origin.
4. A center or vortex point occurs when the eigenvalues are purely imaginary, causing the trajectories to travel in closed paths around the origin in a limitedly stable manner.
The document discusses the concepts of controllability and observability in state space analysis of dynamic systems. It defines controllability as the ability to transfer a system state to any desired state using control inputs. Observability is defined as the ability to identify the system state using output measurements. Gilbert's and Kalman's tests are described to check for complete controllability and observability by examining the system and output matrices. No cancellation of poles and zeros in the transfer function is a necessary condition for complete controllability and observability.
The document discusses the z-transform, which is a mathematical tool used to analyze discrete-time control systems. The z-transform plays a similar role for discrete-time systems as the Laplace transform does for continuous-time systems. Some key properties of the z-transform discussed include the region of convergence, properties and theorems like the shifting theorem and initial/final value theorems, and applications to feedback control systems.
This document provides an introduction to analog filters. It discusses the classification of filters as either digital or analog, and passive or active analog filters. It describes the basic types of filters - lowpass, highpass, bandpass and bandstop. Circuit examples are provided for passive lowpass, highpass, bandpass and bandstop filters using resistors, capacitors and inductors. An example is shown of designing a bandpass filter with a specified center frequency and bandwidth. Basic active filter circuits are also illustrated for lowpass, highpass, bandpass and bandstop filters using op-amps.
The document discusses power flow analysis, which determines the voltage, current, real power, and reactive power at points in an electrical network under normal operating conditions. It provides three key points:
1. Power flow analysis is important for planning, operations, and future expansion of power systems by studying the effects of new loads, generators, or transmission lines.
2. The analysis involves classifying buses as slack, generator, or load buses and formulating the network equations based on the bus admittance matrix.
3. Solving the load flow problem involves determining the complex voltages across all buses given the network configuration and bus demands. This provides critical information for monitoring overloads and voltage deviations.
This document discusses the synthesis of one-port networks using only two element types, inductors and capacitors. It begins by defining network functions as the ratio of output to input phasors in the frequency domain. It describes properties that network functions must satisfy to be stable, including conditions on the poles and zeros. Specifically, it covers properties of L-C immittance functions, which are positive real functions with simple poles and zeros that lie on and interlace along the jω axis. The document provides examples of synthesizing L-C driving point immittances by finding their partial fraction expansions and representing each term with an inductor or capacitor in either series or parallel. It also describes an alternative Cauer method using continued
The document discusses optimal power flow analysis which is power flow analysis with an optimization objective such as minimizing fuel costs or transmission losses. It describes power flow analysis as determining the voltage magnitude and angle for each bus given load and generator conditions. Optimal power flow aims to satisfy nonlinear equality constraints from load flow equations and inequality constraints while optimizing an objective function such as fuel costs. Common solution methods include gradient, Newton-based, and linear programming approaches as well as intelligent methods like artificial neural networks.
The document discusses power system stability, including classifications of stability (steady state, transient, and dynamic) and factors that affect transient stability. It also covers topics like the swing equation, equal area criterion, critical clearing angle, and multi-machine stability studies. Some key points:
1) Power system stability refers to a system's ability to return to normal operating conditions after disturbances like faults or load changes.
2) Transient stability depends on factors like fault duration and location, generator inertia, and pre-fault loading conditions.
3) The equal area criterion states that a system will remain stable if the accelerating and decelerating area segments on the power-angle curve are equal.
4)
This document discusses block diagram reduction techniques. It provides steps to simplify a block diagram and determine the closed loop transfer function. The steps include applying rules such as combining blocks in series and parallel, moving summing points, interchanging summing points, eliminating feedback loops by considering the forward or feedback loop transfer function to be unity. An example problem is worked through to demonstrate these rules and techniques for simplifying block diagrams.
This document provides an introduction to control engineering. It discusses several key points:
1) Control engineering deals with designing systems to control dynamic processes and improve response speed, accuracy, and stability. This includes analyzing both classical and modern control methods.
2) Modern control engineering uses state-space and eigenvector approaches to model multi-input multi-output systems as sets of first-order differential equations.
3) Automatic control systems are commonly used, where a controlled variable is measured and compared to a setpoint to generate an output that achieves the desired result. This reduces costs and improves quality and productivity over manual control.
Automatic generation control (AGC) is a system for adjusting the power output of multiple generators at different power plants, in response to changes in the load. Since a power grid requires that generation and load closely balance moment by moment, frequent adjustments to the output of generators are necessary. The balance can be judged by measuring the system frequency; if it is increasing, more power is being generated than used, which causes all the machines in the system to accelerate. If the system frequency is decreasing, more load is on the system than the instantaneous generation can provide, which causes all generators to slow down.
1. The document discusses power system stability, including classifications of power system states as steady state, dynamic state, and transient state.
2. It describes synchronous machine swing equation and power angle equation, which relate the mechanical power input to the electrical power output of a generator through the power/torque angle.
3. An example calculation is shown to find the steady state power limit of a power system with a generator connected to an infinite bus through a transmission line.
This document discusses DC servo motors. It defines a servo motor as a rotary actuator that allows for precise control of position, velocity, and acceleration using position feedback from an encoder to a controller. Servo motors are used in applications requiring precise motion control, such as robotics, CNC machinery, and automated manufacturing. The mechanism uses the difference between the commanded position and measured position to generate an error signal to rotate the motor in the direction needed to bring the output shaft to the appropriate position.
Modern Control - Lec 04 - Analysis and Design of Control Systems using Root L...Amr E. Mohamed
The document provides an overview of root locus analysis and design of control systems. It begins with an introduction to root locus including motivation, definition, and the basic feedback control system model. It then covers the key rules and steps for constructing and interpreting root loci, including determining asymptotes, breakaway/break-in points, and imaginary axis crossings. Three examples are worked through step-by-step to demonstrate how to apply the rules and steps to sketch root loci for different open-loop transfer functions. The document emphasizes that root locus allows choosing controller parameters to place closed-loop poles in desired performance regions.
Modern Control - Lec07 - State Space Modeling of LTI SystemsAmr E. Mohamed
The document provides an overview of state-space representation of linear time-invariant (LTI) systems. It defines key concepts such as state variables, state vector, state equations, and output equations. Examples are given to show how to derive the state-space models from differential equations describing dynamical systems. Specifically, it shows how to 1) select state variables, 2) write first-order differential equations as state equations, and 3) obtain output equations to fully represent LTI systems in state-space form.
This document contains several examples of using Mason's Rule to calculate transfer functions from signal flow graphs. The examples demonstrate applying Mason's Rule to calculate loop gains, forward path gains, non-touching loops, and the overall transfer function. Signal flow graphs are constructed from block diagrams and the steps of Mason's Rule are systematically worked through.
This document discusses electric drives and AC motor drives. It defines electric drives as systems that use 50% of electrical energy produced and can operate equipment at constant or variable speeds. The main components of electric drives are motors, including DC and AC types, and power sources like batteries or utilities. It also summarizes different types of single-phase and three-phase DC drives classified by their converter configurations. For AC drives, it explains that speed and torque can be controlled through stator voltage, rotor voltage or frequency control. It concludes that variable speed AC drives can increase system efficiency from 15-27% compared to constant speed operation.
The document discusses Model Reference Adaptive Systems (MRAS) for adaptive control. It describes the MIT rule, which is the original approach for MRAS. The MIT rule adjusts controller parameters based on the error between system output and reference model output. Several examples are provided to illustrate parameter adaptation for systems with adjustable gain, first order, and second order dynamics using the MIT rule. The normalized MIT rule is also introduced to improve parameter convergence.
This document provides a preface for a textbook on systems and control. It describes the scope and objectives of the textbook, which is to familiarize readers with dynamical system theory and provide tools for control system design. The textbook can be used for senior/graduate level courses in systems and control, introductory courses in nonlinear systems, or introductory courses in modern automatic control. It includes modeling of dynamical systems from various engineering and scientific disciplines. Analysis methods like Lyapunov stability theory and numerical techniques are discussed. Classical and modern control approaches like fuzzy and neural networks are presented in a unified manner.
Control Design Techniques in Power Electronics Devices (Power Systems) (Heber...Gollapalli Sreenivasulu
This document describes the modeling of several DC-to-DC power converters, including the buck, boost, buck-boost, Cuk, SEPIC and Zeta converters. It presents the modeling approach, which involves deriving state-space models, performing normalization, and determining the equilibrium points and static transfer functions. Experimental prototypes are also discussed for some converters. The goal is to mathematically model these switched power electronics devices as controlled dynamic systems suitable for control design techniques.
Modern Power Systems Analysis (Power Electronics and Power Systems) ( PDFDriv...ASHOKKUMARARJANBHAIP1
This document provides an overview and table of contents for the book "Modern Power Systems Analysis" by Xi-Fan Wang, Yonghua Song, and Malcolm Irving. It discusses changes in power systems and analysis techniques over the last 20 years. The book is intended as a textbook for students and a reference for researchers. It covers topics such as mathematical models, load flow analysis, stochastic security analysis, optimal power flow, HVDC and FACTS devices, generator and load models, and transient and small-signal stability analysis.
[Andrew R. Hileman] Insulation Coordination for Power System.pdfTIRSOROJAS4
This book provides a comprehensive reference on insulation coordination for power systems. It is intended as a teaching text that can be used for a course on insulation coordination methods. The book covers topics such as lightning phenomena, insulation strength, traveling waves, line insulation coordination, and station insulation coordination. It is based on a course originally taught by the author at Westinghouse and Carnegie Mellon University. The book uses probabilistic and statistical methods extensively. It has been used to teach a graduate-level course at Penn State University and for training at several utilities.
This document introduces the Eco-Schools USA program, which aims to educate students about sustainability. It is part of an international program involving over 50 countries. The US program uses a seven step framework to guide schools in developing and implementing sustainable initiatives. Students focus on certain "pathways" or topics, including energy and climate change. Schools can apply for bronze, silver, or green flag awards for their efforts. The introduction emphasizes the importance of educating students about energy and climate change issues so they can address these challenges as future leaders.
NG2S: A Study of Pro-Environmental Tipping Point via ABMsKan Yuenyong
A study of tipping point: much less is known about the most efficient ways to reach such transitions or how self-reinforcing systemic transformations might be instigated through policy. We employ an agent-based model to study the emergence of social tipping points through various feedback loops that have been previously identified to constitute an ecological approach to human behavior. Our model suggests that even a linear introduction of pro-environmental affordances (action opportunities) to a social system can have non-linear positive effects on the emergence of collective pro-environmental behavior patterns.
Egr445 book for Dr. Redkar's Class at ASU, I do not own right to this book, this is for vewing and student purposes.
author : De Silva, Clarence W.
publisher : CRC Press
isbn10 | asin : 0203502787
print isbn13 : 9780203611647
ebook isbn13 : 9780203502785
language : English
subject Mechatronics.
publication date : 2005
lcc : TJ163.12.D45 2005eb
ddc : 621.3
subject : Mechatronics.
2020_Raymond Murray, Keith E. Holbert - Nuclear Energy).pdfTowhidulIslam143592
This document is the preface to the 8th edition of the textbook "Nuclear Energy: An Introduction to the Concepts, Systems, and Applications of Nuclear Processes" by Raymond L. Murray and Keith E. Holbert. It provides biographies of the authors, acknowledges those who assisted with the new edition, and explains the scope and purpose of revising and expanding the textbook. The continued goal is to provide factual information about nuclear energy to inform discussion of this topic, with a focus on expanding coverage of safety, waste, weapons, and other controversial issues.
Advanced Thermodynamics For Engineers - WinterboneSarah Marie
This document provides the preface to the book "Advanced Thermodynamics for Engineers, Second Edition" by Desmond E. Winterbone and Ali Turan. It introduces the motivation for writing the book, which is to provide material beyond the basics of thermodynamics typically covered in undergraduate courses. The book aims to introduce concepts that can help engineers use energy resources more efficiently and apply to a wide range of new technologies. It covers topics like combustion, fuel cells, property relationships, irreversible thermodynamics, and the limits of thermodynamic efficiency. The preface explains the structure and approach taken in the book.
Upper Mud Creek has been negatively affected by urbanization, showing it is less healthy than the control site, Mud Tributary. While Lower Mud Creek shows some improvement over Upper Mud Creek, it is still not as healthy as the untouched Mud Tributary site downstream. The differences in health between the three sites can be explained by the increasing effects of urbanization upstream, though Lower Mud Creek shows some recovery as the urban influences decrease downstream. In conclusion, the Mud Creek ecosystem only partially recovers from the effects of urbanization as it travels further downstream.
The document is a final report for an optimal system operation project. It was authored by Oswaldo Guerra Gomez, a student at Saxion University of Applied Sciences in the Netherlands under the supervision of Mr. Nguyen Trung Thang and Mr. Jan Bollen. The report details the student's research and implementation of the Flower Pollination Algorithm (FPA) to solve the economic load dispatch (ELD) problem of minimizing power generation costs while meeting demand. The student analyzed various optimization algorithms, programmed FPA in MATLAB, and tested it on standard functions and a 6-unit power system, finding it outperformed other algorithms in accuracy and speed. The report also includes a documentary about Vietnam made during the
This document summarizes a presentation about ChemEd X Data, an open web platform containing chemical data to support student learning. The platform contains unstructured data on elements, compounds, and reactions that students can explore to discover trends. It aims to move beyond static pre-cooked data by allowing students to choose data, identify patterns, and find exceptions. The document describes how the data is organized and can be visualized. It also discusses early findings that show the platform improves students' abilities to design experiments, draw conclusions from evidence, and identify exceptions. The overall goal is to enhance higher-order thinking skills.
RMG at the Flame Chemistry Workshop 2014Richard West
Presentation to the 2nd International Workshop on Flame Chemistry, preceding the 35th International Symposium on Combustion, in San Francisco, CA, in August 2014.
Describes recent progress in two projects related to our Reaction Mechanism Generator software.
This document provides an introduction to the textbook "An Introduction to Chemical Engineering Kinetics and Reactor Design" by Charles G. Hill Jr. It discusses the need for chemical engineers to understand chemical reaction kinetics and reactor design. It provides an overview of the textbook's contents, which include introductions to chemical kinetics, reaction engineering, and reactor design. The textbook is designed to teach the fundamentals of these topics to undergraduate chemical engineering students.
The document discusses using the ChemSpider platform to engage students in curating and contributing chemical structure data. It outlines how students could help by checking spectra data, playing identification games, uploading their own spectra from research projects, and depositing structures. Contributions would be peer reviewed and DOI indexed. The goal is to build ChemSpider into a premier open chemistry database while providing educational benefits to students.
This document provides an overview of the Chemical Process Principles I course, including the course outline, recommended textbooks, distribution of grades, learning practices, and objectives. The course covers units, dimensions and conversions, gas laws, stoichiometry, material and energy balances. It aims to introduce calculation techniques in chemical engineering and teach students how to formulate and solve material and energy balances through practice. The major topics will include background information, material balances, behavior of gases/liquids/solids, and energy balances.
Outlining Essays (Grades ) - Introducing Expository WDeja Lewis
Physical therapy plays an important role in prevention programs by treating conditions to avoid worsening, reduce problems, or promote better outcomes. Physical therapists can improve healthcare quality through prevention, treatment, and management of patient impairments. Preventative physical therapy aims to restore function, decrease dependency, and educate patients. Prevention services include primary prevention by recognizing risk factors, secondary prevention through early intervention, and tertiary prevention by managing established conditions.
emotional interface - dehligame satta for youbkldehligame1
Welcome to DelhiGame.in, your premier hub for the latest Satta results and gaming updates in Delhi! Check out our live results https://delhigame.in/ and stay informed with the latest updates https://delhigame.in/past-results/ . Join us to experience the thrill of gaming like never before!
Data analytics is a powerful tool that can transform business decision-making across industries. Contact District 11 Solutions, which specializes in data analytics, to make informed decisions and achieve your business goals.
Introduction to Data Science
1.1 What is Data Science, importance of data science,
1.2 Big data and data Science, the current Scenario,
1.3 Industry Perspective Types of Data: Structured vs. Unstructured Data,
1.4 Quantitative vs. Categorical Data,
1.5 Big Data vs. Little Data, Data science process
1.6 Role of Data Scientist
Why You Need Real-Time Data to Compete in E-CommercePromptCloud
In the fast-paced world of e-commerce, real-time data is crucial for staying competitive. By accessing up-to-date information on market trends, competitor pricing, and customer preferences, businesses can make informed decisions quickly. Real-time data enables dynamic pricing strategies, effective inventory management, and personalized marketing efforts, all of which are essential for meeting customer demands and outperforming competitors. Embrace real-time data to stay agile, optimize your operations, and drive growth in the ever-evolving e-commerce landscape. Get in touch for custom web scraping services: https://bit.ly/3WkqYVm
Decision Making Optimization through Data Mining
Being a Paper Presented at the 9th Accountant Conference of the Institute of Chartered Accountant of Nigeria (ICAN) Canada District from July 16 – 20, 2024.
Docker has revolutionized the way we develop, deploy, and run applications. It's a powerful platform that allows you to package your software into standardized units called containers. These containers are self-contained environments that include everything an application needs to run: code, libraries, system tools, and settings.
Here's a breakdown of what Docker offers:
Faster Development and Deployment:
Spin up new environments quickly: Forget about compatibility issues and dependency management. With Docker, you can create consistent environments for development, testing, and production with ease.
Share and reuse code: Build reusable Docker images and share them with your team or the wider community on Docker Hub, a public registry for Docker images.
Reliable and Consistent Applications:
Cross-platform compatibility: Docker containers run the same way on any system with Docker installed, eliminating compatibility headaches. Your code runs consistently across Linux, Windows, and macOS.
Isolation and security: Each container runs in isolation, sharing only the resources it needs.
Graph Machine Learning - Past, Present, and Future -kashipong
Graph machine learning, despite its many commonalities with graph signal processing, has developed as a relatively independent field.
This presentation will trace the historical progression from graph data mining in the 1990s, through graph kernel methods in the 2000s, to graph neural networks in the 2010s, highlighting the key ideas and advancements of each era. Additionally, recent significant developments, such as the integration with causal inference, will be discussed.
2. Electrical Engineering
Textbook Series
Richard C. Dorf, Series Editor
University of California, Davis
Forthcoming and Published Titles
Applied Vector Analysis
Matiur Rahman and Isaac Mulolani
Continuous Signals and Systems with MATLAB
Taan EIAli and Mohammad A. Karim
Discrete Signals and Systems with MATLAB
Taan EIAIi
Electromagnetics
Edward J. Rothwell and Michael J. Cloud
Optimal Control Systems
Desineni Subbaram Naidu
3. OPTIMAL
CONTROL
SYSTEMS
Desineni Subbaram Naidu
Idaho State Universitv.
Pocatello. Idaho. USA
o
CRC PRESS
Boca Raton London New York Washington, D.C.
5. v
"Because the shape of the whole universe is most perfect
and, in fact, designed by the wisest Creator, nothing
in all of the world will occur in which no maximum or
minimum rule is somehow shining forth. "
Leohard Euler, 1144
6. vi
Dedication
My deceased parents who shaped my life
Desineni Rama Naidu
Desineni Subbamma
and
My teacher who shaped my education
Buggapati A udi Chetty
7. vii
Preface
Many systems, physical, chemical, and economical, can be modeled
by mathematical relations, such as deterministic and/or stochastic differential
and/or difference equations. These systems then change with
time or any other independent variable according to the dynamical relations.
It is possible to steer these systems from one state to another
state by the application of some type of external inputs or controls.
If this can be done at all, there may be different ways of doing the
same task. If there are different ways of doing the same task, then
there may be one way of doing it in the "best" way. This best way can
be minimum time to go from one state to another state, or maximum
thrust developed by a rocket engine. The input given to the system
corresponding to this best situation is called "optimal" control. The
measure of "best" way or performance is called "performance index"
or "cost function." Thus, we have an "optimal control system," when a
system is controlled in an optimum way satisfying a given performance
index. The theory of optimal control systems has enjoyed a flourishing
period for nearly two decades after the dawn of the so-called "modern"
control theory around the 1960s. The interest in theoretical and practical
aspects of the subject has sustained due to its applications to such
diverse fields as electrical power, aerospace, chemical plants, economics,
medicine, biology, and ecology.
Aim and Scope
In this book we are concerned with essentially the control of physical
systems which are "dynamic" and hence described by ordinary differential
or difference equations in contrast to "static" systems, which are
characterized by algebraic equations. Further, our focus is on "deterministic"
systems only.
The development of optimal control theory in the sixties revolved
around the "maximum principle" proposed by the Soviet mathematician
L. S. Pontryagin and his colleagues whose work was published in
English in 1962. Further contributions are due to R. E. Kalman of the
United States. Since then, many excellent books on optimal control
theory of varying levels of sophistication have been published.
This book is written keeping the "student in mind" and intended
to provide the student a simplified treatment of the subject, with an
10. x
ACKNOWLEDGMENTS
The permissions given by
1. Prentice Hall for D. E. Kirk, Optimal Control Theory: An Introduction,
Prentice Hall, Englewood Cliffs, NJ, 1970,
2. John Wiley for F. L. Lewis, Optimal Control, John Wiley & Sons,
Inc., New York, NY, 1986,
3. McGraw-Hill for M. Athans and P. L. Falb, Optimal Control:
An Introduction to the Theory and Its Applications, McGraw-Hill
Book Company, New York, NY, 1966, and
4. Springer-Verlag for H. H. Goldstine, A History of the Calculus of
Variations, Springer-Verlag, New York, NY, 1980,
are hereby acknowledged.
11. xi
AUTHOR'S BIOGRAPHY
Desineni "Subbaram" Naidu received his B.E. degree in Electrical Engineering
from Sri Venkateswara University, Tirupati, India, and M.Tech. and Ph.D.
degrees in Control Systems Engineering from the Indian Institute of Technology
(lIT), Kharagpur, India. He held various positions with the Department of
Electrical Engineering at lIT. Dr. Naidu was a recipient of a Senior National
Research Council (NRC) Associateship of the National Academy of Sciences,
Washington, DC, tenable at NASA Langley Research Center, Hampton,
Virginia, during 1985-87 and at the U. S. Air Force Research Laboratory
(AFRL) at Wright-Patterson Air Force Base (WPAFB), Ohio, during 1998-
99. During 1987-90, he was an adjunct faculty member in the Department of
Electrical and Computer Engineering at Old Dominion University, Norfolk,
Virginia. Since August 1990, Dr. Naidu has been a professor at Idaho State
University. At present he is Director of the Measurement and Control Engineering
Research Center; Coordinator, Electrical Engineering program; and
Associate Dean of Graduate Studies in the College of Engineering, Idaho State
University, Pocatello, Idaho.
Dr. Naidu has over 150 publications including a research monograph, Singular
Perturbation Analysis of Discrete Control Systems, Lecture Notes in
Mathematics, 1985; a book, Singular Perturbation Methodology in Control
Systems, lEE Control Engineering Series, 1988; and a research monograph
entitled, Aeroassisted Orbital Transfer: Guidance and Control Strategies, Lecture
Notes in Control and Information Sciences, 1994.
Dr. Naidu is (or has been) a member of the Editorial Boards of the IEEE
Transaction on Automatic Control, (1993-99), the International Journal of
Robust and Nonlinear Control, (1996-present), the International Journal of
Control-Theory and Advanced Technology (C-TAT), (1992-1996), and a member
of the Editorial Advisory Board of Mechatronics: The Science of Intelligent
Machines, an International Journal, (1992-present).
Professor Naidu is an elected Fellow of The Institute of Electrical and Electronics
Engineers (IEEE), a Fellow of World Innovation Foundation (WIF), an
Associate Fellow of the American Institute of Aeronautics and Astronautics
(AIAA) and a member of several other organizations such as SIAM, ASEE,
etc. Dr. Naidu was a recipient of the Idaho State University Outstanding Researcher
Award for 1993-94 and 1994-95 and the Distinguished Researcher
Award for 1994-95. Professor Naidu's biography is listed (multiple years) in
Who's Who among America's Teachers, the Silver Anniversary 25th Edition
of Who's Who in the West, Who's Who in Technology, and The International
Directory of Distinguished Leadership.
13. Contents
1 Introduction 1
1.1 Classical and Modern Control . . . . . . . . . . . . . . . . . . . . . 1
1.2 Optimization................................. 4
1.3 Optimal Control .............................. 6
1.3.1 Plant ................................. 6
1.3.2 Performance Index ........................ 6
1.3.3 Constraints ............................. 9
1.3.4 Formal Statement of Optimal Control System .... 9
1.4 Historical Tour .............................. 11
1.4.1 Calculus of Variations .................... 11
1.4.2 Optimal Control Theory .................. 13
1.5 About This Book ............................. 15
1.6 Chapter Overview ............................ 16
1.7 Problems ................................... 17
2 Calculus of Variations and Optimal Control 19
2.1 Basic Concepts .............................. 19
2.1.1 Function and Functional .................. 19
2.1.2 Increment ............................. 20
2.1.3 Differential and Variation . . . . . . . . . . . . . . . . . . 22
2.2 Optimum of a Function and a Functional ............ 25
2.3 The Basic Variational Problem ................... 27
2.3.1 Fixed-End Time and Fixed-End State System ... 27
2.3.2 Discussion on Euler-Lagrange Equation ........ 33
2.3.3 Different Cases for Euler-Lagrange Equation .... 35
2.4 The Second Variation . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.5 Extrema of Functions with Conditions .............. 41
2.5.1 Direct Method .......................... 43
2.5.2 Lagrange Multiplier Method ................ 45
2.6 Extrema of Functionals with Conditions ............ 48
2.7 Variational Approach to Optimal Control Systems . . . . . 57
xiii
19. List of Figures
1.1 Classical Control Configuration . . . . . . . . . . . . . . . . . . . . 1
1.2 Modern Control Configuration .................... 3
1.3 Components of a Modern Control System ............ 4
1.4 Overview of Optimization . . . . . . . . . . . . . . . . . . . . . . . . 5
1.5 Optimal Control Problem ....................... 10
2.1 Increment ~f, Differential df, and Derivative j of a
Function f ( t) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.2 Increment ~J and the First Variation 8J of the Func-tional
J .................................... 24
2.3 ( a) Minimum and (b) Maximum of a Function f ( t) . . . . . 26
2.4 Fixed-End Time and Fixed-End State System ........ 29
2.5 A Nonzero g(t) and an Arbitrary 8x(t) ............. 32
2.6 Arc Length ................................. 37
2.7 Free-Final Time and Free-Final State System ......... 59
2.8 Final-Point Condition with a Moving Boundary B(t) .... 63
2.9 Different Types of Systems: (a) Fixed-Final Time and
Fixed-Final State System, (b) Free-Final Time and FixedFinal
State System, (c) Fixed-Final Time and Free-Final
State System, (d) Free-Final Time and Free-Final State
System .................................... 66
2.10 Optimal Controller for Example 2.12 ............... 72
2.11 Optimal Control and States for Example 2.12 ......... 74
2.12 Optimal Control and States for Example 2.13 ......... 77
2.13 Optimal Control and States for Example 2.14 ......... 81
2.14 Optimal Control and States for Example 2.15 ......... 84
2.15 Open-Loop Optimal Control ..................... 94
2.16 Closed-Loop Optimal Control .................... 95
3.1 State and Costate System ...................... 107
3.2 Closed-Loop Optimal Control Implementation ....... 117
X'lX
20. xx
3.3 Riccati Coefficients for Example 3.1 . . . . . . . . . . . . . .. 125
3.4 Closed-Loop Optimal Control System for Example 3.1 126
3.5 Optimal States for Example 3.1. . . . . . . . . . . . . . . . .. 127
3.6 Optimal Control for Example 3.1 ................ 127
3.7 Interpretation of the Constant Matrix P ........... 133
3.8 Implementation of the Closed-Loop Optimal Control:
Infinite Final Time. . . . . . . . . . . . . . . . . . . . . . . . . .. 135
3.9 Closed-Loop Optimal Control System . . . . . . . . . . . .. 138
3.10 Optimal States for Example 3.2. . . . . . . . . . . . . . . . .. 140
3.11 Optimal Control for Example 3.2 ................ 141
3.12 (a) Open-Loop Optimal Controller (OLOC) and
(b) Closed-Loop Optimal Controller (CLOC) ........ 145
4.1 Implementation of the Optimal Tracking System ..... 157
4.2 Riccati Coefficients for Example 4.1 ............... 163
4.3 Coefficients 91(t) and 92(t) for Example 4.1 ......... 164
4.4 Optimal States for Example 4.1 .................. 164
4.5 Optimal Control for Example 4.1 ................ 165
4.6 Riccati Coefficients for Example 4.2 ............... 167
4.7 Coefficients 91(t) and 92(t) for Example 4.2 ......... 168
4.8 Optimal Control and States for Example 4.2 ........ 168
4.9 Optimal Control and States for Example 4.2 ........ 169
4.10 Optimal Closed-Loop Control in Frequency Domain ... 180
4.11 Closed-Loop Optimal Control System with Unity
Feedback. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 184
4.12 Nyquist Plot of Go(jw) ........................ 185
4.13 Intersection of Unit Circles Centered at Origin
and -1 + jO ............................... 186
5.1 State and Costate System. . . . . . . . . . . . . . . . . . . . .. 205
5.2 Closed-Loop Optimal Controller for Linear Discrete-Time
Regulator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 215
5.3 Riccati Coefficients for Example 5.3 ............... 219
5.4 Optimal Control and States for Example 5.3 ........ 220
5.5 Optimal Control and States for Example 5.3 ........ 221
5.6 Closed-Loop Optimal Control for Discrete-Time
Steady-State Regulator System . . . . . . . . . . . . . . . . .. 223
5.7 Implementation of Optimal Control for Example 5.4 . .. 226
5.8 Implementation of Optimal Control for Example 5.4 ... 227
5.9 Riccati Coefficients for Example 5.5. . . . . . . . . . . . . .. 231
21. XXI
5.10 Optimal States for Example 5.5 ................ " 232
5.11 Optimal Control for Example 5.5 ................ 233
5.12 Implementation of Discrete-Time Optimal Tracker .... 239
5.13 Riccati Coefficients for Example 5.6 . . . . . . . . . . . . . .. 240
5.14 Coefficients 91(t) and 92(t) for Example 5.6 ......... 241
5.15 Optimal States for Example 5.6. . . . . . . . . . . . . . . . .. 241
5.16 Optimal Control for Example 5.6 ................ 242
5.17 Closed-Loop Discrete-Time Optimal Control System. . . 243
6.1 (a) An Optimal Control Function Constrained by a
Boundary (b) A Control Variation for Which -8u(t)
Is Not Admissible ........................... 254
6.2 Illustration of Constrained (Admissible) Controls ..... 260
6.3 Optimal Path from A to B . . . . . . . . . . . . . . . . . . . .. 261
6.4 A Multistage Decision Process .................. 262
6.5 A Multistage Decision Process: Backward Solution .... 263
6.6 A Multistage Decision Process: Forward Solution ..... 265
6.7 Dynamic Programming Framework of Optimal State
Feedback Control . . . . . . . . . . . . . . . . . . . . . . . . . . .. 271
6.8 Optimal Path from A to B . . . . . . . . . . . . . . . . . . . . . 290
7.1 Signum Function . . . . . . . . . . . . . . . . . . . . . . . . . . .. 299
7.2 Time-Optimal Control ........................ 299
7.3 Normal Time-Optimal Control System ............. 300
7.4 Singular Time-Optimal Control System ............ 301
7.5 Open-Loop Structure for Time-Optimal Control System 304
7.6 Closed-Loop Structure for Time-Optimal Control System 306
7.7 Possible Costates and the Corresponding Controls .... 309
7.8 Phase Plane Trajectories for u = + 1 (dashed lines) and
u = -1 (dotted lines) ......................... 310
7.9 Switch Curve for Double Integral Time-Optimal Control
System ................................... 312
7.10 Various Trajectories Generated by Four Possible Control
Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 313
7.11 Closed-Loop Implementation of Time-Optimal Control
Law ..................................... 315
7.12 SIMULINK@ Implementation of Time-Optimal
Control Law ............................... 316
7.13 Phase-Plane Trajectory for 1'+: Initial State (2,-2) and
Final State (0,0) ............................ 317
22. xxii
7.14 Phase-Plane Trajectory for 7-: Initial State (-2,2) and
Final State (0,0) ............................ 317
7.15 Phase-Plane Trajectory for R+: Initial State (-1,-1) and
Final State (0,0) ............................ 318
7.16 Phase-Plane Trajectory for R_: Initial State (1,1) and
Final State (0,0) ............................ 318
7.17 Relations Between A2(t) and lu*(t)1 + u*(t)A2(t) ...... 322
7.18 Dead-Zone Function. . . . . . . . . . . . . . . . . . . . . . . . .. 323
7.19 Fuel-Optimal Control ......................... 323
7.20 Switching Curve for a Double Integral Fuel-Optimal
Control System ............................. 324
7.21 Phase-Plane Trajectories for u(t) = 0 .............. 325
7.22 Fuel-Optimal Control Sequences ................. 326
7.23 E-Fuel-Optimal Control. ....................... 327
7.24 Optimal Control as Dead-Zone Function ........... 330
7.25 Normal Fuel-Optimal Control System ............. 331
7.26 Singular Fuel-Optimal Control System ............. 332
7.27 Open-Loop Implementation of Fuel-Optimal Control
System ................................... 333
7.28 Closed-Loop Implementation of Fuel-Optimal Control
System ................................... 334
7.29 SIMULINK@ Implementation of Fuel-Optimal Control
Law ..................................... 334
7.30 Phase-Plane Trajectory for "Y+: Initial State (2,-2) and
Final State (0,0) ............................ 336
7.31 Phase-Plane Trajectory for "Y-: Initial State (-2,2) and
Final State (0,0) ............................ 336
7.32 Phase-Plane Trajectory for R1 : Initial State (1,1) and
Final State (0,0) ............................ 337
7.33 Phase-Plane Trajectory for R3: Initial State (-1,-1) and
Final State (0,0) ............................ 337
7.34 Phase-Plane Trajectory for R2 : Initial State (-1.5,1) and
Final State (0,0) ............................ 338
7.35 Phase-Plane Trajectory for R4: Initial State (1.5,-1) and
Final State (0,0) ............................ 338
7.36 Saturation Function .......................... 343
7.37 Energy-Optimal Control ....................... 344
7.38 Open-Loop Implementation of Energy-Optimal Control
System ................................... 345
23. XXlll
7.39 Closed-Loop Implementation of Energy-Optimal
Control System . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 346
7.40 Relation between Optimal Control u*(t) vs (a) q*(t) and
(b) 0.5A*(t) ................................ 348
7.41 Possible Solutions of Optimal Costate A*(t) ......... 349
7.42 Implementation of Energy-Optimal Control Law ...... 351
7.43 Relation between Optimal Control u*(t) and Optimal
Costate A2 ( t) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 358
25. List of Tables
2.1 Procedure Summary of Pontryagin Principle for Bolza
Problem ................................... 69
3.1 Procedure Summary of Finite-Time Linear Quadratic
Regulator System: Time-Varying Case. . . . . . . . . . . .. 113
3.2 Procedure Summary of Infinite-Time Linear Quadratic
Regulator System: Time-Varying Case. . . . . . . . . . . .. 129
3.3 Procedure Summary of Infinite-Interval Linear Quadratic
Regulator System: Time-Invariant Case . . . . . . . . . . .. 136
4.1 Procedure Summary of Linear Quadratic Tracking System159
4.2 Procedure Summary of Regulator System with Prescribed
Degree of Stability . . . . . . . . . . . . . . . . . . . . . . . . . .. 178
5.1 Procedure Summary of Discrete-Time Optimal Control
System: Fixed-End Points Condition .............. 204
5.2 Procedure Summary for Discrete-Time Optimal Control
System: Free-Final Point Condition ............... 208
5.3 Procedure Summary of Discrete-Time, Linear Quadratic
Regulator System ............................ 214
5.4 Procedure Summary of Discrete-Time, Linear Quadratic
Regulator System: Steady-State Condition . . . . . . . . .. 222
5.5 Procedure Summary of Discrete-Time Linear Quadratic
Tracking System ............................ 238
6.1 Summary of Pontryagin Minimum Principle ......... 257
6.2 Computation of Cost during the Last Stage k = 2 ..... 269
6.3 Computation of Cost during the Stage k = 1,0 ....... 270
6.4 Procedure Summary of Hamilton-Jacobi-Bellman (HJB)
Approach ................................. 280
xxv
26. XXVI
7.1 Procedure Summary of Optimal Control Systems with
State Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . .. 355
27. Chapter 1
Introduction
In this first chapter, we introduce the ideas behind optimization and
optimal control and provide a brief history of calculus of variations and
optimal control. Also, a brief summary of chapter contents is presented.
1.1 Classical and Modern Control
The classical (conventional) control theory concerned with single input
and single output (8180) is mainly based on Laplace transforms theory
and its use in system representation in block diagram form. From
Figure 1.1, we see that
Reference
Input
R(s) +
Error
Signal
- E(s)
Y(s)
R(s)
G(s)
Control BPI
1 + G(s)H(s)
c ompensator ant
.. Gc(s) Input ... G (s)
U(s) p
Feedback
H(s) ...
(1.1.1)
Output
yes)
Figure 1.1 Classical Control Configuration
1
28. 2 Chapter 1: Introduction
where s is Laplace variable and we used
(1.1.2)
Note that
1. the input u(t) to the plant is determined by the error e(t) and
the compensator, and
2. all the variables are not readily available for feedback. In most
cases only one output variable is available for feedback.
The modern control theory concerned with multiple inputs and multiple
outputs (MIMO) is based on state variable representation in terms
of a set of first order differential (or difference) equations. Here, the
system (plant) is characterized by state variables, say, in linear, timeinvariant
form as
x(t) = Ax(t) + Bu(t)
y(t) = Cx(t) + Du(t)
(1.1.3)
(1.1.4)
where, dot denotes differentiation with respect to (w.r.t.) t, x(t), u(t),
and y( t) are n, r, and m dimensional state, control, and output vectors
respectively, and A is nxn state, B is nxr input, Cis mxn output, and D
is mxr transfer matrices. Similarly, a nonlinear system is characterized
by
x(t) = f(x(t), u(t), t)
y(t) = g(x(t), u(t), t).
(1.1.5)
(1.1.6)
The modern theory dictates that all the state variables should be fed
back after suitable weighting. We see from Figure 1.2 that in modern
control configuration,
1. the input u( t) is determined by the controller (consisting of error
detector and compensator) driven by system states x(t) and
reference signal r ( t ) ,
2. all or most of the state variables are available for control, and
3. it depends on well-established matrix theory, which is amenable
for large scale computer simulation.
29. 1.1 Classical and Modern Control 3
Plant
Control Output
Input .. p ..
u(t) y(t)
State
R eference
x(t)
Input
.. C '"
r(t)
Controller
Figure 1.2 Modern Control Configuration
The fact that the state variable representation uniquely specifies the
transfer function while there are a number of state variable representations
for a given transfer function, reveals the fact that state variable
representation is a more complete description of a system.
Figure 1.3 shows components of a modern control system. It shows
three components of modern control and their important contributors.
The first stage of any control system theory is to obtain or formulate
the dynamics or modeling in terms of dynamical equations such as differential
or difference equations. The system dynamics is largely based
on the Lagrangian function. Next, the system is analyzed for its performance
to find out mainly stability of the system and the contributions
of Lyapunov to stability theory are well known. Finally, if the system
performance is not according to our specifications, we resort to design
[25, 109]. In optimal control theory, the design is usually with respect
to a performance index. We notice that although the concepts such as
Lagrange function [85] and V function of Lyapunov [94] are old, the
techniques using those concepts are modern. Again, as the phrase modern
usually refers to time and what is modern today becomes ancient
after a few years, a more appropriate thing is to label them as optimal
control, nonlinear control, adaptive control, robust control and so on.
30. 4 Chapter 1: Introduction
I Modem Control System I
~
~r ~ r
System Dynamics System Analysis System Synthesis
(Modeling) (Perfonnance) (Design)
r r ~
State Function of V Function of H Function of
Lagrange Lyapunov Pontraygin
(1788) (1892) (1956)
Figure 1.3 Components of a Modern Control System
1.2 Optimization
Optimization is a very desirable feature in day-to-day life. We like to
work and use our time in an optimum manner, use resources optimally
and so on. The subject of optimization is quite general in the sense
that it can be viewed in different ways depending on the approach (algebraic
or geometric), the interest (single or multiple), the nature of the
signals (deterministic or stochastic), and the stage (single or multiple)
used in optimization. This is shown in Figure 1.4. As we notice that
the calculus of variations is one small area of the big picture of the optimization
field, and it forms the basis for our study of optimal control
systems. Further, optimization can be classified as static optimization
and dynamic optimization.
1. Static Optimization is concerned with controlling a plant under
steady state conditions, i.e., the system variables are not changing
with respect to time. The plant is then described by algebraic
equations. Techniques used are ordinary calculus, Lagrange multipliers,
linear and nonlinear programming.
2. Dynamic Optimization concerns with the optimal control of
plants under dynamic conditions, i.e., the system variables are
changing with respect to time and thus the time is involved in
system description. Then the plant is described by differential
31. 1.2 Optimization
Calculus and
Lagrange Multipliers
OPTIMIZATION
Algebraic Approach
Multiple Interest
Game Theory
Stochastic
Multiple Stage
Dynamic Programming
Linear and Nonlinear
Programming
Functional
Analysis
Figure 1.4 Overview of Optimization
5
32. 6 Chapter 1: Introduction
(or difference) equations. Techniques used are search techniques,
dynamic programming, variational calculus (or calculus of variations)
and Pontryagin principle.
1.3 Optimal Control
The main objective of optimal control is to determine control signals
that will cause a process (plant) to satisfy some physical constraints
and at the same time extremize (maximize or minimize) a chosen performance
criterion (performance index or cost function). Referring to
Figure 1.2, we are interested in finding the optimal control u*(t) (* indicates
optimal condition) that will drive the plant P from initial state
to final state with some constraints on controls and states and at the
same time extremizing the given performance index J.
The formulation of optimal control problem requires
1. a mathematical description (or model) of the process to be controlled
(generally in state variable form),
2. a specification of the performance index, and
3. a statement of boundary conditions and the physical constraints
on the states and/or controls.
1.3.1 Plant
For the purpose of optimization, we describe a physical plant by a set of
linear or nonlinear differential or difference equations. For example, a
linear time-invariant system is described by the state and output relations
(1.1.3) and (1.1.4) and a nonlinear system by (1.1.5) and (1.1.6).
1.3.2 Performance Index
Classical control design techniques have been successfully applied to linear,
time-invariant, single-input, single output (8180) systems. Typical
performance criteria are system time response to step or ramp input
characterized by rise time, settling time, peak overshoot, and steady
state accuracy; and the frequency response of the system characterized
by gain and phase margins, and bandwidth.
In modern control theory, the optimal control problem is to find a
control which causes the dynamical system to reach a target or fol-
33. 1.3 Optimal Control 7
low a state variable (or trajectory) and at the same time extremize a
performance index which may take several forms as described below.
1. Performance Index for Time-Optimal Control System:
We try to transfer a system from an arbitrary initial state x(to) to
a specified final state x( t f) in minimum time. The corresponding
performance index (PI) is
it!
J = dt = t f - to = t*.
to
(1.3.1 )
2. Performance Index for Fuel-Optimal Control System: Consider
a spacecraft problem. Let u(t) be the thrust of a rocket
engine and assume that the magnitude I u( t) I of the thrust is proportional
to the rate of fuel consumption. In order to minimize
the total expenditure of fuel, we may formulate the performance
index as
it!
J = lu(t)ldt.
to
(1.3.2)
For several controls, we may write it as
(1.3.3)
where R is a weighting factor.
3. Performance Index for Minimum-Energy Control System:
Consider Ui (t) as the current in the ith loop of an electric
network. Then 2:i!1 u;(t)ri (where, ri is the resistance of the ith
loop) is the total power or the total rate of energy expenditure of
the network. Then, for minimization of the total expended energy,
we have a performance criterion as
(1.3.4)
or in general,
it!
J = u'(t)Ru(t)dt
to
(1.3.5)
34. 8 Chapter 1: Introduction
where, R is a positive definite matrix and prime (') denotes transpose
here and throughout this book (see Appendix A for more
details on definite matrices).
Similarly, we can think of minimization of the integral of the
squared error of a tracking system. We then have,
it!
J = x/(t)Qx(t)dt
to
(1.3.6)
where, Xd(t) is the desired value, xa(t) is the actual value, and
x(t) = xa(t) - Xd(t), is the error. Here, Q is a weighting matrix,
which can be positive semi-definite.
4. Performance Index for Terminal Control System: In a terminal
target problem, we are interested in minimizing the error
between the desired target position Xd (t f) and the actual target
position Xa (t f) at the end of the maneuver or at the final time t f.
The terminal (final) error is x ( t f) = Xa ( t f) - Xd ( t f ). Taking care
of positive and negative values of error and weighting factors, we
structure the cost function as
(1.3.7)
which is also called the terminal cost function. Here, F is a positive
semi-definite matrix.
5. Performance Index for General Optimal Control System:
Combining the above formulations, we have a performance index
in general form as
it!
J = x/(tf)Fx(tf) + [X/(t)QX(t) + u/(t)Ru(t)]dt
to
(1.3.8)
or,
it!
J = S(x(tf),tf) + V(x(t),u(t),t)dt
to
(1.3.9)
where, R is a positive definite matrix, and Q and F are positive
semidefinite matrices, respectively. Note that the matrices Q and
R may be time varying. The particular form of performance index
(1.3.8) is called quadratic (in terms of the states and controls)
form.
35. 1.3 Optimal Control 9
The problems arising in optimal control are classified based on the
structure of the performance index J [67]. If the PI (1.3.9) contains
the terminal cost function S(x(t), u(t), t) only, it is called the Mayer
problem, if the PI (1.3.9) has only the integral cost term, it is called
the Lagrange problem, and the problem is of the Bolza type if the PI
contains both the terminal cost term and the integral cost term as in
(1.3.9). There are many other forms of cost functions depending on our
performance specifications. However, the above mentioned performance
indices (with quadratic forms) lead to some very elegant results in
optimal control systems.
1.3.3 Constraints
The control u( t) and state x( t) vectors are either unconstrained or
constrained depending upon the physical situation. The unconstrained
problem is less involved and gives rise to some elegant results. From the
physical considerations, often we have the controls and states, such as
currents and voltages in an electrical circuit, speed of a motor, thrust
of a rocket, constrained as
(1.3.10)
where, +, and - indicate the maximum and minimum values the variables
can attain.
1.3.4 Formal Statement of Optimal Control System
Let us now state formally the optimal control problem even risking repetition
of some of the previous equations. The optimal control problem
is to find the optimal control u*(t) (* indicates extremal or optimal
value) which causes the linear time-invariant plant (system)
x(t) = Ax(t) + Bu(t) (1.3.11)
to give the trajectory x* (t) that optimizes or extremizes (minimizes or
maximizes) a performance index
J = x'(tf)Fx(tf) + J.tJ
[x'(t)Qx(t) + u'(t)Ru(t)]dt (1.3.12)
to
or which causes the nonlinear system
x(t) = f(x(t), u(t), t) (1.3.13)
36. 10 Chapter 1: Introduction
to give the state x*(t) that optimizes the general performance index
itf
J = S(x(tj), tf) + V(x(t), u(t), t)dt
to
(1.3.14)
with some constraints on the control variables u( t) and/or the state
variables x(t) given by (1.3.10). The final time tf may be fixed, or free,
and the final (target) state may be fully or partially fixed or free. The
entire problem statement is also shown pictorially in Figure 1.5. Thus,
Optimal Control System I
+ .. •
Plant I Cost Function II Constraints
, ,
J J ~
.. ~ J* ... ~
J*
..
u*(t) u(t)
r
u*(t) u(t)
(a) Minimum (b) Maximum
Figure 1.5 Optimal Control Problem
we are basically interested in finding the control u*(t) which when
applied to the plant described by (1.3.11) or (1.3.13), gives an optimal
performance index J* described by (1.3.12) or (1.3.14).
The optimal control systems are studied in three stages.
1. In the first stage, we just consider the performance index of the
form (1.3.14) and use the well-known theory of calculus of variations
to obtain optimal functions.
2. In the second stage, we bring in the plant (1.3.11) and try to
address the problem of finding optimal control u*(t) which will
37. 1.4 Historical Tour 11
drive the plant and at the same time optimize the performance
index (1.3.12). Next, the above topics are presented in discretetime
domain.
3. Finally, the topic of constraints on the controls and states (1.3.10)
is considered along with the plant and performance index to obtain
optimal control.
1.4 Historical Tour
We basically consider two stages of the tour: first the development of
calculus of variations, and secondly, optimal control theory [134, 58,
99, 28]1.
1.4.1 Calculus oj Variations
According to a legend [88], Tyrian princess Dido used a rope made
of cowhide in the form of a circular arc to maximize the area to be
occupied to found Carthage. Although the story of the founding of
Carthage is fictitious, it probably inspired a new mathematical discipline,
the calculus of variations and its extensions such as optimal
control theory.
The calculus of variations is that branch of mathematics that deals
with finding a function which is an extremum (maximum or minimum)
of a functional. A functional is loosely defined as a function of a function.
The theory of finding maxima and minima of functions is quite
old and can be traced back to the isoperimetric problems considered
by Greek mathematicians such as Zenodorus (495-435 B.C.) and by
Poppus (c. 300 A.D.). But we will start with the works of Bernoulli. In
1699 Johannes Bernoulli (1667-1748) posed the brachistochrone problem:
the problem of finding the path of quickest descent between two
points not in the same horizontal or vertical line. This problem which
was first posed by Galileo (1564-1642) in 1638, was solved by John,
his brother Jacob (1654- 1705), by Gottfried Leibniz (1646-1716), and
anonymously by Isaac Newton (1642-1727). Leonard Euler (1707-1783)
joined John Bernoulli and made some remarkable contributions, which
influenced Joseph-Louis Lagrange (1736-1813), who finally gave an el-
IThe permission given by Springer-Verlag for H. H. Goldstine, A History of the Calculus
of Variations, Springer-Verlag, New York, NY, 1980, is hereby acknowledged.
38. 12 Chapter 1: Introduction
egant way of solving these types of problems by using the method
of (first) variations. This led Euler to coin the phrase calculus of variations.
Later this necessary condition for extrema of a functional
was called the Euler - the Lagrange equation. Lagrange went on to
treat variable end - point problems introducing the multiplier method,
which later became one of the most powerful tools-Lagrange (or EulerLagrange)
multiplier method-in optimization.
The sufficient conditions for finding the extrema of functionals in calculus
of variations was given by Andrien Marie Legendre (1752-1833)
in 1786 by considering additionally the second variation. Carl Gustav
Jacob Jacobi (1804-1851) in 1836 came up with a more rigorous analysis
of the sufficient conditions. This sufficient condition was later on
termed as the Legendre-Jacobi condition. At about the same time Sir
William Rowan Hamilton (1788-1856) did some remarkable work on
mechanics, by showing that the motion of a particle in space, acted
upon by various external forces, could be represented by a single function
which satisfies two first-order partial differential equations. In 1838
Jacobi had some objections to this work and showed the need for only
one partial differential equation. This equation, called Hamilton-Jacobi
equation, later had profound influence on the calculus of variations and
dynamic programming, optimal control, and as well as on mechanics.
The distinction between strong and weak extrema was addressed by
Karl Weierstrass (1815-1897) who came up with the idea of the field
of extremals and gave the Weierstrass condition, and sufficient conditions
for weak and strong extrema. Rudolph Clebsch (1833-1872) and
Adolph Mayer proceeded with establishing conditions for the more general
class of problems. Clebsch formulated a problem in the calculus of
variations by adjoining the constraint conditions in the form of differential
equations and provided a condition based on second variation.
In 1868 Mayer reconsidered Clebsch's work and gave some elegant results
for the general problem in the calculus of variations. Later Mayer
described in detail the problems: the problem of Lagrange in 1878, and
the problem of Mayer in 1895.
In 1898, Adolf Kneser gave a new approach to the calculus of variations
by using the result of Karl Gauss (1777-1855) on geodesics. For
variable end-point problems, he established the transversality condition
which includes orthogonality as a special case. He along with
Oskar Bolza (1857-1942) gave sufficiency proofs for these problems.
In 1900, David Hilbert (1862-1943) showed the second variation as a
39. 1.4 Historical Tour 13
quadratic functional with eigenvalues and eigenfunctions. Between 1908
and 1910, Gilbert Bliss (1876-1951) [23] and Max Mason looked in
depth at the results of Kneser. In 1913, Bolza formulated the problem
of Bolza as a generalization of the problems of Lagrange and Mayer.
Bliss showed that these three problems are equivalent. Other notable
contributions to calculus of variations were made by E. J. McShane
(1904-1989) [98], M. R. Hestenes [65], H. H. Goldstine and others.
There have been a large number of books on the subject of calculus
of variations: Bliss (1946) [23], Cicala (1957) [37], Akhiezer (1962) [1],
Elsgolts (1962) [47], Gelfand and Fomin (1963) [55], Dreyfus (1966)
[45], Forray (1968) [50], Balakrishnan (1969) [8], Young (1969) [146],
Elsgolts (1970) [46], Bolza (1973) [26], Smith (1974) [126], Weinstock
(1974) [143], Krasnov et al. (1975) [81], Leitmann (1981) [88], Ewing
(1985) [48], Kamien and Schwartz (1991) [78], Gregory and Lin
(1992) [61], Sagan (1992) [118], Pinch (1993) [108], Wan (1994) [141],
Giaquinta and Hildebrandt (1995) [56, 57], Troutman (1996) [136], and
Milyutin and Osmolovskii (1998) [103].
1.4.2 Optimal Control Theory
The linear quadratic control problem has its origins in the celebrated
work of N. Wiener on mean-square filtering for weapon fire control during
World War II (1940-45) [144, 145]. Wiener solved the problem of
designing filters that minimize a mean-square-error criterion (performance
measure) of the form
(1.4.1)
where, e( t) is the error, and E {x} represents the expected value of the
random variable x. For a deterministic case, the above error criterion
is generalized as an integral quadratic term as
J = 1000
e'(t)Qe(t)dt (1.4.2)
where, Q is some positive definite matrix. R. Bellman in 1957 [12]
introduced the technique of dynamic programming to solve discretetime
optimal control problems. But, the most important contribution
to optimal control systems was made in 1956 [25] by L. S. Pontryagin
(formerly of the United Soviet Socialistic Republic (USSR)) and his associates,
in development of his celebrated maximum principle described
40. 14 Chapter 1: Introduction
in detail in their book [109]. Also, see a very interesting article on the
"discovery of the Maximum Principle" by R. V. Gamkrelidze [52], one
of the authors of the original book [109]. At this time in the United
States, R. E. Kalman in 1960 [70] provided linear quadratic regulator
(LQR) and linear quadratic Gaussian (LQG) theory to design optimal
feedback controls. He went on to present optimal filtering and estimation
theory leading to his famous discrete Kalman filter [71] and the
continuous Kalman filter with Bucy [76]. Kalman had a profound effect
on optimal control theory and the Kalman filter is one of the most
widely used technique in applications of control theory to real world
problems in a variety of fields.
At this point we have to mention the matrix Riccati equation that
appears in all the Kalman filtering techniques and many other fields.
C. J. Riccati [114, 22] published his result in 1724 on the solution for
some types of nonlinear differential equations, without ever knowing
that the Riccati equation would become so famous after more than two
centuries!
Thus, optimal control, having its roots in calculus of variations developed
during 16th and 17th centuries was really born over 300 years
ago [132]. For additional details about the historical perspectives on
calculus of variations and optimal control, the reader is referred to some
excellent publications [58, 99, 28, 21, 132].
In the so-called linear quadratic control, the term "linear" refers to
the plant being linear and the term "quadratic" refers to the performance
index that involves the square or quadratic of an error, and/or
control. Originally, this problem was called the mean-square control
problem and the term "linear quadratic" did not appear in the literature
until the late 1950s.
Basically the classical control theory using frequency domain deals
with single input and single output (SIS0) systems, whereas modern
control theory works with time domain for SISO and multi-input and
multi-output (MIMO) systems. Although modern control and hence
optimal control appeared to be very attractive, it lacked a very important
feature of robustness. That is, controllers designed based on LQR
theory failed to be robust to measurement noise, external disturbances
and unmodeled dynamics. On the other hand, frequency domain techniques
using the ideas of gain margin and phase margin offer robustness
in a natural way. Thus, some researchers [115, 95], especially in the
United Kingdom, continued to work on developing frequency domain
41. 1.5 About This Book 15
approaches to MIMO systems.
One important and relevant field that has been developed around
the 1980s is the Hoo-optimal control theory. In this framework, the
work developed in the 1960s and 1970s is labeled as H2-optimal control
theory. The seeds for Hoo-optimal control theory were laid by G. Zames
[148], who formulated the optimal Hoo-sensitivity design problem for
SISO systems and solved using optimal N evanilina-Pick interpolation
theory. An important publication in this field came from a group of four
active researchers, Doyle, Glover, Khargonekar, and Francis[44], who
won the 1991 W. R. G. Baker Award as the best IEEE Transactions
paper. There are many other works in the field of Hoo control ([51, 96,
43, 128, 7, 60, 131, 150, 39, 34]).
1.5 About This Book
This book, on the subject of optimal control systems, is based on the
author's lecture notes used for teaching a graduate level course on this
subject. In particular, this author was most influenced by Athans and
Falb [6], Schultz and Melsa [121], Sage [119], Kirk [79], Sage and White
[120], Anderson and Moore [3] and Lewis and Syrmos [91], and one
finds the footprints of these works in the present book.
There were a good number of books on optimal control published
during the era of the "glory of modern control," (Leitmann (1964) [87],
Tou (1964) [135], Athans and Falb (1966) [6], Dreyfus (1966) [45], Lee
and Markus (1967) [86], Petrov (1968) [106], Sage (1968) [119], Citron
(1969) [38], Luenberger (1969) [93], Pierre (1969) [107], Pun (1969)
[110], Young (1969) [146], Kirk (1970) [79], Boltyanskii [24], Kwakernaak
and Sivan (1972) [84], Warga (1972) [142], Berkovitz (1974) [17],
Bryson and Ho (1975) [30]), Sage and White (1977) [120], Leitmann
(1981) [88]), Ryan (1982) [116]). There has been renewed interest with
the second wave of books published during the last few years (Lewis
(1986) [89], Stengal (1986) [127], Christensen et al. (1987) [36] Anderson
and Moore (1990) [3], Hocking (1991) [66], Teo et al. (1991) [133],
Gregory and Lin (1992) [61], Lewis (1992) [90], Pinch (1993) [108], Dorato
et al. (1995) [42], Lewis and Syrmos (1995) [91]), Saberi et al.
(1995) [117], Sima (1996) [124], Siouris [125], Troutman (1996) [136]
Bardi and Dolcetta (1997) [9], Vincent and Grantham (1997) [139],
Milyutin and Osmolovskii (1998) [103], Bryson (1999) [29], Burl [32],
Kolosov (1999) [80], Pytlak (1999) [111], Vinter (2000) [140], Zelikin
43. 1.7 Problems 17
1. 7 Problems
Problem 1.1 A D.C. motor speed control system is described by a
second order state equation,
:h (t) = 25x2(t)
X2(t) = -400Xl(t) - 200X2(t) + 400u(t) ,
where, Xl(t) = the speed of the motor, and X2(t) = the current in
the armature circuit and the control input u( t) = the voltage input
to an amplifier supplying the motor. Formulate a performance index
and optimal control problem to keep the speed constant at a particular
value.
Problem 1.2 [83] In a liquid-level control system for a storage tank,
the valves connecting a reservoir and the tank are controlled by gear
train driven by a D. C. motor and an electronic amplifier. The dynamics
is described by a third order system
Xl(t) = -2Xl(t)
X2(t) = X3(t)
X3(t) = -10X3(t) + 9000u(t)
where, Xl(t) = is the height in the tank, X2(t) = is the angular position
of the electric motor driving the valves controlling the liquid from
reservoir to tank, X3(t) = the angular velocity of the motor, and u(t) =
is the input to electronic amplifier connected to the input of the motor.
Formulate optimal control problem to keep the liquid level constant at
a reference value and the system to act only if there is a change in the
liquid level.
Problem 1.3 [35] In an inverted pendulum system, it is required to
maintain the upright position of the pendulum on a cart. The linearized
state equations are
Xl(t) = X2(t)
X2(t) = -X3(t) + O.2u(t)
X3(t) = X4(t)
X4(t) = 10x3(t) - O.2u(t)
44. 18 Chapter 1: Introduction
where, Xl (t) = is horizontal linear displacement of the cart, X2(t) = is
linear velocity of the cart, X3(t) = is angular position of the pendulum
from vertical line, X4(t) = is angular velocity, and u(t) = is the horizontal
force applied to the cart. Formulate a performance index to keep
the pendulum in the vertical position with as little energy as possible.
Problem 1.4 [101J A mechanical system consisting of two masses and
two springs, one spring connecting the two masses and the other spring
connecting one of the masses to a fixed point. An input is applied to
the mass not connected to the fixed point. The displacements (XI(t)
and X2 (t)) and the corresponding velocities (X3 (t) and X4 (t)) of the two
masses provide a fourth-order system described by
XI(t) = X3(t)
X2(t) = X4(t)
X3(t) = -4XI(t) + 2X2(t)
X4(t) = XI(t) - X2(t) + u(t)
Formulate a performance index to minimize the errors in displacements
and velocities and to minimize the control effort.
Problem 1.5 A simplified model of an automobile suspension system
is described by
mx(t) + kx(t) = bu(t)
where, x(t) is the position, u(t) is the input to the suspension system
(in the form of an upward force), m is the mass of the suspension
system, and k is the spring constant. Formulate the optimal control
problem for minimum control energy and passenger comfort. Assume
suitable values for all the constants.
Problem 1.6 [112J Consider a continuous stirred tank chemical reactor
described by
XI(t) = -O.lXI(t) - 0.12x2(t)
X2(t) = -0.3XI(t) - 0.012x2(t) - 0.07u(t)
where, the normalized deviation state variables of the linearized model
are Xl (t) = reaction variable, X2 (t) = temperature and the control
variable u(t) = effective cooling rate coefficient. Formulate a suitable
performance measure to minimize the deviation errors and to minimize
the control effort.
45. Chapter 2
Calculus of Variations
and Optimal Control
Calculus of variations (Co V) or variational calculus deals with finding
the optimum (maximum or minimum) value of a functional. Variational
calculus that originated around 1696 became an independent
mathematical discipline after the fundamental discoveries of L. Euler
(1709-1783), whom we can claim with good reason as the founder of
calculus of variations.
In this chapter, we start with some basic definitions and a simple
variational problem of extremizing a functional. We then incorporate
the plant as a conditional optimization problem and discuss various
types of problems based on the boundary conditions. We briefly mention
both the Lagrangian and Hamiltonian formalisms for optimization.
It is suggested that the student reviews the material in Appendices A
and B given at the end of the book. This chapter is motivated by
[47, 79, 46, 143, 81, 48]1.
2.1 Basic Concepts
2.1.1 Function and Functional
We discuss some fundamental concepts associated with functionals along
side with those of functions.
(a) Function: A variable x is a function of a variable quantity t, (writ-
IThe permission given by Prentice Hall for D. E. Kirk, Optimal Control Theory: An Introduction,
Prentice Hall, Englewood Cliffs, NJ, 1970, is hereby acknowledged.
19
46. 20 Chapter 2: Calculus of Variations and Optimal Control
ten as x(t) = !(t)), if to every value of t over a certain range of t there
corresponds a value x; i.e., we have a correspondence: to a number t
there corresponds a number x. Note that here t need not be always
time but any independent variable.
Example 2.1
Consider
x(t) = 2t2 + 1. (2.1.1 )
For t = 1, x = 3, t = 2, x = 9 and so on. Other functions are
x(t) = 2t; X(tb t2) = tt + t§.
N ext we consider the definition of a functional based on that of a
function.
(b) Functional: A variable quantity J is a functional dependent on a
function ! (x), written as J = J (f (x) ), if to each function f (x), there
corresponds a value J, i.e., we have a correspondence: to the function
f (x) there corresponds a number J. Functional depends on several
functions.
Example 2.2
Let x(t) = 2t2 + 1. Then
{I (I 2 5
J(x(t)) = io x(t)dt = io (2t2 + l)dt = 3 + 1 = 3 (2.1.2)
is the area under the curve x(t). If v(t) is the velocity of a vehicle,
then
l ti
J ( v ( t )) = v ( t ) dt
to
(2.1.3)
is the path traversed by the vehicle. Thus, here x(t) and v(t) are
functions of t, and J is a functional of x(t) or v(t).
Loosely speaking, a functional can be thought of as a "function of a
function."
2.1.2 Increment
We consider here increment of a function and a functional.
47. 2.1 Basic Concepts 21
(a) Increment of a Function: In order to consider optimal values
of a function, we need the definition of an increment [47, 46, 79].
DEFINITION 2.1 The increment of the function I, denoted by ~/, is
defined as
~/~/(t + ~t) - I(t). (2.1.4)
It is easy to see from the definition that ~I depends on both the
independent variable t and the increment of the independent variable
~t, and hence strictly speaking, we need to write the increment of a
function as ~/(t, ~t).
Example 2.3
If
find the increment of the function I ( t) .
Solution: The increment ~I becomes
~I ~ I(t + ~t) - I(t)
= (tl + ~iI + t2 + ~t2? - (tl + t2)2
= (tl + ~tl)2 + (t2 + ~t2)2 + 2(iI + ~h)(t2 + ~t2) -
(tI + t§ + 2tlt2)
= 2(tl + t2)~tl + 2(tl + t2)~t2 + (~tl)2 + (~t2)2
(2.1.5)
+2~tl~t2. (2.1.6)
(b) Increment of a Functional: Now we are ready to define the
increment of a functional.
DEFINITION 2.2 The increment of the functional J, denoted by ~J, is
defined as
I ~J~J(x(t) + 8x(t)) - J(x(t))·1 (2.1. 7)
Here 8x(t) is called the variation of the function x(t). Since the increment
of a functional is dependent upon the function x(t) and its
48. 22 Chapter 2: Calculus of Variations and Optimal Control
variation 8x(t), strictly speaking, we need to write the increment as
ilJ(x(t),8x(t)).
Example 2.4
Find the increment of the functional
(2.1.8)
Solution: The increment of J is given by
ilJ ~ J(x(t) + 8x(t)) - J(x(t)),
= it! [2(x(t) + 8x(t))2 + 1] dt _it! [2x2(t) + 1] dt,
it! to to
= [4x(t)8x(t) + 2(8x(t) )2] dt. (2.1.9)
to
2.1.3 Differential and Variation
Here, we consider the differential of a function and the variation of a
functional.
(a) Differential of a Function: Let us define at a point t* the
increment of the function J as
ilf~J(t* + ilt) - J(t*). (2.1.10)
By expanding J (t* + ilt) in a Taylor series about t*, we get
Af = f(t') + (:), At + :, (~n, (At)2 + ... - f(t*). (2.1.11)
Neglecting the higher order terms in ilt,
Af = (:) * At = j(t*)At = df. (2.1.12)
Here, df is called the differential of J at the point t*. j(t*) is the
derivative or slope of J at t*. In other words, the differential dJ is
the first order approximation to increment ilt. Figure 2.1 shows the
relation between increment, differential and derivative.
49. 2.1 Basic Concepts 23
f(t)
f(t* +~t) ....... '. . .. ... . ......... '
[(to) ~:~~~ ~ [ ... :. : ::t ~ .. .1~.
. ~t
:~ .
o t* t*+~t t
Figure 2.1 Increment fl.j, Differential dj, and Derivative j of a
Function j ( t)
Example 2.5
Let j(t) = t2 + 2t. Find the increment and the derivative of the
function j ( t).
Solution: By definition, the increment fl.j is
fl.j £ j(t + fl.t) - j(t),
= (t + fl.t)2 + 2(t + fl.t) - (t2 + 2t),
= 2tfl.t + 2fl.t + ... + higher order terms,
= 2(t + l)fl.t,
= j(t)fl.t.
Here, j(t) = 2(t + 1).
(2.1.13)
(b) Variation of a Functional: Consider the increment of a functional
fl.J£J(x(t) + 8x(t)) - J(x(t)). (2.1.14)
50. 24 Cbapter 2: Calculus of Variations and Optimal Control
Expanding J(x(t) + 8x(t)) in a Taylor series, we get
{)J 1 {)2 J
jj.J = J(x(t)) + -{) 8x(t) + -, {) 2 (8x(t))2 + ... - J(x(t))
x 2. x
{)J 1 {)2J 2
= {)x 8x(t) + 2! {)x2 (8x(t)) + ...
= 8 J + 82 J + ... , (2.1.15)
where,
{)J
8J = {)x 8x(t) and (2.1.16)
are called the first variation (or simply the variation) and the second
variation of the functional J, respectively. The variation 8 J of a functional
J is the linear (or first order approximate) part (in 8x(t)) of the
increment jj.J. Figure 2.2 shows the relation between increment and
the first variation of a functional.
J(x(t»
J(x*(t)+Ox(t» . . . . . . . . . .. ... . ......... ,
. J(x*(t» ... .:. : J ~~ .. .1~.
:.. ~
: ox(t):
o x*(t) x*(t)+ Ox(t) x(t)
Figure 2.2 Increment jj.J and the First Variation 8J of the
Functional J
51. 2.2 Optimum of a Function and a Functional 25
Example 2.6
Given the functional
it!
J(x(t)) = [2x2(t) + 3x(t) + 4]dt,
to
(2.1.17)
evaluate the variation of the functional.
Solution: First, we form the increment and then extract the variation
as the first order approximation. Thus
~J ~ J(x(t) + 8x(t)) - J(x(t)),
it!
= [2(x(t) + 8x(t))2 + 3(x(t) + 8x(t)) + 4)
to
-(2x2(t) + 3x(t) + 4)] dt,
it!
= [4x(t)8x(t) + 2(8x(t))2 + 38x(t)] dt.
to
(2.1.18)
Considering only the first order terms, we get the (first) variation
as
it!
8J(x(t),8x(t)) = (4x(t) + 3)8x(t)dt.
to
(2.1.19)
2.2 Optimum of a Function and a Functional
We give some definitions for optimum or extremum (maximum or minimum)
of a function and a functional [47, 46, 79]. The variation plays
the same role in determining optimal value of a functional as the differential
does in finding extremal or optimal value of a function.
DEFINITION 2.3 Optimum of a Function: A function f (t) is said
to have a relative optimum at the point t* if there is a positive parameter E
such that for all points t in a domain V that satisfy It - t* I < E, the increment
of f(t) has the same sign (positive or negative).
In other words, if
~f = f(t) - f(t*) 2:: 0, (2.2.1)
52. 26 Chapter 2: Calculus of Variations and Optimal Control
f(t f(t) v
0
t* t
f(t)
0
t
o t* t
(a)
m
0 t* t
f(t)
0
t
t*
o~-------.- ----.t
... ~
;.~~., ~ f(t*)<0
(b)
Figure 2.3 (a) Minimum and (b) Maximum of a Function f (t)
then, f(t*) is a relative local minimum. On the other hand, if
b.f = f(t) - f(t*) ~ 0, (2.2.2)
then, f (t*) is a relative local maximum. If the previous relations are
valid for arbitrarily large E, then, f(t*) is said to have a global absolute
optimum. Figure 2.3 illustrates the (a) minimum and (b) maximum of
a function.
It is well known that the necessary condition for optimum of a function
is that the (first) differential vanishes, i.e., df = O. The sufficient
condition
53. 2.3 The Basic Variational Problem
1. for minimum is that the second differential is positive,
i.e., d2 f > 0, and
2. for maximum is that the second differential is negative,
i.e., d2 f < 0.
If d2 f = 0, it corresponds to a stationary (or inflection) point.
27
DEFINITION 2.4 Optimum of a Functional: A functional J is
said to have a relative optimum at x* if there is a positive E such that for all
functions x in a domain n which satisfy Ix - x* I < E, the increment of J has
the same sign.
In other words, if
!1J = J(x) - J(x*) ~ 0, (2.2.3)
then J(x*) is a relative minimum. On the other hand, if
!1J = J(x) - J(x*) ~ 0, (2.2.4)
then, J(x*) is a relative maximum. If the above relations are satisfied
for arbitrarily large E, then, J(x*) is a global absolute optimum.
Analogous to finding extremum or optimal values for functions, in
variational problems concerning functionals, the result is that the variation
must be zero on, an optimal curve. Let us now state the result in
the form of a theorem, known as fundamental theorem of the calculus
of variations, the proof of which can be found in any book on calculus
of variations [47, 46, 79].
THEOREM 2.1
For x*(t) to be a candidate for an optimum, the (first) variation of J must
be zero on x*(t), i.e., 6J(x*(t), 6x(t)) = ° for all admissible values of 6x(t).
This is a necessary condition. As a sufficient condition for minimum, the
second variation 62J > 0, and for maximum 62J < 0.
2.3 The Basic Variational Problem
2.3.1 Fixed-End Time and Fixed-End State System
We address a fixed-end time and fixed-end state problem, where both
the initial time and state and the final time and state are fixed or given
54. 28 Chapter 2: Calculus of Variations and Optimal Control
a priori. Let x(t) be a scalar function with continuous first derivatives
and the vector case can be similarly dealt with. The problem is to find
the optimal function x* (t) for which the functional
it!
J(x(t)) = V(x(t), x(t), t)dt
to
(2.3.1)
has a relative optimum. It is assumed that the integrand V has continuous
first and second partial derivatives w.r.t. all its arguments; to
and t f are fixed (or given a priori) and the end points are fixed, i.e.,
x(t = to) = Xo; x(t = tf) = xf' (2.3.2)
We already know from Theorem 2.1 that the necessary condition for
an optimum is that the variation of a functional vanishes. Hence, in
our attempt to find the optimum of x(t), we first define the increment
for J, obtain its variation and finally apply the fundamental theorem
of the calculus of variations (Theorem 2.1).
Thus, the various steps involved in finding the optimal solution to
the fixed-end time and fixed-end state system are first listed and then
discussed in detail.
• Step 1: Assumption of an Optimum
• Step 2: Variations and Increment
• Step 3: First Variation
• Step 4: Fundamental Theorem
• Step 5: Fundamental Lemma
• Step 6: Euler-Lagrange Equation
• Step 1: Assumption of an Optimum: Let us assume that x*(t) is
the optimum attained for the function x(t). Take some admissible
function xa(t) = x*(t) + 8x(t) close to x*(t), where 8x(t) is the
variation of x*(t) as shown in Figure 2.4. The function xa(t)
should also satisfy the boundary conditions (2.3.2) and hence it
is necessary that
(2.3.3)
55. 2.3 The Basic Variational Problem 29
x(t)
xo ..... .
o
Figure 2.4 Fixed-End Time and Fixed-End State System
• Step 2: Variations and Increment: Let us first define the increment
as
6.J(x*(t), 8x(t)) ~ J(x*(t) + 8x(t), x*(t) + 8x(t), t)
-J(x*(t), x*(t), t)
It!
= V (x*(t) + 8x(t), x*(t) + 8x(t), t) dt
to
It!
- V(x*(t), x*(t), t)dt.
to
(2.3.4)
which by combining the integrals can be written as
It!
6.J(x*(t), 8x(t)) = [V (x*(t) + 8x(t), x*(t) + 8x(t), t)
to
- V(x* (t), x*(t), t)] dt. (2.3.5)
where,
x(t) = d:~t) and 8x(t) = :t {8x(t)} (2.3.6)
Expanding V in the increment (2.3.5) in a Taylor series about
the point x*(t) and x*(t), the increment 6.J becomes (note the
56. 30 Chapter 2: Calculus of Variations and Optimal Control
cancelation of V(x*(t), x*(t), t))
= l' [8V(X*(~~X*(t), t) 6x(t) + 8V(X*(~~ x*(t), t) 6x(t)
~J = ~J(x*(t), 8x(t))
~ {82V( ... ) (8 ())2 82 + V( ... ) (8· ( ))2 2! 8x2 x t + 8x2 X t +
+ 2~:~~·) 6x (t)6x (t) } + .. -] dt. (2.3.7)
Here, the partial derivatives are w.r.t. x(t) and x(t) at the optimal
condition (*) and * is omitted for simplicity .
• Step 3: First Variation: Now, we obtain the variation by retaining
the terms that are linear in 8x(t) and 8x(t) as
8J(x*(t),8x(t)) = it! [8V(X*(t), x*(t), t) 8x(t)
to 8x
8V(x*(t), x*(t), t)8· ( )] d + 8x x t t. (2.3.8)
To express the relation for the first variation (2.3.8) entirely in
terms containing 8x(t) (since 8x(t) is dependent on 8x(t)), we
integrate by parts the term involving 8x(t) as (omitting the arguments
in V for simplicity)
1:' (~~) * 6x(t)dt = 1:' (~~) * ! (6x(t))dt
= 1:' (~~) * d(6x(t)),
= [( ~~) * 6X(t{: _it! 8x(t)~ (8~) dt.
to dt 8x *
(2.3.9)
In the above, we used the well-known integration formula J udv =
uv - J vdu where u = 8V/8X and v = 8x(t)). Using (2.3.9), the
57. 2.3 The Basic Variational Problem 31
relation (2.3.8) for first variation becomes
8J(x*(t),6x(t)) = {' (~:) * 6x(t)dt + [( ~~) * 6X(t)[
_ rtf !i (a~) 8x(t)dt,
lto dt ax *
= rtf [(av) _!i (a~) ]8x(t)dt lto ax * dt ax *
+ [( ~~) * 6x(t)] I:: . (2.3.10)
Using the relation (2.3.3) for boundary variations in (2.3.10), we
get
8J(x*(t),6x(t)) = 1:' [( ~:) * - :t (~~) .l6X(t)dt. (2.3.11)
• Step 4: Fundamental Theorem: We now apply the fundamental
theorem of the calculus of variations (Theorem 2.1), i.e., the variation
of J must vanish for an optimum. That is, for the optimum
x*(t) to exist, 8J(x*(t),8x(t)) = O. Thus the relation (2.3.11)
becomes
rtf [(av) _!i (a~) ]8X(t)dt = O. lto ax * dt ax *
(2.3.12)
Note that the function 8x(t) must be zero at to and tf, but for
this, it is completely arbitrary .
• Step 5: Fundamental Lemma: To simplify the condition obtained
in the equation (2.3.12), let us take advantage of the following
lemma called the fundamental lemma of the calculus of
variations [47, 46, 79].
LEMMA 2.1
If for every function g(t) which is continuous,
ltf
g(t)8x(t)dt = 0
to
(2.3.13)
58. 32 Chapter 2: Calculus of Variations and Optimal Control
where the function 8x(t) is continuous in the interval [to, tf]' then the
function 9 ( t) must be zero everywhere throughout the interval [to, t f] .
(see Figure 2.5.)
Proof: We prove this by contradiction. Let us assume that g(t) is
nonzero (positive or negative) during a short interval [ta, tb]. Next, let
us select 8x(t), which is arbitrary, to be positive (or negative) throughout
the interval where 9 ( t) has a nonzero value. By this selection
of 8x(t), the value of the integral in (2.3.13) will be nonzero. This
contradicts our assumption that g( t) is non-zero during the interval.
Thus g( t) must be identically zero everywhere during the entire interval
[to, tf] in (2.3.13). Hence the lemma.
get)
t
8x(t)
Figure 2.5 A Nonzero g(t) and an Arbitrary 8x(t)
• Step 6: Euler-Lagrange Equation: Applying the previous lemma
to (2.3.12), a necessary condition for x*(t) to be an optimal of
the functional J given by (2.3.1) is
(
av(x*(t),x*(t),t)) _ ~ (av(x*(t),.x*(t),t)) = 0 (2.3.14)
ax * dt ax *
59. 2.3 The Basic Variational Problem 33
or in simplified notation omitting the arguments in V,
(aV) _!i (aV) = 0 ax * dt ax *
(2.3.15)
for all t E [to, tf]. This equation is called Euler equation, first
published in 1741 [126].
A historical note is worthy of mention.
Euler obtained the equation (2.3.14) in 1741 using an elaborate
and cumbersome procedure. Lagrange studied Euler's
results and wrote a letter to Euler in 1755 in which he obtained
the previous equation by a more elegant method of
"variations" as described above. Euler recognized the sim-plicity
and generality of the method of Lagrange and introduced
the name calculus of variations. The all important
fundamental equation (2.3.14) is now generally known as
Euler-Lagrange (E.-L') equation after these two great mathematicians
of the 18th century. Lagrange worked further
on optimization and came up with the well-known Lagrange
multiplier rule or method.
2.3.2 Discussion on Euler-Lagrange Equation
We provide some comments on the Euler-Lagrange equation [47,46].
1. The Euler-Lagrange equation (2.3.14) can be written in many
different forms. Thus (2.3.14) becomes
d
V - - (V·) = 0 x dt x (2.3.16)
where,
Vx = ~: = Vx(x*(t), ±*(t), t); Vi; = ~~ = Vi;(X*(t), x*(t), t).
(2.3.17)
Since V is a function of three arguments x*(t), x*(t), and t, and
60. 34 Chapter 2: Calculus of Variations and Optimal Control
that x*(t) and x*(t) are in turn functions of t, we get
Combining (2.3.16) and (2.3.18), we get an alternate form for
the EL equation as
(2.3.19)
2. The presence of -it and/or x*(t) in the EL equation (2.3.14) means
that it is a differential equation.
3. In the EL equation (2.3.14), the term aV(x*(~i:x*(t),t) is in general
a function of x*(t), x*(t), and t. Thus when this function is
differentiated w.r.t. t, x*(t) may be present. This means that the
differential equation (2.3.14) is in general of second order. This is
also evident from the alternate form (2.3.19) for the EL equation.
4. There may also be terms involving products or powers of x* (t),
x*(t), and x*(t), in which case, the differential equation becomes
nonlinear.
5. The explicit presence of t in the arguments indicates that the
coefficients may be time-varying.
6. The conditions at initial point t = to and final point t = t f leads
to a boundary value problem.
7. Thus, the Euler-Lagrange equation (2.3.14) is, in general, a nonlinear,
time-varying, two-point boundary value, second order, ordinary
differential equation. Thus, we often have a nonlinear
two-point boundary value problem (TPBVP). The solution of the
nonlinear TPBVP is quite a formidable task and often done using
numerical techniques. This is the price we pay for demanding
optimal performance!
61. 2.3 The Basic Variational Problem 35
8. Compliance with the Euler-Lagrange equation is only a necessary
condition for the optimum. Optimal may sometimes not yield
either a maximum or a minimum; just as inflection points where
the derivative vanishes in differential calculus. However, if the
Euler-Lagrange equation is not satisfied for any function, this
indicates that the optimum does not exist for that functional.
2.3.3 Different Cases for Euler-Lagrange Equation
We now discuss various cases of the EL equation.
Case 1: V is dependent of x(t), and t. That is, V = V(x(t), t). Then
Vx = O. The Euler-Lagrange equation (2.3.16) becomes
This leads us to
d
dt (Vx) = o.
' = oV(x*(t), t) = C
Vx
ox
where, C is a constant of integration.
(2.3.20)
(2.3.21)
Case 2: V is dependent of x(t) only. That is, V = V(x(t)). Then
Vx = O. The Euler-Lagrange equation (2.3.16) becomes
d
dt (Vx) = 0 ~ Vx = C. (2.3.22)
In general, the solution of either (2.3.21) or (2.3.22) becomes
(2.3.23)
This is simply an equation of a straight line.
Case 3: V is dependent of x(t) and x(t). That is, V = V(x(t), x(t)).
Then vtx = O. Using the other form of the Euler-Lagrange equation
(2.3.19), we get
Vx - Vxxx*(t) - Vxxx*(t) = O. (2.3.24)
Multiplying the previous equation by x*(t), we have
x*(t) [Vx - Vxxx*(t) - Vxxx*(t)] = o. (2.3.25)
This can be rewritten as ! (V - x*(t)Vx) = 0 ~ V - x*(t)Vx = C. (2.3.26)
62. 36 Chapter 2: Calculus of Variations and Optimal Control
The previous equation can be solved using any of the techniques such
as, separation of variables.
Case 4: V is dependent of x(t), and t, i.e., V = V(x(t), t). Then,
Vi; = 0 and the Euler-Lagrange equation (2.3.16) becomes
8V(x*(t), t) = 0 ax . (2.3.27)
The solution of this equation does not contain any arbitrary constants
and therefore generally speaking does not satisfy the boundary conditions
x(to) and x(tf). Hence, in general, no solution exists for this
variational problem. Only in rare cases, when the function x(t) satisfies
the given boundary conditions x(to) and x(tf), it becomes an optimal
function.
Let us now illustrate the application of the EL equation with a very
simple classic example of finding the shortest distance between two
points. Often, we omit the * (which indicates an optimal or extremal
value) during the working of a problem and attach the same to the final
solution.
Example 2.7
Find the minimum length between any two points.
Solution: It is well known that the solution to this problem is a
straight line. However, we like to illustrate the application of EulerLagrange
equation for this simple case. Consider the arc between
two points A and B as shown in Figure 2.6. Let ds be the small arc
length, and dx and dt are the small rectangular coordinate values.
Note that t is the independent variable representing distance and
not time. Then,
(2.3.28)
Rewriting
ds = VI + x2(t)dt, where x(t) = ~~. (2.3.29)
N ow the total arc length S between two points x (t = to) and x (t =
t f) is the performance index J to be minimized. Thus,
S = J = J ds = rtf VI + x2 (t)dt = rtf V(x(t))dt (2.3.30) Jto Jto
63. 2.3 The Basic Variational Problem 37
x(t)
xo ... 'A::
o
Figure 2.6 Arc Length
where, V(x(t)) = Jl + x2 (t). Note that V is a function of x(t)
only. Applying the Euler-Lagrange equation (2.3.22) to the performance
index (2.3.30), we get
x*(t) _ C VI + X*2(t) - .
Solving this equation, we get the optimal solution as
x*(t) = C1t + C2 .
(2.3.31)
(2.3.32)
This is evidently an equation for a straight line and the constants
C1 and C2 are evaluated from the given boundary conditions. For
example, if x(O) = 1 and x(2) = 5, C1 = 2 and C2 = 1 the straight
line is x*(t) = 2t + 1.
Although the previous example is a simple one,
1. it illustrates the formulation of a performance index from a given
simple specification or a statement, and
2. the solution is well known a priori so that we can easily verify
the application of the Euler-Lagrange equation.
In the previous example, we notice that the integrand V in the functional
(2.3.30), is a function of x(t) only. Next, we take an example,
where, V is a function of x(t), x(t) and t.
64. 38 Chapter 2: Calculus of Variations and Optimal Control
Example 2.8
Find the optimum of
J = l [:i;2(t) - 2tX(t)] dt
that satisfy the boundary (initial and final) conditions
x(O) = 1 and x(2) = 5.
(2.3.33)
(2.3.34)
Solution: In the EL equation (2.3.19), we first identify that V =
x2 (t) - 2tx(t). Then applying the EL equation (2.3.15) to the
performance index (2.3.33) we get
av _ ~ (av) = 0 ----+ -2t - ~ (2x(t)) = 0 ax dt ax dt
----+ x(t) = t.
Solving the previous simple differential equation, we have
t3
x*(t) ="6 + CIt + C2
(2.3.35)
(2.3.36)
where, C1 and C2 are constants of integration. Using the given
boundary conditions (2.3.19) in (2.3.36), we have
4
x(O) = 1 ----+ C2 = 1, x(2) = 5 ----+ C1 = 3' (2.3.37)
With these values for the constants, we finally have the optimal
function as
t 3 4
x*(t) = "6 + "3t + 1. (2.3.38)
Another classical example in the calculus of variations is the brachistochrone
(from brachisto, the shortest, and chrones, time) problem and
this problem is dealt with in almost all books on calculus of variations
[126].
Further, note that we have considered here only the so-called fixedend
point problem where both (initial and final) ends are fixed or given
in advance. Other types of problems such as free-end point problems
are not presented here but can be found in most of the books on the
calculus of variations [79, 46, 81, 48]. However, these free-end point
problems are better considered later in this chapter when we discuss
the optimal control problem consisting of a performance index and a
physical plant.
65. 2.4 The Second Variation 39
2.4 The Second Variation
In the study of extrema of functionals, we have so far considered only
the necessary condition for a functional to have a relative or weak extremum,
i.e., the condition that the first variation vanish leading to
the classic Euler-Lagrange equation. To establish the nature of optimum
(maximum or minimum), it is required to examine the second
variation. In the relation (2.3.7) for the increment consider the terms
corresponding to the second variation [120],
2
J = f :! [( ~~) . (8x(t))2 + (~:~) • (8X(t))2
8
+ 2 (::;x) * 8X(t)8X(t)] dt. (2.4.1)
Consider the last term in the previous equation and rewrite it in terms
of 8x(t) only using integration by parts (f udv = uv - f vdu where,
u = :;¥X8x(t) and v = 8x(t)). Then using 8x(to) = 8x(tf) = 0 for
fixed-end conditions, we get
82 J = ~ rtf [{ (82V) _!i ( 8
2V.) } (8x(t))2
2 ltD 8x2 dt 8x8x
* *
+ (~:~). (8X(t))2] dt. (2.4.2)
According to Theorem 2.1, the fundamental theorem of the calculus of
variations, the sufficient condition for a minimum is 82 J > O. This, for
arbitrary values of 8x(t) and 8x(t), means that
(82V) d (82V)
8x2 * - dt 8x8x * > 0,
(2.4.3)
(82V) 8x2 * > O. (2.4.4)
For maximum, the signs of the previous conditions are reversed. Alternatively,
we can rewrite the second variation (2.4.1) in matrix form
as
2 1 tf . 8x2 8x8± 8x(t)
[
82V 82V]
8 J = 210 [8x(t) 8x(t)] ::rx ~:'; * [8X(t) ] dt
1 rtf . [8X(t)] = "2 ltD [8x(t) 8x(t)]II 8x(t) dt (2.4.5)
66. 40 Chapter 2: Calculus of Variations and Optimal Control
where,
(2.4.6)
If the matrix II in the previous equation is positive (negative) definite,
we establish a minimum (maximum). In many cases since 8x(t) is
arbitrary, the coefficient of (8x(t))2, i.e., 82V /8x2 determines the sign
of 82 J. That is, the sign of second variation agrees with the sign of
82V / 8x2. Thus, for minimization requirement
(2.4.7)
For maximization, the sign of the previous equation reverses. In the
literature, this condition is called Legendre condition [126].
In 1786, Legendre obtained this result of deciding whether a
given optimum is maximum or minimum by examining the
second variation. The second variation technique was further
generalized by Jacobi in 1836 and hence this condition
is usually called Legendre-Jacobi condition.
Example 2.9
Verify that the straight line represents the minimum distance between
two points.
Solution: This is an obvious solution, however, we illustrate the
second variation. Earlier in Example 2.7, we have formulated a
functional for the distance between two points as
(2.4.8)
and found that the optimum is a straight line x*(t) = Clt + C2. To
satisfy the sufficiency condition (2.4.7), we find
x*(t) 1
3/2
. (2.4.9)
[1+x*2(t)]
Since x*(t) is a constant (+ve or -ve) , the previous equation satisfies
the condition (2.4.7). Hence, the distance between two points as
given by x*(t) (straight line) is minimum.
67. 2.5 Extrema of Functions with Conditions 41
Next, we begin the second stage of optimal control. We consider optimization
(or extremization) of a functional with a plant, which is
considered as a constraint or a condition along with the functional. In
other words, we address the extremization of a functional with some
condition, which is in the form of a plant equation. The plant takes
the form of state equation leading us to optimal control of dynamic
systems. This section is motivated by [6, 79, 120, 108].
2.5 Extrema of Functions with Conditions
We begin with an example of finding the extrema of a function under
a condition (or constraint). We solve this example with two methods,
first by direct method and then by Lagrange multiplier method. Let us
note that we consider this simple example only to illustrate some basic
concepts associated with conditional extremization [120].
Example 2.10
A manufacturer wants to maximize the volume of the material
stored in a circular tank subject to the condition that the material
used for the tank is limited (constant). Thus, for a constant
thickness of the material, the manufacturer wants to minimize the
volume of the material used and hence part of the cost for the tank.
Solution: If a fixed metal thickness is assumed, this condition implies
that the cross-sectional area of the tank material is constant.
Let d and h be the diameter and the height of the circular tank.
Then the volume contained by the tank is
V(d, h) = wd2h/4 (2.5.1)
and the cross-sectional surface area (upper, lower and side) of the
tank is
A(d, h) = 2wd2/4 + wdh = Ao. (2.5.2)
Our intent is to maximize V(d, h) keeping A(d, h) = Ao, where Ao
is a given constant. We discuss two methods: first one is called the
Direct Method using simple calculus and the second one is called
Lagrange Multiplier Method using the Lagrange multiplier method.
1 Direct Method: In solving for the optimum value directly, we
eliminate one of the variables, say h, from the volume relation
(2.5.1) using the area relation (2.5.2). By doing so, the condition is
embedded in the original function to be extremized. From (2.5.2),
h
= Ao - wd2/2
7rd . (2.5.3)
68. 42 Chapter 2: Calculus of Variations and Optimal Control
Using the relation (2.5.3) for height in the relation (2.5.1) for volume
(2.5.4)
Now, to find the extrema of this simple calculus problem, we differentiate
(2.5.4) w.r.t. d and set it to zero to get
~o _ ~7rd2 = O. (2.5.5)
Solving, we get the optimal value of d as
d* = J2Ao .
37r
(2.5.6)
By demanding that as per the Definition 2.3 for optimum of a
function, the second derivative of V w.r.t. d in (2.5.4) be negative
for maximum, we can easily see that the positive value of
the square root function corresponds to the maximum value of the
function. Substituting the optimal value of the diameter (2.5.6) in
the original cross-sectional area given by (2.5.2), and solving for
the optimum h *, we get
h* = J2AO
•
37r
(2.5.7)
Thus, we see from (2.5.6) and (2.5.7) that the volume stored by a
tank is maximized if the height of the tank is made equal to its
diameter.
2 Lagrange Multiplier Method: Now we solve the above problem
by applying Lagrange multiplier method. We form a new function
to be extremized by adjoining a given condition to the original
function. The new adjoined function is extremized in the normal
way by taking the partial derivatives w.r. t. all its variables, making
them equal to zero, and solving for these variables which are extremals.
Let the original volume relation (2.5.1) to be extremized
be rewritten as
f(d, h) = 7rd2h/4
and the condition (2.5.2) to be satisfied as
g(d, h) = 27rd2/4 + 7rdh - Ao = O.
(2.5.8)
(2.5.9)
Then a new adjoint function £, (called Lagrangian) is formed as
£'(d, h, -) = f(d, h) + -g(d, h)
= 7rd2h/4 + -(27rd2/4 + 7rdh - Ao) (2.5.10)
69. ----------------------------------
2.5 Extrema of Functions with Conditions 43
where, A, a parameter yet to be determined, is called the Lagrange
multiplier. Now, since the Lagrangian C is a function of three
optimal variables d, h, and A, we take the partial derivatives of
£(d, h, A) w.r.t. each of the variables d, h and A and set them to
zero. Thus,
ac
ad = 7rdh/2 + A(7rd + 7rh) = 0
ac
ah = 7rd2/4 + A(7rd) = 0
ac 2
aA = 27rd /4 + 7rdh - Ao = o.
(2.5.11 )
(2.5.12)
(2.5.13)
Now, solving the previous three relations (2.5.11) to (2.5.13) for
the three variables d*, h *, and A *, we get
d* = J2AO; h* = J2AO
; ,X* = -J Ao . 37r 37r 247r
(2.5.14)
Once again, to maximize the volume of a cylindrical tank, we need
to have the height (h *) equal to the diameter (d*) of the tank. Note
that we need to take the negative value of the square root function
for A in (2.5.14) in order to satisfy the physical requirement that
the diameter d obtained from (2.5.12) as
d = -4A (2.5.15)
is a positive value.
Now, we generalize the previous two methods.
2.5.1 Direct Method
N ow we generalize the preceding method of elimination using differential
calculus. Consider the extrema of a function f(XI, X2) with two
interdependent variables Xl and X2, subject to the condition
(2.5.16)
As a necessary condition for extrema, we have
af af
df = -a dXI + -a dX2 = o. Xl X2
(2.5.17)
However, since dXI and dX2 are not arbitrary, but related by the condition
ag ag
dg = -a dXI + -a dX2 = 0,
Xl X2
(2.5.18)
70. 44 Chapter 2: Calculus of Variations and Optimal Control
it is not possible to conclude as in the case of extremization of functions
without conditions that
and (2.5.19)
in the necessary condition (2.5.17). This is easily seen, since if the
set of extrema conditions (2.5.19) is solved for optimal values xi and
x2' there is no guarantee that these optimal values, would, in general
satisfy the given condition (2.5.16).
In order to find optimal values that satisfy both the condition (2.5.16)
and that of the extrema conditions (2.5.17), we arbitrarily choose one
of the variables, say Xl, as the independent variable. Then X2 becomes
a dependent variable as per the condition (2.5.16). Now, assuming that
8g/8x2 i- 0, (2.5.18) becomes
8g/ 8XI} dX2 = - {
8g/8x2 dXI (2.5.20)
and using (2.5.20) in the necessary condition (2.5.17), we have
(2.5.21)
As we have chosen dXI to be the independent, we now can consider it
to be arbitrary, and conclude that in order to satisfy (2.5.21), we have
the coefficient of dXI to be zero. That is
(%:J (%:J -(%:J (%:J = o. (2.5.22)
Now, the relation (2.5.22) and the condition (2.5.16) are solved simultaneously
for the optimal solutions xi and x2' Equation (2.5.22) can
be rewritten as
=0. (2.5.23)
This is also, as we know, the Jacobian of f and 9 w.r.t. Xl and X2.
This method of elimination of the dependent variables is quite tedious
for higher order problems.