A Solution to the Optimal Tracking Problem for Linear Systems

CORNELIU BOTAN FLORIN OSTAFI ALEXANDRU ONEA

Dept. of Automatic Control and Industrial Informatics
"Gh. Asachi" Technical University of Iasi
Bd. Mangeron. 53A, 6600 Iasi
ROMANIA
Abstract: The paper establishes a new procedure to obtain the solution for the optimal tracking problem based
on dynamic programming. The optimal control refers to a quadratic criterion with finite final time, regarding a
perturbed time-variant linear system. The proposed algorithm can be easier implemented by comparison with
other procedures.
Key-Words: optimal control, linear quadratic, tracking problem
1 Introduction
A perturbed linear time-variant multivariable
system is considered
) t ( x ) t ( C ) t ( y
x ) t ( x , ) t ( w ) t ( u ) t ( B ) t ( x ) t ( A ) t ( x
0
0

+ +
(1)
where
n r m n
) t ( w , ) t ( y , ) t ( u , ) t ( x
are the sate, control, output and disturbance vector,
respectively.
The problem is to ensure that the output vector y(t)
evolves near to a desired trajectory
r
) t ( z and
that the energy consumption has a low level. For
this purpose, it is introduced the criterion
[ ]dt ) t ( u ) t ( P ) t ( u ) t ( e ) t ( Q ) t ( e
2
1
) t ( Se ) t ( e
2
1
J
f
0
t
t
T T
f f
T

+ +
+
(2)
(T denotes the transposition), where S0, Q(t) 0,
P(t)>0 are the weight matrices of appropriate
dimensions and
) t ( x ) t ( C ) t ( z ) t ( y ) t ( z ) t ( e (3)
is the tracking error.
The optimal tracking problem refers to the system
(1) and the criterion (2). If the pair (A,C) is
completely observable, the problem can be
reformulated as one referring to the state vector [1],
and thus the criterion is
[
]dt ) t ( u ) t ( P ) t ( u ) t ( x ) t ( C ) t ( Q ) t ( z 2
) t ( z ) t ( Q ) t ( z ) t ( x ) t ( Q ) t ( x
2
1
) t ( SCx ) t ( z
) t ( Sz ) t ( z
2
1
) t ( x S ) t ( x
2
1
J
T T
t
t
T T
f f
T
f f
T
f f
T
f
0
+
+ +
+
+

(4)
where
0 ) t ( C ) t ( Q ) t ( C ) t ( Q
, 0 ) t ( QC ) t ( C S
T
f f
T


(5)
Let us denote with V(t,x) the minimum value of the
criterion (4) on the interval [t, t
f
]. This function
satisfies the Hamilton-Jacobi-Bellman equation [2]
0 t , x ,
x
) x , t ( V
H
t
) x , t ( V
(
,
\
,
(
j

(6)
and
) t ( SCx ) t ( z
) t ( Sz ) t ( z
2
1
) t ( x S ) t ( x
2
1
) x , t ( V
f f
T
f f
T
f f
T
f

+
(7)

For the above formulated problem
( )
n T
T T
) t ( p ), w Bu Ax ( p
Pu u x Q x
2
1
) u , t , x , p ( H
+ + +
+ +
(8)
The optimal control vector is
) t ( p ) t ( B ) t ( P ) t ( u
T 1
(9)
If V(t,x) is imposed as
) t ( g ) t ( x ) t ( x ) t ( R ) t ( x
2
1
) x , t ( V
~
T
~
T
+ (10)
( ) t ( g
~
depends on z(t)), thus V(t,x) is the solution of
equation (3) only if the symmetric matrix
R
~
verifies the Riccati differential equation
) t ( Q ) t ( R
~
) t ( A
) t ( A ) t ( R
~
) t ( R
~
) t ( B ) t ( P ) t ( B ) t ( R
~
) t ( R
~
T
T 1

(11)
with
S ) t ( R
~
f
(12)
and ) t ( g
~
satisfies a linear differential equation.
Finally, the optimal control becomes
)] t ( g ) t ( x ) t ( R
~
)[ t ( B ) t ( P ) t ( u
~
T 1
+

(13)
Matrix ) t ( R
~
is time variant even in the
cases when the matrices of the system (1) and of
the criterion (2) are constant. It means that the
resulting optimal controller is time variant even in
the case of an invariant linear quadratic tracking
problem. Solving in inverse time of the equation
(11) introduces a supplementary difficult in the
implementation of this controller.
In order to avoid these difficulties, a new
procedure is proposed. This procedure is similar
with one proposed in [3] for a state regulator
problem for an unperturbed system. Note that the
presence of exogenous vectors z(t) and w(t)
complicates the solution, since the additional terms
appear. The proposed procedure is especially useful
in the time invariant case, when a significant
simplification of the implementation is obtained.
2 Main results
Instead of (10), it is proposed V(t,x) in the form
) t ( ) t ( v x ) t ( x ) t ( R ) t ( x
2
1
) x , t ( V
T T
+ + (14)
where the symmetric matrix ) t ( R is a particular
solution to the equation (11), and we denote
S ) t ( R
f
(15)
Note that, in many cases, such a solution can be
beforehand computed off-line and then used in the
real time computing.
Then
) t ( v x R
x
) x , t ( V
) t ( ) t ( v x x ) t ( R x
2
1
t
) x , t ( V
T T
+

+ +

(16)
and, using (8), (9) and (16), yields
( )
( )
, w ) v x R (
QCx z Qz z
2
1
v x R A x
2
1
Ax ) v x R (
2
1
v x R N ) v x R (
2
1
x Q x
2
1
) t , x ,
t
) x , t ( V
( H
T
T T T T
T T
T
+ +
+ + + +
+ + + + +

(17)
where ) t ( B ) t ( P ) t ( B ) t ( N
T 1
(18)
Lemma 1: The minimum cost function V(t,x) can
be written in the form (14), where ) t ( R is a
particular solution to the Riccati equation (11), with
final condition (15). The vector v(t) and the scalar
(t) satisfy the equations:
) t ( h ) t ( v F ) t ( v
T
+ (19)
) t ( SCz ) t ( x ) S S ( ) t ( v
f f f
(20)
where
) t ( R ) t ( N ) t ( A ) t ( F (21)
) t ( w ) t ( R ) t ( z ) t ( Q ) t ( C ) t ( h
T
(22)
and
) t ( z ) t ( Q ) t ( z
2
1
) t ( w ) t ( v ) t ( v ) t ( N ) t ( v
2
1
) t (
T
T T

(23)
) t ( Sz ) t ( z
2
1
) t ( x ) S S )( t ( x
2
1
) t (
f f
T
f f
T
f
+
+
(24)
Proof: Substituting (16) and (17) in (6), yields
0 Nv v
2
1
w v Qz z
2
1
] w R Qz C v ) RN A ( v [ x
x ) Q R A A R NR R R ( x
2
1
T T T
T T T
T T
+ +
+ + + + +
+ + + +

This relation is satisfied for any x, if ) t ( R is

solution to the equation (11) and v(t) and (t)
satisfy (19) and (23), respectively. The final
condition (7) is also satisfied.
Let us denote with ) t , t ( and ) t , t (
f f

the transition matrices for F and F
T
, respectively.
The solution to the equation (19) is
+

d ) ( h ) t , ( ) t ( v ) t , t ( ) t ( v
t
t
f f f
f
(25)

This solution cannot be used because the final
condition ) t ( v
f
is not a priori known. Therefore
the final condition ) t ( v
f
must be expressed as a
function of ) t ( x
0
. In this respect, we observe that
the co-state vector
) t ( v ) t ( x ) t ( R ) t ( p + (26)
can be replaced in (9) and then in (1). Thus the
equations (1) and (24) can be written:
]
]
]
,

,
+
]
]
]
,

]
]
]
,

,
) t ( h
) t ( w
) t ( v
) t ( x
G
) t ( v
) t ( x

, (27)

with
n 2 nx 2
T
) t ( F 0
) t ( N ) t ( F
) t ( G
]
]
]
,

(28)
where N and F are given by (18) and (21),
respectively.
The transition matrix for G(t) can be
expressed as [3]:
]
]
]
,

) t , t ( 0
) t , t ( ) t , t (
) t , t (
f
f 12 f
f
(29)
where ) t ( and ) t ( are the transition matrices for
F and F
T
, respectively, and
12
satisfies the
equation
0 ) t , t (
), t , t ( N ) t , t ( ) t ( F ) t , t (
f f 12
f f 12 f 12

(30)
and it is

d ) t , ( ) ( N ) , t ( ) t , t (
f
t
t
f f 12
(31)
The solution to the system (27) is

]
]
]
,

+
]
]
]
,

,

]
]
]
,

d
) ( h
) ( w
) , t (
) t ( v
) t ( x
) t , t (
) t ( v
) t ( x
t
t
f
f
f
f
(32)
with ) t , t (
f
given by (29).
We are now in position to formulate the following
Theorem: for the formulated tracking problem, the
optimal control is
) t ( u ) t ( u ) t ( u
c f
*
+ , (33)
where u
f
(t)is the feedback component
) t ( x ) t ( R ) t ( B P ) t ( u
T 1
f

(34)
and u
c
(t) is a corrective component
) t ( v B P ) t ( u
T 1
c

(35)
The vector v(t) has two components
) t ( v ) t ( v ) t ( v
e 0
+ , (36)
one depending on the initial state ) t ( x
0
, and one
depending on the exogenous variables
) t , t ( g ) t ( SCz ) t , t ( )] t , t ( g
) t ( SCz ) t , t ( )[ t , t ( W ) t , t ( ) t ( v
) t ( x ) t , t ( M ) S S )( t , t ( W ) t , t ( ) t ( v
f 1 f 0 f 0 2
f f 0 12 f 0 0 e
0 f 0
1
f 0 0 0
+



(37)
where
) t , t ( M ) S S )( t , t ( ) t , t ( W
f
1
f f

(38)
and

d ) ( h ) , t ( ) t , t ( g
t
t
f 1
f
(39)
+

d )] ( h ) , t ( ) ( w ) , t ( [ ) t , t ( g
12
t
t
f 2
f
(40)
Proof: One can express x(t) from (32) and (20)
) t , t ( g
) t ( SCz ) t , t ( ) t ( x ) t , t ( M ) t ( x
f 2
f f 12 f f
+
+
(41)
where
) S S )( t , t ( ) t , t ( ) t , t ( M
f 12 f f
+ (42)
and corresponds to the transition from x(t
f
) to x(t)
for the controlled system without exogenous
variables.
From (41) results
)] t , t ( g ) t ( SCz ) t , t (
) t ( x )[ t , t ( M ) t ( x
f 0 2 f f 0 12
0 f 0
1
f
+
+

(43)
Using (43), the expression (25) of v(t) can be
written as
) t , t ( g )} t ( SCz
)] t , t ( g ) t ( SCz ) t , t (
) t ( x [ M ) S S ){( t , t ( ) t ( v
f 1 f
f 0 2 f f 0 12
0
1
f
+
+
+

(44)
From (44) it is possible to separate the components
(37) depending on the initial state and on the
exogenous variables, respectively.
Remark 1: Using the above relations, it is proved in
[3] that the solution to the Riccati matriceal
differential equation (11) is
) t , t ( M ) S S )( t , t ( ) t ( R ) t ( R
~
f
1
f

+ (45)
Remark 2: The function ) t , t ( g
f 0 2
given by (42)
for t = t
0
can be computed only if the exogenous
vectors z(t) and w(t) are beforehand known.
Therefore, the problem can be solved only under
this assumption, or, at least, the shape of these
vectors is known and their amplitude is estimated at
the beginning of the optimisation process.
3 Optimal controller
The classical solution for the optimal tracking
problem is based on the relation (13), but the
drawbacks specified in the section 1 appear in this
case. A simpler controller implementation can be
done using the relations presented in this paper.
- A first possibility is to use the
implementation based on relation (13), but using
also (45) in order to compute the matrix ) t ( R
~
. Note
that particular solution ) t ( R to the equation (13)
can be computed beforehand.
- Another way based on (45) is to write this
relation so that ) t ( R
~
can be computed in direct time.
If ) t ( R
~
0
would be known, we could write [3]
) t , t ( M )] t ( R ) t ( R
~
)[ t , t ( R ) t ( R
~
0
1
0 0 0 0

+
with
)] t ( R ) t ( R
~
)[ t , t ( ) t , t ( ) t , t ( M
0 0 f 0 12 0 0 0
+
But ) t ( R
~
0
can be computed taking t = t
0
in (45):
) t , t ( M ) S S ( ) t , t ( ) t ( R ) t ( R
~
f 0
1
0 0 0

+
Replacing in the previous relation, yields
) t , t ( M ) t , t ( W ) t , t ( ) t ( R ) t ( R
~
f 0
1
0 f 0 0

+
where W(t
0
,t
f
) is a constant matrix given by (38) for
t = t
0
.
The matrix ) t , t (
0
and the matrices from
) t , t ( M
0 0
can be iteratively computed.
-A more convenient way is based on
relations offered by the above theorem. These
relations are rather complicated, but the most part
of the computing is performed off-line in the design
stage of the controller. The real-time control
implies to compute ) t ( u
f
given by (34) as a usual
feedback component and the corrective component
) t ( u
c
, given by (35). This last component depends
on v(t) given by (44) and contains only two time
variables elements : ) t , t (
0
and ) t , t ( g
f 1
. The
both elements can be recurrently computed. Indeed,
, ) t , t ( ) t , t ( ) t , t (
1 i i 0 1 i 1 i i 0 i i

with I ) t , t (
0 0 0
(the identity matrix). In the
above relation
i
represents the transition matrix at
the moment t
i
and
i
is the transition matrix
which corresponds to two successive moments t
i-1
and t
i
. For the time invariant problem this last
matrix is constant.
The vector ) t , t ( g
f 1
given by (39) must be
firstly expressed in terms of t
0
, in order to ensure a
real time computing. In this respect we observe that

d ) ( h ) , t (
d ) ( h ) , t ( d ) ( h ) , t ( ) t , t ( g
f
0
0
f
t
t
t
t
t
t
f 1
(46)
The last integral in (46) is constant and can
be computed off-line and the first one can be
recurrently computed in real time.
Based on these remarks, results that the
real time computing performed by the optimal
controller is not very complicated.
A significant simplification of the
computing appear in time invariant problems, and
moreover in the case when the exogenous vectors
are constants.
The solution to the Riccati differential
equation in this case is
) t , t ( M ) R S )( t , t ( R ) t ( R
~
f
1
f

+ (47)
where R represents a solution to the Riccati
algebraic matriceal equation
0 Q R A RA RNR
T
(48)
For P>0, Q0, (A,B) stabilizable and (A, Q )
detectable, the unique positive defined solution of
this equation will be chosen.
We also remark that the constant vectors w
and h can pull out from the integrals (39) and (42)
and this fact simplifies the computing.
We note that in this case the structure of
the optimal controller contains only time invariant
blocks.
4 Numerical example
As a simple numerical example was chosen a time
invariant system described by the equation (1) with
]
]
]
,

]
]
]
,

3 . 6
0
B ,
4 . 19 5 . 3
20 0
A
In the criterion (4), the values of the terminal
moments are t
0
= 0 and t
f
= 0.3 and the matrices
are chosen as follows:S=diag(10,0), Q=diag(1,3.1),
P=p=1. The exogenous vectors are z(t)=[54 2.3]
and w(t)=[-28 0].
The optimal controller was designed based on the
relations offered by the above theorem. The
behaviour of the system is indicated in the figure
5. Conclusions
The optimal tracking problem for a linear
time-variant system is studied, tacking into account
the presence of the disturbances.
The proposed algorithms are more
convenient for implementation by comparing with
the usual procedures.
One of the proposed algorithms is based on
the direct time solving of the Riccati equation.
It is also indicated an efficient possibility
of implementation for the optimal controller, using
a usual feedback and a corrective component,
depending on the initial state. This optimal
controller is advantageous especially in the time-
invariant case.
References:
[1] Anderson B.D.O., Moore J.B., Optimal Control.
Linear Quadratic Methods, Prentice Hall, 1990.
[2]. Kalman R.E, Falb P.L., Arbib M.A., Topics in
Mathematical System Theory, McGraw Hill
1969.
[3] Boan C., Onea A., Ostafi F., A Solution to the
LQ-Problem with Finite Final Time, WSES Int.
Conf. on Automation and Information (AITA
2001), Skiatos, Greece, in Advanced in
Automation, Multimedia and Video Systems,
and Modern Computer Science, Ed. Kulev
V.V., DAttellis C.E., Mastorakis N.E.,WSES
Press, 2001, pp.62-66.
[4] Boan C., On the Optimal Output Regulator
Problem for the Linear Systems, Proceedings of
the 1985 International Conference on Control
Systems and Computer Science, Bucharest,
Romania, Vol. I, 1985, pp. 90-94.
[5] Botan C., On the Solution of the Riccati
Differential Matrix Equation, Symp. on
Computational System Analysis, Berlin, Elsever
Science Publishers, 1992, pp. 141-146.