Verification theorem and construction of epsilon-optimal controls for control of abstract evolution equations

Fabbri, Giorgio and Gozzi, Fausto and Swiech, Andrzej (2007): Verification theorem and construction of epsilon-optimal controls for control of abstract evolution equations.

Abstract

We study several aspects of the dynamic programming approach to optimal control of abstract evolution equations, including a class of semilinear partial differential equations. We introduce and prove a verification theorem which provides a sufficient condition for optimality. Moreover we prove sub- and superoptimality principles of dynamic programming and give an explicit construction of $\epsilon$-optimal controls.

• Item Type: MPRA Paper
• Language: English
• Keywords: optimal control of PDE; verification theorem; dynamic programming; $\epsilon$-optimal controls; Hamilton-Jacobi-Bellman equations
• Subjects: C - Mathematical and Quantitative Methods > C6 - Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling > C61 - Optimization Techniques ; Programming Models ; Dynamic Analysis

References

S. Anita, Analysis and control of age-dependent population dynamics, Kluwer Academic Publishers, Dordrecht, (2001).

V. Barbu, Optimal feedback controls for a class of nonlinear distributed parameter systems SIAM J. Control Optim. 21 (1983), no. 6, 871-894.

V. Barbu, Hamilton-Jacobi equations and nonlinear control problems, J. Math. Anal. Appl 120 (1986), no. 2, 494-509.

V. Barbu, Analysis and control of nonlinear infinite-dimensional systems, Mathematics in Science and Engineering, 190. Academic Press, Inc., Boston, MA, 1993.

V. Barbu, Approximation Hamilton-Jacobi equations and suboptimal feedback controllers, Nonlinear analysis and applications (Warsaw, 1994), 27-47, GAKUTO Internat. Ser. Math. Sci. Appl., 7, Tokyo, 1996.

V. Barbu, E. N. Barron and R. Jensen, The necessary conditions for optimal control in Hilbert spaces, J. Math. Anal. Appl. 133 (1988), no. 1, 151-162.

V. Barbu and G. Da Prato, Hamilton-Jacobi equations in Hilbert spaces, Research Notes in Mathematics 86, Pitman, Boston, MA, 1983.

V. Barbu and G. Da Prato, Hamilton-Jacobi equations and synthesis of nonlinear control processes in Hilbert spaces, J. Differential Equations 48 (1983), no. 3, 350-372.

V. Barbu and G. Da Prato, Hamilton-Jacobi equations in Hilbert spaces: variational and semigroup approach, Ann. Mat. Pura Appl. (4) 142 (1985), 303-349 (1986).

V. Barbu and G. Da Prato, A note on a Hamilton-Jacobi equation in Hilbert space Nonlinear Anal. 9 (1985), no. 12, 1337-1345.

V. Barbu, G. Da Prato and C. Popa, Existence and uniqueness of the dynamic programming equation in Hilbert space Nonlinear Anal. 7 (1983), no. 3, 283-299.

M. Bardi and I. Capuzzo-Dolcetta, Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations, Systems & Control: Foundations & Applications, Birkhauser Boston, Inc., Boston, MA, 1997.

E. Barucci and F. Gozzi, Optimal investment in a vintage capital model, Res. Econ. 52 (1998), 159-188. [14] E. Barucci and F. Gozzi, Technology adoption and accumulation in a vintage capital model, J. Econ. 74 (2001) no. 1, 1-30.

P. Cannarsa, Regularity properties of solutions to Hamilton-Jacobi equations in infinite dimensions and nonlinear optimal control, Dfferential Integral Equations 2 (1989), no. 4, 479-493.

P. Cannarsa and O. Carja, On the Bellman equation for the minimum time problem in infinite dimensions, SIAM J. Control Optim. 43 (2004), no. 2, 532-548

P. Cannarsa and Da Prato, G. Nonlinear optimal control with infinite horizon for distributed parameter systems and stationary Hamilton-Jacobi equations, SIAM J. Control Optim. 27 (1989), no. 4, 861-875.

P. Cannarsa and Da Prato, Some results on nonlinear optimal control problems and Hamilton-Jacobi equations in infinite dimensions, J. Funct. Anal. 90 (1990), no. 1, 27-47.

P. Cannarsa and G. Di Blasio, A direct approach to infinite-dimensional Hamilton-Jacobi equations and applications to convex control with state constraints Differential Integral Equations 8 (1995), no. 2, 225-246.

P. Cannarsa, F. Gozzi and H. M. Soner, A dynamic programming approach to nonlinear boundary control problems of parabolic type J. Funct. Anal. 117 (1993), no. 1, 25-61.

P. Cannarsa and H. Frankowska, Value function and optimality conditions for semilinear control problems, Appl. Math. Optim. 26 (1992), no. 2, 139-169.

P. Cannarsa and H. Frankowska, Value function and optimality condition for semilinear control problems. II. Parabolic case, Appl. Math. Optim. 33 (1996), no. 1, 1-33.

P. Cannarsa and M. E. Tessitore, Optimality conditions for boundary control problems of parabolic type, Control and estimation of distributed parameter systems: nonlinear phenomena (Vorau, 1993), 79-96, Internat. Ser. Numer. Math., 118, Birkhauser, Basel, 1994.

P. Cannarsa and M. E. Tessitore, Infinite dimensional Hamilton-Jacobi equations and Dirichlet boundary control problems of parabolic type, SIAM J. Control Optim. 34 (1996), no. 6, pp. 1831-1847.

F. H. Clarke, Yu. S. Ledyaev, E. D. Sontag and A. I. Subbotin, Asymptotic controllability implies feedback stabilization, IEEE Trans. Automat. Control 42 (1997), no. 10, 1394-1407.

F. H. Clarke, Yu. S. Ledyaev, R. J. Stern and P. R. Wolenski, Nonsmooth Analysis and Control Theory, Springer, New York, 1998.

M. G. Crandall, H. Ishii P. L. Lions, User's guide to viscosity solutions of second order partial differential equations, Bull. Am. Math. Soc., New Ser. 27 (1992), no. 1, 1-67.

M. G. Crandall and P. L. Lions, Hamilton-Jacobi equations in infinite dimensions. IV. Hamiltonians with unbounded linear terms, J. Funct. Anal. 90, (1990), 237-283.

M. G. Crandall and P. L. Lions, Hamilton-Jacobi equations in infinite dimensions. V. Unbounded linear terms and B-continuous solutions, J. Funct. Anal. 97, (1991), 417-465.

M. G. Crandall and P. L. Lions, Hamilton-Jacobi equations in infinite dimensions. VI: Nonlinear A and Tataru's method refined, Evolution equations, control theory, and biomathematics (Han sur Lesse, 1991), pp. 51-89, Lecture Notes in Pure and Appl. Math., 155, Dekker, New York, 1994.

M. G. Crandall and P. L. Lions, Hamilton-Jacobi equations in infinite dimensions. VII: The HJB equation is not always satisfied, J. Funct. Anal. 125 (1994) pp. 111-148.

G. Di Blasio, Global solutions for a class of Hamilton-Jacobi equations in Hilbert spaces, Numer. Funct. Anal. Optim. 8 (1985/86), no. 3-4, 261-300.

I. Ekeland and G. Lebourg, Generic Frechet-differentiability and perturbed optimization problems in Banach spaces., Trans. Amer. Math. Soc., 224, (1977), 193-216.

S. Faggian and F. Gozzi, On the dynamic programming approach for optimal control problems of PDE's with age structure, Math. Popul. Stud. 11 (2004), no. 3-4, 233-270

G. Feichtinger, A. Prskawetz and V. M. Veliov, Age-structured optimal control in population economics, Math. Popul. Stud. 65 (2004), no. 4, 373-387.

W. Fleming and R. W. Rishel, Deterministic and stochastic optimal control, Applications of Mathematics, No. 1. Springer-Verlag, Berlin-New York, 1975.

W. Fleming and H. M. Soner, Controlled Markov processes and viscosity solutions, Second edition, Stochastic Modelling and Applied Probability, 25, Springer, New York, 2006.

F. Gozzi, S. S. Sritharan and A. Swiech, Viscosity solutions of dynamic programming equations for the optimal control of the two-dimensional Navier-Stokes equations, Arch. Ration. Mech. Anal. 163 (2002), no. 4, 295-327.

M. Iannelli, Mathematical theory of age-structured population dynamics, Giardini, Pisa, (1995)

M. Iannelli, M. Martcheva and F.A. Milner, Gender-structured population modeling, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, (2005)

H. Ishii, Viscosity solutions for a class of Hamilton-Jacobi equations in Hilbert spaces, J. Funct. Anal., 105 (1992), pp. 301-341.

H. Ishii and S. Koike, On epsilon-optimal controls for state constraint problems, Ann. Inst. H. Poincare Anal. Non Lineaire 17 (2000), no. 4, 473-502.

D. Kelome, Viscosity solutions of second order equations in a separable Hilbert space and applications to stochastic optimal control, Ph.D. The- sis, 2002.

D. Kelome and A. Swiech, Perron's method and the method of relaxed limits for "unbounded" PDE in Hilbert spaces, Studia Math. 176 (2006), no. 3, 249-277.

M. Kocan and P. Soravia, A viscosity approach to infinite-dimensional Hamilton-Jacobi equations arising in optimal control with state constraints, SIAM J. Control Optim. 36 (1998), 1348-1375.

X.Y. Li and J.M. Yong, Optimal control theory for infinite-dimensional systems, Birkhauser Boston, Cambridge, MA, 1995.

P. D. Loewen, Optimal control via nonsmooth analysis, CRM Proceedings & Lecture Notes, 2. American Mathematical Society, Providence, RI, 1993. x+153 pp.

D. Preiss, Differentiability of Lipschitz functions on Banach spaces, J. Funct. Anal. 91 (1990), no. 2, 312-345.

M. Renardy, Polar decomposition of positive operators and problem od Crandall and Lions, Appl. Anal., 57-3 (1995), 383-385

K. Shimano, A class of Hamilton-Jacobi equations with unbounded coefficients in Hilbert spaces, Appl. Math. Optim. 45 (2002), no. 1, 75-98.

S. S. Sritharan, Dynamic programming of the Navier-Stokes equations, Systems Control Lett. 16 (1991), no. 4, 299-307.

A. Swiech, Sub- and superoptimality principles of dynamic programming revisited, Nonlinear Anal. 26 (1996), no. 8, 1429-1436.

D. Tataru, Viscosity solutions of Hamilton-Jacobi equations with unbounded linear terms, J. Math. Anal. Appl. 163 (1992), pp. 345-392.

D. Tataru, Viscosity solutions for the dynamic programming equation, Appl. Math. Optim., 25 (1992), pp. 109-126.

R. Vinter, Optimal control, Systems & Control: Foundations & Applications. Birkhauser Boston, Inc., Boston, MA, 2000.

J. Yong and X.Y. Zhou, Stochastic controls, Springer, New York, (1999).

X. Y. Zhou, Verification theorems within the framework of viscosity solutions, J. Math. Anal. Appl. 177 (1993), no. 1, 208-225.