A new class of adaptive control systems which is not only asymptotically stable but also optimal or sub-optimal to some meaningful cost functionals, is derived in this manuscript. This can be done by relating Lyapunov functions which are utilized in stability analysis of adaptive control, with solutions of Hamilton-Jacobi (-Isaacs) equations in the several optimal control problems. The three types of adaptive optimal control problems are considered; that is, Type 1) disturbance attenuation problems are solved as solutions which attain certain H∞ property from exogenous disturbance to outputs, but no penalty is imposed on control effort, Type 2) quadratic cost functionals which include penalty on control effort and output variables, are to be minimized, and Type 3) disturbance attenuation problems are solved as solutions of certain optimal control problems (or differential games) where control effort is also penalized. Adaptive quadratic (Type 2) and H∞ (Type 1 and 3) optimal (or sub-optimal) control systems are constructed for generalized adaptive control problems.