The mathematical techniques of optimization are fundamental to statistical theory and practice. This volume covers these methods, ranging from classical optimization and Lagrange multipliers, to numerical techniques using gradients or direct search, to linear, nonlinear and dynamic programming using the Kuhn-Tucker conditions or the Pontryagin maximal principle. Variational methods and optimization in function spaces are also discussed, as are stochastic optimization in simulation, including annealing methods. The text features various applications, including: estimates of maximum likelihood; Markov decision processes; programming methods used to optimize monitoring of patients in hospitals; the derivation of the Neyman-pearson lemma; the search for optimal designs; and the simulation of a steel mill.