Mathematical optimization is the engine under machine learning, operations research, and much of quantitative engineering, but its literature ranges from gentle to ferociously abstract. The organizing insight is that convex problems are the ones we can reliably solve, so a good reading order masters the convex world first and only then ventures into the harder, nonconvex terrain.
The path below starts with the convex foundations, builds the classical linear and integer programming theory, and finishes with nonlinear, numerical, and stochastic methods.
The convex foundation
Start with CONVEX OPTIMIZATION by Boyd and Vandenberghe, the modern classic — free online, endlessly clear, and the reason a generation understands why convexity matters. Pair it with Convex Analysis by Rockafellar, the deep and definitive treatment of the underlying theory of convex sets and functions. Boyd builds intuition and applications; Rockafellar supplies the rigor, and between them the central concept of the field is fully in hand.
Linear and integer programming
Next, the classical combinatorial core. Solutions manual for linear programming by Chvátal accompanies his renowned text on the simplex method and LP duality, the foundation of the whole discipline. Theory of Linear and Integer Programming by Schrijver is the authoritative reference on the polyhedral theory behind linear and integer models, the mathematics that makes large-scale planning problems tractable.
Nonlinear and stochastic methods
The final arc is where problems get hard. Nonlinear Programming by Bertsekas is the standard graduate text on optimization beyond the linear case, and Convex Optimization Algorithms, also by Bertsekas, focuses on the methods that actually compute solutions. Numerical Optimization by Nocedal and Wright is the essential practical reference for the algorithms in real solvers. Variational analysis by Rockafellar and Wets pushes the theory further, Interior-Point Polynomial Algorithms in Convex Programming by Nesterov and Nemirovskii gives the theory behind modern solvers, Lectures on modern convex optimization by Ben-Tal and Nemirovski covers conic and robust methods, and Introductory Lectures on Stochastic Optimization by Duchi connects it all to the stochastic methods that train today's models.
Read in this order and optimization becomes one connected theory rather than scattered algorithms. Follow the full path from convexity to stochastic methods.
