Consider the unconstrained minimization problem where is twice differentiable and . Newton’s method is a second-order descent method for finding the minimum. Starting at some initial point, at the th iteration we update the candidate solution with the formula where and are the gradient and Hessian of respectively, and is a step size chosen appropriately […]
Quasi-Newton methods in optimization
Tag: Minimization
Lagrange dual, weak duality and strong duality
Consider an optimization problem in standard form: with the variable . Let be the domain for , i.e. the intersection of the domains of the ‘s and the ‘s. Let denote the optimal value of the problem. The Lagrange dual function is the function defined as the minimum value of the Lagrangian over : The […]
Lagrange dual, weak duality and strong duality
The Lagrange dual function is always concave
Consider an optimization problem in standard form: with the variable . Let be the domain for , i.e. the intersection of the domains of the ‘s and the ‘s. The Lagrangian associated with this problem is the function defined as with domain . The Lagrange dual function is the function defined as the minimum value […]
The Lagrange dual function is always concave