Ya-xiang Yuan
Institute of Computational Math. and Scientific/Engineering Computing
AMSS, Chinese Academy of Sciences
The steepest descent method is the simplest method for minimization that use gradients. It is well known that the steepest descent method with the ``best'' stepsize in the sense of reducing the objective function, namely the exact line search converges only linearly and would lead to zig-zag, giving very slow convergence, particularly when the function is ill-conditioned. However, a surprising result given by Barzilai and Borwein indicates that a specific stepsize would ensure the steepest descent method converging superlinearly for two dimensional problems. This talk will present recent advances on different choices for the steepsize of the steepest descent method, in order to improve the Barzilai and Borwein method.