Some improved nonlinear conjugate gradient methods and application to non-parametric estimation

Abstract

The conjugate gradient method is one of the most important ideas in scientific computing, it is applied to solving linear systems of equations and nonlinear optimization problems. In this paper, based on a variant of Liu-Storey (LS) method, Hestenes-Stiefel (HS) method and Polak-Ribiere-Polyak (PRP) method, three ` modified CG methods ( named MCLS, MCHS and MCPRP ) are presented and analyzed. The three presented methods generate a descent direction and possess good convergence properties under the strong Wolfe line search conditions. Preliminary elementary numerical experiment results are reported, which show that the proposed methods are promising and effective in minimizing some unconstrained optimization problems and each of these modifications outperforms the four famous conjugate gradient methods. Finally, the proposed methods were further extended to solve the problem of the conditional model regression function.