A Derivative-Free Decent Method via Acceleration Parameter for Solving Systems of Nonlinear Equations

An algorithm for solving large-scale systems of nonlinear equations based on the transformation of the Newton method with the line search into a derivative-free descent method is introduced. Main idea used in the algorithm construction is to approximate the Jacobian by an appropriate diagonal matrix. Furthermore, the step length is calculated using inexact line search procedure. Under appropriate conditions, the proposed method is proved to be globally convergent under mild conditions. The numerical results presented show the efficiency of the proposed method. ARTICLE INFORMATION Received: 15-Jul-2019 Revised: 02-Aug-2019 Accepted: 29-Aug-2019


INTRODUCTION
Consider the systems of nonlinear equations: ( ) = 0, (1) where : → is nonlinear map. Among various methods for solving nonlinear equations (1), Newton's method is quite welcome due to its nice properties such as the rapid convergence rate, the decreasing of the function value sequence [10]. However, at each iteration, Newton's method needs the computation of the derivative as well as the solution of some system of linear equations.
The iterative formula of a Newton method is given by +1 = + , = , = 0,1, ⋯, where, is a step length to be computed by a line search technique [2,3], +1 represents a new iterative point, is the previous iteration, while is the search direction to be calculated by solving the following linear system of equations, Where ( ) is the Jacobian matrix of ( ) at . A basic requirement of the line search is to sufficiently decrease the function values i.e ‖ ( + )‖ ≤ ‖ ( )‖.
(3) In fact, this problem can come from an unconstrained optimization problem, a saddle point, and equality constrained problem [6]. Let be a norm function defined by ( ) = 1 2 ‖ ( )‖ 2 (4) where, || · || to stand for the Euclidian norm. Then the nonlinear equations problem (1) is equivalent to the following global optimization problem ( ), ∈ . and condition (3) is equivalent to Furthermore, the search direction is generally required to satisfy the descent condition ∇ ( ) < 0. The derivative-free direction can be obtained in several ways [4,5,7,9,12]. An iterative method that generates a sequence { } satisfying (3) or (5) is called a norm descent method. If is a descent direction of at , then inequality (5) holds for all > 0 sufficiently small. In particular, Newton method with line search is norm descent. For a quasi-Newton method, however, may not be a descent direction of at even if is symmetric and positive definite. To globalize a quasi-Newton method, Li and Fukushima [6] proposed an approximately norm descent line search technique and established global and super-linear convergence of a Gauss-Newton based BFGS method for solving symmetric nonlinear equations. The method in [6] is not norm descent. In addition, the global convergence theorem is established under the assumption that ( ) is uniformly nonsingular.
The drawback of the technique (2) is the need to compute the Jacobian matrix ( ) at every iteration, which will increase the computing difficulty, due to the first-order derivative of the system because sometimes they are not even available or could not be obtained exactly [10] especially for the large-scale problems. Therefore, motivated by [1] the purpose of this article is to develop a derivativefree method with decent direction for solving system of nonlinear equations via ( ) ≈ , Where is an identity matrix. The presented method has a norm descent property without computing the Jacobian matrix with less number of iterations and CPU time that is globally convergent.
The next section of this paper present the proposed method. In section 3, the convergence analysis of the proposed method is given. In Sections 4 and 5, report on some numerical results and a conclusion were given respectively.

Derivation of the Method
The main idea used in the proposed algorithm construction is approximation of the Jacobian in (2) by a diagonal matrix via acceleration parameter. Now, from Taylor's expansion of the first order the approximation of ( +1 ) can be brought as follows: Having in mind that the distance between , and +1 is small enough.
to approximate the gradient ∇ ( ), which avoids computing exact gradient and updated via line search method. It is clear that when ‖ ( )‖ is small, then ≈ ∇ ( ). pre-multiplying both side of (10) by , the relation allows us to compute the parameter +1 in the following way: we can easily show that, our direction is we finally have the general scheme as: (14) we can easily show that, our direction is Therefore, the derivative-free line search used in [1,3,6] is the best choice to compute the step length .
Step 2: Compute ( ) and test a stopping criterion. If yes, then stop; otherwise continue with Step 3.

REMARK 1:
It is clear that the line search (16) is well defined. Otherwise, for any integer > 0, . This leads to a contradiction since η k f(x k ) is positive.
(31) by summing the above inequality, we have (32) so from the level set and fact that { } satisfies (15) then the series is convergent. This implies (29). By similar way we can prove that (30) holds.

Lemma 5.
Suppose that assumption 1 holds and { } is generated by algorithm 1. Then there exists some positive constants 2 such that for all > 0, , we have (33). The proof is completed. We can deduce that for all k (33) hold. Now we are going to establish the following global convergence theorem to show that under some suitable conditions, there exist an accumulation point of { } which is a solution of problem (1). Theorem 1. Suppose that assumption 1 holds,{ } is generated by algorithm 1. Assume further for all > 0, where c is some positive constant. Then The proof is completed.

Numerical Results
In this section, we compared the performance of our method with an improved derivative free method via double direction approach for solving systems of nonlinear equations (IDFDD) [5]. For the both algorithms the following parameters are set ω1 = ω2 = 10 −4 , r = 0.2 and = 1 ( +1) 2 . The employed computational codes was written in Matlab 7.9.0 (R2009b) and run on a personal computer 2.00 GHz CPU processor and 3 GB RAM memory. We stopped the iteration if the total number of iterations exceeds 1000 or ‖ ( )‖ ≤ 10 −4 . We claim that the method fails, and use the symbol " − " to represents failure due to; (i) Memory requirement (ii) Number of iterations exceed 1000. (iii) If ‖ ( )‖ is not a number. The methods were tested on some Benchmark test problems with different initial points. problem 1 and 3 below are from [5] while problem 2 is from [11].        From Tables 1, 2 and 3 we can easily observe that both of these methods attempt to solve the systems of nonlinear equations (1), but the better efficiency and effectiveness of our proposed algorithm was clear for it solves where IDFDD fails. This is quite evident for instance with problem 1. In particular, the ADDA method considerably outperforms the IDFDD for almost all the tested problems, as it has the least number of iterations and CPU time, which are even much less than the CPU for the IDFDD method.

Conclusion
In this paper, a derivative-free decent method via acceleration parameter for solving systems of nonlinear equations is given. It is a fully derivative-free iterative method which possesses global convergence under some appropriate conditions. Numerical comparisons using a set of large-scale test problems show that the proposed method is practically quite effective.