NEW CONJUGATE GRADIENT METHOD FOR ACCELERATED CONVERGENCE AND COMPUTATIONAL EFFICIENCY IN UNCONSTRAINED OPTIMIZATION PROBLEMS
Abstract
Conjugate gradient (CG) algorithms play an important role in solving large-scale unconstrained optimization problems due to their low memory requirements and strong convergence properties. However, many classical CG algorithms suffer from inefficiencies when dealing with complex or ill-conditioned objective functions. This paper addresses this challenge by proposing a new conjugate gradient method that combines the descent direction of traditional CG algorithms with Newton-type updates to improve convergence and computational efficiency. The proposed method is constructed to ensure sufficient descent at each iteration and global convergence under standard assumptions. By integrating the modified Newton update mechanism, the method effectively accelerates convergence without incurring the high computational cost typically associated with full Newton methods. To evaluate the performance of the proposed approach, we conducted extensive numerical experiments using a collection of well-known benchmark functions from the CUTEr test suite. The results show that the new method consistently outperforms the classical Hestenes-Stiefel method in terms of CPU time, number of function evaluations, and iteration count. These findings confirm the method’s potential as an efficient and robust alternative for solving large-scale unconstrained optimization problems.
Downloads
References
H. Akhadkulov, S. M. Noorani, A. B. Saaban, F. M. Alipiah, and H. Alsamir, “NOTES ON MULTIDIMENSIONAL FIXED-POINT THEOREMS,” Demonstr. Math., vol. 50, no. 1, pp. 360–374, Dec. 2017, doi: https://doi.org/10.1515/dema-2017-0033
H. Alsamir et al., “FIXED POINT RESULTS IN METRIC-LIKE SPACES VIA Σ-SIMULATION FUNCTIONS,” Eur. J. Pure Appl. Math., vol. 12, pp. 88–100, Jan. 2019. doi: https://doi.org/10.29020/nybg.ejpam.v12i1.3331
C. Luo, L. Wang, Y. Xie, and B. Chen, “A NEW CONJUGATE GRADIENT METHOD FOR MOVING FORCE IDENTIFICATION OF VEHICLE–BRIDGE SYSTEM,” J. Vib. Eng. Technol., vol. 12, no. 1, pp. 19–36, Jan. 2024. doi: https://doi.org/10.1007/s42417-022-00824-1
Y. Yuan, D. H. K. Tsang, and V. K. N. Lau, “COMBINING CONJUGATE GRADIENT AND MOMENTUM FOR UNCONSTRAINED STOCHASTIC OPTIMIZATION WITH APPLICATIONS TO MACHINE LEARNING,” IEEE Internet Things J., vol. 11, no. 13, pp. 23236–23254, Jul. 2024. doi: https://doi.org/10.1109/JIOT.2024.3376821
S. B. Hanachi, B. Sellami, and M. Belloufi, “A NEW FAMILY OF HYBRID CONJUGATE GRADIENT METHOD FOR UNCONSTRAINED OPTIMIZATION AND ITS APPLICATION TO REGRESSION ANALYSIS,” RAIRO - Oper. Res., vol. 58, no. 1, Art. no. 1, Jan. 2024. doi: https://doi.org/10.1051/ro/2023196
B. A. Hassan, I. A. R. Moghrabi, A. L. Ibrahim, and H. N. Jabbar, “IMPROVED CONJUGATE GRADIENT METHODS FOR UNCONSTRAINED MINIMIZATION PROBLEMS AND TRAINING RECURRENT NEURAL NETWORK,” Eng. Rep., vol. 7, no. 2, p. e70019, 2025. doi: https://doi.org/10.1002/eng2.70019
D. H. Omar, A. L. Ibrahim, M. M. Hassan, B. G. Fathi, and D. A. Sulaiman, “ENHANCED CONJUGATE GRADIENT METHOD FOR UNCONSTRAINED OPTIMIZATION AND ITS APPLICATION IN NEURAL NETWORKS,” Eur. J. Pure Appl. Math., vol. 17, no. 4, pp. 2692–2705, Oct. 2024. doi: https://doi.org/10.29020/nybg.ejpam.v17i4.5354.
B. A. Hassan and H. A. Alashoor, “ON IMAGE RESTORATION PROBLEMS USING NEW CONJUGATE GRADIENT METHODS,” Indones. J. Electr. Eng. Comput. Sci., vol. 29, no. 3, p. 1438, Mar. 2023. doi: https://doi.org/10.11591/ijeecs.v29.i3.pp1438-1445
B. A. Hassan and H. Alashoor, “A NEW TYPE COEFFICIENT CONJUGATE ON THE GRADIENT METHODS FOR IMPULSE NOISE REMOVAL IN IMAGES,” Eur. J. Pure Appl. Math., vol. 15, no. 4, pp. 2043–2053, Oct. 2022. doi: https://doi.org/10.29020/nybg.ejpam.v15i4.4579
B. A. Hassan and A. A. Saad, “EXPLAINING NEW PARAMETERS CONJUGATE ANALYSIS BASED ON THE QUADRATIC MODEL,” J. Interdiscip. Math., vol. 26, no. 6, pp. 1219–1229, 2023. doi: https://doi.org/10.47974/JIM-1620
A. Muhammed Awwal, P. Kumam, K. Sitthithakerngkiet, A. Muhammad, A. Halilu, and S. Ibrahim, “DERIVATIVE-FREE METHOD BASED ON DFP UPDATING FORMULA FOR SOLVING CONVEX CONSTRAINED NONLINEAR MONOTONE EQUATIONS AND APPLICATION,” AIMS Math., vol. 6, pp. 8792–8814, Jun. 2021. doi: https://doi.org/10.3934/math.2021510
M. R. Hestenes and E. Stiefel, “METHODS OF CONJUGATE GRADIENTS FOR SOLVING LINEAR SYSTEMS,” J. Res. Natl. Bur. Stand., vol. 49, no. 6, p. 409, Dec. 1952. doi: https://doi.org/10.6028/jres.049.044
R. Fletcher and C. M. Reeves, “FUNCTION MINIMIZATION BY CONJUGATE GRADIENTS,” Comput. J., vol. 7, no. 2, pp. 149–154, Jan. 1964. doi: https://doi.org/10.1093/comjnl/7.2.149
S. G. Shareef and A. L. Ibrahim, “A NEW CONJUGATE GRADIENT FOR UNCONSTRAINED OPTIMIZATION BASED ON STEP SIZE OF BARZILAI AND BORWEIN,” Sci. J. Univ. Zakho, vol. 4, no. 1, Art. no. 1, Jun. 2016. doi: https://doi.org/10.25271/2016.4.1.29
B. A. Hassan and A. Ahmed A. Abdullah, “IMPROVEMENT OF CONJUGATE GRADIENT METHODS FOR REMOVING IMPULSE NOISE IMAGES,” Indones. J. Electr. Eng. Comput. Sci., vol. 29, no. 1, p. 245, Jan. 2022. doi: https://doi.org/10.11591/ijeecs.v29.i1.pp245-251
J. Sabi’u, S. Ibrahim, P. Kaelo, M. Malik, and S. Kamaruddin, “AN OPTIMAL CHOICE DAI-LIAO CONJUGATE GRADIENT ALGORITHM FOR UNCONSTRAINED OPTIMIZATION AND PORTFOLIO SELECTION,” AIMS Math., vol. 9, pp. 642–664, Jan. 2024. doi: https://doi.org/10.3934/math.2024034
J. K. Liu and S. J. Li, “NEW HYBRID CONJUGATE GRADIENT METHOD FOR UNCONSTRAINED OPTIMIZATION,” Appl. Math. Comput., vol. 245, pp. 36–43, Oct. 2014. doi: https://doi.org/10.1016/j.amc.2014.07.096
B. A. Hassan, I. A. R. Moghrabi, T. A. Ameen, R. M. Sulaiman, and I. M. Sulaiman, “IMAGE NOISE REDUCTION AND SOLUTION OF UNCONSTRAINED MINIMIZATION PROBLEMS VIA NEW CONJUGATE GRADIENT METHODS,” Mathematics, vol. 12, no. 17, Art. no. 17, Jan. 2024. doi: https://doi.org/10.3390/math12172754
Z. Salleh and A. Alhawarat, “AN EFFICIENT MODIFICATION OF THE HESTENES-STIEFEL NONLINEAR CONJUGATE GRADIENT METHOD WITH RESTART PROPERTY,” J. Inequalities Appl., vol. 2016, no. 1, p. 110, Apr. 2016. doi: https://doi.org/10.1186/s13660-016-1049-5
G. Yuan, Wei ,Zengxin, and Q. and Zhao, “A MODIFIED POLAK–RIBIÈRE–POLYAK CONJUGATE GRADIENT ALGORITHM FOR LARGE-SCALE OPTIMIZATION PROBLEMS,” IIE Trans., vol. 46, no. 4, pp. 397–413, Apr. 2014. doi: https://doi.org/10.1080/0740817X.2012.726757
Z. Wei, G. Li, and L. Qi, “NEW NONLINEAR CONJUGATE GRADIENT FORMULAS FOR LARGE-SCALE UNCONSTRAINED OPTIMIZATION PROBLEMS,” Appl. Math. Comput., vol. 179, no. 2, pp. 407–430, Aug. 2006. doi: https://doi.org/10.1016/j.amc.2005.11.150
B. A. Hassan, M. Alnaimy, S. Ibrahim, and A. Abubakar, AN ALTERNATIVE MODIFIED QUASI-NEWTON EQUATION FOR SOLVING UNCONSTRAINED OPTIMIZATION PROBLEMS. 2022, p. 205. doi: https://doi.org/10.1109/IT-ELA57378.2022.10107966
P. Wolfe, “CONVERGENCE CONDITIONS FOR ASCENT METHODS. II: SOME CORRECTIONS,” SIAM Rev., vol. 13, no. 2, pp. 185–188, Apr. 1971. doi: https://doi.org/10.1137/1013035
P. Wolfe, “CONVERGENCE CONDITIONS FOR ASCENT METHODS,” SIAM Rev., Jul. 2006. doi: https://doi.org/10.1137/1011036
L. Nazareth, “A CONJUGATE DIRECTION ALGORITHM WITHOUT LINE SEARCHES,” J. Optim. Theory Appl., vol. 23, no. 3, pp. 373–387, Nov. 1977. doi: https://doi.org/10.1007/BF00933447
B. A. Hassan and A. A. Saad, “ELASTIC CONJUGATE GRADIENT METHODS TO SOLVE ITERATION PROBLEMS,” J. Interdiscip. Math., vol. 26, no. 6, pp. 1207–1217, 2023. doi: https://doi.org/10.47974/JIM-1619
N. I. M. Gould, D. Orban, and P. L. Toint, “CUTER AND SIFDEC: A CONSTRAINED AND UNCONSTRAINED TESTING ENVIRONMENT, REVISITED,” ACM Trans Math Softw, vol. 29, no. 4, pp. 373–394, Dec. 2003, doi: https://doi.org/10.1145/962437.962439
N. Andrei, “AN UNCONSTRAINED OPTIMIZATION TEST FUNCTIONS COLLECTION,” 2008. Accessed: Feb. 26, 2025. [Online]. Available: https://www.semanticscholar.org/paper/An-Unconstrained-Optimization-Test-Functions-Andrei/3bcfd42ec838188f3366d75d3568c0638a79c8c4
J. J. Moré, B. S. Garbow, and K. E. Hillstrom, “TESTING UNCONSTRAINED OPTIMIZATION SOFTWARE,” ACM Trans Math Softw, vol. 7, no. 1, pp. 17–41, Mar. 1981. doi: https://doi.org/10.1145/355934.355936
E. D. Dolan and J. J. Moré, “BENCHMARKING OPTIMIZATION SOFTWARE WITH PERFORMANCE PROFILES,” Math. Program., vol. 91, no. 2, pp. 201–213, Jan. 2002. doi: https://doi.org/10.1007/s101070100263
B. A. Hassan and H. M. Sadiq, “A NEW FORMULA ON THE CONJUGATE GRADIENT METHOD FOR REMOVING IMPULSE NOISE IMAGES”, Bulletin of the South Ural State University. Ser. Mathematical Modelling, Programming & Computer Software (Bulletin SUSU MMCS), 2022, vol. 15, no. 4, pp. 123-130. doi: https://doi.org/10.14529/mmp220412
B. A. Hassan and H. Sadiq, “Efficient new conjugate gradient methods for removing impulse noise images”. European Journal of Pure and Applied Mathematics, 15(4), 2011–2021. doi: https://doi.org/10.29020/nybg.ejpam.v15i4.4568
Copyright (c) 2025 Basim A. Hassan, Alaa Luqman Ibrahim, Thaair A. Ameen, Ibrahim Mohammed Sulaiman
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this Journal agree to the following terms:
- Author retain copyright and grant the journal right of first publication with the work simultaneously licensed under a creative commons attribution license that allow others to share the work within an acknowledgement of the work’s authorship and initial publication of this journal.
- Authors are able to enter into separate, additional contractual arrangement for the non-exclusive distribution of the journal’s published version of the work (e.g. acknowledgement of its initial publication in this journal).
- Authors are permitted and encouraged to post their work online (e.g. in institutional repositories or on their websites) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published works.
1.gif)
