Vol 19 No 4 (2025): BAREKENG: Journal of Mathematics and Its Application
Articles

NUMERICAL AND CONVERGENCE ANALYSIS OF AN ENHANCED DAI-LIAO METHOD FOR UNCONSTRAINED OPTIMIZATION

pdf
  • Basim A. Hassan,
  • Ibrahim Mohammed Sulaiman,
  • Yeldez J. Subhi
Basim A. Hassan
Department of Mathematics, College of Computers Sciences and Mathematics, University of Mosul, Iraq
Ibrahim Mohammed Sulaiman
Faculty of Education and Arts, Sohar University, Oman
Yeldez J. Subhi
Department of Renewable Energy Techniques Engineering, College of Oil and Gas Techniques Engineering Northern technical University, Iraq
DOI: https://doi.org/10.30598/barekengvol19iss4pp2993-3004
Published September 1, 2025
Keywords
  • Enhancement Dai-Liao,
  • Convergence property,
  • Iteration problems
How to Cite
[1]
B. A. Hassan, I. M. Sulaiman, and Y. J. Subhi, “NUMERICAL AND CONVERGENCE ANALYSIS OF AN ENHANCED DAI-LIAO METHOD FOR UNCONSTRAINED OPTIMIZATION”, BAREKENG: J. Math. & App., vol. 19, no. 4, pp. 2993-3004, Sep. 2025.

Copyright (c) 2025 Basim A. Hassan, Sulaiman Mohammed Ibrahim, Yeldez J. Subhi

Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Abstract

Iterative algorithms play an important role in mathematical optimization, particularly in solving large-scale unconstrained optimization problems. The conjugate gradient (CG) methods are widely used due to their low memory requirements and efficiency. However, their performance highly depends on the choice of parameters that influence search directions and convergence speed. Despite their advantages, traditional CG algorithms sometimes suffer from slow convergence or poor accuracy, especially for ill-conditioned problems. The selection of conjugate gradient parameters significantly influences the performance, and there is a need to develop improved strategies to enhance solution accuracy and efficiency. This study constructs a new conjugate gradient parameter using the curvature condition to refine search directions and accelerate convergence. The proposed approach ensures a more effective balance between descent properties and numerical stability. Preliminary numerical experiments demonstrate that the proposed method outperforms classical CG variants regarding convergence rate and accuracy. The improved search directions lead to faster and more reliable optimization solutions. The newly developed conjugate gradient formula contributes to a more robust and efficient optimization. This advancement enhances the applicability of CG methods in solving complex optimization problems, paving the way for more effective computational efficiency.

 

pdf

Downloads

Download data is not yet available.

References

  1. M. Wang, W. Fu, X. He, S. Hao, and X. Wu, “A SURVEY ON LARGE-SCALE MACHINE LEARNING,” IEEE Trans. Knowl. Data Eng., vol. 34, no. 6, pp. 2574–2594, Jun. 2022, doi: https://doi.org/10.1109/TKDE.2020.3015777.
  2. Y. Shi et al., “MACHINE LEARNING FOR LARGE-SCALE OPTIMIZATION IN 6G WIRELESS NETWORKS,” IEEE Commun. Surv. Tutor., vol. 25, no. 4, pp. 2088–2132, 2023, doi: https://doi.org/10.1109/COMST.2023.3300664
  3. W. Zhang, X. Gu, L. Tang, Y. Yin, D. Liu, and Y. Zhang, “APPLICATION OF MACHINE LEARNING, DEEP LEARNING AND OPTIMIZATION ALGORITHMS IN GEOENGINEERING AND GEOSCIENCE: COMPREHENSIVE REVIEW AND FUTURE CHALLENGE,” Gondwana Res., vol. 109, pp. 1–17, Sep. 2022, doi: https://doi.org/10.1016/j.gr.2022.03.015.
  4. K. Uba kamfa, S. Ibrahim, M. Awang Nawi, M. Waziri, and M. Mamat, “PERFORMANCE EVALUATION OF A NOVEL CONJUGATE GRADIENT METHOD FOR TRAINING FEED FORWARD NEURAL NETWORK,” Math. Model. Comput., vol. 10, Apr. 2023, doi: https://doi.org/10.23939/mmc2023.02.326.
  5. U. A. Omesa, I. M. Sulaiman, M. Malik, B. A. Hassan, W. M. Yusuf, and and M. Mamat, “A THREE-TERM CONJUGATE GRADIENT METHOD FOR NON-CONVEX FUNCTIONS WITH APPLICATIONS FOR HEAT TRANSFER,” Sci. J. King Faisal Univ. Basic Appl. Sci., 2023, Accessed: Apr. 12, 2025. [Online]. Available: https://services.kfu.edu.sa/ScientificJournal/en/Home/ContentsDetails/40702 doi: https://doi.org/10.37575/b/sci/220053
  6. N. Salihu, P. Kumam, A. Muhammed Awwal, S. Ibrahim, and T. Seangwattana, “The global convergence of spectral RMIL conjugate gradient method for unconstrained optimization with applications to robotic model and image recovery,” PLoS ONE, vol. 18, p. e0281250., Mar. 2023, doi: https://doi.org/10.1371/journal.pone.0281250.
  7. R. B. Yunus, N. Zainuddin, H. Daud, R. Kannan, S. A. Abdul Karim, and M. M. Yahaya, “A MODIFIED STRUCTURED SPECTRAL HS METHOD FOR NONLINEAR LEAST SQUARES PROBLEMS AND APPLICATIONS IN ROBOT ARM CONTROL,” Mathematics, vol. 11, no. 14, Art. no. 14, Jan. 2023, doi: https://doi.org/10.3390/math11143215.
  8. B. A. Hassan and H. A. Alashoor, “ON IMAGE RESTORATION PROBLEMS USING NEW CONJUGATE GRADIENT METHODS,” Indones. J. Electr. Eng. Comput. Sci., vol. 29, no. 3, p. 1438, Mar. 2023, doi: https://doi.org/10.11591/ijeecs.v29.i3.pp1438-1445
  9. B. A. Hassan and H. Alashoor, “A NEW TYPE COEFFICIENT CONJUGATE ON THE GRADIENT METHODS FOR IMPULSE NOISE REMOVAL IN IMAGES,” Eur. J. Pure Appl. Math., vol. 15, no. 4, pp. 2043–2053, Oct. 2022, doi: https://doi.org/10.29020/nybg.ejpam.v15i4.4579.
  10. B. A. Hassan, I. A. R. Moghrabi, T. A. Ameen, R. M. Sulaiman, and I. M. Sulaiman, “IMAGE NOISE REDUCTION AND SOLUTION OF UNCONSTRAINED MINIMIZATION PROBLEMS VIA NEW CONJUGATE GRADIENT METHODS,” Mathematics, vol. 12, no. 17, Art. no. 17, Jan. 2024, doi: https://doi.org/10.3390/math12172754
  11. M. R. Hestenes and E. Stiefel, “METHODS OF CONJUGATE GRADIENTS FOR SOLVING LINEAR SYSTEMS,” J. Res. Natl. Bur. Stand., vol. 49, no. 6, p. 409, Dec. 1952, doi: https://doi.org/10.6028/jres.049.044
  12. R. Fletcher and C. M. Reeves, “FUNCTION MINIMIZATION BY CONJUGATE GRADIENTS,” Comput. J., vol. 7, no. 2, pp. 149–154, Jan. 1964, doi: https://doi.org/10.1093/comjnl/7.2.149
  13. E. Polak and G. Ribiere, “NOTE SUR LA CONVERGENCE DE MÉTHODES DE DIRECTIONS CONJUGUÉES,” Rev. Fr. Inform. Rech. Opérationnelle Sér. Rouge, vol. 3, no. 16, Art. no. 16, 1969, doi: https://doi.org/10.1051/m2an/196903R100351.
  14. B. T. Polyak, “THE CONJUGATE GRADIENT METHOD IN EXTREMAL PROBLEMS,” USSR Comput. Math. Math. Phys., vol. 9, no. 4, pp. 94–112, Jan. 1969, doi: https://doi.org/10.1016/0041-5553(69)90035-4.
  15. Y. H. Dai and Y. Yuan, “A NONLINEAR CONJUGATE GRADIENT METHOD WITH A STRONG GLOBAL CONVERGENCE PROPERTY,” SIAM J. Optim., vol. 10, no. 1, pp. 177–182, Jan. 1999, doi: https://doi.org/10.1137/S1052623497318992.
  16. R. Fletcher, “INTRODUCTION,” IN PRACTICAL METHODS OF OPTIMIZATION, John Wiley & Sons, Ltd, 2000, pp. 1–11. doi: https://doi.org/10.1002/9781118723203.ch1
  17. B. A. Hassan, M. Alnaimy, S. Ibrahim, and A. Abubakar, AN ALTERNATIVE MODIFIED QUASI-NEWTON EQUATION FOR SOLVING UNCONSTRAINED OPTIMIZATION PROBLEMS. 2022, p. 205. doi: https://doi.org/10.1109/IT-ELA57378.2022.10107966
  18. G. B. Fotopoulos, P. Popovich, and N. H. Papadopoulos, “REVIEW NON-CONVEX OPTIMIZATION METHOD FOR MACHINE LEARNING,” Oct. 02, 2024, arXiv: arXiv:2410.02017. doi: https://doi.org/10.48550/arXiv.2410.02017
  19. M. Danilova et al., “RECENT THEORETICAL ADVANCES IN NON-CONVEX OPTIMIZATION,” in High-Dimensional Optimization and Probability: With a View Towards Data Science, A. Nikeghbali, P. M. Pardalos, A. M. Raigorodskii, and M. Th. Rassias, Eds., Cham: Springer International Publishing, 2022, pp. 79–163. doi: https://doi.org/10.1007/978-3-031-00832-0_3
  20. G. Yuan, J. Lu, and Z. Wang, “THE PRP CONJUGATE GRADIENT ALGORITHM WITH A MODIFIED WWP LINE SEARCH AND ITS APPLICATION IN THE IMAGE RESTORATION PROBLEMS,” Appl. Numer. Math., vol. 152, pp. 1–11, Jun. 2020, doi: https://doi.org/10.1016/j.apnum.2020.01.019
  21. Y. -H. Dai and L. -Z. Liao, “NEW CONJUGACY CONDITIONS AND RELATED NONLINEAR CONJUGATE GRADIENT METHODS,” Appl. Math. Optim., vol. 43, no. 1, pp. 87–101, Jan. 2001, doi: https://doi.org/10.1007/s002450010019.
  22. J. Sabi’u, S. Ibrahim, P. Kaelo, M. Malik, and S. Kamaruddin, “AN OPTIMAL CHOICE DAI-LIAO CONJUGATE GRADIENT ALGORITHM FOR UNCONSTRAINED OPTIMIZATION AND PORTFOLIO SELECTION,” AIMS Math., vol. 9, pp. 642–664, Jan. 2024, doi: https://doi.org/10.3934/math.2024034
  23. B. A. Hassan, Z. M. Abdullah, and S. A. Hussein, “ROBUST PARAMETERS FOR CONJUGATE GRADIENT METHOD IN UNCONSTRAINED OPTIMIZATION,” in 2022 International Conference on Data Science and Intelligent Computing (ICDSIC), Nov. 2022, pp. 257–260. doi: https://doi.org/10.1109/ICDSIC56987.2022.10076040
  24. S. P. S, I. Branislav, M. Haifeng, and M. Dijana, “A SURVEY OF GRADIENT METHODS FOR SOLVING NONLINEAR OPTIMIZATION,” Electron. Res. Arch., vol. 28, no. 4, pp. 1573–1624, Dec. 2020, doi: https://doi.org/10.3934/era.2020115.
  25. S. Babaie-Kafaki, “A SURVEY ON THE DAI–LIAO FAMILY OF NONLINEAR CONJUGATE GRADIENT METHODS,” RAIRO - Oper. Res., vol. 57, no. 1, Art. no. 1, Jan. 2023, doi: https://doi.org/10.1051/ro/2022213
  26. M. Y. Waziri, K. Ahmed, and J. Sabi’u, “A DAI–LIAO CONJUGATE GRADIENT METHOD VIA MODIFIED SECANT EQUATION FOR SYSTEM OF NONLINEAR EQUATIONS,” Arab. J. Math., vol. 9, no. 2, pp. 443–457, Aug. 2020, doi: https://doi.org/10.1007/s40065-019-0264-6
  27. B. A. Hassan and A. A. Saad, “ELASTIC CONJUGATE GRADIENT METHODS TO SOLVE ITERATION PROBLEMS,” J. Interdiscip. Math., vol. 26, no. 6, pp. 1207–1217, 2023, doi: https://doi.org/10.47974/JIM-1619
  28. B. A. Hassan, H. N. Jabbar, and Y. A. Laylani, “UPSCALING PARAMETERS FOR CONJUGATE GRADIENT METHOD IN UNCONSTRAINED OPTIMIZATION,” J. Interdiscip. Math., vol. 26, no. 6, pp. 1171–1180, 2023, doi: https://doi.org/10.47974/JIM-1615.
  29. B. A. Hassan and A. A. Saad, “EXPLAINING NEW PARAMETERS CONJUGATE ANALYSIS BASED ON THE QUADRATIC MODEL,” J. Interdiscip. Math., vol. 26, no. 6, pp. 1219–1229, 2023, doi: https://doi.org/10.47974/JIM-1620
  30. B. A. Hassan, R. Moghrabi, and S. Ibrahim, “NEW CONJUGATE GRADIENT IMAGE PROCESSING METHODS 5,” Asian-Eur. J. Math., vol. 16, Mar. 2023, doi: https://doi.org/10.1142/S1793557123500997
  31. I. M. Sulaiman, Sukono, S. Supian, and M. Mamat, “NEW CLASS OF HYBRID CONJUGATE GRADIENT COEFFICIENTS WITH GUARANTEED DESCENT AND EFFICIENT LINE SEARCH,” IOP Conf. Ser. Mater. Sci. Eng., vol. 621, no. 1, p. 012021, Oct. 2019, doi: https://doi.org/10.1088/1757-899X/621/1/012021
  32. B. A. Hassan and A. Ahmed A. Abdullah, “IMPROVEMENT OF CONJUGATE GRADIENT METHODS FOR REMOVING IMPULSE NOISE IMAGES,” Indones. J. Electr. Eng. Comput. Sci., vol. 29, no. 1, p. 245, Jan. 2022, doi: https://doi.org/10.11591/ijeecs.v29.i1.pp245-251
  33. A. Perry, “TECHNICAL NOTE—A MODIFIED CONJUGATE GRADIENT ALGORITHM,” Oper. Res., vol. 26, no. 6, pp. 1073–1078, Dec. 1978, doi: https://doi.org/10.1287/opre.26.6.1073
  34. G. Zoutendijk, “NONLINEAR PROGRAMMING, COMPUTATIONAL METHODS,” Integer Nonlinear Program., pp. 37–86, 1970.
  35. N. Andrei, “AN UNCONSTRAINED OPTIMIZATION TEST FUNCTIONS COLLECTION,” 2008. Accessed: Feb. 26, 2025. [Online]. Available: https://www.semanticscholar.org/paper/An-Unconstrained-Optimization-Test-Functions-Andrei/3bcfd42ec838188f3366d75d3568c0638a79c8c4
  36. E. D. Dolan and J. J. Moré, “BENCHMARKING OPTIMIZATION SOFTWARE WITH PERFORMANCE PROFILES,” Math. Program., vol. 91, no. 2, pp. 201–213, Jan. 2002, doi: https://doi.org/10.1007/s101070100263