NUMERICAL AND CONVERGENCE ANALYSIS OF AN ENHANCED DAI-LIAO METHOD FOR UNCONSTRAINED OPTIMIZATION
Abstract
Iterative algorithms play an important role in mathematical optimization, particularly in solving large-scale unconstrained optimization problems. The conjugate gradient (CG) methods are widely used due to their low memory requirements and efficiency. However, their performance highly depends on the choice of parameters that influence search directions and convergence speed. Despite their advantages, traditional CG algorithms sometimes suffer from slow convergence or poor accuracy, especially for ill-conditioned problems. The selection of conjugate gradient parameters significantly influences the performance, and there is a need to develop improved strategies to enhance solution accuracy and efficiency. This study constructs a new conjugate gradient parameter using the curvature condition to refine search directions and accelerate convergence. The proposed approach ensures a more effective balance between descent properties and numerical stability. Preliminary numerical experiments demonstrate that the proposed method outperforms classical CG variants regarding convergence rate and accuracy. The improved search directions lead to faster and more reliable optimization solutions. The newly developed conjugate gradient formula contributes to a more robust and efficient optimization. This advancement enhances the applicability of CG methods in solving complex optimization problems, paving the way for more effective computational efficiency.
Downloads
References
M. Wang, W. Fu, X. He, S. Hao, and X. Wu, “A SURVEY ON LARGE-SCALE MACHINE LEARNING,” IEEE Trans. Knowl. Data Eng., vol. 34, no. 6, pp. 2574–2594, Jun. 2022, doi: https://doi.org/10.1109/TKDE.2020.3015777.
Y. Shi et al., “MACHINE LEARNING FOR LARGE-SCALE OPTIMIZATION IN 6G WIRELESS NETWORKS,” IEEE Commun. Surv. Tutor., vol. 25, no. 4, pp. 2088–2132, 2023, doi: https://doi.org/10.1109/COMST.2023.3300664
W. Zhang, X. Gu, L. Tang, Y. Yin, D. Liu, and Y. Zhang, “APPLICATION OF MACHINE LEARNING, DEEP LEARNING AND OPTIMIZATION ALGORITHMS IN GEOENGINEERING AND GEOSCIENCE: COMPREHENSIVE REVIEW AND FUTURE CHALLENGE,” Gondwana Res., vol. 109, pp. 1–17, Sep. 2022, doi: https://doi.org/10.1016/j.gr.2022.03.015.
K. Uba kamfa, S. Ibrahim, M. Awang Nawi, M. Waziri, and M. Mamat, “PERFORMANCE EVALUATION OF A NOVEL CONJUGATE GRADIENT METHOD FOR TRAINING FEED FORWARD NEURAL NETWORK,” Math. Model. Comput., vol. 10, Apr. 2023, doi: https://doi.org/10.23939/mmc2023.02.326.
U. A. Omesa, I. M. Sulaiman, M. Malik, B. A. Hassan, W. M. Yusuf, and and M. Mamat, “A THREE-TERM CONJUGATE GRADIENT METHOD FOR NON-CONVEX FUNCTIONS WITH APPLICATIONS FOR HEAT TRANSFER,” Sci. J. King Faisal Univ. Basic Appl. Sci., 2023, Accessed: Apr. 12, 2025. [Online]. Available: https://services.kfu.edu.sa/ScientificJournal/en/Home/ContentsDetails/40702 doi: https://doi.org/10.37575/b/sci/220053
N. Salihu, P. Kumam, A. Muhammed Awwal, S. Ibrahim, and T. Seangwattana, “The global convergence of spectral RMIL conjugate gradient method for unconstrained optimization with applications to robotic model and image recovery,” PLoS ONE, vol. 18, p. e0281250., Mar. 2023, doi: https://doi.org/10.1371/journal.pone.0281250.
R. B. Yunus, N. Zainuddin, H. Daud, R. Kannan, S. A. Abdul Karim, and M. M. Yahaya, “A MODIFIED STRUCTURED SPECTRAL HS METHOD FOR NONLINEAR LEAST SQUARES PROBLEMS AND APPLICATIONS IN ROBOT ARM CONTROL,” Mathematics, vol. 11, no. 14, Art. no. 14, Jan. 2023, doi: https://doi.org/10.3390/math11143215.
B. A. Hassan and H. A. Alashoor, “ON IMAGE RESTORATION PROBLEMS USING NEW CONJUGATE GRADIENT METHODS,” Indones. J. Electr. Eng. Comput. Sci., vol. 29, no. 3, p. 1438, Mar. 2023, doi: https://doi.org/10.11591/ijeecs.v29.i3.pp1438-1445
B. A. Hassan and H. Alashoor, “A NEW TYPE COEFFICIENT CONJUGATE ON THE GRADIENT METHODS FOR IMPULSE NOISE REMOVAL IN IMAGES,” Eur. J. Pure Appl. Math., vol. 15, no. 4, pp. 2043–2053, Oct. 2022, doi: https://doi.org/10.29020/nybg.ejpam.v15i4.4579.
B. A. Hassan, I. A. R. Moghrabi, T. A. Ameen, R. M. Sulaiman, and I. M. Sulaiman, “IMAGE NOISE REDUCTION AND SOLUTION OF UNCONSTRAINED MINIMIZATION PROBLEMS VIA NEW CONJUGATE GRADIENT METHODS,” Mathematics, vol. 12, no. 17, Art. no. 17, Jan. 2024, doi: https://doi.org/10.3390/math12172754
M. R. Hestenes and E. Stiefel, “METHODS OF CONJUGATE GRADIENTS FOR SOLVING LINEAR SYSTEMS,” J. Res. Natl. Bur. Stand., vol. 49, no. 6, p. 409, Dec. 1952, doi: https://doi.org/10.6028/jres.049.044
R. Fletcher and C. M. Reeves, “FUNCTION MINIMIZATION BY CONJUGATE GRADIENTS,” Comput. J., vol. 7, no. 2, pp. 149–154, Jan. 1964, doi: https://doi.org/10.1093/comjnl/7.2.149
E. Polak and G. Ribiere, “NOTE SUR LA CONVERGENCE DE MÉTHODES DE DIRECTIONS CONJUGUÉES,” Rev. Fr. Inform. Rech. Opérationnelle Sér. Rouge, vol. 3, no. 16, Art. no. 16, 1969, doi: https://doi.org/10.1051/m2an/196903R100351.
B. T. Polyak, “THE CONJUGATE GRADIENT METHOD IN EXTREMAL PROBLEMS,” USSR Comput. Math. Math. Phys., vol. 9, no. 4, pp. 94–112, Jan. 1969, doi: https://doi.org/10.1016/0041-5553(69)90035-4.
Y. H. Dai and Y. Yuan, “A NONLINEAR CONJUGATE GRADIENT METHOD WITH A STRONG GLOBAL CONVERGENCE PROPERTY,” SIAM J. Optim., vol. 10, no. 1, pp. 177–182, Jan. 1999, doi: https://doi.org/10.1137/S1052623497318992.
R. Fletcher, “INTRODUCTION,” IN PRACTICAL METHODS OF OPTIMIZATION, John Wiley & Sons, Ltd, 2000, pp. 1–11. doi: https://doi.org/10.1002/9781118723203.ch1
B. A. Hassan, M. Alnaimy, S. Ibrahim, and A. Abubakar, AN ALTERNATIVE MODIFIED QUASI-NEWTON EQUATION FOR SOLVING UNCONSTRAINED OPTIMIZATION PROBLEMS. 2022, p. 205. doi: https://doi.org/10.1109/IT-ELA57378.2022.10107966
G. B. Fotopoulos, P. Popovich, and N. H. Papadopoulos, “REVIEW NON-CONVEX OPTIMIZATION METHOD FOR MACHINE LEARNING,” Oct. 02, 2024, arXiv: arXiv:2410.02017. doi: https://doi.org/10.48550/arXiv.2410.02017
M. Danilova et al., “RECENT THEORETICAL ADVANCES IN NON-CONVEX OPTIMIZATION,” in High-Dimensional Optimization and Probability: With a View Towards Data Science, A. Nikeghbali, P. M. Pardalos, A. M. Raigorodskii, and M. Th. Rassias, Eds., Cham: Springer International Publishing, 2022, pp. 79–163. doi: https://doi.org/10.1007/978-3-031-00832-0_3
G. Yuan, J. Lu, and Z. Wang, “THE PRP CONJUGATE GRADIENT ALGORITHM WITH A MODIFIED WWP LINE SEARCH AND ITS APPLICATION IN THE IMAGE RESTORATION PROBLEMS,” Appl. Numer. Math., vol. 152, pp. 1–11, Jun. 2020, doi: https://doi.org/10.1016/j.apnum.2020.01.019
Y. -H. Dai and L. -Z. Liao, “NEW CONJUGACY CONDITIONS AND RELATED NONLINEAR CONJUGATE GRADIENT METHODS,” Appl. Math. Optim., vol. 43, no. 1, pp. 87–101, Jan. 2001, doi: https://doi.org/10.1007/s002450010019.
J. Sabi’u, S. Ibrahim, P. Kaelo, M. Malik, and S. Kamaruddin, “AN OPTIMAL CHOICE DAI-LIAO CONJUGATE GRADIENT ALGORITHM FOR UNCONSTRAINED OPTIMIZATION AND PORTFOLIO SELECTION,” AIMS Math., vol. 9, pp. 642–664, Jan. 2024, doi: https://doi.org/10.3934/math.2024034
B. A. Hassan, Z. M. Abdullah, and S. A. Hussein, “ROBUST PARAMETERS FOR CONJUGATE GRADIENT METHOD IN UNCONSTRAINED OPTIMIZATION,” in 2022 International Conference on Data Science and Intelligent Computing (ICDSIC), Nov. 2022, pp. 257–260. doi: https://doi.org/10.1109/ICDSIC56987.2022.10076040
S. P. S, I. Branislav, M. Haifeng, and M. Dijana, “A SURVEY OF GRADIENT METHODS FOR SOLVING NONLINEAR OPTIMIZATION,” Electron. Res. Arch., vol. 28, no. 4, pp. 1573–1624, Dec. 2020, doi: https://doi.org/10.3934/era.2020115.
S. Babaie-Kafaki, “A SURVEY ON THE DAI–LIAO FAMILY OF NONLINEAR CONJUGATE GRADIENT METHODS,” RAIRO - Oper. Res., vol. 57, no. 1, Art. no. 1, Jan. 2023, doi: https://doi.org/10.1051/ro/2022213
M. Y. Waziri, K. Ahmed, and J. Sabi’u, “A DAI–LIAO CONJUGATE GRADIENT METHOD VIA MODIFIED SECANT EQUATION FOR SYSTEM OF NONLINEAR EQUATIONS,” Arab. J. Math., vol. 9, no. 2, pp. 443–457, Aug. 2020, doi: https://doi.org/10.1007/s40065-019-0264-6
B. A. Hassan and A. A. Saad, “ELASTIC CONJUGATE GRADIENT METHODS TO SOLVE ITERATION PROBLEMS,” J. Interdiscip. Math., vol. 26, no. 6, pp. 1207–1217, 2023, doi: https://doi.org/10.47974/JIM-1619
B. A. Hassan, H. N. Jabbar, and Y. A. Laylani, “UPSCALING PARAMETERS FOR CONJUGATE GRADIENT METHOD IN UNCONSTRAINED OPTIMIZATION,” J. Interdiscip. Math., vol. 26, no. 6, pp. 1171–1180, 2023, doi: https://doi.org/10.47974/JIM-1615.
B. A. Hassan and A. A. Saad, “EXPLAINING NEW PARAMETERS CONJUGATE ANALYSIS BASED ON THE QUADRATIC MODEL,” J. Interdiscip. Math., vol. 26, no. 6, pp. 1219–1229, 2023, doi: https://doi.org/10.47974/JIM-1620
B. A. Hassan, R. Moghrabi, and S. Ibrahim, “NEW CONJUGATE GRADIENT IMAGE PROCESSING METHODS 5,” Asian-Eur. J. Math., vol. 16, Mar. 2023, doi: https://doi.org/10.1142/S1793557123500997
I. M. Sulaiman, Sukono, S. Supian, and M. Mamat, “NEW CLASS OF HYBRID CONJUGATE GRADIENT COEFFICIENTS WITH GUARANTEED DESCENT AND EFFICIENT LINE SEARCH,” IOP Conf. Ser. Mater. Sci. Eng., vol. 621, no. 1, p. 012021, Oct. 2019, doi: https://doi.org/10.1088/1757-899X/621/1/012021
B. A. Hassan and A. Ahmed A. Abdullah, “IMPROVEMENT OF CONJUGATE GRADIENT METHODS FOR REMOVING IMPULSE NOISE IMAGES,” Indones. J. Electr. Eng. Comput. Sci., vol. 29, no. 1, p. 245, Jan. 2022, doi: https://doi.org/10.11591/ijeecs.v29.i1.pp245-251
A. Perry, “TECHNICAL NOTE—A MODIFIED CONJUGATE GRADIENT ALGORITHM,” Oper. Res., vol. 26, no. 6, pp. 1073–1078, Dec. 1978, doi: https://doi.org/10.1287/opre.26.6.1073
G. Zoutendijk, “NONLINEAR PROGRAMMING, COMPUTATIONAL METHODS,” Integer Nonlinear Program., pp. 37–86, 1970.
N. Andrei, “AN UNCONSTRAINED OPTIMIZATION TEST FUNCTIONS COLLECTION,” 2008. Accessed: Feb. 26, 2025. [Online]. Available: https://www.semanticscholar.org/paper/An-Unconstrained-Optimization-Test-Functions-Andrei/3bcfd42ec838188f3366d75d3568c0638a79c8c4
E. D. Dolan and J. J. Moré, “BENCHMARKING OPTIMIZATION SOFTWARE WITH PERFORMANCE PROFILES,” Math. Program., vol. 91, no. 2, pp. 201–213, Jan. 2002, doi: https://doi.org/10.1007/s101070100263
Copyright (c) 2025 Basim A. Hassan, Sulaiman Mohammed Ibrahim, Yeldez J. Subhi
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this Journal agree to the following terms:
- Author retain copyright and grant the journal right of first publication with the work simultaneously licensed under a creative commons attribution license that allow others to share the work within an acknowledgement of the work’s authorship and initial publication of this journal.
- Authors are able to enter into separate, additional contractual arrangement for the non-exclusive distribution of the journal’s published version of the work (e.g. acknowledgement of its initial publication in this journal).
- Authors are permitted and encouraged to post their work online (e.g. in institutional repositories or on their websites) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published works.
1.gif)
