AN IMPROVED HYBRID CONJUGATE GRADIENT METHOD WITH SPECTRAL STRATEGY AND ITS APPLICATIONS IN COVID-19 PREDICTION
Abstract
This paper introduces a hybrid conjugate gradient (CG) method for unconstrained optimization with a spectral strategy, inspired by key advancements in existing CG techniques. The proposed method guarantees a descent direction at every iteration, regardless of the line search scheme employed. Its global convergence is rigorously established under the Wolfe line search conditions. Numerical experiments on benchmark optimization problems demonstrate that the proposed method outperforms the FR and RMIL methods across multiple performance metrics. Furthermore, its effectiveness is showcased in a neural network framework for predicting chickenpox and COVID-19 infection cases, highlighting its practical applicability in real-world scenarios.
Downloads
References
K. U. Kamfa, A. A. Mustafa Mamat, M. Rivaie, P. L. B. Ghazali, and Z. Salleh, “ANOTHER MODIFIED CONJUGATE GRADIENT COEFFICIENT WITH GLOBAL CONVERGENCE PROPERTIES,” Applied Mathematical Sciences, vol. 9, no. 37, pp. 1833–1844, 2015, doi: https://doi.org/10.12988/ams.2015.411996.
K. K. Ibrahim Mohammed Sulaiman Mustafa Mamat and M. Danlami, “A DESCENT MODIFICATION OF CONJUGATE GRADIENT METHOD FOR OPTIMIZATION MODELS,” Iraqi Journal of Science, vol. 7, no. 61, 2020, [Online]. Available: https://ijs.uobaghdad.edu.iq/index.php/eijs/article/view/1654
M. Rivaie, M. Mamat, L. W. June, and I. Mohd, “A NEW CLASS OF NONLINEAR CONJUGATE GRADIENT COEFFICIENTS WITH GLOBAL CONVERGENCE PROPERTIES,” Appl Math Comput, vol. 218, no. 22, pp. 11323–11332, 2012, doi: https://doi.org/10.1016/j.amc.2012.05.030 .
N. Andrei, “NONLINEAR CONJUGATE GRADIENT METHODS FOR UNCONSTRAINED OPTIMIZATION.,” Springer, Berlin, 2020.doi: https://doi.org/10.1007/978-3-030-42950-8
Z. Dai, “COMMENTS ON A NEW CLASS OF NONLINEAR CONJUGATE GRADIENT COEFFICIENTS WITH GLOBAL CONVERGENCE PROPERTIES,” Appl Math Comput, vol. 276, pp. 297–300, 2016, doi: https://doi.org/10.1016/j.amc.2015.11.085 .
E. Polak and G. Ribiere, “NOTE SUR LA CONVERGENCE DE MÉTHODES DE DIRECTIONS CONJUGUÉES.,” ESAIM: Math. Model. Numer. Anal. , vol. R1, no. 3, pp. 35–43, 1969.doi: https://doi.org/10.1051/m2an/196903R100351
E. Stiefel, “METHODS OF CONJUGATE GRADIENTS FOR SOLVING LINEAR SYSTEMS.,” J. Res. Natl. Bur. Stand., vol. 49, pp. 409–435, 1952.doi: https://doi.org/10.6028/jres.049.044
M. J. D. Fletcher R Powell, “A RAPIDLY CONVERGENT DESCENT METHOD FOR MINIMIZATION.,” Comput. J., vol. 2, no. 49, pp. 163–168, 1963.doi: https://doi.org/10.1093/comjnl/6.2.163
Y. H. Dai and Y. Yuan, “A NONLINEAR CONJUGATE GRADIENT METHOD WITH A STRONG GLOBAL CONVERGENCE PROPERTY,” SIAM Journal on Optimization, vol. 10, no. 1, pp. 177–182, 1999, doi: https://doi.org/10.1137/S1052623497318992.
E. Polak and G. Ribiere, “THE CONJUGATE GRADIENT METHOD IN EXTREMAL PROBLEMS.,” USSR Comput. Math. Math. Phys., vol. 4, no. 9, pp. 94–112, 1969.doi: https://doi.org/10.1016/0041-5553(69)90035-4
Y. Liu and C. Storey, “EFFICIENT GENERALIZED CONJUGATE GRADIENT ALGORITHMS,” part 1: theory. J. Optim. Theory Appl. , vol. 1, no. 69, pp. 129–137, 1991.doi: https://doi.org/10.1007/BF00940464
Z. Wei, S. Yao, and L. Liu, “THE CONVERGENCE PROPERTIES OF SOME NEW CONJUGATE GRADIENT METHODS,” Appl Math Comput, vol. 183, no. 2, pp. 1341–1350, 2006, doi: https://doi.org/10.1016/j.amc.2006.05.150 .
N. Andrei, “OPEN PROBLEMS IN NONLINEAR CONJUGATE GRADIENT ALGORITHMS FOR UNCONSTRAINED OPTIMIZATION.,” Bulletin of the Malaysian Mathematical Sciences Society. Second Series, vol. 34, no. 2, pp. 319–330, 2011.
K. Kamilu, M. Sulaiman, A. Muhammad, A. Mohamad, and M. Mamat, “PERFORMANCE EVALUATION OF A NOVEL CONJUGATE GRADIENT METHOD FOR TRAINING FEED FORWARD NEURAL NETWORK,” Math. Model. Comp, vol. 10, pp. 326–337, 2023.doi: https://doi.org/10.23939/mmc2023.02.326
G. Zoutendijk, “COMPUTATIONAL METHODS IN NONLINEAR PROGRAMMING,” Studies in optimization, vol. 1, p. 125, 1970.
M. J. D. Powell, “RESTART PROCEDURES FOR THE CONJUGATE GRADIENT METHOD,” Math Program, vol. 12, pp. 241–254, 1977.doi: https://doi.org/10.1007/BF01593790
M. J. D. Powell, “SOME CONVERGENCE PROPERTIES OF THE CONJUGATE GRADIENT METHOD,” Math Program, vol. 11, pp. 42–49, 1976.doi: https://doi.org/10.1007/BF01580369
H. Zhang and W. W. Hager, “A NONMONOTONE LINE SEARCH TECHNIQUE AND ITS APPLICATION TO UNCONSTRAINED OPTIMIZATION,” SIAM Journal on Optimization, vol. 14, no. 4, pp. 1043–1056, 2004, doi: https://doi.org/10.1137/S1052623403428208.
W. Z. & C. X. Huang S., “A NEW NONMONOTONE LINE SEARCH TECHNIQUE FOR UNCONSTRAINED OPTIMIZATION,” Numer Algor, vol. 68, pp. 671–689, 2015, doi: https://doi.org/10.1007/s11075-014-9866-4 .
Y. Dai, J. Han, G. Liu, D. Sun, H. Yin, and Y. X. Yuan, “CONVERGENCE PROPERTIES OF NONLINEAR CONJUGATE GRADIENT METHODS.,” SIAM J. Optim. , vol. 2, no. 10, pp. 345–358, 2000.doi: https://doi.org/10.1137/S1052623494268443
T.-A. D. S. C, “EFFICIENT HYBRID CONJUGATE GRADIENT TECHNIQUES.,” J. Optim. Theory Appl., vol. 2, no. 64, pp. 379–397, 1990.doi: https://doi.org/10.1007/BF00939455
M. Al-Baali, “ DESCENT PROPERTY AND GLOBAL CONVERGENCE OF THE FLETCHER–REEVES METHOD WITH INEXACT LINE SEARCH,” IMA J. Numer. Anal., vol. 5, no. 1, pp. 121–124, 1985.doi: https://doi.org/10.1093/imanum/5.1.121
J. C. N. j Gilbert, “GLOBAL CONVERGENCE PROPERTIES OF CONJUGATE GRADIENT METHODS FOR OPTIMIZATION,” SIAM J. Optim., vol. 2, no. 1, pp. 21–42, 1992.doi: https://doi.org/10.1137/0802003
M. M. Sulaiman I M, “A NEW CONJUGATE GRADIENT METHOD WITH DESCENT PROPERTIES AND ITS APPLICATION TO REGRESSION ANALYSIS.,” J. Numer. Anal. Ind. Appl. Math., vol. 1–2, no. 14, pp. 25–39, 2020.
E. Ali and S. Mahdi, “A FAMILY OF DEVELOPED HYBRID FOUR-TERM CONJUGATE GRADIENT ALGORITHMS FOR UNCONSTRAINED OPTIMIZATION WITH APPLICATIONS IN IMAGE RESTORATION,” Symmetry (Basel), vol. 15, no. 6, 2023, doi: https://doi.org/10.3390/sym15061203.
W. W. Hager and H. Zhang, “A SURVEY OF NONLINEAR CONJUGATE GRADIENT METHODS,”,” Pacific Journal of Optimization, vol. 2, pp. 35–58, 2006.
Y. F. Hu and C. Storey, “GLOBAL CONVERGENCE RESULT FOR CONJUGATE GRADIENT METHODS,” J Optim Theory Appl, vol. 71, no. 2, pp. 399–405, 1991.doi: https://doi.org/10.1007/BF00939927
M. Malik, M. Mamat, S. S. Abas, I. M. Sulaiman, and F. Sukono, “A NEW COEFFICIENT OF THE CONJUGATE GRADIENT METHOD WITH THE SUFFICIENT DESCENT CONDITION AND GLOBAL CONVERGENCE PROPERTIES,” Engineering Letters, vol. 28, no. 3, pp. 62–72, 2020.
L. : Armijo, “MINIMIZATION OF FUNCTIONS HAVING LIPSCHITZ CONTINUOUS FIRST PARTIAL DERIVATIVES,” Pac. J. Math, vol. 16, pp. 1–3, 1966.doi: https://doi.org/10.2140/pjm.1966.16.1
A. A. : Goldstein, “GOLDSTEIN ON STEEPEST DESCEN.,” Pac. J. Math, vol. 3, pp. 147–151, 1965.doi: https://doi.org/10.1137/0303013
P. Wolfe, “CONVERGENCE CONDITIONS FOR ASCENT METHODS.,” SIAM Review, vol. 11, pp. 226–235, 1969.doi: https://doi.org/10.1137/1011036
X. S. Zhong Wan Kok Lay Teo and C. Hu, “NEW BFGS METHOD FOR UNCONSTRAINED OPTIMIZATION PROBLEM BASED ON MODIFIED ARMIJO LINE SEARCH,” Optimization, vol. 63, no. 2, pp. 285–304, 2014, doi: https://doi.org/10.1080/02331934.2011.644284.
Z. S. Zhou w Chalabi, “MODIFICATIONS OF THE WOLFE LINE SEARCH RULES TO SATISFY SECOND-ORDER OPTIMALLY CONDITIONS IN UNCONSTRAINED OPTIMIZATION.,” SIAM Review, vol. 11, pp. 226–235, 1969.
G. M. Al-Naemi, “A NEW MODIFIED HS ALGORITHM WITH STRONG POWELL-WOLFE LINE SEARCH FOR UNCONSTRAINED OPTIMIZATION,” Eastern-European Journal of Enterprise Technologies, vol. 4, no. 2, pp. 14–21, 2022.doi: https://doi.org/10.15587/1729-4061.2022.254017
W. W. Z. H. Hager, “A SURVEY OF NONLINEAR CONJUGATE GRADIENT METHODS.,” Pac. J. Optim., vol. 1, no. 2, pp. 35–58, 2006.
I. M. Sulaiman, M. Malik, A. M. Awwal, P. Kumam, M. Mamat, and S. Al-Ahmad, “ON THREE-TERM CONJUGATE GRADIENT METHOD FOR OPTIMIZATION PROBLEMS WITH APPLICATIONS ON COVID-19 MODEL AND ROBOTIC MOTION CONTROL,” Advances in Continuous and Discrete Models, vol. 2022, no. 1, p. 1, 2022.doi: https://doi.org/10.1186/s13662-021-03638-9
M. Malik, I. M. Sulaiman, A. B. Abubakar, G. Ardaneswari, and Sukono, “A NEW FAMILY OF HYBRID THREE-TERM CONJUGATE GRADIENT METHOD FOR UNCONSTRAINED OPTIMIZATION WITH APPLICATION TO IMAGE RESTORATION AND PORTFOLIO SELECTION,” AIMS Mathematics, vol. 8, no. 1, pp. 1–28, 2023, doi: https://doi.org/10.3934/math.2023001.
Y. Yamakawa, H. Sato, and K. Aihara, “MODIFIED ARMIJO LINE-SEARCH IN RIEMANNIAN OPTIMIZATION WITH REDUCED COMPUTATIONAL COST,” 2023. [Online]. Available: https://arxiv.org/abs/2304.02197
K. Amini and P. Faramarzi, “GLOBAL CONVERGENCE OF A MODIFIED SPECTRAL THREE-TERM CG ALGORITHM FOR NONCONVEX UNCONSTRAINED OPTIMIZATION PROBLEMS,” J Comput Appl Math, vol. 417, p. 114630, 2023, doi: https://doi.org/10.1016/j.cam.2022.114630 .
A. Alhawarat, G. Alhamzi, I. Masmali, and Z. Salleh, “A DESCENT FOUR-TERM CONJUGATE GRADIENT METHOD WITH GLOBAL CONVERGENCE PROPERTIES FOR LARGE-SCALE UNCONSTRAINED OPTIMISATION PROBLEMS,” Math Probl Eng, vol. 2021, no. 1, p. 6219062, 2021.doi: https://doi.org/10.1155/2021/6219062
N. Andrei, “HYBRID CONJUGATE GRADIENT ALGORITHM FOR UNCONSTRAINED OPTIMIZATION,” J Optim Theory Appl, vol. 141, pp. 249–264, 2009.doi: https://doi.org/10.1007/s10957-008-9505-0
M. Lotfi and S. M. Hosseini, “AN EFFICIENT HYBRID CONJUGATE GRADIENT METHOD WITH SUFFICIENT DESCENT PROPERTY FOR UNCONSTRAINED OPTIMIZATION,” Optim Methods Softw, vol. 37, no. 5, pp. 1725–1739, 2022.doi: https://doi.org/10.1080/10556788.2021.1977808
S. C. Hu Y.F., “GLOBAL CONVERGENCE RESULT FOR CONJUGATE GRADIENT METHODS.,” J. Optim. Theory Appl., vol. 2, no. 71, pp. 399–405, 1991.doi: https://doi.org/10.1007/BF00939927
J. Jian, L. Han, and X. Jiang, “A HYBRID CONJUGATE GRADIENT METHOD WITH DESCENT PROPERTY FOR UNCONSTRAINED OPTIMIZATION,” Appl Math Model, vol. 39, no. 3, pp. 1281–1290, 2015.doi: https://doi.org/10.1016/j.apm.2014.08.008
Yunus, R. B. and K. Kamilu, and M. S. I. and M. Mustafa, “A NOVEL THREE TERM CONJUGATE GRADIENT METHOD FOR UNCONSTRAINED OPTIMIZATION VIA SHIFTED VARIABLE METRIC APPROACH WITH APPLICATION,” Springer International Publishing, 2022, pp. 581–596. doi: https://doi.org/10.1007/978-3-031-04028-3_37.
X. Z. Jiang, G. D. Ma, and J. B. Jian, “A NEW GLOBAL CONVERGENT CONJUGATE GRADIENT METHOD WITH WOLFE LINE SEARCH,” Chinese Journal of Engineering Mathematics, vol. 28, no. 6, pp. 779–786, 2011.
E. D. Dolan and J. J. Moré, “BENCHMARKING OPTIMIZATION SOFTWARE WITH PERFORMANCE PROFILES,” Math Program, vol. 91, pp. 201–213, 2002.doi: https://doi.org/10.1007/s101070100263
F. Shen, J. Chao, and J. Zhao, “FORECASTING EXCHANGE RATE USING DEEP BELIEF NETWORKS AND CONJUGATE GRADIENT METHOD,” Neurocomputing, vol. 167, pp. 243–253, 2015.doi: https://doi.org/10.1016/j.neucom.2015.04.071
Copyright (c) 2025 Kamilu Kamfa, Rabiu Bashir Yunus, Muhammad Auwal Lawan
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this Journal agree to the following terms:
- Author retain copyright and grant the journal right of first publication with the work simultaneously licensed under a creative commons attribution license that allow others to share the work within an acknowledgement of the work’s authorship and initial publication of this journal.
- Authors are able to enter into separate, additional contractual arrangement for the non-exclusive distribution of the journal’s published version of the work (e.g. acknowledgement of its initial publication in this journal).
- Authors are permitted and encouraged to post their work online (e.g. in institutional repositories or on their websites) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published works.
1.gif)
