RIDGE LEAST ABSOLUTE DEVIATION PERFORMANCE IN ADDRESSING MULTICOLLINEARITY AND DIFFERENT LEVELS OF OUTLIER SIMULTANEOUSLY
Abstract
If there is multicollinearity and outliers in the data, the inference about parameter estimation in the LS method will deviate due to the inefficiency of this method in estimating. To overcome these two problems simultaneously, it can be done using robust regression, one of which is ridge least absolute deviation method. This study aims to evaluate the performance of the ridge least absolute deviation method in surmounting multicollinearity in divers sample sizes and percentage of outliers using simulation data. The Monte Carlo study was designed in a multiple regression model with multicollinearity (ρ=0.99) between variables and and outliers 10%, 20%, 30% on response variables with different sample sizes (n = 25, 50,75,100,200; =0, and β=1 otherwise). The existence of multicollinearity in the data is done by calculating the correlation value between the independent variables and the VIF value. Outlier detection is done by using boxplot. Parameter estimation was carried out using the RLAD and LS methods. Furthermore, a comparison of the MSE values of the two methods is carried out to see which method is better in overcoming multicollinearity and outliers. The results showed that RLAD had a lower MSE than LS. This signifies that RLAD is more precise in estimating the regression coefficients for each sample size and various outlier levels studied.
Downloads
References
S. Chatterjee, A.S. Hadi, and B. Price, Regression Analysis by Example, 4th ed., vol. 95, no. 452. New Jersey: Jhon Wiley & Sons Inc., 2000.
I. Pardoe, Applied Regression Modelling, 3rd ed., New York: Wiley, 2020.
D.C. Montgomery, E.A. Peck and G.G. Vinning., Introduction to Linear Regression Analysis. New York: A Wiley Intersection Publication, 2012.
M.H. Kutner et al., Applied Linear Statistical Models., 5th ed. New York: McGraw-Hill, 2005.
D. Gibbons, “A Simulation Study of some Ridge Estimators,” J. Am. Stat. Assoc., vol. 76, pp. 131–139, 1981.
M. E. El-Salam, “The Efficiency of Some Robust Ridge Regression for Handling Multicollinearity and Non-Normals Errors Problems.,” Appl. Math. Sci., vol. 77, no. 7, pp. 3831 – 3846, 2013.
A. Hoerl and R. Kennard, “Ridge Regression: Biased Estimation for Nonorthogonal Problems,” Technometrics, vol. 12, pp. 55–67, 1970.
N. Herawati, K. Nisa, E. Setiawan, Nusyirwan, and Tiryono, “Regularized Multiple Regression Methods to Deal with Severe Multicollinearity,” Int. J. Stat. Appl., vol. 8, no. 4, pp. 167–172, 2018.
E. Setiawan, N. Herawati, K. Nisa, Nusyirwan, and S. Saidi, “Handling Full Multicollinearity and Various Numbers of Outliers using Robust Ridge Regression,” Sci.Int.(Lahore), vol. 31, no. 2, pp. 201–204, 2019.
N. Herawati, K. Nisa, and Nusyirwan. " Selecting the Method to Overcome Partial and Full Multicollinearity in Binary Logistic Model," International Journal of Statistoics and Apllications, vol. 10, no. 3, pp. 55–59, 2020.
A.M. Leroy and P.J. Rousseeuw, Robust Regression and Outlier Detection. Jhon Wiley & Sons, 1987
H. Samkar and O. Alpu, “Ridge Regression Based on Some Robust Estimators.,” J. Mod. Appl. Stat. Methods, vol. 17, no. 9, pp. 495–501, 2010.
A.E. Hoerl, “Application of ridge analysis to regression problems,” Chem. Eng. Prog., vol. 58, pp. 54–59, 1962.
M. El-Dereny and N. I. Rashwan, “Solving Multicollinearity Problem Using Ridge Regression Models.,” Int. J. Contemp. Math. Sci., vol. 6, no. 12, pp. 585 – 600, 2011.
G.C. McDonald and D.I. Galarneau, “A Monte Carlo evaluation of some ridge type estimators.,” J. Amer. Stat. Assoc., vol. 20, pp. 407–416, 1975.
Copyright (c) 2022 Netti Herawati, Subian Saidi, Dorrah Azis
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this Journal agree to the following terms:
- Author retain copyright and grant the journal right of first publication with the work simultaneously licensed under a creative commons attribution license that allow others to share the work within an acknowledgement of the work’s authorship and initial publication of this journal.
- Authors are able to enter into separate, additional contractual arrangement for the non-exclusive distribution of the journal’s published version of the work (e.g. acknowledgement of its initial publication in this journal).
- Authors are permitted and encouraged to post their work online (e.g. in institutional repositories or on their websites) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published works.
