Identifying the Limitation of Stepwise Selection for Variable Selection in Regression Analysis
American Journal of Theoretical and Applied Statistics
Volume 4, Issue 5, September 2015, Pages: 414-419
Received: Jul. 25, 2015; Accepted: Aug. 6, 2015; Published: Sep. 18, 2015
Views 6485      Downloads 211
Akinwande Michael Olusegun, Department of Mathematics, Ahmadu Bello University, Zaria, Nigeria
Hussaini Garba Dikko, Department of Mathematics, Ahmadu Bello University, Zaria, Nigeria
Shehu Usman Gulumbe, Department of Mathematics, Usman Danfodiyo University, Sokoto, Nigeria
Article Tools
Follow on us
In application, one major difficulty a researcher may face in fitting a multiple regression is the problem of selecting significant relevant variables, especially when there are many independent variables to select from as well as having in mind the principle of parsimony; a comparative study of the limitation of stepwise selection for selecting variables in multiple regression analysis was carried out. Regression analysis in its bi-variate and multiple cases and stepwise selection (forward selection, backward elimination and stepwise selection) was employed for this study comparing the zero-order correlations and Beta (β) weights to give a clearer picture of the limitation of stepwise selection. Subsequently, from the comparisons, it was evident that including the suspected predictor (suppressor) variable that was not significant in the bi-variate case as suggested by the stepwise selection improved the beta weight of other predictors in the model and the overall predictability of the model as argued.
Stepwise Selection, Suppression Effect, Regressor Weights, Correlation
To cite this article
Akinwande Michael Olusegun, Hussaini Garba Dikko, Shehu Usman Gulumbe, Identifying the Limitation of Stepwise Selection for Variable Selection in Regression Analysis, American Journal of Theoretical and Applied Statistics. Vol. 4, No. 5, 2015, pp. 414-419. doi: 10.11648/j.ajtas.20150405.22
Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2013). Applied multiple regression/correlation analysis for the behavioral sciences (Revised ed.). New York: Routledge.
Conger, A. J. (1974). A revised definition for suppressor variables: A guide to their identification and interpretation. Educational and Psychological Measurement , 35-46.
Darlington, R. B. (1968). Multiple regression in psychological research and practice. Psychological Bulletin , 161-182.
Horst, P. (1941). The prediction of personal adjustment. Social Science Research Council Bulletin , 431-436.
John, N., William, W., & Michael, H. K. (1983). Stepwise Selection. In N. John, W. William, & H. K. Michael, Applied Linear regression Models (pp. 430-434). Illinois: Richard D Irwin Inc.
Lancaster, B. P. (1999). Defining and interpreting suppressor effects: Advantages and limitations. Southwest Educational Research Association, San Antonio , 1-21.
Liebscher, G. (2012). A Universal Selection Method in Linear Regression Models. Open Journal of Statistics , 153-162.
Loukas, A. P. (2005). Early adolescent social and overt aggression: Examining the roles of social anxiety and maternal psychological control. Journal of Youth and Adolescence , 335-345.
Mendershausen, H. (1939). Clearing variates in confluence analysis. Journal of the American Statistical Association , 93-105.
Nathans, L. L. (2012). Interpreting Multiple Linear Regression: A Guidebook of Variable Importance. Practical Assessment, Research & Evaluation , 17, 123-136.
Pedhazur, E. J. (1997). Multiple regression in behavioral research. New York: Holt, Rinehart & Winston.
Shanta, P., & Williams, E. (2010). Suppressor Variables in Social Work Research: Ways to Identify in Multiple Regression Models. Journal of the Society for Social Work and Research , 28-40.
Shieh, G. (2006). Suppression situations in multiple linear regression. Educational and Psychological Measurement , 435-447.
Yao, J. (2013). Precision Analysis and Parameter Inversion in the Stepwise Deployment of a Mixed Constellation. Open Journal of Statistics , 390-397.
Science Publishing Group
1 Rockefeller Plaza,
10th and 11th Floors,
New York, NY 10020
Tel: (001)347-983-5186