Model Selection Techniques for Multiple Linear Regression Models

Abstract

Statistics is an important tool for researchers in almost every field that impacts modern life. Multiple linear regression analysis is one of the most important tools available to these researchers. A difficult, but frequently encountered problem in multiple regression analysis, is model selection. Classical model selection techniques included forward selection, backward elimination, and stepwise regression. Many new techniques have become available with the tremendous advances that have been made in computational power. These techniques include Mallow’s Cp, Akaike’s Information Criterion (AIC), Sawa’s Bayesian Criterion (BIC), Schwartz’ Bayesian Criterion (SBC) and many others. This study focused on the Akaike’s Information Criterion, Sawa’s Bayesian Criterion and Schwartz’ Bayesian Criterion. A simulation of several situations was conducted to try to answer two important questions. First, how good are these techniques? Second, are there any characteristics the researcher can use to determine which technique to use? The results indicated that there are some situations where the answers to these questions are clear cut but in other situations the results are somewhat unpredictable.