Predictive Models Based on Reduced Input Space That Uses Rejected Variables

Because the number of variables is often tremendous in data mining applications, variable selection or dimension reduction is essential to produce models with acceptable accuracy and generalization. After applying variable selection methods, data miners often consider only selected variables for their tasks. However, the input space represented by the rejected variables might still contain potential for positive contribution to the model. This paper demonstrates the use of projection methods to combine both the selected variable space and the rejected variable space. The new input space, which is made by adding projections of the rejected variables to the set of selected variables, reduces the loss of input variable information while keeping interpretability of important individual variables.