Behaviormetrika
Online ISSN : 1349-6964
Print ISSN : 0385-7417
ISSN-L : 0385-7417
Articles
MODIFIED RULE ENSEMBLE METHOD FOR BINARY DATA AND ITS APPLICATIONS
Toshio ShimokawaLi LiKun YanShinnichi KitamuraMasashi Goto
Author information
JOURNALS RESTRICTED ACCESS

2014 Volume 41 Issue 2 Pages 225-244

Details
Abstract

Ensemble learning, which combines multiple base learners to improve statistical prediction accuracy, is frequently used in statistical science and data mining. However, because of their “black box” nature, ensemble learning models are difficult to interpret. A recently proposed rule ensemble method known as RuleFit presents the base learner as a production rule and also generates a measure that influences the response variable. The RuleFit method for binary response applies a squared-error ramp loss function, and base learners are weighted by shrinkage regression using the lasso method. Thus, RuleFit is not constructed by a logistic regression model. Moreover, highly correlated pairs of base learners may be excessively pruned by the lasso method. In this study, we solved the excess pruning problem by constructing RuleFit within a logistic regression framework, weighting the base learners by elastic net. The effectiveness ofour proposed RuleFit model is illustrated through a real data set. In small-scale simulations, this method demonstrated higher predictive performance than the original RuleFit model.

Information related to the author
© 2014 The Behaviormetric Society
Previous article Next article
feedback
Top