Graph-Based Model-Selection Framework for Large Ensembles

The intuition behind ensembles is that different prediciton models compensate each other’s errors if one combines them in an appropriate way. In case of large ensembles a lot of different prediction models are available. However, many of them may share similar error characteristics, which highly depress the compensation effect. Thus the selection of an appropriate subset of models is crucial. In this paper, we address this problem. As major contribution, for the case if a large number of models is present, we propose a graph-based framework for model selection while paying special attention to the interaction effect of models. In this framework, we introduce four ensemble techniques and compare them to the state-of-the-art in experiments on publicly available real-world data.