Wundervald, Bruna and Parnell, Andrew and Domijan, Katarina (2020) Generalizing Gain Penalization for Feature Selection in Tree-Based Models. IEEE Access, 8. pp. 190231-190239. ISSN 2169-3536

Preview

Download (4MB) | Preview

Abstract

We develop a new approach for feature selection via gain penalization in tree-based models. First, we show that previous methods do not perform sufficient regularization and often exhibit sub-optimal out-of-sample performance, especially when correlated features are present. Instead, we develop a new gain penalization idea that exhibits a general local-global regularization for tree-based models. The new method allows for full flexibility in the choice of feature-specific importance weights, while also applying a global penalization. We validate our method on both simulated and real data, exploring how the hyperparameters interact and we provide the implementation as an extension of the popular R package ranger.

Item Type:	Article
Keywords:	Dimensionality reduction; feature selection; gain penalization; tree-models;
Academic Unit:	Faculty of Science and Engineering > Research Institutes > Hamilton Institute
Item ID:	15346
Identification Number:	https://doi.org/10.1109/ACCESS.2020.3032095
Depositing User:	Katarina Domijan
Date Deposited:	25 Jan 2022 17:00
Journal or Publication Title:	IEEE Access
Publisher:	IEEE
Refereed:	Yes
URI:	Publisher
Use Licence:	This item is available under a Creative Commons Attribution Non Commercial Share Alike Licence (CC BY-NC-SA). Details of this licence are available here

Repository Staff Only(login required)

Item control page

Downloads

Origin of downloads