MURAL - Maynooth University Research Archive Library



    Generalizing Gain Penalization for Feature Selection in Tree-Based Models


    Wundervald, Bruna and Parnell, Andrew C. and Domijan, Katarina (2020) Generalizing Gain Penalization for Feature Selection in Tree-Based Models. IEEE Access, 8. pp. 190231-190239. ISSN 2169-3536

    [img]
    Preview
    Download (4MB) | Preview


    Share your research

    Twitter Facebook LinkedIn GooglePlus Email more...



    Add this article to your Mendeley library


    Abstract

    We develop a new approach for feature selection via gain penalization in tree-based models. First, we show that previous methods do not perform sufficient regularization and often exhibit sub-optimal out-of-sample performance, especially when correlated features are present. Instead, we develop a new gain penalization idea that exhibits a general local-global regularization for tree-based models. The new method allows for full flexibility in the choice of feature-specific importance weights, while also applying a global penalization. We validate our method on both simulated and real data, exploring how the hyperparameters interact and we provide the implementation as an extension of the popular R package ranger.

    Item Type: Article
    Keywords: Dimensionality reduction; feature selection; gain penalization; tree-models;
    Academic Unit: Faculty of Science and Engineering > Research Institutes > Hamilton Institute
    Item ID: 15346
    Identification Number: https://doi.org/10.1109/ACCESS.2020.3032095
    Depositing User: Katarina Domijan
    Date Deposited: 25 Jan 2022 17:00
    Journal or Publication Title: IEEE Access
    Publisher: IEEE
    Refereed: Yes
    URI:

    Repository Staff Only(login required)

    View Item Item control page

    Downloads

    Downloads per month over past year