Published by the Foundation for Open Access Statistics Editors-in-chief: Bettina Grün, Torsten Hothorn, Edzer Pebesma, Achim Zeileis    ISSN 1548-7660; CODEN JSSOBK
Authors: Daniel Conn, Tuck Ngun, Gang Li, Christina M. Ramirez
Title: Fuzzy Forests: Extending Random Forest Feature Selection for Correlated, High-Dimensional Data
Abstract: In this paper we introduce fuzzy forests, a novel machine learning algorithm for ranking the importance of features in high-dimensional classification and regression problems. Fuzzy forests is specifically designed to provide relatively unbiased rankings of variable importance in the presence of highly correlated features, especially when the number of features, p, is much larger than the sample size, n (p n). We introduce our implementation of fuzzy forests in the R package, fuzzyforest. Fuzzy forests works by taking advantage of the network structure between features. First, the features are partitioned into separate modules such that the correlation within modules is high and the correlation between modules is low. The package fuzzyforest allows for easy use of the package WGCNA (weighted gene coexpression network analysis, alternatively known as weighted correlation network analysis) to form modules of features such that the modules are roughly uncorrelated. Then recursive feature elimination random forests (RFE-RFs) are used on each module, separately. From the surviving features, a final group is selected and ranked using one last round of RFE-RFs. This procedure results in a ranked variable importance list whose size is pre-specified by the user. The selected features can then be used to construct a predictive model.

Page views:: 3937. Submitted: 2016-03-22. Published: 2019-10-31.
Paper: Fuzzy Forests: Extending Random Forest Feature Selection for Correlated, High-Dimensional Data     Download PDF (Downloads: 1375)
fuzzyforest_1.0.6.tar.gz: R source package Download (Downloads: 90; 651KB) Replication materials Download (Downloads: 90; 3MB)

DOI: 10.18637/jss.v091.i09

This work is licensed under the licenses
Paper: Creative Commons Attribution 3.0 Unported License
Code: GNU General Public License (at least one of version 2 or version 3) or a GPL-compatible license.