The discriminating power of random features

Stefano Rovetta*, Francesco Masulli, Maurizio Filippone

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Input selection is found as a part of several machine learning tasks, either to improve performance or as the main goal. For instance, gene selection in bioinformatics is an input selection problem. However, as we prove in this paper, the reliability of input selection in the presence of high-dimensional data is affected by a small-sample problem. As a consequence of this effect, even completely random inputs have a chance to be selected as very useful, even if they are not relevant from the point of view of the underlying model. We express the probability of this event as a function of data cardinality and dimensionality, discuss the applicability of this analysis, and compute the probability for some data sets. We also show, as an illustration, some experimental results obtained by applying a specific input selection algorithm, previously presented by the authors, which show how inputs known to be random are consistently selected by the method.

Original languageEnglish (US)
Title of host publicationNeural Nets WIRN09 - Proceedings of the 19th Italian Workshop on Neural Nets
PublisherIOS Press
Pages3-10
Number of pages8
ISBN (Print)9781607500728
DOIs
StatePublished - 2009

Publication series

NameFrontiers in Artificial Intelligence and Applications
Volume204
ISSN (Print)0922-6389
ISSN (Electronic)1879-8314

ASJC Scopus subject areas

  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'The discriminating power of random features'. Together they form a unique fingerprint.

Cite this