Margin-adaptive model selection in statistical learning

Arlot, Sylvain & Bartlett, Peter L. (2011) Margin-adaptive model selection in statistical learning. Bernoulli.

View at publisher


A classical condition for fast learning rates is the margin condition, first introduced by Mammen and Tsybakov. We tackle in this paper the problem of adaptivity to this condition in the context of model selection, in a general learning framework. Actually, we consider a weaker version of this condition that allows one to take into account that learning within a small model can be much easier than within a large one. Requiring this “strong margin adaptivity” makes the model selection problem more challenging. We first prove, in a general framework, that some penalization procedures (including local Rademacher complexities) exhibit this adaptivity when the models are nested. Contrary to previous results, this holds with penalties that only depend on the data. Our second main result is that strong margin adaptivity is not always possible when the models are not nested: for every model selection procedure (even a randomized one), there is a problem for which it does not demonstrate strong margin adaptivity.

Impact and interest:

3 citations in Scopus
3 citations in Web of Science®
Search Google Scholar™

Citation counts are sourced monthly from Scopus and Web of Science® citation databases.

These databases contain citations from different subsets of available publications and different time periods and thus the citation count from each is usually different. Some works are not in either database and no count is displayed. Scopus includes citations from articles published in 1996 onwards, and Web of Science® generally from 1980 onwards.

Citations counts from the Google Scholar™ indexing service can be viewed at the linked Google Scholar™ search.

ID Code: 45500
Item Type: Journal Article
Refereed: Yes
Keywords: Adaptivity, Classification, Empirical minimization, Empirical risk minimization, Local Rademacher complexity, Margin condition, Model selection, Oracle inequalities, Statistical learning
ISSN: 1350-7265
Subjects: Australian and New Zealand Standard Research Classification > MATHEMATICAL SCIENCES (010000) > STATISTICS (010400)
Divisions: Past > QUT Faculties & Divisions > Faculty of Science and Technology
Past > Schools > Mathematical Sciences
Deposited On: 29 Aug 2011 22:10
Last Modified: 29 Aug 2011 22:11

Export: EndNote | Dublin Core | BibTeX

Repository Staff Only: item control page