Sparseness vs estimating conditional probabilities : some asymptotic results

Bartlett, Peter L. & Tewari, Ambuj (2007) Sparseness vs estimating conditional probabilities : some asymptotic results. Journal of Machine Learning Research, 8(April), pp. 775-790.

View at publisher


One of the nice properties of kernel classifiers such as SVMs is that they often produce sparse solutions. However, the decision functions of these classifiers cannot always be used to estimate the conditional probability of the class label. We investigate the relationship between these two properties and show that these are intimately related: sparseness does not occur when the conditional probabilities can be unambiguously estimated. We consider a family of convex loss functions and derive sharp asymptotic results for the fraction of data that becomes support vectors. This enables us to characterize the exact trade-off between sparseness and the ability to estimate conditional probabilities for these loss functions.

Impact and interest:

28 citations in Scopus
21 citations in Web of Science®
Search Google Scholar™

Citation counts are sourced monthly from Scopus and Web of Science® citation databases.

These databases contain citations from different subsets of available publications and different time periods and thus the citation count from each is usually different. Some works are not in either database and no count is displayed. Scopus includes citations from articles published in 1996 onwards, and Web of Science® generally from 1980 onwards.

Citations counts from the Google Scholar™ indexing service can be viewed at the linked Google Scholar™ search.

ID Code: 43999
Item Type: Journal Article
Refereed: Yes
Additional Information: Fulltext freely available see link above
Additional URLs:
Keywords: kernel methods, estimating conditional probabilities, sparseness, support vector machines, OAVJ
ISSN: 1533-7928
Subjects: Australian and New Zealand Standard Research Classification > INFORMATION AND COMPUTING SCIENCES (080000) > ARTIFICIAL INTELLIGENCE AND IMAGE PROCESSING (080100)
Australian and New Zealand Standard Research Classification > PSYCHOLOGY AND COGNITIVE SCIENCES (170000) > COGNITIVE SCIENCE (170200)
Divisions: Past > QUT Faculties & Divisions > Faculty of Science and Technology
Past > Schools > Mathematical Sciences
Copyright Owner: Copyright 2007 Journal of Machine Learning Research
Deposited On: 17 Aug 2011 21:45
Last Modified: 29 Feb 2012 14:34

Export: EndNote | Dublin Core | BibTeX

Repository Staff Only: item control page