On Ordinal VC-Dimension and Some Notions of Complexity

Martin, Eric, Sharma, Arun, & Stephan, Fran (2006) On Ordinal VC-Dimension and Some Notions of Complexity. Theoretical Computer Science, 364(1), pp. 62-76.

View at publisher

Abstract

We generalize the classical notion of Vapnik–Chernovenkis (VC) dimension to ordinal VC-dimension, in the context of logical learning paradigms. Logical learning paradigms encompass the numerical learning paradigms commonly studied in Inductive Inference. A logical learning paradigm is defined as a set W of structures over some vocabulary, and a set D of first-order formulas that represent data. The sets of models of ϕ in W, where ϕ varies over D, generate a natural topology W over W.

We show that if D is closed under boolean operators, then the notion of ordinal VC-dimension offers a perfect characterization for the problem of predicting the truth of the members of D in a member of W, with an ordinal bound on the number of mistakes. This shows that the notion of VC-dimension has a natural interpretation in Inductive Inference, when cast into a logical setting. We also study the relationships between predictive complexity, selective complexity—a variation on predictive complexity—and mind change complexity. The assumptions that D is closed under boolean operators and that W is compact often play a crucial role to establish connections between these concepts.

We then consider a computable setting with effective versions of the complexity measures, and show that the equivalence between ordinal VC-dimension and predictive complexity fails. More precisely, we prove that the effective ordinal VC-dimension of a paradigm can be defined when all other effective notions of complexity are undefined. On a better note, when W is compact, all effective notions of complexity are defined, though they are not related as in the noncomputable version of the framework.

Impact and interest:

1 citations in Scopus
Search Google Scholar™
1 citations in Web of Science®

Citation counts are sourced monthly from Scopus and Web of Science® citation databases.

These databases contain citations from different subsets of available publications and different time periods and thus the citation count from each is usually different. Some works are not in either database and no count is displayed. Scopus includes citations from articles published in 1996 onwards, and Web of Science® generally from 1980 onwards.

Citations counts from the Google Scholar™ indexing service can be viewed at the linked Google Scholar™ search.

ID Code: 23758
Item Type: Journal Article
Refereed: Yes
Keywords: Generalized VC-dimension; Inductive inference; Predictive complexity
DOI: 10.1016/j.tcs.2006.07.041
ISSN: 0304-3975
Divisions: Past > QUT Faculties & Divisions > Faculty of Science and Technology
Deposited On: 17 Jun 2009 14:07
Last Modified: 29 Feb 2012 13:30

Export: EndNote | Dublin Core | BibTeX

Repository Staff Only: item control page