Learning with symmetric label noise: The importance of being unhinged

van Rooyen, Brendan, Menon, Aditya Krishna, & Williamson, Robert C. (2015) Learning with symmetric label noise: The importance of being unhinged. In Advances in Neural Information Processing Systems 28 (NIPS 2015), Neural Information Processing Systems Foundation, Inc., Palais des Congrès de Montréal, Montréal, Canada, pp. 10-18.

View at publisher (open access)

Abstract

Convex potential minimisation is the de facto approach to binary classification. However, Long and Servedio [2008] proved that under symmetric label noise (SLN), minimisation of any convex potential over a linear function class can result in classification performance equivalent to random guessing. This ostensibly shows that convex losses are not SLN-robust. In this paper, we propose a convex, classification-calibrated loss and prove that it is SLN-robust. The loss avoids the Long and Servedio [2008] result by virtue of being negatively unbounded. The loss is a modification of the hinge loss, where one does not clamp at zero; hence, we call it the unhinged loss. We show that the optimal unhinged solution is equivalent to that of a strongly regularised SVM, and is the limiting solution for any convex potential; this implies that strong l2 regularisation makes most standard learners SLN-robust. Experiments confirm the unhinged loss’ SLN-robustness.

Impact and interest:

0 citations in Scopus
Search Google Scholar™

Citation counts are sourced monthly from Scopus and Web of Science® citation databases.

These databases contain citations from different subsets of available publications and different time periods and thus the citation count from each is usually different. Some works are not in either database and no count is displayed. Scopus includes citations from articles published in 1996 onwards, and Web of Science® generally from 1980 onwards.

Citations counts from the Google Scholar™ indexing service can be viewed at the linked Google Scholar™ search.

ID Code: 91573
Item Type: Conference Paper
Refereed: Yes
Additional URLs:
Divisions: Current > Research Centres > ARC Centre of Excellence for Mathematical & Statistical Frontiers (ACEMS)
Current > Schools > School of Mathematical Sciences
Current > QUT Faculties and Divisions > Science & Engineering Faculty
Copyright Owner: Copyright 2015 Neural Information Processing Systems Foundation, Inc.
Deposited On: 04 Jan 2016 03:08
Last Modified: 05 Jan 2016 05:31

Export: EndNote | Dublin Core | BibTeX

Repository Staff Only: item control page