In search of good probability assessors: An experimental comparison of elicitation rules for confidence judgments

Hollard, Guillaume, Massoni, Sebastien, & Vergnaud, Jean-Christophe (2016) In search of good probability assessors: An experimental comparison of elicitation rules for confidence judgments. Theory and Decision, 80(3), pp. 363-387.

View at publisher


In this paper, we use an experimental design to compare the performance of elicitation rules for subjective beliefs. Contrary to previous works in which elicited beliefs are compared to an objective benchmark, we consider a purely subjective belief framework (confidence in one’s own performance in a cognitive task and a perceptual task). The performance of different elicitation rules is assessed according to the accuracy of stated beliefs in predicting success. We measure this accuracy using two main factors: calibration and discrimination. For each of them, we propose two statistical indexes and we compare the rules’ performances for each measurement. The matching probability method provides more accurate beliefs in terms of discrimination, while the quadratic scoring rule reduces overconfidence and the free rule, a simple rule with no incentives, which succeeds in eliciting accurate beliefs. Nevertheless, the matching probability appears to be the best mechanism for eliciting beliefs due to its performances in terms of calibration and discrimination, but also its ability to elicit consistent beliefs across measures and across tasks, as well as its empirical and theoretical properties.

Impact and interest:

0 citations in Scopus
Search Google Scholar™
1 citations in Web of Science®

Citation counts are sourced monthly from Scopus and Web of Science® citation databases.

These databases contain citations from different subsets of available publications and different time periods and thus the citation count from each is usually different. Some works are not in either database and no count is displayed. Scopus includes citations from articles published in 1996 onwards, and Web of Science® generally from 1980 onwards.

Citations counts from the Google Scholar™ indexing service can be viewed at the linked Google Scholar™ search.

ID Code: 85894
Item Type: Journal Article
Refereed: Yes
DOI: 10.1007/s11238-015-9509-9
ISSN: 0040-5833
Divisions: Current > QUT Faculties and Divisions > QUT Business School
Current > Schools > School of Economics & Finance
Copyright Owner: Copyright 2015 Springer Science+Business Media
Deposited On: 21 Jul 2015 23:30
Last Modified: 11 Mar 2016 01:28

Export: EndNote | Dublin Core | BibTeX

Repository Staff Only: item control page