DMCA
Crowdsourced judgement elicitation with endogenous proficiency (2013)
| Venue: | ACM International World Wide Web Conference (WWW |
| Citations: | 9 - 0 self |
BibTeX
@INPROCEEDINGS{Dasgupta13crowdsourcedjudgement,
author = {Anirban Dasgupta and Arpita Ghosh},
title = {Crowdsourced judgement elicitation with endogenous proficiency},
booktitle = {ACM International World Wide Web Conference (WWW},
year = {2013}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Crowdsourcing is now widely used to replace judgement or evaluation by an expert authority with an aggregate evaluation from a number of non-experts, in applications ranging from rating and categorizing online content all the way to evaluation of student assignments in massively open online courses (MOOCs) via peer grading. A key issue in these settings, where direct monitoring of both effort and accuracy is infeasible, is incentivizing agents in the ‘crowd ’ to put in effort to make good evaluations, as well as to truthfully report their evaluations. We study the design of mechanisms for crowdsourced judgement elicitation when workers strategically choose both their reports and the effort they put into their evaluations. This leads to a new family of information elicitation problems with unobservable ground truth, where an agent’s proficiency — the probability with which she correctly evaluates the underlying ground
Keyphrases
crowdsourced judgement elicitation endogenous proficiency peer grading key issue direct monitoring underlying ground unobservable ground truth aggregate evaluation student assignment information elicitation problem good evaluation agent proficiency online content open online course expert authority new family
