Results 1 -
2 of
2
Approval voting and incentives in crowdsourcing
- In Proc. of 32nd ICML
, 2015
"... Abstract The growing need for labeled training data has made crowdsourcing an important part of machine learning. The quality of crowdsourced labels is, however, adversely affected by three factors: (1) the workers are not experts; (2) the incentives of the workers are not aligned with those of the ..."
Abstract
-
Cited by 1 (1 self)
- Add to MetaCart
(Show Context)
Abstract The growing need for labeled training data has made crowdsourcing an important part of machine learning. The quality of crowdsourced labels is, however, adversely affected by three factors: (1) the workers are not experts; (2) the incentives of the workers are not aligned with those of the requesters; and (3) the interface does not allow workers to convey their knowledge accurately, by forcing them to make a single choice among a set of options. In this paper, we address these issues by introducing approval voting to utilize the expertise of workers who have partial knowledge of the true answer, and coupling it with a ("strictly proper") incentive-compatible compensation mechanism. We show rigorous theoretical guarantees of optimality of our mechanism together with a simple axiomatic characterization. We also conduct preliminary empirical studies on Amazon Mechanical Turk which validate our approach.
Lie on the Fly: Iterative Voting
"... Manipulation can be performed when intermediate voting results are known; voters might attempt to vote strategically and try and manipulate the re-sults during an iterative voting process. When only partial voting preferences are available, preference elicitation is necessary. In this paper, we com- ..."
Abstract
- Add to MetaCart
Manipulation can be performed when intermediate voting results are known; voters might attempt to vote strategically and try and manipulate the re-sults during an iterative voting process. When only partial voting preferences are available, preference elicitation is necessary. In this paper, we com-bine two approaches of iterative processes: iterative preference elicitation and iterative voting and study the outcome and performance of a setting where manipulative voters submit partial preferences. We provide practical algorithms for manipulation un-der the Borda voting rule and evaluate those using different voting centers: the Careful voting center that tries to avoid manipulation and the Naive vot-ing center. We show that in practice, manipulation happens in a low percentage of the settings and has a low impact on the final outcome. The Careful voting center reduces manipulation even further. 1
