Citation Context

...sting wages) in order to calibrate algorithms that approximate the ground truth related to the crowdsourcing task [2]. Benefit of explicit feedback and information exchange between workers is studied =-=[4, 6]-=- to improve worker self-coordination, but no existing research incorporates these aspects in a dynamic and interactive environment, nor are there optimized solutions for ground truth discovery, consid...

Citation Context

...aborative computing, Computer-supported cooperative work, Web-based interactionsINTRODUCTIONsA major research challenge in crowdsourcing has been increasing crowd worker engagement and output qualitys=-=[4,8,16,19,20,21,26,33,37]-=-. One significant issue has beensthat crowd platform features aimed at lowering marketstransaction costs – such as simplified work histories, deidentification of worker names, and lack of long-term co...

Citation Context

...lity traits from Facebook activity [8]. The work [9] also uses personality traits predicted from task completion to categorise workers into five different classes and evaluate their performance with respect to those classes. In [9] the relationship between personality traits and the successful completion of tasks from a relevance labelling scenario have been investigated. One particularly interesting result of [9] is that successful completion of such a labelling task is strongly correlated to the Openness trait (the trait describing among others a person’s curiosity and creativity). The work [10] discovered that workers revealing demographic information outperform anonymous workers in a simple word-counting task scenario, in particular when cooperation between different workers is needed. In [11] Satzger et Approach 1.a.i 1.a.ii 1.b 2.a 2.b Sarasua et al. [5] X Bozzon et al. [6] X Difallah et al. [7] X Kazai et al. [9] X X Huang and Fu [10] X Satzger et al. [11] X X Khazankin et al. [12] X Amati et al. [13] X X TABLE I. COMPARING APPROACHES al. introduce auction-based task assignment for crowdsourcing scenario where workers bid for tasks they like to solve. Besides this they also incl...

Citation Context

...sting wages) in order to calibrate algorithms that approximate the ground truth related to the crowdsourcing task [2]. Benefit of explicit feedback and information exchange between workers is studied =-=[4, 6]-=- to improve worker self-coordination, but no existing research incorporates these aspects in a dynamic and interactive environment, nor are there optimized solutions for ground truth discovery, consid...

Citation Context

...o account when designing an incentive management system. For example, the finding that the transparency of actors and processes in a socio-technical system will likely improve the overall performance =-=[34]-=- for pringl translates to the requirement of portability and transparency of incentives. The findings of [35] indicate that for performing more intellectually challenging tasks smaller groups of exper...

Citation Context

... suggests that making workers’ rewards co-dependent on each other can significantly increase the quality of their work. This suggests the potency of a peer-dependent reward scheme for quality control =-=[19]-=-. In a teamwork-based scheme, paired workers are rewarded based on their average work whereas in a competition-based scheme, the paired worker who performs the best gets the entire reward. Herein, we ...

Citation Context

... through real data from Amazon Mechanical Turk [130], a paid crowdsourcing microtask platform. The approach is analyzed under different crowdsourcing models including the peer-dependent reward scheme =-=[61]-=- and the dependent observations model [111]. In the process, an ordering principle 11 for the quality of crowds is also developed. For systems with peer-dependent reward schemes, it is observed that h...