Best Paper Award
The paper “Clarity is a Worthwhile Quality: On the Role of Task Clarity in Microtask Crowdsourcing” [1] has won the Douglas Engelbart Best Paper Award at the 28th ACM Conference on Hypertext and Social Media (Hypertext 2017) [2] conference, Prague, Czech Republic.
The paper is co-authored by Jie Yang (PhD student in the Web Information Systems group), Alessandro Bozzon (his supervisor), and Ujwal Gadiraju (a PhD from the Leibniz University Hannover, and former WIS MsC student).
In this work, we highlight the role of clarity as a characterising property of tasks in crowdsourcing. We show that clarity is a problem for crowd workers, that it is coherently perceived by them, and that it is affected by the type of the task. We then propose a set of features to capture task clarity, and use the acquired labels to train and validate a supervised machine learning model for task clarity prediction. Finally, we perform a long-term analysis of the evolution of task clarity on Amazon Turk, and show that clarity is not a property suitable for temporal characterisation.