MURAL - Maynooth University Research Archive Library



    Effects of Expertise Assessment on the Quality of Task Routing in Human Computation


    ul Hassan, Umair, Ó Riain, Seán and Curry, Edward (2013) Effects of Expertise Assessment on the Quality of Task Routing in Human Computation. In: 2nd International Workshop on Social Media for Crowdsourcing and Human Computation, January 2013, Paris, France.

    [thumbnail of UuH_effects.pdf]
    Preview
    Text
    UuH_effects.pdf

    Download (915kB) | Preview

    Abstract

    Human computation systems are characterized by the use of human workers to solve computationally difficult problems. Expertise profiling involves assessment and representation of a worker’s expertise, in order to route human computation tasks to appropriate workers. This paper studies the relationship between the assessment workload on workers and the quality of task routing. Three expertise assessment approaches were compared with the help of a user study, using two different groups of human workers. The first approach requests workers to provide self-assessment of their knowledge. The second approach measures the knowledge of workers through their performance against tasks with known responses. We propose a third approach based on a combination of self-assessment and task-assessment. The results suggest that the self-assessment approach requires minimum assessment workload from workers during expertise profiling. By comparison, the task-assessment approach achieved the highest response rate and accuracy. The proposed approach requires less assessment workload, while achieving the response rate and accuracy similar to the task-assessment approach.
    Item Type: Conference or Workshop Item (Paper)
    Academic Unit: Faculty of Social Sciences > School of Business
    Item ID: 16015
    Depositing User: Souleiman Hasan
    Date Deposited: 30 May 2022 13:44
    Refereed: Yes
    URI: https://mural.maynoothuniversity.ie/id/eprint/16015
    Use Licence: This item is available under a Creative Commons Attribution Non Commercial Share Alike Licence (CC BY-NC-SA). Details of this licence are available here

    Repository Staff Only (login required)

    Item control page
    Item control page

    Downloads

    Downloads per month over past year

    Origin of downloads