MURAL - Maynooth University Research Archive Library

    Are self-assessments reliable indicators of topic knowledge?

    Cole, Michael J. and Zhang, Xiangmin and Liu, Jinging and Liu, Chang and Belkin, Nicholas J. and Bierig, Ralf and Gwizdka, Jacek (2011) Are self-assessments reliable indicators of topic knowledge? Proceedings of the American Society for Information Science and Technology, 47 (1). pp. 1-10. ISSN 00447870

    Download (239kB) | Preview

    Share your research

    Twitter Facebook LinkedIn GooglePlus Email more...

    Add this article to your Mendeley library


    Self-assessment of topic/task knowledge is a human metacognitive capacity that impacts information behavior, for example through selection of learning and search strategies. It is often used as a measure in experiments for evaluation of results and those measurements are taken to be generally reliable. We conducted a user study (n=40) to test this by constructing a concept-based topic knowledge representation for each participant and then comparing it with the participant judgment of their topic knowledge elicited with Likert-scale questions. The tasks were in the genomics domain and knowledge representations were constructed from the MeSH thesaurus terms that indexed relevant documents for five topics. The participants rated their familiarity with the topic, the anticipated task difficulty, the amount of learning gained during the task, and made other knowledge-related judgments associated with the task. Although there is considerable variability over individuals, the results provide evidence that these self-assessed topic knowledge measures are correlated in the expected way with the independently-constructed topic knowledge measure. We argue the results provide evidence for the general validity of topic knowledge self-assessment and discuss ways to further explore knowledge self-assessment and its reliability for prediction of individual knowledge levels.

    Item Type: Article
    Keywords: User studied; methodology; knowledge representation;
    Academic Unit: Faculty of Science and Engineering > Computer Science
    Faculty of Science and Engineering > Research Institutes > Hamilton Institute
    Item ID: 15218
    Identification Number:
    Depositing User: Ralf Bierig
    Date Deposited: 11 Jan 2022 12:46
    Journal or Publication Title: Proceedings of the American Society for Information Science and Technology
    Publisher: ASIST
    Refereed: Yes
    Use Licence: This item is available under a Creative Commons Attribution Non Commercial Share Alike Licence (CC BY-NC-SA). Details of this licence are available here

    Repository Staff Only(login required)

    View Item Item control page


    Downloads per month over past year

    Origin of downloads