MURAL - Maynooth University Research Archive Library

    The Scholarly Influence of the CLEF eHealth Initiative by the Conference and Labs of the Evaluation Forum: Review and Bibliometric Study of the 2012-2017 Outcomes.

    Suominen, Hanna and Kelly, Liadh and Goeuriot, Lorraine (2018) The Scholarly Influence of the CLEF eHealth Initiative by the Conference and Labs of the Evaluation Forum: Review and Bibliometric Study of the 2012-2017 Outcomes. JMIR Research Protocols, 7 (7). ISSN 1929-0748

    Download (340kB) | Preview

    Share your research

    Twitter Facebook LinkedIn GooglePlus Email more...

    Add this article to your Mendeley library


    Background: The eHealth initiative of the Conference and Labs of the Evaluation Forum (CLEF) has aimed since 2012 to gather researchers working on health text analytics and to provide them with annual workshop, shared development challenges/tasks, benchmark datasets, and software for processing and evaluation. The overall purpose of this initiative is to ease and support patients, their next-ofkin, clinical staff, health scientists, and healthcare policy makers in accessing, understanding, using, and authoring health information in a multilingual setting. Objective: This original research paper reports on the outcomes of the first six installations of CLEF eHealth from 2012 to 2017. The focus is on measuring and analysing the scholarly influence by reviewing CLEF eHealth papers, together with relevant citation metrics. Methods: A review and bibliometric study of the CLEF eHealth proceedings, working notes, and author-declared paper extensions was conducted. Citation data for these publications were collected from Google Scholar. Citation content analysis was used for the publications and their citations. Results: The large number of registrations, submissions, and citations demonstrate the substantial community interest in the tasks and their resources. In total, 718 teams have registered their interest in the tasks, leading to 130 teams submitting to the 15 tasks. 184 papers using CLEF eHealth data generated 1,299 citations, yielding a total scholarly citation influence of almost 963,000 citations for the 741 co-authors and included authors from 33 countries across the world. The tasks’ evaluation outcomes contribute to the knowledge of the difficulty of the research challenges the tasks address and the applicability of particular methods in solving these challenges, with typically statistically significant improvements in processing quality. Conclusions: These outcomes encourage continuing to develop these technologies to address patient needs. Consequently, data and tools have been opened for future research and development and the CLEF eHealth initiative continues to run new challenges.

    Item Type: Article
    Keywords: Evaluation Studies as Topic; Health Records; Information Extraction; Information Storage and Retrieval; Information Visualization; Patient Education as Topic; Speech Recognition; Systematic Reviews; Test-set Generation; Text Classification;
    Academic Unit: Faculty of Science and Engineering > Computer Science
    Faculty of Science and Engineering > Research Institutes > Hamilton Institute
    Item ID: 13240
    Identification Number:
    Depositing User: Liadh Kelly
    Date Deposited: 17 Sep 2020 11:01
    Journal or Publication Title: JMIR Research Protocols
    Publisher: JMIR Publications
    Refereed: Yes
    Use Licence: This item is available under a Creative Commons Attribution Non Commercial Share Alike Licence (CC BY-NC-SA). Details of this licence are available here

    Repository Staff Only(login required)

    View Item Item control page


    Downloads per month over past year

    Origin of downloads