MURAL - Maynooth University Research Archive Library

    An Analysis of Evaluation Campaigns in ad-hoc Medical Information Retrieval: CLEF eHealth 2013 and 2014

    Goeuriot, Lorraine and Jones, Gareth J.F. and Kelly, Liadh and Leveling, Johannes and Lupu, Mihai and Palotti, Joao and Zuccon, Guido (2018) An Analysis of Evaluation Campaigns in ad-hoc Medical Information Retrieval: CLEF eHealth 2013 and 2014. Information Retrieval Journal. ISSN 1386-4564

    Download (522kB) | Preview
    Official URL:

    Share your research

    Twitter Facebook LinkedIn GooglePlus Email more...

    Add this article to your Mendeley library


    Since its inception in 2013, one of the key contributions of the CLEF eHealth evaluation campaign has been the organization of an ad-hoc information retrieval (IR) benchmarking task. This IR task evaluates systems intended to support laypeople searching for and understanding health infor- mation. Each year the task provides registered participants with standard IR test collections consisting of a document collection and topic set. Partici- pants then return retrieval results obtained by their IR systems for each query, which are assessed using a pooling procedure. In this article we focus on CLEF eHealth 2013 and 2014's retrieval task, which saw topics created based on pa- tients' information needs associated with their medical discharge summaries. We overview the task and datasets created, and the results obtained by par- ticipating teams over these two years. We then provide a detailed comparative analysis of the results, and conduct an evaluation of the datasets in the light of these results. This two-fold study of the evaluation campaign teaches us about technical aspects of medical IR, such as the e�ectiveness of query expansion; the quality and characteristics of CLEF eHealth IR datasets, such as their reliability; and how to run an IR evaluation campaign in the medical domain.

    Item Type: Article
    Academic Unit: Faculty of Science and Engineering > Computer Science
    Item ID: 9452
    Depositing User: Liadh Kelly
    Date Deposited: 08 May 2018 16:01
    Journal or Publication Title: Information Retrieval Journal
    Publisher: Springer Netherlands
    Refereed: Yes
      Use Licence: This item is available under a Creative Commons Attribution Non Commercial Share Alike Licence (CC BY-NC-SA). Details of this licence are available here

      Repository Staff Only(login required)

      View Item Item control page


      Downloads per month over past year

      Origin of downloads