Overview of the CLEF eHealth evaluation lab 2016

Liadh Kelly, Lorraine Goeuriot, Hanna Suominen, Aurèlie Nèvèol, João Palotti, Guido Zuccon

Research output: A Conference proceeding or a Chapter in BookConference contribution

33 Citations (Scopus)

Abstract

In this paper we provide an overview of the fourth edition of the CLEF eHealth evaluation lab. CLEF eHealth 2016 continues our evaluation resource building efforts around the easing and support of patients, their next-of-kins and clinical staff in understanding, accessing and authoring eHealth information in a multilingual setting. This year’s lab offered three tasks: Task 1 on handover information extraction related to Australian nursing shift changes, Task 2 on information extraction in French corpora, and Task 3 on multilingual patient-centred information retrieval considering query variations. In total 20 teams took part in these tasks (3 in Task 1, 7 in Task 2 and 10 in Task 3). Herein, we describe the resources created for these tasks, evaluation methodology adopted and provide a brief summary of participants to this year’s challenges and some results obtained. As in previous years, the organizers have made data and tools associated with the lab tasks available for future research and development.

Original languageEnglish
Title of host publicationExperimental IR Meets Multilinguality, Multimodality, and Interaction
Subtitle of host publication7th International Conference of the CLEF Association, CLEF 2016, Proceedings
EditorsNorbert Fuhr, Paulo Quaresma, Teresa Gonçalves, Birger Larsen, Krisztian Balog, Craig Macdonald, Linda Cappellato, Nicola Ferro
Place of PublicationCham, Switzerland
PublisherSpringer
Pages255-266
Number of pages12
Volume9822
ISBN (Electronic)9783319445649
ISBN (Print)9783319445632
DOIs
Publication statusPublished - 2016
Event7th International Conference of the CLEF Association, CLEF 2016 - Evora, Evora, Portugal
Duration: 5 Sep 20168 Sep 2016
http://clef2016.clef-initiative.eu/ (Conference website)

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9822 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference7th International Conference of the CLEF Association, CLEF 2016
Abbreviated titleCLEF 2016
CountryPortugal
CityEvora
Period5/09/168/09/16
OtherCLEF 2016 is the seventh CLEF conference continuing the popular CLEF campaigns which have run since 2000 contributing to the systematic evaluation of information access systems, primarily through experimentation on shared tasks. Building on the format first introduced in 2010, CLEF 2016 consists of an independent peer-reviewed conference on a broad range of issues in the fields of multilingual and multimodal information access evaluation, and a set of labs and workshops designed to test different aspects of mono and cross-language Information retrieval systems.
Together, the conference and the lab series will maintain and expand upon the CLEF tradition of community-based evaluation and discussion on evaluation issues
Internet address

Fingerprint

Information Extraction
Nursing
Evaluation
Information retrieval
Resources
Authoring
Handover
Research and Development
Information Retrieval
Continue
Query
Methodology
Corpus

Cite this

Kelly, L., Goeuriot, L., Suominen, H., Nèvèol, A., Palotti, J., & Zuccon, G. (2016). Overview of the CLEF eHealth evaluation lab 2016. In N. Fuhr, P. Quaresma, T. Gonçalves, B. Larsen, K. Balog, C. Macdonald, L. Cappellato, ... N. Ferro (Eds.), Experimental IR Meets Multilinguality, Multimodality, and Interaction: 7th International Conference of the CLEF Association, CLEF 2016, Proceedings (Vol. 9822, pp. 255-266). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9822 LNCS). Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-319-44564-9_24
Kelly, Liadh ; Goeuriot, Lorraine ; Suominen, Hanna ; Nèvèol, Aurèlie ; Palotti, João ; Zuccon, Guido. / Overview of the CLEF eHealth evaluation lab 2016. Experimental IR Meets Multilinguality, Multimodality, and Interaction: 7th International Conference of the CLEF Association, CLEF 2016, Proceedings. editor / Norbert Fuhr ; Paulo Quaresma ; Teresa Gonçalves ; Birger Larsen ; Krisztian Balog ; Craig Macdonald ; Linda Cappellato ; Nicola Ferro. Vol. 9822 Cham, Switzerland : Springer, 2016. pp. 255-266 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{8a4b8f0fcefa43059dc7d94fb63fc376,
title = "Overview of the CLEF eHealth evaluation lab 2016",
abstract = "In this paper we provide an overview of the fourth edition of the CLEF eHealth evaluation lab. CLEF eHealth 2016 continues our evaluation resource building efforts around the easing and support of patients, their next-of-kins and clinical staff in understanding, accessing and authoring eHealth information in a multilingual setting. This year’s lab offered three tasks: Task 1 on handover information extraction related to Australian nursing shift changes, Task 2 on information extraction in French corpora, and Task 3 on multilingual patient-centred information retrieval considering query variations. In total 20 teams took part in these tasks (3 in Task 1, 7 in Task 2 and 10 in Task 3). Herein, we describe the resources created for these tasks, evaluation methodology adopted and provide a brief summary of participants to this year’s challenges and some results obtained. As in previous years, the organizers have made data and tools associated with the lab tasks available for future research and development.",
keywords = "Entity linking, Evaluation, Information extraction, Information retrieval, Medical informatics, Nursing records, Patient handoff/handover, Self-diagnosis, Speech recognition, Test-set generation, Text classification, Text segmentation",
author = "Liadh Kelly and Lorraine Goeuriot and Hanna Suominen and Aur{\`e}lie N{\`e}v{\`e}ol and Jo{\~a}o Palotti and Guido Zuccon",
year = "2016",
doi = "10.1007/978-3-319-44564-9_24",
language = "English",
isbn = "9783319445632",
volume = "9822",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer",
pages = "255--266",
editor = "Norbert Fuhr and Paulo Quaresma and Teresa Gon{\cc}alves and Birger Larsen and Krisztian Balog and Craig Macdonald and Linda Cappellato and Nicola Ferro",
booktitle = "Experimental IR Meets Multilinguality, Multimodality, and Interaction",
address = "Netherlands",

}

Kelly, L, Goeuriot, L, Suominen, H, Nèvèol, A, Palotti, J & Zuccon, G 2016, Overview of the CLEF eHealth evaluation lab 2016. in N Fuhr, P Quaresma, T Gonçalves, B Larsen, K Balog, C Macdonald, L Cappellato & N Ferro (eds), Experimental IR Meets Multilinguality, Multimodality, and Interaction: 7th International Conference of the CLEF Association, CLEF 2016, Proceedings. vol. 9822, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9822 LNCS, Springer, Cham, Switzerland, pp. 255-266, 7th International Conference of the CLEF Association, CLEF 2016, Evora, Portugal, 5/09/16. https://doi.org/10.1007/978-3-319-44564-9_24

Overview of the CLEF eHealth evaluation lab 2016. / Kelly, Liadh; Goeuriot, Lorraine; Suominen, Hanna; Nèvèol, Aurèlie; Palotti, João; Zuccon, Guido.

Experimental IR Meets Multilinguality, Multimodality, and Interaction: 7th International Conference of the CLEF Association, CLEF 2016, Proceedings. ed. / Norbert Fuhr; Paulo Quaresma; Teresa Gonçalves; Birger Larsen; Krisztian Balog; Craig Macdonald; Linda Cappellato; Nicola Ferro. Vol. 9822 Cham, Switzerland : Springer, 2016. p. 255-266 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9822 LNCS).

Research output: A Conference proceeding or a Chapter in BookConference contribution

TY - GEN

T1 - Overview of the CLEF eHealth evaluation lab 2016

AU - Kelly, Liadh

AU - Goeuriot, Lorraine

AU - Suominen, Hanna

AU - Nèvèol, Aurèlie

AU - Palotti, João

AU - Zuccon, Guido

PY - 2016

Y1 - 2016

N2 - In this paper we provide an overview of the fourth edition of the CLEF eHealth evaluation lab. CLEF eHealth 2016 continues our evaluation resource building efforts around the easing and support of patients, their next-of-kins and clinical staff in understanding, accessing and authoring eHealth information in a multilingual setting. This year’s lab offered three tasks: Task 1 on handover information extraction related to Australian nursing shift changes, Task 2 on information extraction in French corpora, and Task 3 on multilingual patient-centred information retrieval considering query variations. In total 20 teams took part in these tasks (3 in Task 1, 7 in Task 2 and 10 in Task 3). Herein, we describe the resources created for these tasks, evaluation methodology adopted and provide a brief summary of participants to this year’s challenges and some results obtained. As in previous years, the organizers have made data and tools associated with the lab tasks available for future research and development.

AB - In this paper we provide an overview of the fourth edition of the CLEF eHealth evaluation lab. CLEF eHealth 2016 continues our evaluation resource building efforts around the easing and support of patients, their next-of-kins and clinical staff in understanding, accessing and authoring eHealth information in a multilingual setting. This year’s lab offered three tasks: Task 1 on handover information extraction related to Australian nursing shift changes, Task 2 on information extraction in French corpora, and Task 3 on multilingual patient-centred information retrieval considering query variations. In total 20 teams took part in these tasks (3 in Task 1, 7 in Task 2 and 10 in Task 3). Herein, we describe the resources created for these tasks, evaluation methodology adopted and provide a brief summary of participants to this year’s challenges and some results obtained. As in previous years, the organizers have made data and tools associated with the lab tasks available for future research and development.

KW - Entity linking

KW - Evaluation

KW - Information extraction

KW - Information retrieval

KW - Medical informatics

KW - Nursing records

KW - Patient handoff/handover

KW - Self-diagnosis

KW - Speech recognition

KW - Test-set generation

KW - Text classification

KW - Text segmentation

UR - http://www.scopus.com/inward/record.url?scp=84984804764&partnerID=8YFLogxK

UR - http://www.mendeley.com/research/overview-clef-ehealth-evaluation-lab-2016

U2 - 10.1007/978-3-319-44564-9_24

DO - 10.1007/978-3-319-44564-9_24

M3 - Conference contribution

SN - 9783319445632

VL - 9822

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 255

EP - 266

BT - Experimental IR Meets Multilinguality, Multimodality, and Interaction

A2 - Fuhr, Norbert

A2 - Quaresma, Paulo

A2 - Gonçalves, Teresa

A2 - Larsen, Birger

A2 - Balog, Krisztian

A2 - Macdonald, Craig

A2 - Cappellato, Linda

A2 - Ferro, Nicola

PB - Springer

CY - Cham, Switzerland

ER -

Kelly L, Goeuriot L, Suominen H, Nèvèol A, Palotti J, Zuccon G. Overview of the CLEF eHealth evaluation lab 2016. In Fuhr N, Quaresma P, Gonçalves T, Larsen B, Balog K, Macdonald C, Cappellato L, Ferro N, editors, Experimental IR Meets Multilinguality, Multimodality, and Interaction: 7th International Conference of the CLEF Association, CLEF 2016, Proceedings. Vol. 9822. Cham, Switzerland: Springer. 2016. p. 255-266. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-44564-9_24