Information extraction to improve standard compliance

The case of clinical handover

Liyuan Zhou, Hanna Suominen

Research output: A Conference proceeding or a Chapter in BookConference contribution

1 Citation (Scopus)

Abstract

Clinical handover refers to healthcare workers transferring responsibility and accountability for patient care, e.g., between shifts or wards. Safety and quality health standards call for this process to be systematically structured across the organisation and synchronous with its documentation. This paper evaluates information extraction as a way to help comply with these standards. It implements the handover process of first specifying a structured handover form, whose hierarchy of headings guides the handover narrative, followed by the technology filling it out objectively and almost instantly for proofing and sign-off. We trained a conditional random field with 8 feature types on 101 expert-annotated documents to 36-class classify. This resulted in good generalization to an independent set of 50 validation and 50 test documents that we now release: 77.9% F1 in filtering out irrelevant information, up to 98.4% F1 for the 35 classes for relevant information, and 52.9% F1 after macro-averaging over these 35 classes, whilst these percentages were 86.2, 100.0, and 70.2 for the leave-one-document-out cross-validation across the first set of 101 documents. Also as a result of this study, the validation and test data were released to support further research.

Original languageEnglish
Title of host publicationAI 2015, Advances in Artificial Intelligence
Subtitle of host publication28th Australasian Joint Conference Canberra, ACT, Australia, November 30 – December 4, 2015 Proceedings
EditorsBernhard Pfahringer, Jochen Renz
Place of PublicationCham, Switzerland
PublisherSpringer
Pages644-649
Number of pages6
Volume9457
ISBN (Electronic)9783319263502
ISBN (Print)9783319263496
DOIs
Publication statusPublished - 2015
Event28th Australasian Joint Conference on Artificial Intelligence, AI 2015: Advances in Artificial Intelligence (AI 2015) - Canberra, Canberra, Australia
Duration: 30 Nov 20154 Dec 2015

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9457
ISSN (Print)03029743
ISSN (Electronic)16113349

Conference

Conference28th Australasian Joint Conference on Artificial Intelligence, AI 2015
Abbreviated titleAI 2015
CountryAustralia
CityCanberra
Period30/11/154/12/15

Fingerprint

Handover
Information Extraction
Compliance
Macros
Health
Accountability
Conditional Random Fields
Independent Set
Cross-validation
Healthcare
Averaging
Percentage
Filtering
Safety
Classify
Standards
Evaluate
Class

Cite this

Zhou, L., & Suominen, H. (2015). Information extraction to improve standard compliance: The case of clinical handover. In B. Pfahringer, & J. Renz (Eds.), AI 2015, Advances in Artificial Intelligence: 28th Australasian Joint Conference Canberra, ACT, Australia, November 30 – December 4, 2015 Proceedings (Vol. 9457, pp. 644-649). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9457). Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-319-26350-2_57
Zhou, Liyuan ; Suominen, Hanna. / Information extraction to improve standard compliance : The case of clinical handover. AI 2015, Advances in Artificial Intelligence: 28th Australasian Joint Conference Canberra, ACT, Australia, November 30 – December 4, 2015 Proceedings. editor / Bernhard Pfahringer ; Jochen Renz. Vol. 9457 Cham, Switzerland : Springer, 2015. pp. 644-649 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{c13862826c1b4f8a9ca0ebf490b42117,
title = "Information extraction to improve standard compliance: The case of clinical handover",
abstract = "Clinical handover refers to healthcare workers transferring responsibility and accountability for patient care, e.g., between shifts or wards. Safety and quality health standards call for this process to be systematically structured across the organisation and synchronous with its documentation. This paper evaluates information extraction as a way to help comply with these standards. It implements the handover process of first specifying a structured handover form, whose hierarchy of headings guides the handover narrative, followed by the technology filling it out objectively and almost instantly for proofing and sign-off. We trained a conditional random field with 8 feature types on 101 expert-annotated documents to 36-class classify. This resulted in good generalization to an independent set of 50 validation and 50 test documents that we now release: 77.9{\%} F1 in filtering out irrelevant information, up to 98.4{\%} F1 for the 35 classes for relevant information, and 52.9{\%} F1 after macro-averaging over these 35 classes, whilst these percentages were 86.2, 100.0, and 70.2 for the leave-one-document-out cross-validation across the first set of 101 documents. Also as a result of this study, the validation and test data were released to support further research.",
keywords = "Artificial intelligence applications, Clinical handover, Computer systems evaluation, Information extraction, Test-set generation",
author = "Liyuan Zhou and Hanna Suominen",
year = "2015",
doi = "10.1007/978-3-319-26350-2_57",
language = "English",
isbn = "9783319263496",
volume = "9457",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer",
pages = "644--649",
editor = "Bernhard Pfahringer and Jochen Renz",
booktitle = "AI 2015, Advances in Artificial Intelligence",
address = "Netherlands",

}

Zhou, L & Suominen, H 2015, Information extraction to improve standard compliance: The case of clinical handover. in B Pfahringer & J Renz (eds), AI 2015, Advances in Artificial Intelligence: 28th Australasian Joint Conference Canberra, ACT, Australia, November 30 – December 4, 2015 Proceedings. vol. 9457, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9457, Springer, Cham, Switzerland, pp. 644-649, 28th Australasian Joint Conference on Artificial Intelligence, AI 2015, Canberra, Australia, 30/11/15. https://doi.org/10.1007/978-3-319-26350-2_57

Information extraction to improve standard compliance : The case of clinical handover. / Zhou, Liyuan; Suominen, Hanna.

AI 2015, Advances in Artificial Intelligence: 28th Australasian Joint Conference Canberra, ACT, Australia, November 30 – December 4, 2015 Proceedings. ed. / Bernhard Pfahringer; Jochen Renz. Vol. 9457 Cham, Switzerland : Springer, 2015. p. 644-649 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9457).

Research output: A Conference proceeding or a Chapter in BookConference contribution

TY - GEN

T1 - Information extraction to improve standard compliance

T2 - The case of clinical handover

AU - Zhou, Liyuan

AU - Suominen, Hanna

PY - 2015

Y1 - 2015

N2 - Clinical handover refers to healthcare workers transferring responsibility and accountability for patient care, e.g., between shifts or wards. Safety and quality health standards call for this process to be systematically structured across the organisation and synchronous with its documentation. This paper evaluates information extraction as a way to help comply with these standards. It implements the handover process of first specifying a structured handover form, whose hierarchy of headings guides the handover narrative, followed by the technology filling it out objectively and almost instantly for proofing and sign-off. We trained a conditional random field with 8 feature types on 101 expert-annotated documents to 36-class classify. This resulted in good generalization to an independent set of 50 validation and 50 test documents that we now release: 77.9% F1 in filtering out irrelevant information, up to 98.4% F1 for the 35 classes for relevant information, and 52.9% F1 after macro-averaging over these 35 classes, whilst these percentages were 86.2, 100.0, and 70.2 for the leave-one-document-out cross-validation across the first set of 101 documents. Also as a result of this study, the validation and test data were released to support further research.

AB - Clinical handover refers to healthcare workers transferring responsibility and accountability for patient care, e.g., between shifts or wards. Safety and quality health standards call for this process to be systematically structured across the organisation and synchronous with its documentation. This paper evaluates information extraction as a way to help comply with these standards. It implements the handover process of first specifying a structured handover form, whose hierarchy of headings guides the handover narrative, followed by the technology filling it out objectively and almost instantly for proofing and sign-off. We trained a conditional random field with 8 feature types on 101 expert-annotated documents to 36-class classify. This resulted in good generalization to an independent set of 50 validation and 50 test documents that we now release: 77.9% F1 in filtering out irrelevant information, up to 98.4% F1 for the 35 classes for relevant information, and 52.9% F1 after macro-averaging over these 35 classes, whilst these percentages were 86.2, 100.0, and 70.2 for the leave-one-document-out cross-validation across the first set of 101 documents. Also as a result of this study, the validation and test data were released to support further research.

KW - Artificial intelligence applications

KW - Clinical handover

KW - Computer systems evaluation

KW - Information extraction

KW - Test-set generation

UR - http://www.scopus.com/inward/record.url?scp=84952653252&partnerID=8YFLogxK

UR - http://www.mendeley.com/research/information-extraction-improve-standard-compliance-case-clinical-handover

U2 - 10.1007/978-3-319-26350-2_57

DO - 10.1007/978-3-319-26350-2_57

M3 - Conference contribution

SN - 9783319263496

VL - 9457

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 644

EP - 649

BT - AI 2015, Advances in Artificial Intelligence

A2 - Pfahringer, Bernhard

A2 - Renz, Jochen

PB - Springer

CY - Cham, Switzerland

ER -

Zhou L, Suominen H. Information extraction to improve standard compliance: The case of clinical handover. In Pfahringer B, Renz J, editors, AI 2015, Advances in Artificial Intelligence: 28th Australasian Joint Conference Canberra, ACT, Australia, November 30 – December 4, 2015 Proceedings. Vol. 9457. Cham, Switzerland: Springer. 2015. p. 644-649. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-26350-2_57