Capturing patient information at nursing shift changes

Methodological evaluation of speech recognition and information extraction

Hanna Suominen, Maree Johnson, Liyuan Zhou, Paula Sanchez, Raul Sirel, Jim Basilakis, Leif Hanlen, Dominique Estival, Linda Dawson, Barbara Kelly

    Research output: Contribution to journalArticle

    15 Citations (Scopus)

    Abstract

    Objective: We study the use of speech recognition and information extraction to generate drafts of Australian nursinghandover documents. Methods Speech recognition correctness and clinicians' preferences were evaluated using 15 recorder-microphone combinations, six documents, three speakers, Dragon Medical 11, and five survey/interview participants. Information extraction correctness evaluation used 260 documents, six-class classification for each word, two annotators, and the CRF++ conditional random field toolkit. Results A noise-cancelling lapel-microphone with a digital voice recorder gave the best correctness (79%). This microphone was also the most preferred option by all but one participant. Although the participants liked the small size of this recorder, their preference was for tablets that can also be used for document proofing and sign-off, among other tasks. Accented speech was harder to recognize than native language and a male speaker was detected better than a female speaker. Information extraction was excellent in filtering out irrelevant text (85% F1) and identifying text relevant to two classes (87% and 70% F1). Similarly to the annotators' disagreements, there was confusion between the remaining three classes, which explains the modest 62% macro-averaged F1. Discussion We present evidence for the feasibility of speech recognition and information extraction to support clinicians' in entering text and unlock its content for computerized decision-making and surveillance in healthcare. Conclusions The benefits of this automation include storing all information; making the drafts available and accessible almost instantly to everyone with authorized access; and avoiding information loss, delays, and misinterpretations inherent to using a ward clerk or transcription services.

    Original languageEnglish
    Pages (from-to)e48-e66
    JournalJournal of the American Medical Informatics Association
    Volume22
    Issue numbere1
    DOIs
    Publication statusPublished - 2015

    Fingerprint

    Information Storage and Retrieval
    Nursing
    Automation
    Tablets
    Noise
    Decision Making
    Language
    Interviews
    Delivery of Health Care
    Recognition (Psychology)

    Cite this

    Suominen, Hanna ; Johnson, Maree ; Zhou, Liyuan ; Sanchez, Paula ; Sirel, Raul ; Basilakis, Jim ; Hanlen, Leif ; Estival, Dominique ; Dawson, Linda ; Kelly, Barbara. / Capturing patient information at nursing shift changes : Methodological evaluation of speech recognition and information extraction. In: Journal of the American Medical Informatics Association. 2015 ; Vol. 22, No. e1. pp. e48-e66.
    @article{79890c1eb067488c931c053831222db0,
    title = "Capturing patient information at nursing shift changes: Methodological evaluation of speech recognition and information extraction",
    abstract = "Objective: We study the use of speech recognition and information extraction to generate drafts of Australian nursinghandover documents. Methods Speech recognition correctness and clinicians' preferences were evaluated using 15 recorder-microphone combinations, six documents, three speakers, Dragon Medical 11, and five survey/interview participants. Information extraction correctness evaluation used 260 documents, six-class classification for each word, two annotators, and the CRF++ conditional random field toolkit. Results A noise-cancelling lapel-microphone with a digital voice recorder gave the best correctness (79{\%}). This microphone was also the most preferred option by all but one participant. Although the participants liked the small size of this recorder, their preference was for tablets that can also be used for document proofing and sign-off, among other tasks. Accented speech was harder to recognize than native language and a male speaker was detected better than a female speaker. Information extraction was excellent in filtering out irrelevant text (85{\%} F1) and identifying text relevant to two classes (87{\%} and 70{\%} F1). Similarly to the annotators' disagreements, there was confusion between the remaining three classes, which explains the modest 62{\%} macro-averaged F1. Discussion We present evidence for the feasibility of speech recognition and information extraction to support clinicians' in entering text and unlock its content for computerized decision-making and surveillance in healthcare. Conclusions The benefits of this automation include storing all information; making the drafts available and accessible almost instantly to everyone with authorized access; and avoiding information loss, delays, and misinterpretations inherent to using a ward clerk or transcription services.",
    keywords = "Computer systems evaluation, Information extraction, Nursing records, Patient handoff, Speech recognition software",
    author = "Hanna Suominen and Maree Johnson and Liyuan Zhou and Paula Sanchez and Raul Sirel and Jim Basilakis and Leif Hanlen and Dominique Estival and Linda Dawson and Barbara Kelly",
    year = "2015",
    doi = "10.1136/amiajnl-2014-002868",
    language = "English",
    volume = "22",
    pages = "e48--e66",
    journal = "Journal of the American Medical Informatics Association : JAMIA",
    issn = "1067-5027",
    publisher = "Oxford University Press",
    number = "e1",

    }

    Suominen, H, Johnson, M, Zhou, L, Sanchez, P, Sirel, R, Basilakis, J, Hanlen, L, Estival, D, Dawson, L & Kelly, B 2015, 'Capturing patient information at nursing shift changes: Methodological evaluation of speech recognition and information extraction', Journal of the American Medical Informatics Association, vol. 22, no. e1, pp. e48-e66. https://doi.org/10.1136/amiajnl-2014-002868

    Capturing patient information at nursing shift changes : Methodological evaluation of speech recognition and information extraction. / Suominen, Hanna; Johnson, Maree; Zhou, Liyuan; Sanchez, Paula; Sirel, Raul; Basilakis, Jim; Hanlen, Leif; Estival, Dominique; Dawson, Linda; Kelly, Barbara.

    In: Journal of the American Medical Informatics Association, Vol. 22, No. e1, 2015, p. e48-e66.

    Research output: Contribution to journalArticle

    TY - JOUR

    T1 - Capturing patient information at nursing shift changes

    T2 - Methodological evaluation of speech recognition and information extraction

    AU - Suominen, Hanna

    AU - Johnson, Maree

    AU - Zhou, Liyuan

    AU - Sanchez, Paula

    AU - Sirel, Raul

    AU - Basilakis, Jim

    AU - Hanlen, Leif

    AU - Estival, Dominique

    AU - Dawson, Linda

    AU - Kelly, Barbara

    PY - 2015

    Y1 - 2015

    N2 - Objective: We study the use of speech recognition and information extraction to generate drafts of Australian nursinghandover documents. Methods Speech recognition correctness and clinicians' preferences were evaluated using 15 recorder-microphone combinations, six documents, three speakers, Dragon Medical 11, and five survey/interview participants. Information extraction correctness evaluation used 260 documents, six-class classification for each word, two annotators, and the CRF++ conditional random field toolkit. Results A noise-cancelling lapel-microphone with a digital voice recorder gave the best correctness (79%). This microphone was also the most preferred option by all but one participant. Although the participants liked the small size of this recorder, their preference was for tablets that can also be used for document proofing and sign-off, among other tasks. Accented speech was harder to recognize than native language and a male speaker was detected better than a female speaker. Information extraction was excellent in filtering out irrelevant text (85% F1) and identifying text relevant to two classes (87% and 70% F1). Similarly to the annotators' disagreements, there was confusion between the remaining three classes, which explains the modest 62% macro-averaged F1. Discussion We present evidence for the feasibility of speech recognition and information extraction to support clinicians' in entering text and unlock its content for computerized decision-making and surveillance in healthcare. Conclusions The benefits of this automation include storing all information; making the drafts available and accessible almost instantly to everyone with authorized access; and avoiding information loss, delays, and misinterpretations inherent to using a ward clerk or transcription services.

    AB - Objective: We study the use of speech recognition and information extraction to generate drafts of Australian nursinghandover documents. Methods Speech recognition correctness and clinicians' preferences were evaluated using 15 recorder-microphone combinations, six documents, three speakers, Dragon Medical 11, and five survey/interview participants. Information extraction correctness evaluation used 260 documents, six-class classification for each word, two annotators, and the CRF++ conditional random field toolkit. Results A noise-cancelling lapel-microphone with a digital voice recorder gave the best correctness (79%). This microphone was also the most preferred option by all but one participant. Although the participants liked the small size of this recorder, their preference was for tablets that can also be used for document proofing and sign-off, among other tasks. Accented speech was harder to recognize than native language and a male speaker was detected better than a female speaker. Information extraction was excellent in filtering out irrelevant text (85% F1) and identifying text relevant to two classes (87% and 70% F1). Similarly to the annotators' disagreements, there was confusion between the remaining three classes, which explains the modest 62% macro-averaged F1. Discussion We present evidence for the feasibility of speech recognition and information extraction to support clinicians' in entering text and unlock its content for computerized decision-making and surveillance in healthcare. Conclusions The benefits of this automation include storing all information; making the drafts available and accessible almost instantly to everyone with authorized access; and avoiding information loss, delays, and misinterpretations inherent to using a ward clerk or transcription services.

    KW - Computer systems evaluation

    KW - Information extraction

    KW - Nursing records

    KW - Patient handoff

    KW - Speech recognition software

    UR - http://www.scopus.com/inward/record.url?scp=84947492008&partnerID=8YFLogxK

    U2 - 10.1136/amiajnl-2014-002868

    DO - 10.1136/amiajnl-2014-002868

    M3 - Article

    VL - 22

    SP - e48-e66

    JO - Journal of the American Medical Informatics Association : JAMIA

    JF - Journal of the American Medical Informatics Association : JAMIA

    SN - 1067-5027

    IS - e1

    ER -