Attention-Based Bidirectional Long Short-Term Memory Networks for Extracting Temporal Relationships from Clinical Discharge Summaries

Ghada Alfattni, Goran Nenadic, Niels Peek

Research output: Contribution to journalArticlepeer-review

42 Downloads (Pure)

Abstract

Temporal relation extraction between health-related events is a widely studied task in clinical Natural Language Processing (NLP). The current state-of-the-art methods mostly rely on engineered features (i.e., rule-based modelling) and sequence modelling, which often encodes a source sentence into a single fixed-length context. An obvious disadvantage of this fixed-length context design is its incapability to model longer sentences, as important temporal information in the clinical text may appear at different positions. To address this issue, we propose an Attention-based Bidirectional Long Short-Term Memory (Att-BiLSTM) model to enable learning the important semantic information in long source text segments and to better determine which parts of the text are most important. We experimented with two embeddings and compared the performances to traditional state-of-the-art methods that require elaborate linguistic pre-processing and hand-engineered features. The experimental results on the i2b2 2012 temporal relation test corpus show that the proposed method achieves a significant improvement with an F-score of 0.811, which is at least 10% better than state-of-the-art in the field. We show that the model can be remarkably effective at classifying temporal relations when provided with word embeddings trained on corpora in a general domain. Finally, we perform an error analysis to gain insight into the common errors made by the model.
Original languageEnglish
Article number103915
JournalJournal of Biomedical Informatics
Volume123
Early online date29 Sept 2021
DOIs
Publication statusPublished - 1 Nov 2021

Keywords

  • Clinical text
  • Deep learning
  • Discharge summaries
  • NLP
  • TLINKs

Fingerprint

Dive into the research topics of 'Attention-Based Bidirectional Long Short-Term Memory Networks for Extracting Temporal Relationships from Clinical Discharge Summaries'. Together they form a unique fingerprint.

Cite this