Efficient large language models for the NHS and psychiatry

Thesis

Electronic Health Records (EHRs) contain large volumes of unstructured clinical text, produced for a large variety of reasons. The clinical notes within EHR can be full of technical jargon and have a low signal-to-noise ratio. This poses a challenge for healthcare providers like the UK National Health Service (NHS), as making use of these texts to produce insights can be labour-intensive and requires domain expertise. This thesis explores methods for developing efficient, cost-effective, be- spoke Large Language Models (LLMs) to understand NHS clinical text, enabling improved patient management and treatment. An enhanced pretraining regime, using contrastive loss on NHS clinical data, enabled the creation of NHS-specific LLMs within a day on a single GPU. These models outperformed open-source LLMs, facilitating faster adaptation to downstream clinical NLP tasks.

Traditional LLM fine-tuning is computationally expensive and challenging with larger models. Efficient adaptation methods, such as prompt learning, were devel- oped and employed, reducing computational and storage requirements by up to 98% while maintaining state-of-the-art performance on several clinical NLP tasks. The bespoke NHS LLMs and efficient adaptation methods were applied to a digital triage system for secondary mental health referrals. This system aimed to improve transparency, accuracy, and efficiency in routing patients to appropriate care pathways based on their clinical information. The resulting model processed variable-length patient referral text and produced triage team recommendations with an explainability tool to enhance interpretability. Crucially, the triage model remained cost-effective and feasible in resource-constrained environments. This work evaluates the feasibility, utility, and potential benefits of developing specialized LLMs for NHS clinical text processing, discussing implications for enhancing patient care and clinical decision support.

Authors: Taylor, N

• DOI: 10.5287/ora-xonzd55kd
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: embeddings; large language model; decision support; mental health; digital triage
• Subjects: Deep learning (Machine learning)