Victor OK Li, ScD | The University of Hong Kong; Jacqueline CK Lam, PhD (Co-Lead); Yang Han, PhD; Lawrence Cheung, PhD; James B Rowe, B.M. B.Ch., PhD; Jocelyn Downey, PhD; David C Rubinsztein,BSc (Med), PhDCompetition Sponsor: Research Grants Council of the Hong Kong Special Administrative Region, China
Award Year: 2024
Timely LOAD prediction is critical to slowing disease progression. Existing works based on clinical biomarkers are generally invasive, time-consuming and expensive, focussing on diagnosis instead of prognosis. In our multi-modal study, linguistic markers are strong early-stage predictors of LOAD. However, speech datasets are typically small, lacking longitudinal representation and linguistic variation, making it difficult to train LOAD models to accurately predict LOAD. Our study revolutionises LOAD prediction via the development of an LLM-driven transformer model trained on sufficient amount of normal control (NC)/mild cognitive impairment (MCI)/LOAD speech samples. First, we develop a novel LLM-driven model to generate new speech and transcription samples based on limited NC/MCI/LOAD samples from DementiaBank and Framingham Heart Study. Particularly, more longitudinal Chinese and English speech samples of NC/MCI/LOAD will be generated from original samples to overcome data shortage. Second, our LOAD speech data are converted into speech and text embeddings capturing the linguistic information of pre-trained LLMs, subsequently fed into our Transformer for accurate prediction of timings for LOAD conversions (including pre-symptomatic LOAD), and identification of reliable linguistic markers discriminating these different conversion states. This has the advantage of increasing/enhancing (1) data volume, diversity, and temporal variability to improve accuracy, robustness and generalizability of LOAD prediction, (2) understanding of language-specific and cross-linguistic markers characterizing disease states (particularly Chinese and English contexts), (3) our detection of MCI/LOAD states, and prediction of the timings of pre-symptomatic onset. Our preliminary results show that LLM+Transformer (with-Generated-Transcription-Data) significantly outperforms Transformer Baseline (without-Generated-Transcription-Data) by 6% in NC/MCI/LOAD detection, achieving 91% accuracy.