EHR-Based Mobile and Web Platform

for Chronic Disease Risk Prediction Using Large Language Multimodal Models

📋 CIKM'24 - ACM International Conference on Information and Knowledge Management

Chun-Chieh Liao^1†, Wei-Ting Kuo^1†, I-Hsuan Hu¹, Yen-Chen Shih²,
Jun-En Ding¹, Feng Liu¹, Fang-Ming Hung^3*

¹Stevens Institute of Technology, ²Northeastern University,
³Far Eastern Memorial Hospital, Taiwan
^†Equal Contribution, ^*Corresponding Author

📄 Paper 🔗 DOI 🎥 Demo Video 📖 arXiv

Abstract

Traditional diagnosis of chronic diseases involves in-person consultations with physicians to identify the disease. However, there is a lack of research focused on predicting and developing application systems using clinical notes and blood test values. We collected five years of Electronic Health Records (EHRs) from Taiwan's hospital database between 2017 and 2021 as an AI database.

Furthermore, we developed an EHR-based chronic disease prediction platform utilizing Large Language Multimodal Models (LLMMs), successfully integrating with frontend web and mobile applications for prediction. This prediction platform can also connect to the hospital's backend database, providing physicians with real-time risk assessment diagnostics.

Our platform represents a significant advancement in making AI-powered chronic disease prediction accessible through intuitive mobile and web interfaces, bridging the gap between advanced AI research and practical clinical deployment.

Key Features & Innovations

Comprehensive platform combining advanced AI with user-friendly interfaces for clinical deployment

🤖

Large Language Multimodal Models

Integrates clinical notes and blood test data using state-of-the-art language models including BERT, BiomedBERT, Flan-T5, and GPT-2 for comprehensive disease prediction.

📱

Cross-Platform Accessibility

Native mobile and web applications provide healthcare professionals with real-time access to patient risk assessments anywhere, anytime.

🏥

Hospital Integration

Seamlessly connects to hospital backend databases for real-time EHR data processing and immediate clinical decision support.

📊

Explainable AI

Utilizes SHAP values to provide interpretable insights, highlighting key clinical factors contributing to disease risk predictions.

⚡

Real-Time Processing

Asynchronous processing architecture ensures fast response times for clinical workflows without disrupting patient care.

🔒

HIPAA Compliant

Secure data handling with de-identified patient records and robust privacy protections for clinical deployment.

Clinical Performance

Validated on 5 years of EHR data from Far Eastern Memorial Hospital (2017-2021)

1.42M

Clinical Notes

Processed for training

387K

Lab Results

Laboratory test records

1,505

Test Items

Different lab parameters

0.83

Best F1 Score

Heart disease prediction

0.70

Diabetes F1

Using GPT-2 backbone

Disease Types

Diabetes, Heart Disease, Hypertension

System Architecture

Modern cloud-based architecture designed for scalability and clinical integration

Data Collection

EHR data preprocessing and physician annotation with comprehensive quality checks

Multimodal Fusion

Integration of clinical notes and laboratory values using attention mechanisms

Model Training

Large language models fine-tuned for medical domain with multimodal capabilities

Platform Deployment

React frontend, Django backend with PostgreSQL database on AWS infrastructure

Citation

@inproceedings{liao2024ehr,
  title={EHR-Based Mobile and Web Platform for Chronic Disease Risk Prediction Using Large Language Multimodal Models},
  author={Liao, Chun-Chieh and Kuo, Wei-Ting and Hu, I-Hsuan and Shih, Yen-Chen and Ding, Jun-En and Liu, Feng and Hung, Fang-Ming},
  booktitle={Proceedings of the 33rd ACM International Conference on Information and Knowledge Management},
  pages={4404--4408},
  year={2024},
  publisher={ACM},
  doi={10.1145/3627673.3679227},
  url={https://arxiv.org/abs/2406.18087}
}