INTERPRETABLE BIMODAL MACHINE LEARNING FOR MULTICLASS DETECTION OF GALLBLADDER AND LIVER DISEASES USING ULTRASOUND IMAGING AND BIOCHEMICAL MARKERS

Abstract

Diagnosis of gallbladder and biliary tract disease remains challenging and there are several diseases that have similar biochemical profiles and subtle differences in the structure seen on ultrasound. In this study, a bimodal machine-learning framework is proposed based on features extracted from ultrasound-generated images and chemical markers for multiple class detection. Images are obtained from the open-access UIdataGB collection (10,692 images, 1,782 patients) and a 512-dimensional representation is extracted from each image utilizing a VGG16 convolutional neural network. The image features are then concatenated with 7 biochemical and demographic variables and used to classify using an XGBoost model. The framework was evaluated on 23 classes of gallbladder and liver disease, and bimodal learning was compared to image-only and biochemical-only baselines as well as nine popular classical and ensemble classifiers such as logistic regression, random forest, support-vector machines, k-nearest neighbors, naïve bayes, AdaBoost, and LightGBM. The macro F1-scores and macro ROC–AUCs for the Bimodal XGBoost Classifier, CNN (with an image), biochemical Classifier (only biochemical data), and LightGBM (next strongest model) were 0.655, 0.623, 0.648, and 0.978, respectively. Gallstones (F1 0.92), gallbladder carcinoma (F1 0.85) and acute cholecystitis (F1 0.78) were the three conditions that demonstrated strong class level performance. Five-fold stratified cross-validation gave a mean F1 score of 0.655 ± 0.021, and SHAP analysis was applied to measure the relative importance of imaging and biochemical features in model predictions, revealing that the imaging features contributed ~60% of the model's decision weight, while the biochemical features contributed ~40%.