Evaluation of large language models for carbohydrate counting in type 1 diabetes mellitus: a comparative study with patient estimations
Diabetes Research and Clinical Practice, cilt.236, 2026 (SCI-Expanded, Scopus)
- Yayın Türü: Makale / Tam Makale
- Cilt numarası: 236
- Basım Tarihi: 2026
- Doi Numarası: 10.1016/j.diabres.2026.113279
- Dergi Adı: Diabetes Research and Clinical Practice
- Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, BIOSIS, CINAHL, EMBASE, Index Islamicus, MEDLINE
- Anahtar Kelimeler: Artificial intelligence in diabetes, Carbohydrate counting, Type 1 diabetes mellitus
- Ankara Yıldırım Beyazıt Üniversitesi Adresli: Evet
Özet
Aims: To evaluate large language models (LLMs) accuracy in carbohydrate (CHO) counting (CC) and compare their performance with estimations by patients with type-1 diabetes mellitus (T1DM). Methods: This cross-sectional study included 14 adults with T1DM with advanced CC training and > 70% accuracy on the AdultCarbQuiz. Eighteen main meals and four snacks were prepared by dietitians using standardized descriptions and two-angle photographs. Dietitian-determined values served as the reference. CHO estimations by patients and LLMs (ChatGPT-5.2, Gemini-3 Pro, and Claude Sonnet-4.5) were compared. LLM performance was evaluated in two-stages: image-only (stage-1) and image-plus-detailed-description (stage-2) inputs. Performance was assessed using mean absolute error (MAE), mean absolute percentage error (MAPE), counting accuracy (CA; ≤10 g), Wilcoxon signed-rank test, and Bland–Altman analysis. Results: Patients showed MAE 8.31 g, MAPE 13.82%, and CA 73.74%. In Stage-1, Gemini had the lowest MAE (6.55 g), and MAPE (12.54%), while Claude achieved the highest CA (81.8%). In Stage-2, Gemini outperformed patients (p < 0.001) with MAE 2.14 g, MAPE 3.21%, CA 100%, and mean bias − 1.05 g. Other model–stage combinations showed performance comparable to patients. Conclusion: LLMs, particularly Gemini-3 Pro, may support CC when detailed inputs are provided; however, findings should be interpreted cautiously given the small, highly selected sample.