切换至 "中华医学电子期刊资源库"

中华口腔医学研究杂志(电子版) ›› 2026, Vol. 20 ›› Issue (01) : 17 -24. doi: 10.3877/cma.j.issn.1674-1366.2026.01.003

数智口腔专栏·论著

深度学习与多模态大语言模型在笑线分类任务中的应用初探
孟笑菲1, 龚卓弘2, 万耀文3, 李沛达3, 胡琪琪1, 邱龙诗语4, 刘恒毅4, 谢伟丽1,()   
  1. 1哈尔滨医科大学附属第一医院口腔修复科,哈尔滨 150001
    2香港大学牙医学院修复齿科,香港 999077
    3中山大学计算机学院,广州 510006
    4中山大学附属口腔医院,光华口腔医学院,广东省口腔医学重点实验室,广东省口腔疾病临床医学研究中心,广州 510055
  • 收稿日期:2025-12-09 出版日期:2026-02-01
  • 通信作者: 谢伟丽

Preliminary exploration on the application of the classification of dental smile lines via deep learning and multimodal large language model

Xiaofei Meng1, Zhuohong Gong2, Yaowen Wan3, Peida Li3, Qiqi Hu1, Longshiyu Qiu4, Hengyi Liu4, Weili Xie1,()   

  1. 1Department of Prosthodontics, The First Affiliated Hospital of Harbin Medical University, Harbin 150001, China
    2Restorative Dental Sciences, Faculty of Dentistry, The University of Hong Kong, Hong Kong 999077, China
    3School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou 510006, China
    4Hospital of Stomatology, Guanghua School of Stomatology, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Stomatology, Guangdong Provincial Clinical Research Center of Oral Diseases, Guangzhou 510055, China
  • Received:2025-12-09 Published:2026-02-01
  • Corresponding author: Weili Xie
引用本文:

孟笑菲, 龚卓弘, 万耀文, 李沛达, 胡琪琪, 邱龙诗语, 刘恒毅, 谢伟丽. 深度学习与多模态大语言模型在笑线分类任务中的应用初探[J/OL]. 中华口腔医学研究杂志(电子版), 2026, 20(01): 17-24.

Xiaofei Meng, Zhuohong Gong, Yaowen Wan, Peida Li, Qiqi Hu, Longshiyu Qiu, Hengyi Liu, Weili Xie. Preliminary exploration on the application of the classification of dental smile lines via deep learning and multimodal large language model[J/OL]. Chinese Journal of Stomatological Research(Electronic Edition), 2026, 20(01): 17-24.

目的

口腔微笑美学评估是临床治疗规划的关键环节,笑线作为关键美学指标,其精准分类对优化美学修复重建治疗方案具有重要意义。然而,笑线分类需要对唇、龈和齿之间的复杂关系准确判断,医师分析的主观性可能造成误诊。本研究旨在以笑线分类任务为口腔微笑美学智能化初探,对比卷积神经网络(CNN)、大语言模型(LLM)及不同层次医师在口腔微笑线分类任务中的表现。

方法

以公开高质量人脸数据集FFHQ为基础,经图像预处理与标准化标注后构建含1 000张样本的微笑图像标注数据集,标注内容为高、中、低笑线3种类型。采用7种经典CNN模型(VGG16、ResNet34等)与5种代表性多模态LLM(Qwen系列、LLaVA-1.5-7B)进行训练、验证和测试,通过准确率、精确率、召回率及F1分数比较模型性能,并与不同层级临床医师的评估结果进行对比。

结果

在7种常用CNN模型中,ResNet152模型总体表现最优,分类准确率达83.30%,显著优于其他CNN模型及多模态LLM;高级口腔医师分类准确率为83.00%,与ResNet152性能接近。注意力热力图显示ResNet152模型关注区域与医师相似。

结论

CNN模型在笑线分类任务中具备更高的临床应用潜力,可达到专家水平;LLM的医学图像精细分类能力仍需优化。本研究为口腔美学智能评估系统的开发提供了实验依据与技术参考。

Objective

Smile aesthetic evaluation is a critical component of clinical treatment planning. As a key aesthetic indicator, the precise classification of the smile line is crucial for optimizing restorative and reconstructive treatment plans. However, classifying smile line requires accurate assessment of complex relationships among the lips, gingiva and teeth, and analysis by dentists involves a degree of subjectivity and chances for misdiagnosis. This study aimed to investigate smile line classification by comparing the performance of convolutional neural networks (CNNs) and large language models (LLMs) , as well as clinicians of varying expertise levels, in this task.

Methods

Based on the publicly available high-quality FFHQ facial dataset, a smile image annotation dataset comprising 1 000 samples was constructed following image preprocessing and standardized annotations of three types: high, medium and low smile line. Seven classic CNN models (VGG16, ResNet34, etc.) and five representative multimodal LLMs (Qwen series, LLaVA 1.5-7B) were employed for training, validation, and testing. Model performance was evaluated using accuracy, precision, recall, and F1 scores, and compared against assessments made by clinicians of different seniority levels.

Results

Among the seven commonly used CNN models, the ResNet152 model demonstrated optimal overall performance, achieving a mean classification accuracy of 83.30%, which significantly outperformed other CNN models and multimodal LLMs. Senior dentists achieved a classification accuracy of 83.00%, comparable to the performance of ResNet152. Heatmaps demonstrate similar attention regions between ResNet152 and dental practitioners.

Conclusions

CNN models demonstrated substantial clinical potential in smile line classification tasks, attaining expert-level performance. In contrast, large language models required further optimization for medical image fine-grained classification. This study provided experimental evidence and technical insights for developing intelligent aesthetic assessment systems in dentistry.

图1 基于卷积神经网络(CNN)和大语言模型(LLM)进行笑线自动分类的模式图
图2 7种卷积神经网络(CNN)模型统计学分析结果  a模型性能与ResNet152之间的差异无统计学意义(P>0.05);b模型性能与ResNet152相比差异具有统计学意义(P<0.05)
图3 5种大语言模型(LLM)统计学分析结果  a模型性能与LLaVA-1.5-7B之间的差异无统计学意义(P>0.05);b模型性能与LLaVA-1.5-7B相比差异具有统计学意义(P<0.05)
表1 各级医师与最佳卷积神经网络(CNN)和大语言模型(LLM)在测试集上的性能比较
图4 不同层次口腔医师/医学生与人工智能模型进行笑线分类任务的混淆矩阵图 A ~ C:医学生、初级医师、高级医师混淆矩阵图;D:ResNet152混淆矩阵图;E:LLaVA-1.5-7B混淆矩阵图。
图5 最佳卷积神经网络(CNN)模型(ResNet152网络)进行笑线分类任务的热力图 A ~ B:高笑线示例图片及热力图;C ~ D:中笑线示例图片及热力图;E ~ F:低笑线示例图片及热力图。
[1]
Lukez APavlic ATrinajstic Zrinski M,et al. The unique contribution of elements of smile aesthetics to psychosocial well-being[J]. J Oral Rehabil201542(4):275-281. DOI:10.1111/joor.12250.
[2]
Pham TAVNguyen PA. Morphological features of smile attractiveness and related factors influence perception and gingival aesthetic parameters[J]. Int Dent J202272(1):67-75. DOI:10.1016/j.identj.2021.02.001.
[3]
Wang CHu WJLiang LZ,et al. Esthetics and smile-related characteristics assessed by laypersons[J]. J Esthet Restor Dent201830(2):136-145. DOI:10.1111/jerd.12356.
[4]
许砚耕,张艳玲,胡文杰,等.以微笑美观为导向的口腔软组织美学评价方法概述[J].口腔医学202545(1):18-24. DOI:10.13591/j.cnki.kqyx.2025.01.004.
[5]
Lee SJin GPark JH,et al. Evaluation metric of smile classification by peri-oral tissue segmentation for the automation of digital smile design[J]. J Dent2024145:104871. DOI:10.1016/j.jdent.2024.104871.
[6]
Liu MQXu ZNMao WY,et al. Deep learning-based evaluation of the relationship between mandibular third molar and mandibular canal on CBCT[J]. Clin Oral Investig202226(1):981-991. DOI:10.1007/s00784-021-04082-5.
[7]
Zeng PSong RLin Y,et al. Abnormal maxillary sinus diagnosing on CBCT images via object detection and 'straight-forward' classification deep learning strategy[J]. J Oral Rehabil202350(12):1465-1480. DOI:10.1111/joor.13585.
[8]
Cui ZFang YMei L,et al. A fully automatic AI system for tooth and alveolar bone segmentation from cone-beam CT images [J]. Nat Commun202213(1):2096. DOI:10.1038/s41467-022-29637-2.
[9]
Gong ZLi XShi M,et al. Measuring the binary thickness of buccal bone of anterior maxilla in low-resolution cone-beam computed tomography via a bilinear convolutional neural network [J]. Quant Imaging Med Surg202313(12):8053-8066. DOI:10.21037/qims-23-744.
[10]
Chen XZhou CZhu Y,et al. Detecting glaucoma in highly myopic eyes from fundus photographs using deep convolutional neural networks[J]. Clin Exp Ophthalmol202553(5):502-515. DOI:10.1111/ceo.14498.
[11]
Biswas SS. Role of chat GPT in public health[J]. Ann Biomed Eng202351(5):868-869. DOI:10.1007/s10439-023-03172-7.
[12]
Russell BCTorralba AMurphy KP,et al. LabelMe:A database and web-based tool for image annotation[J]. Int J Comput Vision200877(1):157-173. DOI:10.1007/s11263-007-0090-8.
[13]
Tjan AHMiller GDThe JG. Some esthetic factors in a smile[J]. J Prosthet Dent198451(1):24-28. DOI:10.1016/s0022-3913(84)80097-9.
[14]
Selvaraju RRCogswell MDas A,et al. Grad-CAM:Visual explanations from deep networks via gradient-based localization[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision(ICCV). Venice,2017:22-29.

URL    
[15]
Das NHussain EMahanta LB. Automated classification of cells into multiple classes in epithelial tissue of oral squamous cell carcinoma using transfer learning and convolutional neural network[J]. Neural Netw2020128:47-60. DOI:10.1016/j.neunet.2020.05.003.
[16]
Setzer FCShi KJZhang Z,et al. Artificial intelligence for the computer-aided detection of periapical lesions in cone-beam computed tomographic images[J]. J Endod202046(7):987-993. DOI:10.1016/j.joen.2020.03.025.
[17]
Xu XLiu CZheng Y. 3D tooth segmentation and labeling using deep convolutional neural networks[J]. IEEE Trans Vis Comput Graph201925(7):2336-2348. DOI:10.1109/tvcg.2018.2839685.
[18]
Khan MKazmi SMRKhan FR,et al. Analysis of different characteristics of smile[J]. BDJ Open20206:6. DOI:10.1038/s41405-020-0032-x.
[19]
Cunha JFernandes GVOFernandes JCH,et al. The interference of age and gender on smile characterization analyzed on six parameters:A clinical-photographic pilot study[J]. Medicina (Kaunas)202359(3):595. DOI:10.3390/medicina59030595.
[20]
Shi MGong ZZeng P,et al. Multi-quantifying maxillofacial traits via a demographic parity-based AI model[J]. BME Front20245:0054. DOI:10.34133/bmef.0054.
[1] 张振奇, 卢漫, 齐艺涵, 庄敏, 胡紫玥, 王璐. 基于DeepSeek大语言模型的胃癌和直肠癌超声报告结构化及T分期自动评估研究[J/OL]. 中华医学超声杂志(电子版), 2025, 22(11): 1055-1061.
[2] 张振奇, 齐艺涵, 王璐, 胡紫玥, 李婷婷, 卢漫. 大语言模型DeepSeek-R1在甲状腺超声报告质量控制中的初步应用[J/OL]. 中华医学超声杂志(电子版), 2025, 22(09): 832-837.
[3] 江瑶, 蒋程, 余翔, 谭莹, 温昕, 温慧莹, 彭桂艳, 李胜利. 基于注意力机制改进的子宫解剖结构检测与分割多任务模型的性能评估[J/OL]. 中华医学超声杂志(电子版), 2025, 22(08): 703-710.
[4] 李婷, 郭超, 李晨曦, 李柔演, 张一涵, 张雨晗, 陈琰, 斯琴高娃, 龚忠诚. 基于深度学习的上颌阻生尖牙自动分割与体积量化研究[J/OL]. 中华口腔医学研究杂志(电子版), 2026, 20(01): 9-16.
[5] 杨雯林, 吴元魁. 影像组学在胰腺神经内分泌瘤诊疗中的研究进展[J/OL]. 中华普通外科学文献(电子版), 2025, 19(06): 426-432.
[6] 梅昊楠, 杨瑞, 刘修恒. 人工智能辅助病理学图像分析在前列腺癌诊断中的研究进展[J/OL]. 中华腔镜泌尿外科杂志(电子版), 2026, 20(01): 1-7.
[7] 希龙夫, 薛荣泉. 人工智能在肝胆胰肿瘤诊治中应用与进展[J/OL]. 中华腔镜外科杂志(电子版), 2025, 18(03): 166-171.
[8] 赵婷, 易晓芳. 人工智能在子宫腺肌病微创治疗中的应用进展[J/OL]. 中华腔镜外科杂志(电子版), 2025, 18(03): 172-176.
[9] 施薇薇, 楼微华, 狄文, 严斌, 张楠, 王酉. 人工智能驱动的腔镜外科发展:研究进展与未来趋势[J/OL]. 中华腔镜外科杂志(电子版), 2025, 18(03): 177-183.
[10] 唐玥, 陈家璐, 覃德龙, 李宗龙, 汤朝晖, 全志伟. 基于AI的多模态影像在肝癌诊治中应用及面临挑战[J/OL]. 中华肝脏外科手术学电子杂志, 2026, 15(01): 4-9.
[11] 鲁莽, 马晓璐, 沈浮, 王颢, 邵成伟, 张卫, 陆建平, 陆海迪. 基于磁共振的深度学习重建方法在直肠癌术前评估中的应用研究[J/OL]. 中华结直肠疾病电子杂志, 2025, 14(05): 445-456.
[12] 郭寒川, 王乾宇, 吴斌. 人工智能在神经识别的研究进展及直肠癌自主神经保护的应用[J/OL]. 中华结直肠疾病电子杂志, 2025, 14(03): 273-276.
[13] 王柯云, 孙雅佳, 李甜, 张钰哲, 郑颖, 张伟光, 王倩, 董哲毅. 糖尿病肾脏疾病早期发生风险预测模型的研究进展[J/OL]. 中华肾病研究电子杂志, 2025, 14(04): 218-225.
[14] 代巍巍, 沈伟, 刘彦, 周黎明. 大语言模型在重症医学领域的应用与展望[J/OL]. 中华重症医学电子杂志, 2025, 11(03): 221-225.
[15] 薛怡宁, 兰雅迪, 刘兆宇, 史磊, 赵琪, 许洪伟. 基于图像的人工智能在胃癌中的研究进展[J/OL]. 中华消化病与影像杂志(电子版), 2025, 15(06): 670-675.
阅读次数
全文


摘要


AI


AI小编
你好!我是《中华医学电子期刊资源库》AI小编,有什么可以帮您的吗?