DIGITAL ORTHODONTICS

Machine-learning–based detection of
degenerative temporomandibular joint
diseases using lateral cephalograms
Xinyi Fang,a,b Xin Xiong,a Jiu Lin,a Yange Wu,a Jie Xiang,a and Jun Wanga
Chengdu, Sichuan, and Hangzhou, Zhejiang, China

Introduction: Degenerative temporomandibular joint diseases (DJDs) are common diseases in dental practice,
characterized by a series of degenerative processes in the temporomandibular joint. Early clinical detection of
DJD by dental practitioners can be beneficial to prevent or alleviate the further progression of the disease. This
study aimed to develop a cephalogram-based multidimensional nomogram to screen DJD. Methods: A total of
502 patients (170 normal and 332 with DJD) were randomly assigned to a training set (n 5 351) or a validation set
(n 5 151). Thirty-six cephalometric parameters were extracted from the cephalograms to be used as input for a
predictive machine-learning algorithm. Multivariable logistic regression was used to construct a combined model
for visualization in the form of a nomogram. Receiver operating characteristic curve, calibration testing, and
decision curve analyses were conducted to evaluate the performance of the combined model. Results: A
Ceph score consisting of 22 cephalometric parameters were significantly associated with DJD (P\0.01). A com-
bined model that consisted of Ceph scores and clinical features (including age, gender, limited mouth opening,
crepitus, etc.) performed well in the receiver operating characteristic curve (area under the curve, 0.893), cali-
bration test, and decision curve analyses, indicating its potential clinical value. Conclusions: This study con-
structed and verified a multidimensional nomogram consisting of Ceph scores and clinical features, which
may contribute to the clinical screening of DJD in dental practice. Future studies are needed to test the reliability
of the model with similar parameters. (Am J Orthod Dentofacial Orthop 2023;163:260-71)

D
egenerative temporomandibular joint diseases joint cartilage loss, osteoproliferative body formation,
(DJDs) are common pathologic conditions and subchondral bone remodeling and hardening. DJD
affecting the temporomandibular joint (TMJ) is often accompanied by pain, dysfunction, joint
with a prevalence of 11%-20% among temporomandib- noises, and compromised oral health–related quality of
ular disorder (TMD) patients.1-3 DJD is characterized by a life.4,5 DJD with joint pain is classified as temporoman-
series of degenerative processes in the TMJ, including dibular joint osteoarthritis (TMJOA), whereas DJD
without joint pain is classified as osteoarthrosis.2 Ac-
a
State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral cording to a previous study, 35% of the population
Diseases, West China Hospital of Stomatology, Sichuan University, Chengdu, Si- with minimal condyle and/or eminence flattening
chuan, China.
b show no symptoms.6 They may experience various
Department of Orthodontics, Hospital of Stomatology, Key Laboratory of Oral
Biomedical Research of Zhejiang Province, School of Stomatology, Zhejiang Uni- morphologic and functional deformities when the
versity School of Medicine, Hangzhou, Zhejiang, China. breakdown of the TMJ begins.7 Thus, screening for
Xinyi Fang and Xin Xiong are joint first authors and contributed equally to this
DJD in the dental clinic during early, more treatable
work.
All authors have completed and submitted the ICMJE Form for Disclosure of Po- stages is challenging and important.
tential Conflicts of Interest, and none were reported. The diagnosis of DJD is typically made on the basis of
This research was funded by the National Natural Science Foundation of China
a radiographic examination of the condyle and articular
(nos. 81771114 and 81970967), the Sichuan Science and Technology Program
(no. 2020YFS0173), and the Major Special Science and Technology Project of Si- eminence.4 TMJ with erosive resorption, attrition, scle-
chuan Province (no. 2022ZDZX0031). rosis, cyst-like changes, and osteophyte formation is
Address correspondence to: Jun Wang, State Key Laboratory of Oral Diseases,
classified as DJD.4 Orthopantomography (OPG), trans-
National Clinical Research Center for Oral Diseases, West China Hospital of Sto-
matology, Sichuan University, No. 14, 3rd section, People’s South Rd, Chengdu, cranial oblique lateral radiography, cone-beam
Sichuan 610041, China; e-mail, wangjunv@scu.edu.cn. computed tomography (CBCT), and magnetic resonance
Submitted, February 2022; revised and accepted, October 2022.
imaging are often used for the detection of DJD.8-10
0889-5406/$36.00
Ó 2022 by the American Association of Orthodontists. All rights reserved. Among them, CBCT is recognized for its advantages in
https://doi.org/10.1016/j.ajodo.2022.10.015

260
Fang et al 261

an accurate, detailed presentation of the TMJ and can be MATERIAL AND METHODS
used to demonstrate the degree and location of TMJ This retrospective study was approved by the Ethics
damage.11 However, CBCT is not commonly performed Committee of West China Hospital of Stomatology Si-
in clinical screening for DJD because of its high radiation chuan University (WCHSIRB-2020-418). All patients
dosage and cost. Generally, OPG is the most typical and their legal guardians were informed of the possibility
method for screening of the maxillofacial region.12 that their records might be used for research purposes,
However, this method of DJD diagnosis is low in accu- and oral informed consent was obtained. Patients from
racy and sensitivity because of image distortion and whom both CBCT of the TMJs and a lateral cephalogram
overlap.12 were obtained during the period from 2018 to 2021 were
Lateral cephalograms are another plain radiograph randomly evaluated. Only the first scan accompanied by
commonly obtained in the dental clinic. They were orig- CBCT was included for training the AI model for patients
inally used in diagnosis and evaluation in orthodontic with $1 lateral cephalogram. All the patients’ personal
practice and are widely used for evaluating craniomaxil- information was deidentified.
lofacial morphology.13-15 Lateral cephalograms are not The inclusion criteria included (1) patients aged $18
used for imaging the TMJ because they are a 2- years, (2) patients with TMJ-related clinical information,
dimensional modality. Several investigations have re- and (3) CBCT conducted within 1 week before/after
ported that patients with DJD show special cephalo- lateral cephalogram. The exclusion criteria included (1)
metric features compared with normal patients.13,16 patients with cephalogram or CBCT from which features
Patients with severe DJD have a short ramus height could not be successfully extracted; (2) patients with tu-
and backward-rotated mandible observable on cephalo- mor or maxillofacial deformity that could cause joint
grams.17 Therefore, the use of lateral cephalograms can deformity; (3) patients with a history of orthodontic
assist in DJD diagnosis. In addition, the standard land- treatment, plastic surgery, or other craniofacial surgery;
marks used in cephalometric tracing make up for the and (4) patients with a history of TMJ treatment. Ulti-
problem of morphologic deviations in OPG application. mately, 502 patients were included in this study. Among
Analyzing and summarizing cephalometric parameters them, 351 patients (59 male and 292 female), with an
strongly related to DJD would be helpful for clinical average age of 32.29 6 10.90 years (age range, 18-69
diagnosis. years), were randomly assigned to training set. The other
Considering the high prevalence of degenerative alter- 151 patients (29 male and 122 female), with an average
ation in TMD patients and the lack of simple and sensitive age of 30.14 6 9.51 years (age range, 18-64 years),
clinical diagnostic features,18 an accurate tool for primary constituted the validation set. The TMD clinical exami-
diagnosis of DJD would be helpful for clinical practice. As nation, including evaluation of limited mouth opening,
an emerging field attracting numerous researchers’ inter- deviation, clicking, and crepitus, was performed by the 2
est, radiomic feature-based artificial intelligence (AI) TMJ specialists (with 8 and 10 years of experience,
models are widely used in diagnostics, decision-making, respectively) as well according to DC/TMD criteria. Any
and outcome prediction in clinics.19-22 Several disagreement was resolved through consultation with a
investigators have constructed AI-based models for the third specialist with 20 years of experience diagnosing
detection of TMJOA using CBCT and OPG.19-21,23-25 and treating TMJ disorders. In addition, self-reported
However, the limited access to CBCT and variability of symptoms were extracted from the medical records. A
panoramic features among different devices restricts the flow diagram of this study is presented in Figure 1.
application of these models. Moreover, their trained CBCT images were obtained from all patients using
deep-learning models cannot explain and analyze the re- the 3D Accuitomo scanner (J Morita Corp, Kyoto, Japan).
sults.26 Following the “as low as reasonably achievable” Scan settings were as follows: 90 kVp, 5 mA, exposure
principle with respect to radiation exposure, it is unethical time of 17.5 seconds, voxel size of 0.25 mm, slice thick-
to regularly perform CBCT on every patient for the ness of 0.25 mm, a field of view of 140 mm 3 100 mm,
screening of DJD. As most patients seeking orthodontic and 360 rotation. The criteria for CBCT diagnosis of
treatment are already subject to cephalograms, a cephalo- DJD were developed with reference to previous litera-
metric parameter-based DJD detection procedure may ture: (1) normal-sized condyle with osteosclerosis and/
assist in the primary diagnosis of DJD without requiring or joint flattening; (2) deformed condyle with a subcu-
additional radiation exposure. Therefore, we sought to taneous cyst, cortical resorption, or extensive osteoscle-
build and verify a cephalometric parameter-based predic- rosis.27 Patients with unilateral or bilateral TMJ
tive machine-learning algorithm to assist in the clinical osteoarthritis were designated as DJD (Supplementary
screening of DJD. Fig 1). Two TMJ specialists (with 4 and 5 years of

American Journal of Orthodontics and Dentofacial Orthopedics February 2023  Vol 163  Issue 2
262 Fang et al

TMJ Feature Feature Model Model

Evaluation Exreaction Selection Analysis Evaluation

Normal Cephalometric LASSO Nomogram

Ceph score
(n=170) tracing regression
Points

Gender

34 35 33 30 28 27 23 20 20 16 12 6 5 4 0
P<0.001 Age

75

1.35
Limited
mouth opening

Deviation

1.30
Binomial Deviance
Clicking

1.25
Crepitus

1.20
Pain

70
Ceph score

1.15

Ceph score
Total Points

1.10
Risk of DJD

1.05
−8 −6 −4 −2
Calibration curve
log (λ)
65

1.0
30 23 13 2

DJD 1.0

0.8
(n=332) Cephalometric
Coefficients

Actual DJD rate

0.5

0.6
60
parameters Normol DJD

0.4
0.0

Features Number

0.2
−0.5

Apparent
Bias−corrected

All 36 Ideal

0.0
−8 −6 −4 −2

ROC curves
log (λ)
0.0 0.2 0.4 0.6 0.8 1.0
Crainal base 3
Selected Predicted event probability

Maxillary skeletal 2
parameters Training set
Decision curve
Ar-Go-Me
1.0
Mandibular skeletal 6

1.0
Combined model
Y-Axis Clinical model
PP-OP All
None
L1-Me
ANB
0.8

Random

0.8
U6-PP
Maxillary/Mandibular 2 S-Ar-Go
Sensitivity

Go-Me

Net benefit
FMIA
0.6

0.6
allocation
PP-FH
Vertical dimension 11 U1-SN
LL-EP
N-S-Ar
0.4

0.4
Dc-Xi-Pm

(7:3) Dentoalveolar 10
MP-OP)
Co-A
S-Ar
AUC
0.2

0.2
Combined : 0.893 (0.847-0.919)
S-N Ceph score : 0.861 (0.822-0.900)
Overbite
Training set (n = 351) Test set (n = 151) Clinical : 0.701(0.645-0.756)
Facial profile 2 Y-Axis Length
ANS-Xi-Pm
0.0

0.0
TMJOA (n = 228) TMJOA (n = 104) FMA
Normal (n = 123) Normal (n = 47) 0.0 0.2 0.4 0.6 0.8 1.0
0.0 0.2 0.4 0.6 0.8 1.0
-0.4 -0.2 0.0 0.2 0.4
1 − Specificity Threshold probability
Coefficients

Fig 1. Flowchart depicting the design of the study.

experience, respectively) were assigned to evaluate these cephalometric parameters in the normal and DJD
TMJ independently using the CBCT without access to groups are listed in Supplementary Table I. To assess
the patients’ cephalograms. Any disagreement was the reproducibility of the cephalometric parameters, 2
resolved through consultation with a third specialist orthodontists traced 50 randomly selected cephalograms
with 20 years of experience diagnosing and treating and repeated these tracings after 30 days. The interclass
TMJ disorders. and intraclass correlation coefficients (ICCs) and Bland-
Cephalometric parameter extraction was performed Altman analysis were computed to evaluate the intraob-
on lateral cephalograms, which were acquired using server and interobserver reproducibility of cephalometric
the same radiographic machine. Cephalometric tracing parameters.
was conducted with the software Uceph (version 4.4.2; The least absolute shrinkage and selection operator
Yacent, China) by 2 orthodontists (X.F. and X.X.) with 5 (LASSO) logistic regression algorithm was applied to
years of experience each (Supplementary Fig 2). The or- choose the most effective and reproducible DJD-
thodontists were blinded to the clinical information and related cephalometric parameters out of the 36 listed
diagnosis of all patients. A total of 36 cephalometric pa- cephalometric parameters.28 Cephalometric parameters
rameters were digitized and divided into 7 categories: with nonzero coefficients in the LASSO regression were
cranial base relationship (3 parameters), size and posi- selected to generate a risk score (referred to as a Ceph
tion of the maxilla (2 parameters), size and position of score) in the training set. The Ceph score was then
the mandible (6 parameters), the relationship between used to predict the TMJ status in both the training
maxilla and mandible (2 parameters), vertical dimension and validation sets using a Mann-Whitney U test. The
(11 parameters), dentoalveolar features (10 parameters) predictive accuracy of the Ceph score was evaluated by
and facial profile (2 parameters). The reference plane the receiver operating characteristic (ROC) test and
was the Frankfort horizontal plane. Detailed values of area under the curve (AUC) in both sets.

February 2023  Vol 163  Issue 2 American Journal of Orthodontics and Dentofacial Orthopedics
Fang et al 263

Multivariable logistic regression analysis was con- randomly assigned to the training and testing groups
ducted on the training set to create a clinical model at a ratio of 7:3. Clinical characteristics and cephalo-
with the following factors: age, gender, and the experi- metric measurement values of included patients are pre-
ence of limited mouth opening, deviation, clicking, crep- sented in Table I and Supplementary Table II. The
itus, joint pain, and orofacial pain. intraobserver ICCs and the interobserver ICCs were
Ceph scores were combined with the clinical model both .0.75. Bland-Altman plots of ANB, FH-PP, and
above to develop a diagnostic model for DJD. A com- S-N are shown in Supplementary Figure 3. Bias in the
bined model with all features was constructed on the Bland-Altman analysis ranged from 0.08 to 0.61. All
basis of a multivariate logistic regression model. A P values in the Bland-Altman analysis were .0.05, rep-
nomogram was generated on the basis of the combined resenting acceptable intraobserver and interobserver
model in the training set using the R package rms as a reproducibility of cephalometric parameters. No signifi-
quantitative tool to predict the individual probability cant difference was found in any clinical feature be-
of DJD. tween the training and the validation sets, which
We compared the discriminatory performance of the indicated excellent equivalency between the 2 sets. Sig-
3 established models (clinical model, Ceph score, and nificant differences were found in gender and experience
combined model) using ROC curves and AUC values in of limited mouth opening and crepitus between the
both the training and validation sets. Then, calibration normal and DJD groups in both the training and test
curves and the Hosmer-Lemeshow test were also em- sets (Table I).
ployed to evaluate the calibration of the combined A total of 36 cephalometric parameters were used for
model. To further evaluate the efficiency of the com- the LASSO analysis. LASSO regression was employed to
bined model for different demographic characteristics, choose the optimized cephalometric parameters for the
a stratified analysis was conducted. The performance cephalometrics model. In this manner, 22 cephalometric
of the combined model was evaluated by the AUC values parameters with nonzero coefficients were selected to
in different subgroups, including gender (male or fe- construct the cephalometrics signature (Fig 2, A-C),
male), sagittal dimension (skeletal Class I, skeletal Class which consisted of 7 vertical dimension parameters, 5
II, or skeletal Class III), and vertical dimension (low angle, dentoalveolar parameters, 4 mandibular skeletal param-
average angle, or high angle). The calibration curves eters, 3 cranial base parameters, 1 facial profile param-
were plotted using R package rms. eter, 1 maxillary skeletal parameter, and 1 maxillary/
To evaluate the clinical application potential of our mandibular parameter. Using all these cephalometric pa-
combined model, we conducted a decision curve analysis rameters, a formula for calculating a risk score (Ceph
(DCA) to further assess the net benefit acquired from the score) was established on the basis of a multi-logistic
deployment of the clinical model and the combined regression, as presented in the Supplementary Material.
model. The performance of these 2 models was analyzed The Ceph score of each patient was calculated. A sig-
at different threshold probabilities, and the model with nificant difference was detected between patients with
the larger region under the curve was chosen for better and without DJD in the training set (P \0.001, Fig 3,
clinical outcomes.29 A), which was verified in the validation set (P \0.001,
Fig 3, C). The AUC value of Ceph scores was 0.861
Statistical analysis (95% confidence interval (CI), 0. 822-0.900, Fig 3, B)
in the training set and 0.812 (95% CI, 0.738-0.886;
Normally distributed variables were compared using Fig 3, D) in the validation set, which demonstrated
Student’s t test. Continuous variables that were not that Ceph scores have a good discriminatory ability.
normally distributed were compared using the Mann- A combined model was constructed using multivari-
Whitney U test. The discrete variables were analyzed us- able regression analysis on the basis of traditional clin-
ing the chi-square test. All the statistical analyses used in ical features and Ceph scores. A detailed description of
this study were performed with SPSS (version 20; IBM, the features is shown in Table II. Ceph scores, the expe-
Armonk, NY), R (4.1.0; R Foundation for Statistical rience of limited mouth opening, and crepitus were all
Computing, Vienna, Austria), and EmpowerStats significantly correlated with DJD. A nomogram based
2.2.0.11 (X&Y solutions, Inc, Boston, Mass). P \0.05 on these features was established (Fig. 4, A). The ROC
was considered statistically significant. test showed that the combined model scored 0.893
(95% CI, 0.847-0.919) on the training set, 0.828 (95%
RESULTS CI, 0.757-0.899) on the validation set, and thus per-
The study workflow is presented in Figure 1. A total formed better than the clinical model, which scored
of 502 patients were included in this study and were 0.701 (95% CI, 0.645-0.756) on the training set and

American Journal of Orthodontics and Dentofacial Orthopedics February 2023  Vol 163  Issue 2
264 Fang et al

Table I. Characteristics of patients in the training and validation set

Training set (n 5 351) Test set (n 5 151)

Characteristics DJD (n 5 228) Normal (n 5 123) P values DJD (n 5 104) Normal (n 5 47) P values
Age, mean 6 standard deviation 31.81 6 10.43 33.17 6 11.75 0.189 30.33 6 9.58 29.74 6 9.56 0.803
Gender, n (%)
Male 28 (12.3) 31 (25.2) 0.002 25 (24.0) 4 (8.5) 0.018
Female 200 (87.7) 92 (74.8) 79 (76.0) 43 (91.5)
Limited mouth opening, n (%)
Yes 88 (38.6) 29 (23.6) 0.003 34 (32.7) 8 (17.0) 0.034
No 140 (61.4) 94 (76.4) 70 (67.3) 39 (83.0)
Deviation, n (%)
Yes 104 (45.6) 60 (48.8) 0.324 50 (48.1) 29 (61.7) 0.084
No 124 (54.4) 63 (51.2) 54 (51.9) 18 (38.3)
Clicking, n (%)
Yes 171 (75.0) 82 (66.7) 0.063 84 (80.8) 34 (72.3) 0.171
No 57 (25.0) 41 (33.3) 20 (19.2) 13 (27.7)
Crepitus, n (%)
Yes 86 (37.7) 16 (13.0) \0.0001 33 (31.7) 8 (17.0) 0.043
No 142 (62.3) 107 (87.0) 71 (68.3) 39 (83.0)
Pain, n (%)
Yes 188 (82.5) 101 (82.1) 0.523 87 (83.7) 37 (78.7) 0.303
No 40 (17.5) 22 (17.9) 17 (16.3) 10 (21.3)
Ceph score, median (interquartile range) 68.62 (67.86-69.32) 66.76 (65.98-67.60) \0.0001 68.14 (67.17-69.15) 66.74 (66.11-67.25) \0.0001

0.603 (95% CI, 0.505-0.701) on the validation set (Figs model and its nomogram displayed superb resolution
4, B and C). Significant differences were shown in AUC ability on the training and validation sets in the ROC
values between the combined model and the clinical test. The AUC values of the Ceph score (0.861 in the
model (P \0.001), which confirmed the combined training set, 0.812 in the validation set) and the com-
model’s superior predictive performance. In addition, bined model (0.893 in the training set, 0.828 in the vali-
the combined model revealed a good capacity for iden- dation set) were significantly higher than those of
tifying DJD in different subgroups (Table III), indicating the clinical model (0.701 in the training set, 0.603 in
its utility for recognizing the disease in different stratifi- the validation set). Furthermore, the DCA showed that
cation contexts. the combined model could be more effective for clinical
The calibration curves of the combined model be- decision-making than the clinical model. The standard
tween predicted and actual DJD showed good consis- and well-developed landmarks used in cephalography
tency in both the training and validation sets (Figs 5, make it possible for this cephalometric parameter-
A and B). According to the Hosmer-Lemeshow test, based combined model to be widely used in diagnosing
nonsignificant P values were found in both the training DJD.
set (P 5 0.863) and validation set (P 5 0.881), demon- We established that cephalometrics could distinguish
strating that the combined model achieved acceptable patients with DJD from normal patients through a
goodness of fit. comprehensive analysis of cephalometric parameters,
The DCA for the combined model and the traditional including cranial base relationship, size and position of
clinical model is presented in Figure 6. The decision the maxilla, size and position of the mandible, the rela-
curve demonstrated that the combined model would tionship between maxilla and mandible, vertical dimen-
be more beneficial than the “treat all patients” strategy sion, dentoalveolar features, and facial profile.
or the “treat none” strategy when the threshold proba- Considering potential doubts that could arise regarding
bility was .0.312. the reproducibility and robustness of cephalometric pa-
rameters, we took strong precautions to ensure the ob-
DISCUSSION jectivity and reproducibility of our cephalometric
In this study, we built and validated a combined model. The diagnosis of DJD and cephalometric feature
model consisting of a cephalometric parameter-based extraction used in this study have been commonly
Ceph score and traditional clinical features, which can applied and verified in previous studies. Two orthodon-
assist in the clinical screening of DJD. This combined tists traced the cephalograms based on the established

February 2023  Vol 163  Issue 2 American Journal of Orthodontics and Dentofacial Orthopedics
Fang et al 265

A 34 35 33 30 28 27 23 20 20 16 12 6 5 4 0 B 30 23 13 2
1.35

1.0
1.30
Binomial Deviance

Coefficients
1.25

0.5
1.20
1.15

0.0
1.10

−0.5
1.05

−8 −6 −4 −2
−8 −6 −4 −2
log (λ) log (λ)

C Ar-Go-Me(Gonial/JawAngle,°)
Y-Axis(SGn-FH,°)
PP-OP(°)
Cephalometric parameters

L1-Me(mm)
ANB(°)
U6-PP(mm)
S-Ar-Go(ArticularAngle,°)
Go-Me(Mandibular body length,mm)
FMIA(L1-FH)
PP-FH(°)
U1-SN(°)
LL-EP(mm)
N-S-Ar(SaddleAngle,°)
Dc-Xi-Pm(°)
MP-OP(°)
Co-A (Midface Length,mm)
S-Ar (Posterior Cranial Base,mm)
S-N (Anterior Cranial Base,mm)
Overbite (mm)
Y-Axis Length (mm)
ANS-Xi-Pm (°)
FMA (FH-MP)

-0.4 -0.2 0.0 0.2 0.4

Coefficients

Fig 2. Cephalometrics feature selection using the LASSO binary logistic regression model: A, Tuning
parameter (l) selection in the LASSO model used 10-fold cross-validation via minimum criteria. Dotted
vertical lines were drawn at the optimal values using the minimum criteria and the l standard error of the
minimum criteria (1-standard error [SE] criteria). The optimal l value of 0.0027 with log (l) 5 5.8992
was selected (1-SE criteria) according to 10-fold cross-validation; B, LASSO coefficient profiles of the
36 cephalometrics features. The dotted line was plotted at the value selected using 10-fold cross-
validation in A, in which optimal l resulted in 22 nonzero coefficients; C, 22 cephalometric features
were selected for the signature building.

standards, and both ICCs and the Bland-Altman analysis All 36 cephalometric parameters from 7 categories
were used to evaluate subjectivity and operator error. describing the relationships between cranial and maxil-
Random errors for the digital tracing were observed in lofacial soft and hard tissues and occlusion were sub-
the Bland-Altman analysis, which might decrease the jected to analysis to rule out the influences of scale
reliability of the model. Future researchers are encour- differences. To prevent overfitting, parameters less
aged to test the reliability of the model with similar correlated with DJD were excluded from the final calcu-
cephalometric parameters. lation. Using all the above methods, a comparatively

American Journal of Orthodontics and Dentofacial Orthopedics February 2023  Vol 163  Issue 2
266 Fang et al

P<0.001

1.0
A 75 B

0.8
Training set

Training set
70

0.6
Sensitivity
Ceph score

0.4
65

0.2
AUC ： 0.861 (0.822-0.900)

0.0
60 0.0 0.2 0.4 0.6 0.8 1.0
Normol DJD
1 − Specificity
C 75 D
P<0.001

1.0
0.8
Validation set

70 Validation set

Sensitivity
Ceph score

0.6
0.4
65
0.2

AUC : 0.812 (0.738-0.886)

60
0.0

Normol DJD 0.0 0.2 0.4 0.6 0.8 1.0

1 − Specificity

Fig 3. The ROC curves of the Ceph score in the (A) training set and the (B) validation set. The box-dot
plots of the Ceph score in the training set (A) and the validation set (C). Plots (B) and (D) show the ROC
curves of the Ceph score in the training and validation sets, respectively.

Table II. Risk factor for DJD

Combined model (95% CI) Clinical model (95% CI)

Variable Odds ratio P values DJD P values

Age 0.9781(0.9520-1.0049) 0.1085 0.9847 (0.9636-1.0064) 0.1656
Gender 0.8911(0.4149-1.9139) 0.7676 0.4451 (0.2418-0.8193) 0.0093
Limited mouth opening 2.1320(1.0890-4.1739) 0.0272 1.8815 (1.1117-3.1842) 0.0185
Deviation 0.9991(0.5459-1.8287) 0.9977 0.7347 (0.4558-1.1844) 0.2058
Clicking 1.4780 (0.7591-2.8777) 0.2504 1.4814 (0.8750-2.5080) 0.1435
Crepitus 4.2697 (2.0247-9.0040) \0.0001 4.0898 (2.2162-7.5471) \0.0001
Pain 1.2900 (0.5956-2.7940) 0.5184 0.7766 (0.4122-1.4630) 0.4339
Ceph score 3.5658 (2.6574-4.7848) \0.0001 NA NA
NA, not available.

evidence-based cephalometric model was built for DJD investigators have reported that a tendency toward
screening in dental practice. mandibular counterclockwise rotation, mandibular ret-
In our cephalometric model, 22 cephalometric pa- rognathism, and a shorter mandible are associated
rameters with strong correlations with osseous abnor- with TMJOA.13,16,30,31 Moreover, in the sagittal dimen-
malities were extracted, most of which were vertical sion, larger ANB angles were positively related to
dimension parameters. Specifically, counterclockwise TMJOA, which is consistent with the higher prevalence
rotation of the occlusal plane and short posterior facial of TMJOA observed in patients with Class II patients
height were closely related to DJD. Similarly, several malocclusion compared with those with Class I and Class

February 2023  Vol 163  Issue 2 American Journal of Orthodontics and Dentofacial Orthopedics
Fang et al 267

A
Points

Gender

Age

Limited
mouth opening

Deviation

Clicking

Crepitus

Pain

Ceph score

Total Points

Risk of DJD

B Training set
C Validation set
1.0

1.0
0.8

0.8
Sensitivity

Sensitivity
0.6

0.6
0.4

0.4

AUC
AUC
0.2

0.2

Combined : 0.893 (0.847-0.919) Combined: 0.828 (0.757-0.899)

Ceph score : 0.861 (0.822-0.900)
Clinical : 0.701(0.645-0.756)
Ceph score: 0.812 (0.738-0.886)
Clinical: 0.603 (0.505-0.701)
0.0

0.0

0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0

1 − Specificity 1 − Specificity

Fig 4. Combined model for the prediction of DJD: A, The nomogram was developed in the training set,
with the Ceph score and clinical features. Plots (B) and (C) show the ROC curves for the combined
model, Ceph score, and clinical model in the training and validation sets, respectively.

III malocclusion.32 On one hand, a hyperdivergent skel- resorption.17 Regarding mandibular skeletal parameters,
etal pattern and mandibular retrognathism could result we found that increased articular angle (S-AR-GO) is
from aggressive condylar resorption. It has been reported positively associated with DJD. A larger articular angle,
that the digastric muscle and the mylohyoid muscle reflecting a backward position of the mandible, might
retract the mandible downward and backward in pa- cause an unfavorable muscle attachment position and
tients with short mandibular ramus height.17,33 On the abnormal muscle activity, leading to tooth extrusion
other hand, these 2 clinical features have been regarded and subsequent counterclockwise mandibular rota-
as risk factors for TMD.34 In patients with DJD, compres- tion.35,36 Moreover, abnormal overbite has also been
sive deflection loaded on the condyle is produced with shown to be related to DJD. Although controversy re-
masticatory muscular force during the interdigitation mains regarding the relationship between overbite and
of teeth, which may result in compressive condyle TMD, it is undisputed that normal occlusal guidance is

American Journal of Orthodontics and Dentofacial Orthopedics February 2023  Vol 163  Issue 2
268 Fang et al

Table III. The AUC values of the combined model for stratified analysis in different subgroups
Subgroup Patients AUC values (95% CI)
Gender
Male (n 5 88) DJD (n 5 53) 0.812 (0.717-0.907)
Normal (n 5 35)
Female (n 5 414) DJD (n 5 279) 0.877 (0.841-0.912)
Normal (n 5 135)
Sagittal dimension
Skeletal Class I (n 5 256) DJD (n 5 148) 0.816 (0.763-0.869)
Normal (n 5 108)
Skeletal Class II (n 5196) DJD (n 5 160) 0.885 (0.820-0.949)
Normal (n 5 36)
Skeletal Class III (n 5 50) DJD (n 5 24) 0.899 (0.806-0.992)
Normal (n 5 26)
Vertical dimension
Low angle (n 5 167) DJD (n 5 98) 0.847 (0.786-0.909)
Normal (n 5 69)
Average angle (n 5 264) DJD (n 5 174) 0.855 (0.807-0.902)
Normal (n 5 90)
High angle (n 5 71) DJD (n 5 60) 0.949 (0.882-1.000)
Normal (n 5 11)

A Training set B Validation set

1.0
1.0
0.8

0.8
Actual DJD rate
Actual DJD rate

0.6

0.6
0.4

0.4
0.2

Apparent Apparent
Bias−corrected Bias−corrected
0.2

Ideal Ideal
0.0

0.0 0.2 0.4 0.6 0.8 1.0 0.2 0.4 0.6 0.8 1.0

Predicted event probability Predicted event probability

Fig 5. The calibration curves in the training set (A) and validation set (B) represent the predictive per-
formance of the combined model. Gray represents a perfect prediction, and pink represents the predic-
tive performance of the combined model.

important to a healthy muscle-occlusal system and the prevalence of crepitus is significantly higher in pa-
TMJ.37-39 tients with TMJOA,41 and our results are consistent
Among all clinical factors included in the combined with this finding. Usually, clicking indicates disc
model, the history of limited mouth opening (odds ratio, displacement with reduction,42 whereas crepitus is
2.132; 95% CI, 1.089-4.174; P 5 0.027) and crepitus more often related to erosion and the formation of sub-
(odds ratio, 4.270; 95% CI, 2.025-9.004; P \0.0011) chondral cysts and osteophytes in the condyles,43 indi-
were significantly correlated with DJD (Table II). Limited cating a late stage of degeneration. Pain is a
mouth opening is a typical symptom of disc displace- complicated situation that can derive from muscles,
ment without reduction. The prevalence of early-stage the TMJ or other orofacial areas.44 The relationship be-
DJD increases from 24% to 60% 1 month after limited tween pain and DJD is inconclusive because around one-
mouth opening occurs.40 Previous studies implied that third of patients with DJD might not suffer from

February 2023  Vol 163  Issue 2 American Journal of Orthodontics and Dentofacial Orthopedics
Fang et al 269

software or a Web site, and using this algorithm to

1.0
Combined model
Clinical model
All
None analyze the cephalograms of patients could help clini-
cians to assess the joint status of patients and identify
0.8

patients with a higher likelihood of developing DJD. Af-

ter identifying patients with high odds of DJD, the
Net benefit
0.6

dentist could perform a more comprehensive clinical ex-

amination and imaging of the TMJ or refer the patient to
a TMJ specialist.
0.4

Several limitations of this study should be noted.

First, this research was conducted in a single hospital.
0.2

Although the cephalometric parameters based on stan-

dard and well-developed landmarks performed well in
numerous cephalometric analyses,13,14,16 further inves-
0.0

tigations based on multicentric databases are still

0.0 0.2 0.4 0.6 0.8 1.0 required to validate the accuracy and robustness of
Threshold probability
this model. Second, the cephalometric tracing was con-
ducted by 2 orthodontists, but this might be acceptable
because the landmarks were identified strictly on the ba-
Fig 6. DCA for the combined model compared with the
clinical model. The y-axis represents the standardized sis of cephalometric analysis criteria that are accepted
net benefit. The x-axis represents the threshold probabil- internationally. Moreover, the intraobserver ICCs and
ity. Gray represents the possibility that all patients had the interobserver ICCs confirmed good intraobserver
DJD. Black represents the possibility that no patients and interobserver reproducibility of cephalometric pa-
had DJD. The decision curves in the validation set rameters. Finally, to achieve data integrity, only the
showed that using the combined model to predict DJD most active features were included in the predictive
adds more benefit than treating all or no patients when model, and it is possible that some other relevant ceph-
the threshold probability is .0.312. alometric features were excluded.

pain,45,46 and conversely, patients with orofacial pain

might not have osseous changes in the TMJ.43,47 There- CONCLUSIONS
fore, it is reasonable that pain symptoms alone exhibited A novel cephalogram-based combined model was
limited ability in our model to predict DJD because of the constructed and verified to screen for DJD. The model
ambiguity of pain-related symptoms. However, with the consists of Ceph scores and clinical features (including
contribution of the Ceph score and clinical factors, the age, gender, limited mouth opening, crepitus, etc.).
combined model achieved better diagnostic ability. This combined model could enhance the accuracy of
Compared with previous research using radiographic DJD screening and increase the optimization of
examination to diagnose TMJOA,19-21,23 our combined decision-making in dental practice. Future studies are
model is advantageous for several reasons. First, the needed to test the reliability of the model with similar
clinical application potential of this combined model cephalometric parameters.
under various circumstances was validated, as the strat-
ified analysis performed for further evaluation showed.
In addition to cephalometric parameters, conventional AUTHOR CREDIT STATEMENT
clinical features were adopted in the combined model. Xinyi Fang contributed to conceptualization, meth-
The combination of cephalometric and clinical features odology, software, validation, formal analysis, investiga-
offers a convenient DJD screening method without any tion, original draft preparation, and visualization; Xin
special imaging beyond what is routinely done. There Xiong contributed to conceptualization, methodology,
is no need for specialized equipment such as CBCT, software, investigation, and original draft preparation;
which is not commonly available in an average dental Jiu Lin contributed to validation, formal analysis, inves-
clinic. Within limiting radiation exposure, the screening tigation, original draft preparation, and visualization;
of DJD is completed. Some newly graduated or general Yange Wu contributed to resources and data curation;
dentists may have limited knowledge about DJD and Jie Xiang contributed to resources and data curation;
have difficulty diagnosing TMJ status in their orthodon- and Jun Wang contributed to manuscript review and ed-
tic practice.48,49 An algorithm based solely on cephalo- iting, supervision, project administration, and funding
metric measurements in our study can be built into acquisition.

American Journal of Orthodontics and Dentofacial Orthopedics February 2023  Vol 163  Issue 2
270 Fang et al

SUPPLEMENTARY DATA 18. Kim K, Wojczy nska A, Lee JY. The incidence of osteoarthritic
change on computed tomography of Korean temporomandibular
Supplementary data associated with this article can disorder patients diagnosed by RDC/TMD; a retrospective study.
be found, in the online version, at https://doi.org/10. Acta Odontol Scand 2016;74:337-42.
1016/j.ajodo.2022.10.015. 19. Choi E, Kim D, Lee JY, Park HK. Artificial intelligence in detecting
temporomandibular joint osteoarthritis on orthopantomogram.
REFERENCES Sci Rep 2021;11:10246.
20. Lee KS, Kwak HJ, Oh JM, Jha N, Kim YJ, Kim W, et al. Automated
1. Mejersj€ o C, Hollender L. Radiography of the temporomandibular detection of TMJ osteoarthritis based on artificial intelligence. J
joint in female patients with TMJ pain or dysfunction. A seven Dent Res 2020;99:1363-7.
year follow-up. Acta Radiol Diagn (Stockh) 1984;25:169-76. 21. Kim D, Choi E, Jeong HG, Chang J, Youm S. Expert system for
2. Tanaka E, Detamore MS, Mercuri LG. Degenerative disorders of the mandibular condyle detection and osteoarthritis classification in
temporomandibular joint: etiology, diagnosis, and treatment. J panoramic imaging using R-CNN and CNN. Appl Sci 2020;10:
Dent Res 2008;87:296-307. 7464.
3. Valesan LF, Da-Cas CD, Reus JC, Denardin ACS, Garanhani RR, 22. Minston W, Benchimol D, Jacobs R, Lund B, Kr€ uger Weiner C,
Bonotto D, et al. Prevalence of temporomandibular joint disorders: Coucke W, et al. Pre-surgical radiographic and clinical features
a systematic review and meta-analysis. Clin Oral Investig 2021;25: as predictors for temporomandibular joint discectomy prognosis.
441-53. Oral Dis 2022;28:2185-93.
4. Wang XD, Zhang JN, Gan YH, Zhou YH. Current understanding of 23. de Dumast P, Mirabel C, Cevidanes L, Ruellas A, Yatabe M,
pathogenesis and treatment of TMJ osteoarthritis. J Dent Res Ioshida M, et al. A Web-based system for neural network based
2015;94:666-73. classification in temporomandibular joint osteoarthritis. Comput
5. Yap AU, Zhang MJ, Cao Y, Lei J, Fu KY. Comparison of psycholog- Med Imaging Graph 2018;67:45-54.
ical states and oral health-related quality of life of patients with 24. Jung W, Lee KE, Suh BJ, Seok H, Lee DW. Deep learning for oste-
differing severity of temporomandibular disorders. J Oral Rehabil oarthritis classification in temporomandibular joint. Oral Dis 2021.
2022;49:177-85. 25. Emshoff R, Bertram A, Hupp L, Rudisch A. A logistic analysis pre-
6. Brooks SL, Westesson PL, Eriksson L, Hansson LG, Barsotti JB. diction model of TMJ condylar erosion in patients with TMJ
Prevalence of osseous changes in the temporomandibular joint arthralgia. BMC Oral Health 2021;21:374.
of asymptomatic persons without internal derangement. Oral 26. Holzinger A. From machine learning to explainable AI. 2018 World
Surg Oral Med Oral Pathol 1992;73:118-22. Symposium on Digital Intelligence for Systems and Machines.
7. Zarb GA, Carlsson GE. Temporomandibular disorders: osteoar- DISA); 2018. p. 55-66.
thritis. J Orofac Pain 1999;13:295-306. 27. Schiffman E, Ohrbach R, Truelove E, Look J, Anderson G,
8. Whyte A, Boeddinghaus R, Bartley A, Vijeyaendra R. Imaging of the Goulet JP, et al. Diagnostic criteria for temporomandibular disor-
temporomandibular joint. Clin Radiol 2021;76:76.e21-35. ders (DG/TMD) for clinical and research applications: recommen-
9. Xiong X, Ye Z, Tang H, Wei Y, Nie L, Wei X, et al. MRI of temporo- dations of the international RDC/TMD consortium network and
mandibular joint disorders: recent advances and future directions. orofacial pain Special Interest Group. J Oral Facial Pain Headache
J Magn Reson Imaging 2021;54:1039-52. 2014;28:6-27.
10. Nicolielo LFP, Van Dessel J, Shaheen E, Letelier C, Codari M, 28. Tibshirani R. Regression shrinkage and selection via the lasso. J R
Politis C, et al. Validation of a novel imaging approach using Stat Soc B (Methodol) 1996;58:267-88.
multi-slice CT and cone-beam CT to follow-up on condylar re- 29. Van Calster B, Wynants L, Verbeek JFM, Verbakel JY,
modeling after bimaxillary surgery. Int J Oral Sci 2017;9:139-44. Christodoulou E, Vickers AJ, et al. Reporting and interpreting de-
11. Larheim TA, Abrahamsson AK, Kristensen M, Arvidsson LZ. cision curve analysis: a guide for investigators. Eur Urol 2018;74:
Temporomandibular joint diagnostics using CBCT. Dentomaxillo- 796-804.
fac Radiol 2015;44:20140235. 30. Nogami S, Yamauchi K, Satomi N, Yamaguchi Y, Yokota S, Abe Y,
12. White SC, Pharoah MJ. Oral Radiology: Principles and Interpreta- et al. Risk factors related to aggressive condylar resorption after or-
tion. 6th ed. St Louis: Mosby; 1982. p. 2009. thognathic surgery for females: retrospective study. Cranio 2017;
13. Kang JH, Yang IH, Hyun HK, Lee JY. Dental and skeletal matura- 35:250-8.
tion in female adolescents with temporomandibular joint osteoar- 31. Sun ZP, Zou BS, Zhao YP, Ma XC. Craniofacial morphology of Chi-
thritis. J Oral Rehabil 2017;44:879-88. nese patients with bilateral temporomandibular joint osteoarthro-
14. Janson G, Mendes LM, Junqueira CH, Garib DG. Soft-tissue sis. Chin J Dent Res 2011;14:21-7.
changes in Class II malocclusion patients treated with extractions: 32. Krisjane Z, Urtane I, Krumina G, Neimane L, Ragovska I. The prev-
a systematic review. Eur J Orthod 2016;38:631-7. alence of TMJ osteoarthritis in asymptomatic patients with dento-
15. Mi JP, He P, Shi K, Feng SY, Chen XZ, He QQ, et al. Cephalometric facial deformities: a cone-beam CT study. Int J Oral Maxillofac
craniofacial features of patients with Sagliker syndrome: a primary Surg 2012;41:690-5.
analysis of our experience. Ann Transl Med 2021;9:963. 33. Al-Moraissi EA, Wolford LM. Does temporomandibular joint pa-
16. Chen S, Lei J, Fu KY, Wang X, Yi B. Cephalometric analysis of the thology with or without surgical management affect the stability
facial skeletal morphology of female patients exhibiting skeletal of counterclockwise rotation of the maxillomandibular complex
Class II deformity with and without temporomandibular joint os- in orthognathic surgery? A systematic review and meta-analysis.
teoarthrosis. PLoS One 2015;10:e0139743. J Oral Maxillofac Surg 2017;75:805-21.
17. Kajii TS, Fujita T, Sakaguchi Y, Shimada K. Osseous changes of the 34. Miller JR, Mancl L, Critchlow C. Severe retrognathia as a risk factor
mandibular condyle affect backward-rotation of the mandibular for recent onset painful TMJ disorders among adult females. J Or-
ramus in Angle Class II orthodontic patients with idiopathic thod 2005;32:249-56: discussion 247.
condylar resorption of the temporomandibular joint. Cranio 35. Aymach Z, Kawamura H. Facilitating ramus lengthening following
2019;37:264-71. mandibular-dependent surgical closing of a skeletal open bite with

February 2023  Vol 163  Issue 2 American Journal of Orthodontics and Dentofacial Orthopedics
Fang et al 271

short ramus: a new modified technique. J Craniomaxillofac Surg 43. Arayasantiparb R, Mitrirattanakul S, Kunasarapun P,
2012;40:169-72. Chutimataewin H, Netnoparat P, Sae-Heng W. Association of
36. Dinsdale A, Liang Z, Thomas L, Treleaven J. Is jaw muscle activity radiographic and clinical findings in patients with temporoman-
impaired in adults with persistent temporomandibular disorders? A dibular joints osseous alteration. Clin Oral Investig 2020;24:221-7.
systematic review and meta-analysis. J Oral Rehabil 2021;48: 44. Gil-Martınez A, Paris-Alemany A, Lopez-de-Uralde-Villanueva I,
487-516. La Touche R. Management of pain in patients with temporoman-
37. Ishizaki K, Suzuki K, Mito T, Tanaka EM, Sato S. Morphologic, dibular disorder (TMD): challenges and solutions. J Pain Res 2018;
functional, and occlusal characterization of mandibular lateral 11:571-87.
displacement malocclusion. Am J Orthod Dentofacial Orthop 45. Campos MI, Campos PS, Cangussu MC, Guimar~aes RC, Line SR.
2010;137(454):e451-9: discussion 454-5. Analysis of magnetic resonance imaging characteristics and pain
38. John MT, Hirsch C, Drangsholt MT, Mancl LA, Setz JM. Overbite in temporomandibular joints with and without degenerative
and overjet are not related to self-report of temporomandibular changes of the condyle. Int J Oral Maxillofac Surg 2008;37:
disorder symptoms. J Dent Res 2002;81:164-9. 529-34.
39. Olliver SJ, Broadbent JM, Thomson WM, Farella M. Occlusal fea- 46. Liang X, Liu S, Qu X, Wang Z, Zheng J, Xie X, et al. Evaluation of
tures and TMJ clicking: A 30-year evaluation from a cohort study. trabecular structure changes in osteoarthritis of the temporoman-
J Dent Res 2020;99:1245-51. dibular joint with cone beam computed tomography imaging. Oral
40. Lei J, Han J, Liu M, Zhang Y, Yap AU, Fu KY. Degenerative tempo- Surg Oral Med Oral Pathol Oral Rad 2017;124:315-22.
romandibular joint changes associated with recent-onset disc 47. Yi Y, Zhou X, Xiong X, Wang J. Neuroimmune interactions in pain-
displacement without reduction in adolescents and young adults. ful TMD: mechanisms and treatment implications. J Leukoc Biol
J Craniomaxillofac Surg 2017;45:408-13. 2021;110:553-63.
41. Abrahamsson AK, Kristensen M, Arvidsson LZ, Kvien TK, 48. Porto F, Harrell R, Fulcher R, Gonzales T. Knowledge and beliefs
Larheim TA, Haugen IK. Frequency of temporomandibular joint regarding temporomandibular disorders among orthodontists.
osteoarthritis and related symptoms in a hand osteoarthritis Am J Orthod Dentofacial Orthop 2019;156:475-84.
cohort. Osteoarthritis Cartilage 2017;25:654-7. 49. Al-Huraishi HA, Meisha DE, Algheriri WA, Alasmari WF,
42. Dias IM, Cordeiro PC, Devito KL, Tavares ML, Leite IC, Tesch Rde S. Alsuhaim AS, Al-Khotani AA. Newly graduated dentists’ knowl-
Evaluation of temporomandibular joint disc displacement as a risk edge of temporomandibular disorders compared to specialists in
factor for osteoarthrosis. Int J Oral Maxillofac Surg 2016;45:313-7. Saudi Arabia. BMC Oral Health 2020;20:272.

American Journal of Orthodontics and Dentofacial Orthopedics February 2023  Vol 163  Issue 2
271.e1 Fang et al

SUPPLEMENTARY DATA 0.07146 3 FMIA (L1-FH) 1 0.24926 3 U6.PP.MM 1

Calculation formulas for Ceph score, clinical model, and 0.26686 3 L1-Me (mm) 0.1302 3 Overbite (mm) 1
combined model 0.0174 3 LL-EP (mm)
Ceph score 5 0.00776 3 N-S-Ar (SaddleAngle,  ) 1 Clinical score 5 0.81005 0.80942 3 (1, Gender 5
0.20498 3 S-Ar-Go (ArticularAngle,  ) 0.09766 3 male) 0.01537 3 Age 1 0.63207 3 (1, Limited mouth
S-N (Anterior Cranial Base, mm) 0.06531 3 S-Ar (Pos- opening 5 yes) 0.30825 3 (1, Deviation 5 yes) 1
terior Cranial Base, mm) 0.02252 3 Co-A (Midface 0.39300 3 (1, Clicking 5 yes) 1 1.40849 3 (1,
Length, mm) 1 0.3145 3 Ar-Go-Me (Gonial/JawAngle, Crepitus 5 yes) 0.25286 3 (1, Pain 5 yes)

) 0.009 3 Dc-Xi-Pm ( ) 1 0.2901 3 Y-Axis (SGn-FH, Nomoscore 5 85.75810 0.11527 3 (Gender 5

) 0.22797 3 Y-Axis Length (mm) 1 0.20486 3 Go- male) 0.02214 3 Age 1 0.75705 3 (1, Limited mouth
Me (Mandibular body length, mm) 1 0.26303 3 ANB opening 5 yes) 0.00087 3 (1, Deviation 5 yes) 1
0.29693 3 FMA (FH-MP) 1 0.03459 3 PP-FH ( ) 0.39070 3 (1, Clicking 5 yes) 1 1.45155 3 (1,
0.02125 3 MP-OP ( ) 1 0.27777 3 PP-OP ( ) Crepitus 5 yes) 1 0.25464 3 (1, Pain 5 yes) 1
0.27614 3 ANS-Xi-Pm ( ) 1 0.03083 3 U1-SN ( ) 1 1.27139 3 Ceph score.

Supplementary Fig 1. Images of health TMJ and TMJs with degenerative joint diseases: A, Flat-
tening; B, Erosion; C, Osteophyte; D, Subcutaneous cyst; E, Extensive sclerosis; F, Erosion (coronal
view); G, Erosion (transverse view); H, l healthy joint; K, Healthy joints (transverse view).

February 2023  Vol 163  Issue 2 American Journal of Orthodontics and Dentofacial Orthopedics
Fang et al 271.e2

Supplementary Fig 2. Cephalometric tracing.

American Journal of Orthodontics and Dentofacial Orthopedics February 2023  Vol 163  Issue 2
271.e3 Fang et al

Supplementary Fig 3. Bland-Altman plots demonstrate the bias for cephalometric variables: A-C,
ANB; D-F, FH-PP; G-I, S-N. Only 3 parameters were shown in this figure. A, D, and G, Showed intra-
observer error for observer 1. B, E, and H, Showed intraobserver error for observer 2. C, F, and I,
Showed interobserver error. Red indicates the 95% limits of test-retest agreement. Black indicates
the mean of the differences, which are close to 0, indicating low test-retest bias.

February 2023  Vol 163  Issue 2 American Journal of Orthodontics and Dentofacial Orthopedics
Fang et al 271.e4

Supplementary Table I. Detailed description of extracted cephalometrics features

Variables DJD Normal P value
Cranial base
N-S-Ar (saddle angle) ( ) 125.2 6 5.4 126.6 6 4.9 0.004
S-N (anterior cranial base) (mm) 62.2 6 3.1 63.3 6 3.3 \0.001
S-Ar (posterior cranial base) (mm) 32.4 6 3.2 33.2 6 3.3 0.013
Maxillary skeletal
Co-A (midface length) (mm) 80.1 6 4.6 81.1 6 6.1 0.044
SNA ( ) 82.2 6 3.5 81.5 6 3.5 0.033
Mandibular skeletal
SNB ( ) 77.5 6 3.8 78.1 6 4.0 0.089
S-Ar-Go (articular angle) ( ) 151.9 6 6.7 148.6 6 5.9 \0.001
Ar-Go-Me (gonial/Jaw angle) ( ) 117.7 6 6.5 116.8 6 5.6 0.121
Dc-Xi-Pm ( ) 36. 9 6 6.0 37.3 6 4.9 0.467
Ar-Go (ramus height) (mm) 45.8 6 4.4 48.9 6 4.5 \0.001
Go'-Me (mandibular body length) (mm) 68.5 6 4.7 68. 8 6 5.2 0.592
Maxillary/Mandibular
ANB ( ) 4.7 6 2.8 3.4 6 2.2 \0.001
Wits (mm) 0.9 6 3.8 0.2 6 2.9 0.043
Vertical dimension
FMA (FH-MP) 25.0 6 5.6 22.8 6 5.2 \0.001
SN-MP ( ) 34.9 6 6.1 32.4 6 5.9 \0.001
PP-FH ( ) 0.17 6 2.8 0.2 6 2.7 0.759
OP-FH ( ) 8.7 6 3.9 7.5 6 3.7 0.001
MP-OP ( ) 16.3 6 4.2 15.4 6 3.8 0.018
PP-OP ( ) 8.5 6 3.6 6. 8 6 3.2 \0.001
ANS-Xi-Pm ( ) 47.2 6 4.9 47.0 6 4.4 0.637
N-Me (anterior face height) (mm) 114.8 6 6.2 115.3 6 7.3 0.400
S-Go (mm) 75.8 6 5.9 79.1 6 6.3 \0.001
Y-axis (SGn-FH) ( ) 61.5 6 3.6 60.7 6 3.2 0.012
Y-axis length (mm) 115. 4 6 6.2 117.2 6 7.5 0.004
Dentoalveolar
U1-L1 (interincisal angle) ( ) 125.6 6 12.8 126.2 6 10.2 0.625
U1-SN ( ) 102.2 6 8.6 103.5 6 8.1 0.087
FMIA ( ) 57. 7 6 9.2 59.3 6 7.0 0.040
IMPA (L1-MP) 97.3 6 8.1 97.8 6 6.5 0.487
U6-PP (mm) 22.5 6 2.2 22.8 6 2.3 0.173
L6-MP (mm) 31.9 6 2.8 31.9 6 2.8 0.971
U1-ANS (mm) 28.7 6 2.6 28.1 6 2.6 0.011
L1-Me (mm) 40.0 6 3.1 39.8 6 3.3 0.508
Overjet (mm) 4.2 6 1.8 3.7 6 1.4 0.006
Overbite (mm) 2.6 6 2.0 2.5 6 1.4 0.671
Facial profile
UL-EP (mm) 0.7 6 2.8 0.1 6 2.2 0.001
LL-EP (mm) 0.8 6 2.8 0.8 6 2.3 0.876

Note. Values are presented as mean 6 standard deviation.

American Journal of Orthodontics and Dentofacial Orthopedics February 2023  Vol 163  Issue 2
271.e5 Fang et al

Supplementary Table II. Diagnostic performance of models on the training and validation sets
Training set (n 5 332) Test set (n 5 141)

AUC
Models Sensitivity Specificity Accuracy AUC (95% CI) Sensitivity Specificity Accuracy (95% CI)
Clinical model 0.513 0.805 0.615 0.701 (0.645-0.756) 0.606 0.638 0.616 0.603 (0.505-0.701)
Ceph score 0.750 0.846 0.784 0.861 (0.822-0.900) 0.702 0.894 0.762 0.812 (0.738-0.886)
Combined model 0.873 0.756 0.832 0.883 (0.847-0.919) 0.702 0.872 0.755 0.828 (0.757-0.899)

February 2023  Vol 163  Issue 2 American Journal of Orthodontics and Dentofacial Orthopedics