Original research

Classification of diabetic maculopathy

based on optical coherence tomography
images using a Vision
Transformer model
Liwei Cai ‍ ‍,1 Chi Wen,2 Jingwen Jiang,3 Congbi Liang,1 Hongmei Zheng,1 Yu Su,1
Changzheng Chen ‍ ‍1

To cite: Cai L, Wen C, ABSTRACT

Jiang J, et al. Classification Purpose To develop a Vision Transformer model to detect WHAT IS ALREADY KNOWN ON THIS TOPIC
of diabetic maculopathy different stages of diabetic maculopathy (DM) based on ⇒ Artificial intelligence (AI) has been used to detect dif-
based on optical coherence ferent macular diseases based on optical coherence
optical coherence tomography (OCT) images.
tomography images using a tomography (OCT) images. However, this new grad-
Vision Transformer model.
Methods After removing images with poor quality, a total
of 3319 OCT images were extracted from the Eye Center ing system for diabetic maculopathy (DM) has not
BMJ Open Ophthalmology
2023;8:e001423. doi:10.1136/ of the Renmin Hospital of Wuhan University and randomly yet been used in deep learning (DL) research, which
bmjophth-2023-001423 split the images into training and validation sets in a 7:3 may be able to predict the treatment outcome and
ratio. All macular cross-­sectional scan OCT images were visual prognosis of DM better in the future.
► Additional supplemental collected retrospectively from the eyes of DM patients from
material is published online WHAT THIS STUDY ADDS
2016 to 2022. One of the OCT stages of DM, including
only. To view, please visit the ⇒ Our DL model based on Vision Transformer demon-
early diabetic macular oedema (DME), advanced DME,
journal online (http://​dx.​doi.​ strated a relatively high accuracy in the detection of
org/​10.​1136/​bmjophth-​2023-​ severe DME and atrophic maculopathy, was labelled on the
collected images, respectively. A deep learning (DL) model OCT grading of DM, which can help with patients in
001423).
based on Vision Transformer was trained to detect four a preliminary screening to identify groups with se-
OCT grading of DM. rious conditions.
LC and CW contributed equally. Results The model proposed in our paper can provide HOW THIS STUDY MIGHT AFFECT RESEARCH,
an impressive detection performance. We achieved an
Received 27 July 2023 PRACTICE OR POLICY
accuracy of 82.00%, an F1 score of 83.11%, an area
Accepted 22 November 2023 ⇒ Our model can help ophthalmologists to develop
under the receiver operating characteristic curve (AUC)
of 0.96. The AUC for the detection of four OCT grading personalised treatment plans for DM patients. These
(ie, early DME, advanced DME, severe DME and atrophic results emphasise the potential of AI in reducing the
maculopathy) was 0.96, 0.95, 0.87 and 0.98, respectively, necessary time of clinical diagnosis, assisting clini-
with an accuracy of 90.87%, 89.96%, 94.42% and cal decision-­making and guaranteeing the cure rate
95.13%, respectively, a precision of 88.46%, 80.31%, in the future.
89.42% and 87.74%, respectively, a sensitivity of 87.03%,
88.18%, 63.39% and 89.42%, respectively, a specificity of
93.02%, 90.72%, 98.40% and 96.66%, respectively and
which is caused by fluid accumulation in the
an F1 score of 87.74%, 84.06%, 88.18% and 88.57%, macula due to a breakdown of the blood–
respectively. retinal barrier.2 3 DME is the most common
Conclusion Our DL model based on Vision Transformer cause of visual impairment in people with
demonstrated a relatively high accuracy in the detection diabetes, and its global prevalence is expected
of OCT grading of DM, which can help with patients in to increase from nearly 18.83 million in 2020
© Author(s) (or their a preliminary screening to identify groups with serious to 28.61 million in 2045.4 Patients with DME
employer(s)) 2023. Re-­use conditions. These patients need a further test for an
permitted under CC BY-­NC. No
can be at great risk of irreversible vision loss
accurate diagnosis, and a timely treatment to obtain a if not treated promptly.2 Early diagnosis and
commercial re-­use. See rights
and permissions. Published by good visual prognosis. These results emphasised the timely treatment can effectively protect and
potential of artificial intelligence in assisting clinicians in
BMJ. restore the vision of DR patients.5
For numbered affiliations see
developing therapeutic strategies with DM in the future.
Previously, based on the location of retinal
end of article. thickening and hard exudates, DME was clas-
sified into involved central and non-­involved
Correspondence to INTRODUCTION central types.6 According to patterns of DME
Dr Changzheng Chen; ​ Diabetic retinopathy (DR) is one of the most on optical coherence tomography (OCT)
whuchenchzh@​163.​com and
common complications of diabetes.1 At any examination, DME can be divided into
Dr Yu Su; ​sy_​daisy1206@​163.​ time during the progression of DR, patients diffused retinal thickening (DRT), cystoid
com may develop diabetic macular oedema (DME), macular oedema (CME) and serous retinal

Cai L, et al. BMJ Open Ophth 2023;8:e001423. doi:10.1136/bmjophth-2023-001423 1

 Open access

detachment (SRD).7 All these classifications only focus images of DM patients centred at the fovea were extracted
on the location and its relationship with the fovea of from an OCT device (Optovue RTVue, Optovue,
macular thickening, or the overall patterns on OCT, Fremont, California, USA) for training and validating
lacking the assessment of macular atrophy and failing the DL model. After randomly splitting the dataset into
to consider the alteration in subtle structure of different training and validation sets in a 7:3 ratio, the training data
manifestations of DME, which cannot meet the needs of and the validation data were completely independent of
treatment. each other. In the model’s training and testing phases, we
In recent years, with the widespread application of made no distinction between the patient’s left and right
spectral domain OCT (SD-­OCT), more biomarkers have eyes. It was not appropriate or possible to involve patients
been identified in DME, which are of great significance or the public in the design, or conduct, or reporting, or
for the treatment and prognosis.8–10 The location and size dissemination plans of our research.
of intraretinal cysts (IRC) correlated with visual acuity at
baseline in DME and the cysts of the inner nuclear layer Grading of DM
were more sensitive to corticosteroid or anti-­VEGF treat- Seven qualitative and quantitative characteristics of DM
ment than the outer nuclear layer.11 12 Visual acuity was are considered and scored according to the grading
closely related to central photoreceptor damage and system called TCED-­HFV, including foveal thickness (T),
the percentage of ellipsoid zone (EZ) destruction, and intraretinal cyst (C), EZ and/or ELM status (E), presence
whether the photoreceptor status was restored deter- of DRIL (D), number of hyper-­reflective foci (H), subfo-
mined the final visual acuity.13 14 The greater the range veal fluid (F), and vitreoretinal relationship (V).16 Based
of disorganisation of the inner retinal layers (DRIL) in on the first four variables, namely T, C, E and D, disease
DME eyes at baseline, the worse the prognosis of vision.15 can be classified into four distinct stages, that is, early
In 2019, an international panel of experts attempted to DME, advanced DME, severe DME and atrophic macu-
combine more OCT-­related morphological features with lopathy (figure 1). Different stages reflect the severity of
central subfoveal thickness (CST) to elaborate a new the disease.
grading system for diabetic maculopathy (DM) appli-
cable to clinical and scientific research.16 DM includes all Image labelling
the phenotypes of macular involvement in DR irrespec- Before training, all OCT images were graded into four
tive of the presence of macular thickening. According stages by trained graders with increasing expertise for
to foveal thickness, the size of the IRC, the EZ and/or verification and correction of image labels. A trained
external limiting membrane (ELM) status, and the pres- grader (LC) excluded images with low image quality.
ence of DRIL, DM is classified as early DME, advanced These images were taken of improper positioning during
DME, severe DME and atrophic maculopathy. Therefore, image acquisition or scans with strong motion arte-
this classification may be able to predict the treatment facts, causing misalignment and blurring of sections.
outcome and visual prognosis of DM better in the future. Then, two retinal specialists (YS and HZ) independently
Patients with DM often need to take regular OCT exam- labelled each image, and images with a clear consensus
inations to record the occurrence and development of annotation between ophthalmologists were taken into
the disease. The increasing number of patients with DM the sample. Images with different grading opinions were
makes it a significant burden for clinicians to manually adjudicated by a senior retinal expert (CC) with more
determine the presence or progression of DM on OCT than 20 years of experience and the final labels were also
images.4 Artificial intelligence (AI) that can help with imported into the database.
screening may reduce the burden on ophthalmologists.
Intelligent systems have been developed for diagnosing Images preprocessing
and classifying DME based on OCT images.17 18 In these The preprocessing part was used to enhance the effec-
studies, AI was only used to detect different macular tive area of OCT images, suppress background noise,
diseases and the overall morphology of DME, and this increase the number of training samples and improve
new grading system has not yet been used in research. In the generalisation ability and robustness of the model.
this study, we aimed to build a deep learning (DL)-­based We chose our segmentation method to simply extract the
training AI system to automatically classify DM images effective region of the OCT images and perform pixel-­
based on the novel classification standard, in order to level enhancement. On the one hand, this was to reduce
help ophthalmologists develop personalised treatment the noise interference. On the other hand, we wanted to
plans for patients with DM. make the model focus more on the effective region of
the images.
MATERIALS AND METHODS The background area occupied most of the OCT
Image dataset image and there was a certain amount of noise overall,
In this study, all completely anonymous retinal OCT which may affect model training. Therefore, we hoped
images were selected from retrospective cohorts of adult to suppress image noise to make it easier for the network
patients from the Eye Center of the Renmin Hospital of to focus on the effective area of the OCT image and
Wuhan University between 2016 and 2022. 4076 OCT converge faster.

2 Cai L, et al. BMJ Open Ophth 2023;8:e001423. doi:10.1136/bmjophth-2023-001423

 Open access

Figure 1 Representative optical coherence tomography images. (A) Early diabetic macular oedema (DME). (B) Advanced
DME. (C) Severe DME. (D) Atrophic maculopathy.

To this end, we proposed to use the Otsu method to 1. Based on an unbiased linear layer, we calculated the
binarise the image, and fuse the resulting binary image logits using the weights of the classifier and the input
with the original image at a certain ratio. This enhanced feature vectors with L2-­Norm.
the effective area of the image and suppressed back- 2. We combined the CrossEntropy Loss and logits with
ground noise. L2-­Norm, which will keep the magnitude of logits a
More specifically, first, according to the interclass constant during training, to create a new loss function.
variance of the histogram of the OCT image, a binary Concretely, the formula of the logits calculated by clas-
segmentation was performed to obtain a binary image, sifier that we proposed was as follows:
denoted as P. The original image was denoted as T. The wT x
dot product operation was performed on P and T and g=K
‍ ||w||||x|| ‍
add up P and T to get the final result. The formula was where $g$ is denoted as the logits output of the back-
as follows: bone, $w$ is the weight parameters of the classifier and x
stands for the feature input of the classsifier, K is a hyper-
parameter.
‍ Iprepro = (T · P) · α + T · (1 − α)‍ And the new loss function can be defined as:
The value of $\alpha$ is between 0 and 1, set manually. g
n
Then, conventional data augmentation methods were ∑ e ||g||
used to enhance the preprocessed images. Specific L=− yi log gj
measures will be introduced in the model validation i=1 ∑k ||g||
section. ‍ j=1 e ‍
Where the temperature parameter $\tau$ controlls the
Model training magnitude of the logits and $y_i$ represents the label.
In the training part, guided by the idea of normalisation,
we designed a classifier and loss function to alleviate the Model inference
problem of network overconfidence and improve its In practice, the distribution of OCT images often varied
robustness. between training and test data, potentially leading to a
Considering that OCT images were prone to noise, decrease in the model’s performance. To address this
we chose the Vision Transformer as the backbone to issue, we have introduced an adaptive mechanism that
extract features from OCT images, which was more allowed the model to dynamically update its parameters
robust to noise compared with convolutional neural based on test data, thereby enhancing both its perfor-
network (CNN), as the Vision Transformer can better mance and generalisation ability. For each test sample,
mine global information through its self-­attention mech- we performed multiple random data augmentation oper-
anism and had less bias towards local texture features. ations, such as rotation, cropping and flipping, to obtain
The features extracted by the backbone were classified diversified test samples and improve the robustness and
through our self-­designed classifier. As there was a long-­ generalisation ability of the model.
tailed problem in the dataset, we redesigned the classifier For each test sample x, a series of random enhance-
and loss function using some normalisation techs. The ment operations were performed m times to obtain a
specific implementation method was as follows: sample set $X=\{x_1,x_2,…,x_m\}$. Take this set as a batch

Cai L, et al. BMJ Open Ophth 2023;8:e001423. doi:10.1136/bmjophth-2023-001423 3

 Open access

and input into the model to obtain the output distribu- Model validation
tion $p(y|x)$. Here, every single $x \in X$ must predict Some metrics have been used to show model perfor-
a label $y\in Y$, but it was important to note that $y$ was mance. The correlation between the true labels and
not the ground truth. We hoped that the model could the predicted labels from our model was depicted as a
maintain relatively stable in the predictions of the same confusion matrix, which was used to calculate the accu-
sample under numerous data augmentation operations, racy, precision, sensitivity, specificity and F1 score for
which meant an improvement of the model’s robustness. image recognition. We also used the area under (AUC)
To achieve this goal, we took the entropy of the average the receiver operating characteristic (ROC) curve to
output distribution of the model as the optimised goal. evaluate the accuracy of the model in detecting the four
By minimising the entropy value, the parameters of the stages. All the metrics mentioned above were calculated
model are updated. Then we input the original sample x using TorchMetrics. We used TorchMetrics to generate
into the model and got the final result. Specifically, the the ROC curve for each stage of DM. This was done by
formula for the optimisation objective was as follows: taking the model’s predictions for all the validation data
∑
H=− p̂(y|x) log p̂(y|x) and the corresponding ground truth labels as input. This
y∈Y approach was also used for other metrics mentioned in
‍ ‍
the paper. For the generation of the ROC curve of the
Note that for every test sample x, the model’s parame-
multiclass classifiers, TorchMetrics employed the One-­vs-­
ters that update in test time will not be saved.
Rest (OvR) strategy. This strategy treated each class as the
Experiment positive class and all other classes as the negative class,
In order to verify the generalisation of our proposed calculating the ROC curve for each class separately. The
schemes, we conducted experiments on the dataset cut-­
off values were determined dynamically by Torch-
constructed by ourselves. Metrics based on all the validation data, varying for each
First, the preprocessing method we proposed was category, eliminating the need for manual setting.
applied to the OCT images with the $\alpha$ value set
to 0.2 to obtain enhanced OCT images. Then, some RESULTS
conventional data augmentation methods were applied A total number of 4076 OCT images were collected. After
to these OCT images after they were resized to 224×224. removing images with severe artefacts causing misalign-
Specifically, includes ‘RandomResizedCrop’, ‘Random- ment and blurring of sections or significant image
HorizontalFlip’, ‘RandomVerticalFlip’, ‘GaussianBlur’ resolution reductions, 3319 OCT images were used in
and ‘Normalise’. The model architecture we proposed this study, which were 1254 images of early DME, 991
using vision transformer will use these images and Adam images of advanced DME, 672 images of severe DME and
optimizer to train. The loss function was the one we 402 images of atrophic maculopathy. Among these, 70%
proposed above, with the temperature parameter set to of images were randomly selected as training dataset to
1.0. The learning rate was set to 0.001, weight decay to establish our model, 30% of images were selected as vali-
0.0005, batch size to 128 and the K value of the classifier dation dataset. The number of images for each stages in
was set to 8. training and validation set is shown in table 1. The repre-
Stochastic gradient descent (SGD) was a very common sentative enhanced OCT images are shown in online
optimiser for model training in machine learning, which supplemental file 1.
meant that it updated the model parameters by using a On the validation dataset, we constructed, we achieved
random subset of the data at each iteration. When it came an accuracy of 82.00%, an F1 score of 83.11%, an AUC
to test time, we used SGD optimiser with 0.001 learning of 0.96. The confusion matrix of the classification results
rate, perform 32 data augmentation operations on each is shown in figure 2. And accuracy,precision, sensitivity
sample, specifically including: ‘random rotation’, ‘histo- and specificity for every stage are shown in table 2. AUC
gram equalisation’, ‘invert pixels’, ‘colour quantisation’, and F1 score for every stage are shown in online supple-
‘shear image along x-­ axis or y-­
axis’, ‘translate image mental file 1. The AUC for the detection of early DME
along x-­axis or y-­axis’. was 0.96, with an accuracy of 90.87%, a precision of

Table 1 The distribution of images

DM stages No of total images No of training images No of validation images

Early DME 1254 884 370

Advanced DME 991 695 296
Severe DME 402 290 112
Atrophic maculopathy 672 464 208
DM, diabetic maculopathy; DME, diabetic macular oedema

4 Cai L, et al. BMJ Open Ophth 2023;8:e001423. doi:10.1136/bmjophth-2023-001423

 Open access

Figure 2 Confusion matrixes of the model. DM, diabetic maculopathy; DME, diabetic macular oedema.

88.46%, a sensitivity of 87.03%, a specificity of 93.02% method we proposed can effectively classify patients with
and an F1 score of 87.74%. The AUC for the detection DM into different stages.
of advanced DME was 0.95, with an accuracy of 89.96%, a
precision of 80.31%, a sensitivity of 88.18%, a specificity
DISCUSSION
of 90.72% and an F1 score of 84.06%. The AUC for the
In this study, we developed a DL model based on vision
detection of severe DME was 0.87, with an accuracy of
transformer for DM grading based on OCT-­ related
94.42%, a precision of 89.42%, a sensitivity of 63.39%,
morphological features. We achieved an accuracy of
a specificity of 98.40% and an F1 score of 88.18%. The
82.00%, an F1 score of 83.11% and an AUC of 0.96. Our
AUC for the detection of atrophic maculopathy was 0.98,
research showed that the accuracy of our model in this
with an accuracy of 95.13%, a precision of 87.74%, a
novel grading system was promising, which can help with
sensitivity of 89.42%, a specificity of 96.66% and an F1
patients in a preliminary screening to identify groups
score of 88.57%. The ROC curve of the classification
with serious conditions. As this classification may be able
results is shown in figure 3. The results showed that the
to predict the treatment outcome and visual prognosis
of DM better in the future, our model can help ophthal-
mologists to develop personalised treatment plans for
Table 2 Accuracy, precision, sensitivity and specificity of
patients with DM.
our model in validation dataset
DM at four different stages reflects the severity of
Accuracy Precision Sensitivity Specificity
DM stages (%) (%) (%) (%)
the disease. Early DME usually corresponds to a short
duration of hyperglycaemic state.16 So most of the time
Early DME 90.87 88.46 87.03 93.02 patients can maintain a good vision if they can take good
Advanced 89.96 80.31 88.18 90.72 control of their blood glucose. EZ/ELM state is different
DME between advanced and severe DME. In the former one,
Severe DME 94.42 89.42 63.39 98.40 EZ/ELM may be damaged but still visible, and the layers
Atrophic 95.13 87.74 89.42 96.66 of the inner retina are usually recognisable. In the latter
maculopathy one, the internal retinal layers and/or EZ/ELM are
mostly destroyed and undetectable. These two groups
DM, diabetic maculopathy; DME, diabetic macular oedema
of patients may have distinct differences in treatment

Cai L, et al. BMJ Open Ophth 2023;8:e001423. doi:10.1136/bmjophth-2023-001423 5

 Open access

Figure 3 ROC curve analysis results of our model. DME, diabetic macular oedema; ROC, receiver operating characteristic.

response and visual prognosis and should be distin- currently required expertise and are time-­ consuming.
guished.16 Patients with advanced DME should be treated Thus, it is particularly beneficial and promising to
promptly. Anti-­VEGF treatment may prevent progression develop an intelligent system for the DM grading based
of the disease into next stage with ELM and/or EZ being on this new system to assist the clinical decision-­making
recovered and CST decreasing to normal values. While processes in patients.
once the disease progresses into severe DME, it may be With the continuous development of DL technology,
difficult in resolution of oedema despite positive treat- now we all have more opportunities to achieve automatic
ment, and finally may inevitably develop into atrophy diagnosis and classification of diseases. Numerous studies
stage. Macular atrophy is characterised by complete EZ/ have demonstrated the expert performance of DL tech-
ELM destruction and DRIL, usually as a result of long-­ nology in detecting DME. For instance, Alqudah19
term macular oedema, and has a poor visual outcome.16 proposed a multiclassification model based on SD-­OCT
Hence, this novel grading system can assist the ophthal- for four types of retinal diseases (age-­related macular
mologists in predicting the prognosis of patients with degeneration, choroidal neovascularisation, DME and
DM in their clinical work, and personalised therapeutic drusen) as well as normal cases. The proposed CNN
strategies could be made according to the OCT grading. architecture with softmax classifier correctly identified
Especially in the former two stages, taking good control 99.17% of DME cases overall. Zhang et al20 proposed a
of blood glucose and timely treatment are significant to multiscale DL model, which were divided into two parts:
promote recovery and prevent them from progressing self-­enhancement model and disease detection model,
into the more severe stages. For these patients, early with achieving 94.5% accuracy in identifying DME.
screening and long-­term follow-­up can maintain a better Meanwhile, they proved that this model provided a better
vision outcome. However, detection and grading of DM ability to recognise low-­quality medical images. Wu et al21

6 Cai L, et al. BMJ Open Ophth 2023;8:e001423. doi:10.1136/bmjophth-2023-001423

 Open access

trained a DL model using Visual Geometry Group 16 always had poor vision after progressing into this stage,
(VGG16) network as the backbone to detect three OCT resulting in worsen image quality. However, our model
morphologies of DME, including DRT, CME and SRD. can help with patients in a preliminary screening to iden-
The accuracy was 93.0%, 95.1% and 98.8%, respectively. tify groups with serious conditions. These patients need a
All the above studies indicated that DL model had good further test for an accurate diagnosis, and a timely treat-
feasibility and application prospects in diagnosing DME. ment to prevent further deterioration in time. Overall,
However, there is still a lack of DL model for automatic the result achieved by our DL model was promising and
detection for this OCT-­based grading of DM. Meanwhile, encouraging.
it should be noted that most of the above studies were Although our model showed great potential, there are
based on CNN. The advantage of CNN is that it can still several limitations in the study. First, OCT images
extract image features well, which has been verified by only obtained from the Optovue RTVue imaging system
a large number of scholars. However, there is still little in our study. The model needs to be further validated
research on Visual Transformer, which has better classifi- by images from different OCT equipment. Second, We
cation capabilities than CNN to solve image classification only perform the classification training in this model.
problems.22 In the future, studies can train models to predict treat-
In current study, we trained a DL model using Vision ment outcomes based on this new grading system. Finally,
Transformer as the backbone to detect this novel grading type of data we used only included images from one eye
in OCT images. Vision Transformer proposed in 2020 is a centre. More OCT images from other multicentre trials
new image classification model, which is considered to be in the future can be used to improve our model.
the best image classification model at present, showing In conclusion, our DL model based on Vision Trans-
better performance than traditional CNN model.22 former demonstrated a relatively high accuracy in the
Vision Transformer is not dependent on any CNN and detection of the different OCT-­based stages of DM. This
is completely based on transformer structure designed DM grading model can reduce the burden on clinical
with different feature extraction methods from CNN.23 ophthalmologists and provide a reference in making
Research has proved the recognition ability of Vision personalised therapeutic strategies. These results empha-
Transformer for OCT images is stronger than CNN models sise the potential of AI in reducing the necessary time of
and traditional machine learning algorithms.23 In the clinical diagnosis, assisting clinical decision-­making and
accuracy comparison of the same test set between Vision
guaranteeing the cure rate in the future.
Transformer and four CNN models: VGG16, Resnet50,
Densenet121 and EfficentNet, Vision Transformer has Author affiliations
the highest classification accuracy of 99.69%. Meanwhile, 1
Department of Ophthalmology, Renmin Hospital of Wuhan University, Wuhan,
both VGG16 and Vision Transformer are faster than Hubei, China
2
other CNN models in the recognition speed of a single Wuhan University School of Computer Science, Wuhan, Hubei, China
3
image.23 Although our result was slightly less impressive State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun
Yat-­sen University, Guangdong Provincial Key Laboratory of Ophthalmology and
than the previous studies using other DL architectures Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases,
to detect DME based on OCT images and the detection Guangzhou, Guangdong, China
of the OCT patterns. It can be more complicated and
challenging than distinguishing DME from other retinal Contributors LC, JJ and CL collected data. LC and CW analysed and interpreted
diseases or simply detecting the overall patterns of DME, the data, and writing the original draft. HZ provided resources. YS and CC provided
with less obvious differences in characteristics and subtle resources, supervised study and reviewed the draft. CC acted as the guarantor of
the work. All authors reviewed the manuscript.
lesions between different OCT grading.
Funding The authors have not declared a specific grant for this research from any
To our knowledge, this is the first article to detect the funding agency in the public, commercial or not-­for-­profit sectors.
severity of DM according to the novel classification stan-
Competing interests None declared.
dard based on OCT images by DL and the first article
to use Vision Transformer to detect DM. As mentioned Patient consent for publication Not applicable.
above, this classification may be able to predict the treat- Ethics approval This study was performed in compliance with the Declaration of
Helsinki and was approved by the Clinical Research Ethics Committee of Renmin
ment outcome and visual prognosis of DM better in Hospital of Wuhan University (ethics number WDRY2021-­K034). The need for written
the future and help ophthalmologists develop precise informed consent was waived by the Clinical Research Ethics Committee of Renmin
treatment plans for patients. And as the Vision Trans- Hospital of Wuhan University due to the retrospective nature of the study.
former can better mine global information through its Provenance and peer review Not commissioned; externally peer reviewed.
self-­attention mechanism and has less bias towards local Data availability statement Data are available on reasonable request. The
texture features, it is more robust to noise compared datasets generated during the current study are not publicly available for data
with CNN commonly used in past studies. So our protection reasons but are available from the corresponding author on reasonable
model combined with these advantages is very prom- request.
ising for detecting OCT images of DM or other retinal Supplemental material This content has been supplied by the author(s). It has
not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been
diseases. Our model had a slightly lower performance in peer-­reviewed. Any opinions or recommendations discussed are solely those
predicting severe DME. Possibly because there were fewer of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and
images compared with other stages, and patients almost responsibility arising from any reliance placed on the content. Where the content

Cai L, et al. BMJ Open Ophth 2023;8:e001423. doi:10.1136/bmjophth-2023-001423 7

 Open access

includes any translated material, BMJ does not warrant the accuracy and reliability 10 Sun JK, Lin MM, Lammer J, et al. Disorganization of the retinal inner
of the translations (including but not limited to local regulations, clinical guidelines, layers as a predictor of visual acuity in eyes with center-­involved
terminology, drug names and drug dosages), and is not responsible for any error diabetic macular edema. JAMA Ophthalmol 2014;132:1309–16.
and/or omissions arising from translation and adaptation or otherwise. 11 Deák GG, Bolz M, Ritter M, et al. A systematic correlation between
morphology and functional alterations in diabetic macular edema.
Open access This is an open access article distributed in accordance with the Invest Ophthalmol Vis Sci 2010;51:6710–4.
Creative Commons Attribution Non Commercial (CC BY-­NC 4.0) license, which 12 Karst SG, Lammer J, Mitsch C, et al. Detailed analysis of retinal
permits others to distribute, remix, adapt, build upon this work non-­commercially, morphology in patients with diabetic macular edema (DME)
and license their derivative works on different terms, provided the original work is randomized to ranibizumab or triamcinolone treatment. Graefes Arch
Clin Exp Ophthalmol 2018;256:49–58.
properly cited, appropriate credit is given, any changes made indicated, and the 13 Maheshwary AS, Oster SF, Yuson RMS, et al. The association
use is non-­commercial. See: http://creativecommons.org/licenses/by-nc/4.0/. between percent disruption of the photoreceptor inner segment-­
outer segment junction and visual acuity in diabetic macular edema.
ORCID iDs Am J Ophthalmol 2010;150:63–67.
Liwei Cai http://orcid.org/0000-0003-1258-232X 14 Yanyali A, Bozkurt KT, Macin A, et al. Quantitative assessment of
Changzheng Chen http://orcid.org/0000-0002-7281-552X photoreceptor layer in eyes with resolved edema after pars plana
vitrectomy with internal limiting membrane removal for diabetic
macular edema. Ophthalmologica 2011;226:57–63.
15 Moein H-­R, Novais EA, Rebhun CB, et al. Optical coherence
tomography angiography to detect macular capillary ischemia in
REFERENCES patients with inner retinal changes after resolved diabetic macular
1 Wilkinson CP, Ferris FL, Klein RE, et al. Proposed international edema. Retina 2018;38:2277–84.
clinical diabetic retinopathy and diabetic macular edema disease 16 Panozzo G, Cicinelli MV, Augustin AJ, et al. An optical coherence
severity scales. Ophthalmology 2003;110:1677–82. tomography-­based grading of diabetic maculopathy proposed by an
2 Ciulla TA, Amador AG, Zinman B. Diabetic retinopathy and diabetic international expert panel: the European school for advanced studies
macular edema: pathophysiology, screening, and novel therapies. in ophthalmology classification. Eur J Ophthalmol 2020;30:8–18.
Diabetes Care 2003;26:2653–64. 17 Lu W, Tong Y, Yu Y, et al. Deep learning-­based automated
3 Chiu SJ, Allingham MJ, Mettu PS, et al. Kernel regression based classification of multi-­categorical abnormalities from optical
segmentation of optical coherence tomography images with diabetic coherence tomography images. Transl Vis Sci Technol
macular edema. Biomed Opt Express 2015;6:1172–94. 2018;7:41:41.:.
4 Teo ZL, Tham Y-­C, Yu M, et al. Global prevalence of diabetic 18 Tsuji T, Hirose Y, Fujimori K, et al. Classification of optical coherence
retinopathy and projection of burden through 2045: systematic tomography images using a capsule network. BMC Ophthalmol
review and meta-­analysis. Ophthalmology 2021;128:1580–91. 2020;20:114.
5 Hariprasad SM, Mieler WF, Grassi M, et al. Vision-­related quality 19 Alqudah AM. AOCT-­NET: a convolutional network automated
of life in patients with diabetic macular oedema. Br J Ophthalmol classification of multiclass retinal diseases using spectral-­domain
2008;92:89–92. optical coherence tomography images. Med Biol Eng Comput
6 Scott IU, Danis RP, Bressler SB, et al. Effect of focal/grid 2020;58:41–53.
photocoagulation on visual acuity and retinal thickening in eyes with 20 Zhang Q, Liu Z, Li J, et al. Identifying diabetic macular edema
non-­center-­involved diabetic macular edema. Retina 2009;29:613–7. and other retinal diseases by optical coherence tomography
7 Otani T, Kishi S, Maruyama Y. Patterns of diabetic macular image and multiscale deep learning. Diabetes Metab Syndr Obes
edema with optical coherence tomography. Am J Ophthalmol 2020;13:4787–800.
1999;127:688–93. 21 Wu Q, Zhang B, Hu Y, et al. Detection of morphologic patterns of
8 Reznicek L, Cserhati S, Seidensticker F, et al. Functional and diabetic macular edema using a deep learning approach based on
morphological changes in diabetic macular edema over the optical coherence tomography images. Retina 2021;41:1110–7.
course of anti-­vascular endothelial growth factor treatment. Acta 22 Bazi Y, Bashmal L, Rahhal MMA, et al. Vision transformers for
Ophthalmol 2013;91:e529–36. remote sensing image classification. Remote Sensing 2021;13:516.
9 De S, Saxena S, Kaur A, et al. Sequential restoration of external 23 Jiang Z, Wang L, Wu Q, et al. Computer-­aided diagnosis of
limiting membrane and ellipsoid zone after intravitreal anti-­VEGF retinopathy based on vision transformer. J Innov Opt Health Sci
therapy in diabetic macular oedema. Eye (Lond) 2021;35:1490–5. 2022;15.

8 Cai L, et al. BMJ Open Ophth 2023;8:e001423. doi:10.1136/bmjophth-2023-001423