Assessing the Feasibility of Handheld Cameras to Increase Access to Teleretinal Diabetic Retinopathy Screenings in Safety Net Clinics in Los Angeles

Abstract

Background: Handheld cameras may increase access to teleretinal diabetic retinopathy screenings (TDRS), as they are more economical and nimbler than traditional desktop cameras. However, their use in safety net clinical settings is less understood. Methods: The Los Angeles County Department of Public Health collaborated with the Los Angeles County Department of Health Services (DHS) to pilot the use of handheld cameras in DHS’s TDRS program. This study assessed screening metrics and interviewed TDRS coordinators and photographers about their experiences using these cameras. Results: Handheld cameras were harder to operate and performed less optimally on key screening metrics (e.g., image quality) compared to desktop cameras. However, this challenge appeared to be linked to the selected model rather than to all handheld cameras. The adoption of handheld cameras in DHS was aided by their integration into an existing agency infrastructure that already supported the use of desktop cameras; these operational supports included an established workflow, a data platform, and experienced photographers. Conclusions: The use of handheld cameras for TDRS is a promising practice for assuring equitable access to screenings. Handheld cameras are portable, smaller in size, and lower in cost than desktop cameras. Future projects should invest in and assess the performance of higher-quality models of these cameras, especially their use in safety net clinical settings.

1. Introduction

Diabetic retinopathy (DR) is the leading cause of blindness in working-age American adults [1,2]. While early detection and treatment can prevent over 90% of vision loss among patients with diabetes [3,4], less than half of diabetic patients in the United States (U.S.) receive the recommended annual DR screening, with even lower adherence among safety net patients [5,6]. It is well known that the burden of vision impairment from diabetes disproportionately affects low-income populations [7]. In the U.S., safety net health care organizations are vital to providing care to low-income and vulnerable populations, which includes patients who are uninsured, under-insured, or lack the financial means to get eye care. A greater proportion of safety net patients with major chronic conditions such as diabetes, hypertension, and elevated cholesterol experience unmet health needs [8], which results in an increased risk of complications like DR. Barriers to DR screening compliance in this population have been described elsewhere [9]. In addition to a lack of health literacy and other factors associated with the social determinants of health, system attributes such as a shortage of eye care specialists, long wait times for an appointment [10], and a lack of proximity to a screening site [6] also contribute to this noncompliance.

These barriers can be addressed by teleretinal DR screenings (TDRS) during primary care visits, wherein a photographer (typically a medical assistant or licensed vocational nurse) captures retinal images on-site and digitally transmits them to a certified image reader for remote evaluation [1,11]. The benefits of teleretinal DR screening both in the U.S. and abroad have been extensively described elsewhere in the literature [2,10,11,12,13]. It has been shown that by capturing images at the primary care site rather than having patients travel to a separate site to be screened by an eye care provider, TDRS can shorten wait times and eliminate financial and geographic barriers to screening [9,10,14].

Despite the convenience of TDRS, the high-resolution desktop cameras that are commonly used in the clinical setting are bulky and expensive, making them infeasible for clinics with limited space or financial resources [15]. Additionally, desktop cameras are stationary, so patients with mobility challenges often find it difficult to adjust their positioning for optimal alignment with the camera.

Handheld cameras could help ameliorate these issues and increase equitable access to screenings, as they are more economical and nimbler than desktop cameras [12,16]. However, their use in safety net clinical settings has not been well documented: little is known about the feasibility or potential use in safety net sites.

To assess the feasibility of using handheld cameras at primary care clinics within a major safety net system, we conducted a pilot study of handheld cameras, integrating their use into an established teleretinal screening program in Los Angeles County that currently utilizes desktop cameras. We comparatively assessed screening metrics (e.g., image quality) from both handheld and desktop cameras and interviewed TDRS program coordinators and photographers to identify key facilitators and barriers to handheld camera use.

2. Methods

2.1. Setting

The Los Angeles County Department of Public Health (DPH) collaborated with the Los Angeles County Department of Health Services (DHS) to increase access to DHS’s TDRS program by piloting the use of handheld cameras as part of the Centers for Disease Control and Prevention-supported Solutions for Healthier Communities initiative in Los Angeles County. DHS is the second largest safety net system in the U.S., serving approximately 750,000 unique patients annually across 26 health centers and four hospitals. The DHS TDRS program has been described in detail elsewhere [10]; briefly, it has operated for over 10 years and screens approximately 2000 patients each month, employing 56 certified medical assistant or licensed vocational nurse photographers using 17 Topcon NW400 and NW8 (Topcon Corporation, Tokyo, Japan) and Canon CR-2 AF Digital (Canon U.S.A., Inc., Huntington, New York, NY, USA) desktop cameras across 16 primary care clinics. The program uses a single-drop dilation protocol (Mydriacil 0.5–1%) to minimize ungradable images.

DHS leadership, including the TDRS program’s executive director, selected the DHS sites to participate in this project. They chose both sites that were and were not already participating in the TDRS program using desktop cameras. Other considerations included whether the site had patient populations that may especially benefit from handheld cameras (e.g., patients with mobility issues), whether there was site buy-in and capacity to participate, and whether the sites were geographically distributed across Los Angeles County. Among sites already participating in the TDRS program, those with high screening volumes were prioritized.

Four brick-and-mortar clinics and two mobile van clinics were initially selected. However, the mobile van clinics, which were intended to serve patients experiencing homelessness, encountered administrative delays during the project’s timeframe. Thus, only the four brick-and-mortar clinics participated. Three of these four sites were already participating in the TDRS program.

2.2. Cameras

All four sites received the same handheld camera model: the Horus 200 Handheld Fundus Camera (Medimaging Integrated Solution, Inc. [MiiS], Hsinchu, Taiwan). The model was selected because it was the only camera that met grant funding requirements: available in the United States at a per-camera cost not to exceed approximately $5000. Photographers at the three sites who were proficient with the use of desktop cameras for TDRS were asked to conduct desktop screenings per usual care protocols and additionally conduct handheld camera screenings if time permitted and if the patient agreed to a second set of images being taken during the same appointment. The desktop camera that photographers used was either the Topcon TRC-NW400 Non-Mydriatic Retinal Camera or the Canon CR-2 AF Non-Mydriatic Retinal Camera.

2.3. Training Process

Through self-training and trial and error, the DHS TDRS program coordinator and assistant coordinator familiarized themselves with the handheld camera model. After reading the user manual, they developed a training protocol for use in the field. Because the original purchase agreement did not require the camera vendor to provide training or technical support, this training protocol relied heavily on the experiences of the DHS coordinators and on the workflow of the existing TDRS program. After developing the protocol, the coordinators trained two photographers on the operation of the handheld camera model; these two photographers were from the four brick-and-mortar clinic sites that subsequently hosted the pilot study. One photographer was a certified medical assistant, while the other was a licensed vocational nurse. The TDRS program coordinator was a registered nurse, and the assistant program coordinator was a certified medical assistant. At the time of the training, both photographers were already proficient with desktop cameras and were conducting screenings for the TDRS program. The training largely focused on learning the handheld camera features, observing how the two coordinators were using the handheld device in the field, and performing test screenings.

2.4. Inclusion/Exclusion Criteria

Patients were eligible for the TDRS program based on the following criteria: diagnosed with diabetes, able to sit up and remain still for retinal photography, no eye exams or not actively followed by an ophthalmologist/optometrist within the last year, and no acute vision loss or major eye complaints. Patients were included in this analysis if they were at least 18 years old, had agreed to be screened with handheld cameras, and ultimately were screened with these cameras. Patients screened with only desktop cameras were excluded from the analysis.

2.5. Data Collection

Screenings occurred between January and September 2023. Screenings were conducted by either one of the two photographers or one of the TDRS program coordinators. Evaluators from DPH, with support from TDRS staff, extracted patient-level data from EyePACS (EyePACS, Inc., Santa Cruz, CA, USA), an online platform that the DHS TDRS program uses to capture, store, and transmit retinal images and associated patient information for readers. Information extracted from this platform included the following: patient demographics (e.g., race/ethnicity, date of birth), health characteristics based on the photographer’s review of the patient’s chart (e.g., years with diabetes, hemoglobin A1c [HbA1c]), patient’s last eye exam (based on photographer’s chart review and confirmation by patient), and the reader’s evaluation of the images (e.g., diagnosis, image quality). To assess interrater reliability, the project’s three readers independently evaluated the handheld versus desktop image sets from the same three randomly selected patients.

Key informant interviews were conducted with the coordinators and photographers in September 2023. DPH evaluators initially interviewed the two program coordinators jointly in the same session. The TDRS assistant program coordinator then interviewed the two photographers separately at each of their regular clinic locations. Based on a guide developed internally, interview questions explored facilitators and barriers to using handheld cameras in the field, coordinating and training to become more familiar with the handheld camera model, and other aspects related to the general usage of these devices.

The program coordinator met monthly with DPH evaluators to discuss implementation progress, identify and troubleshoot challenges, and monitor data collection.

2.6. Data Analysis

After performing data cleaning to ensure that the data were complete and accurate, descriptive statistics were generated to describe patient characteristics and screening results. Reader ratings for the three randomly selected handheld and desktop image sets were assessed for interrater reliability using the Kappa statistic. Thematic analysis of the three interviews were carried out using a deductive coding process. Protocols and materials for this feasibility assessment and pilot study of handheld cameras were reviewed and approved by the Institutional Review Board for the Los Angeles County Departments of Public Health and Health Services.

3. Results

3.1. Patient Participants of the Feasibility Assessment and Pilot Study of Handheld Cameras

In total, 69 patients from the four brick-and-mortar clinics participated in the feasibility assessment and pilot study of handheld cameras (see

Table 1. Patient participants of the feasibility assessment and pilot study of handheld cameras.

Characteristic	Total (n = 69)
Age
Mean (standard deviation)	57.0 (11.5)
Range (minimum, maximum)	29, 76
Race/ethnicity
Latino	58 (84.1%)
Asian/Pacific Islander	5 (7.3%)
Black	1 (1.5%)
Other	5 (7.3%)
Gender
Male	29 (42.0%)
Female	40 (58.0%)
Hemoglobin A1c (HbA1c)
Mean (standard deviation)	7.6 (1.6)
Range (minimum, maximum)	5.1, 12.7
Diabetes diagnosis
1 year or less	14 (20.3%)
2–5 years ago	22 (31.9%)
6–10 years ago	16 (23.2%)
Over 10 years ago	17 (24.6%)
Diabetic control 1
Excellent	30 (43.5%)
Good	7 (10.1%)
Moderate	8 (11.6%)
Fair	7 (10.1%)
Poor	17 (24.6%)
Hypertension status
Controlled hypertension	49 (71.0%)
Uncontrolled hypertension	4 (5.8%)
No hypertension	15 (21.7%)
Unknown	1 (1.5%)
Last eye exam 2
Never	15 (21.7%)
Within the past 2 years	38 (55.1%)
2 to 5 years ago	15 (21.7%)
Over 5 years ago	1 (1.5%)

¹ The photographer reviewed the patient’s chart and assigned a rating based on a standardized scale used at all Los Angeles County Department of Health Services (DHS) sites. ² The photographer reviewed the patient’s chart for their eye exam history at DHS and then asked the patient to confirm that no eye exams outside of DHS had been completed within the last 12 months.

). Their mean age was 57 years (±11.5). Most (84.1%) were Latino, and over half (57.0%) were female. The mean HbA1c was 7.6% (±1.6). Half (47.8%) were diagnosed with diabetes six or more years ago. Nearly half (43.5%) had excellent control of their diabetes. Nearly three-fourths (71.0%) also had controlled hypertension. A fifth (21.7%) had never had an eye exam previously.

3.2. Image Quality and Diagnosis

All 69 patients in the sample were screened with handheld cameras. There were 69 handheld image sets included in the overall analysis. For comparison, there were also 57 desktop image sets taken of the same patients who were imaged at the three participating TDRS sites that were already a part of the DHS program prior to the pilot study.

Overall, handheld cameras produced lower image quality ratings than desktop cameras (Figure 1). Only 1.4% of handheld image sets received a rating higher than adequate, compared to 42.1% of desktop image sets. Nearly two-thirds (63.8%) of handheld image sets were rated insufficient for either any or full interpretation, compared to 5.3% of desktop image sets. The distinction between being insufficient for any versus full interpretation is worth noting. If an image quality is insufficient to make any diagnostic conclusions (i.e., there were no gradable elements present), that was categorized as “insufficient for any interpretation”. However, if, due to the image quality, some but not all elements of disease were gradable, this was considered “insufficient for full interpretation”, as some diagnostic conclusions were possible, though a complete assessment could not be made.

Sample retinal images taken by the TDRS program coordinators of their own eyes are shown in Figure 2. The difference in image quality is noticeable: the handheld image sets are blurrier, darker, and less detailed than the desktop image sets. However, the second set of images from handheld cameras showed improvement, suggesting a need to learn how to adjust and position these portable devices so they can capture gradable images that are comparable to the image quality of desktop cameras.

Image readers were less likely to arrive at a diagnosis based on handheld camera image sets compared to desktop camera image sets (Figure 3). About a quarter (24.6%) of handheld image sets were given no diagnosis or were deemed ungradable, compared to 1.8% of desktop image sets. The diagnoses of no apparent DR, mild non-proliferative DR (NPDR), severe NPDR, and proliferative DR were all lower in percentage among the handheld image sets than for the desktop image sets. However, the diagnosis of moderate NPDR was slightly higher among handheld image sets than among desktop image sets.

To determine if screening metrics differed by camera type, image sets from handheld and desktop cameras were compared against one another. Most handheld-desktop image pairs (78.9%) had different image quality ratings, though only a third of handheld-desktop image pairs (35.1%) had a different diagnosis. Among these, there was no discernable pattern in terms of which camera type actually resulted in an interpretation of a more severe diagnosis—i.e., handheld image sets yielded a diagnosis that was (a) more severe than for the desktop image sets in five of the image pairs; (b) less severe in five other image pairs; and (c) neither more or less severe in the remaining ten image pairs (e.g., “ungradable”, “no diagnosis”, or “missing”).

The Kappa statistic revealed that readers typically had a lower level of agreement regarding diagnoses or image quality for the handheld image sets versus the desktop image sets (

Table 2. Interrater reliability of readers’ evaluation of handheld and desktop image sets.

	Kappa	Level of Agreement
Handheld
Image quality	−0.2857	Less than slight
Diagnosis	−0.1250	Less than slight
Desktop
Image quality	0.1346	Slight
Diagnosis	0.3077	Fair

).

3.3. Facilitators

Results from the feasibility assessment and pilot study of handheld cameras identified several facilitators that may help DHS expand its TDRS program. For instance, the pilot study was integrated into an existing TDRS program infrastructure that already has many operational supports in place, including an established workflow and a data platform. Additionally, the program coordinators and photographers who participated in the pilot study were experienced camera users, having already been trained on desktop cameras. This field experience helped to ease the transition from desktop to handheld cameras—the latter, as it turned out, took greater skill to operate. Staff familiarity with the general screening process and workflow likely reduced the training time needed to learn the policies, protocols, and approaches to capturing/handling images taken by the handheld cameras.

The pilot study also followed an existing DHS TDRS protocol that dilates patients’ eyes before screening. This expected DHS practice, developed to help photographers capture higher quality images of the retina, likely improved the visualization of the vascular structure and vessel features and made it easier for the readers to issue a diagnosis. Without dilating the eyes, the image quality from handheld cameras would have been lower.

3.4. Barriers

Results from the feasibility assessment and pilot study of handheld cameras also revealed several barriers that the TDRS program had to overcome, generating lessons learned that could and should be considered as DHS looks to expand its screening program. First, the handheld camera model lacked a detailed instruction manual or technical support from the vendor, leading to some difficulties and delays in gaining familiarity with these cameras. The manual provided by the manufacturer contained limited information and did not include guidelines or techniques for capturing high-quality images in the field.

Second, handheld cameras took longer to set up and use than expected. For example, photographers had to repeatedly take the same image because the camera model frequently did not operate as anticipated—e.g., the screen sometimes remained dark despite the camera being turned on.

Third, while the camera model has an automatic function (i.e., once focused on a fixed target, it can capture the image clearly), photographers had a hard time maneuvering the device accurately or quickly enough for the function to work. Thus, most images from the pilot study were captured manually; this, unfortunately, came at a cost of lower quality images, as even the slightest movement of the handheld model, such as manually pressing the button to capture the image, led to loss of focus and a suboptimal image. Even when the TDRS program coordinators tried capturing images of each other as model patients (i.e., staying still, not blinking, opening their eyes as wide as possible, focusing their gaze in the exact fixation needed for each image), the image quality remained low. Paradoxically, these suboptimal images undermine the intended convenience of the handheld cameras—i.e., because of the poor image quality of the handheld image sets, many patients at the site where only the handhelds were used were later asked to return for a follow-up screening using desktop cameras.

Fourth, handheld cameras had a short battery life, resulting in photographers needing to charge these devices frequently, inadvertently disrupting the clinic workflow.

Fifth, significant staff time was required to coordinate and implement the pilot study, particularly given the challenges with operating handheld cameras. This extra time added to the staff’s existing responsibilities. However, this appears to be more of an issue with program start-up than with program maintenance or sustainability, as the amount of staff time required to oversee and operate the cameras is expected to diminish as the program matures and evolves.

Finally, the intended flexibility of handheld cameras did not come to fruition during the pilot study. Their unpredictable performance ultimately required patients to sit still for extended periods of time to capture readable images. This inconvenience did not benefit those with mobility issues, as they already struggle to position correctly for desktop cameras.

4. Discussion

Handheld cameras performed less optimally on key screening metrics compared to desktop cameras. Among patients who were screened with both camera types, there was substantial divergence in image quality and, to a lesser extent, diagnosis.

The limited performance of handheld cameras appeared to be associated with the model that was selected, rather than with all handheld camera models. The selected model lacked a detailed manual or vendor support, required substantial setup time, operated unpredictably, and had suboptimal image quality. Indeed, recent studies using other handheld camera models saw better results [17]: in one study, image quality was insufficient in only approximately 15% of cases, compared to 64% in the present feasibility assessment and pilot study [18]. However, this 15% is still significantly higher than what is observed for desktop cameras in the DHS TDRS program.

The feasibility of using handheld cameras largely depends on selecting the right model. The results of our pilot study highlight several criteria that should be considered when selecting a handheld camera model. For example, there should be adequate funding to purchase high-quality handheld cameras, keeping in mind that the technology is steadily improving and the cost of these various models should decrease over time. Another consideration is the staff who are being asked to operate these cameras. A handheld camera program can achieve greater success if the intended users of these cameras (e.g., program coordinators and/or photographers) are given the opportunity to weigh in on camera maneuverability and which model to select, as they are the most familiar with the specific considerations and logistics of imaging their defined patient population(s). Lastly, handheld cameras should be purchased from a vendor (or manufacturer) that can provide guidance on optimal equipment use—e.g., providing a detailed manual, offering on-site or virtual training, and having technical support that is readily available.

4.1. Lessons Learned

The lessons learned from this feasibility assessment and pilot study of handheld cameras underscore several administrative and logistical factors that healthcare organizations interested in using these devices should consider.

First, organizations should determine whether there is an existing TDRS program already in place, complete with infrastructure and staff who are experienced in teleretinal screening, or if a new TDRS program needs to be developed. Integrating a handheld camera workflow into an existing TDRS program may be more straightforward than creating a new program from scratch. However, for organizations that currently do not use a desktop camera due to either space or budget constraints, developing a TDRS program from the ground up may still be worthwhile. Connecting with other organizations that already have a TDRS program may help provide valuable guidance on successful implementation.

Second, organizations should ensure that there are enough program coordinators with sufficient dedicated time to oversee implementation, lead trainings, and provide ongoing support for handheld camera use, given the different skillset that is required to use these devices compared to desktop cameras. In particular, when the handheld camera program spans multiple clinic sites, it is especially important to have an appropriate number of coordinators to be able to quickly respond to on-site troubleshooting requests.

Third, organizations should give photographers adequate time to train and conduct screenings using handheld cameras. There is a learning curve to using handheld cameras. In our pilot study, as photographers gained increased familiarity and experience using these cameras, they were able to overcome some of the early challenges encountered.

Fourth, organizations should confirm there is buy-in among all those involved—including clinic leadership, program coordinators, and photographers—to run a handheld camera program.

Finally, organizations should establish an implementation and quality improvement plan to roll out the handheld cameras, assessing for progress and impact of these cameras on teleretinal screenings in the field. Continuous quality monitoring of camera performance could help identify gaps in program implementation and operation, helping to facilitate corrections or mid-course adjustments when needed.

4.2. Limitations

The sample size of our feasibility assessment and pilot study was relatively small and consisted of only safety net patients in Los Angeles County. Thus, the results may not be generalizable outside of DHS or the county. Also, multiple delays during the pilot study —related to camera acquisition, training, and site readiness—led to fewer screenings and participating sites than originally anticipated.

4.3. Conclusions

Handheld cameras may increase equitable access to DR screenings due to their portability, small size, and lower cost compared to desktop cameras. In particular, they may benefit hard-to-reach communities where access to TDRS is harder to come by. The key challenges encountered in our pilot study, such as suboptimal image quality, appeared to be attributable to the specific camera model we used rather than to all handheld cameras available in the market. Future assessments and program refinements should focus on health system strategies that can mitigate many of these implementation challenges and on demonstrating the value of investing in both higher-quality handheld cameras and an operational infrastructure that can support the population health goals of the intended TDRS program.