Developing a Color Identification Assistance Application for Individuals with Color Blindness

Color blindness is as color vision deficiency which affects an individual's ability to see or differentiate between certain colors. This condition affects approximately 8% of males and 0.5% of females, with males being more commonly affected due to their genetics [4].

The most prevalent forms of color blindness are red-green color blindness, which includes protanomaly, protanopia, deuteranomaly, and deuteranopia, and blue-yellow color blindness, which includes tritanomaly and tritanopia. The rarest form is complete color blindness, known as achromatopsia, where individuals see the world in shades of gray [9]. Despite the challenges it presents, individuals with color blindness develop strategies to compensate this limitation in their daily activities. However, the condition can restrict access to certain occupations and activities that rely heavily on color discrimination [19].

Color blindness can be detected through various diagnostic tests [1], the most common of which are the Ishihara color test and the Farnsworth-Munsell 100 hue test. Both tests follow a similar structure in which the user must interact with colored images, and their results are compared with those from charts, thus determining whether the user has color blindness, its type, and, in some cases, its degree.

Recent advancements in technology have led to the development of aids for individuals with color blindness such as specialized glasses and lenses [17, 20], accessibility options in the operating systems or commercial smartphone applications that enhance color discrimination (e.g. NowYouSee [5], Color Blind Pal [7] or Spectra [13]). These tools work by altering the light spectrum or the way colors are displayed. Frequently the apps apply an image processing to the entire image captured by the camera which can be confusing for the user, as it may also change the perception of other non-conflicting colors. Further, Iqbal et al. [11] commented the challenges faced by color-blind users while operating mobile-phones due to the uniform adaptive features provided by the mobile vendors.

Mining the literature, we also find research works contributing to this domain. Lau et al. [14] proposed a smartphone app based on the key idea of converting color contrasts in a static image to contrasts seen over time. As the user interacts with the device's touchscreen, the image colors are continually changing to help distinguishing between problematic colors and judge the color of objects. They transformed the pixel colors based on a simple shear and translation operation. They tested their solution with five users resulting in the improvement of the color identification. Elrefae [6] presented a smartphone based experimental comparison of color correction algorithms for protanopia, deuteranopia, and tritanopia. They compared the LMS daltonization algorithm, color-blind filter service (CBFS) algorithm, LAB color corrector algorithm, and the shifting color algorithm, but they did not evaluate them with users. Lausegger et al. [15] developed OmniColor, which used a slightly modified version on the daltonization algorithm on a Google Glass application, therefore, requiring specified hardware. They tested their application with five users presenting protanomaly, improving the identification of the Ishihara plates.

In this work we present the development of a mobile app addressed to people with color blindness. The app comprises two distinct parts. On the one hand, we implemented the Ishihara color test to allow users to find out (or confirm) if they present color blindness. On the other hand, we developed a camera interface, in which the colors that usually cause confusion for people with color blindness are modified to be easily distinguishable. We modify only the colors that cause problems according to the type of color blindness, aiming at improving the user experience compared to other existing applications. Developing tools for color blindness not only advances our understanding of human vision but also drives innovation in creating more inclusive environments and technologies that accommodate the diverse ways people perceive the world.

The work is organized as follows. Section 2 describes the app, its design and implementation. Section 3 presents a usability evaluation with users. Section 4 presents a comparison with another similar app. Finally, the concluding section synthesizes the key takeaways from this research and outlines future working lines.

The solution proposed is an app developed for Android (version 5.0.0 (Lollipop) or later) which comprises two modules: the color test and the camera interface. We implemented the Ishihara color test [12] due to its speed and effectiveness. The test consists of a series of plates filled with dots of different colors and sizes. Within these plates, digits or patterns are embedded in a color that contrasts with the background, which can only be correctly identified if the viewer has normal color vision. From the 24-plates version, we used the set of 17 plates with digits for quick screening (see Fig: 1). Interpretation of the Ishihara test is straightforward: if the user identifies 13 or more plates, it is determined that they do not present color blindness. Conversely, correctly identifying 9 or fewer plates suggests the presence of some form of color blindness. In the rare instance where the user's correct answers fall between 10 and 12 plates, it is advisable to either retake the test or seek alternative testing methods. The test also includes plates designed to identify deuteranopia or protanopia. While highly effective for detecting red-green color blindness, this test is not designed to diagnose blue-yellow color blindness or the complete absence of color vision. For those conditions, although the prevalence is rare [11], other types of tests are required. If the type of color blindness could not be determined, the application will set the configuration for deuteranopia as it is the most common type of color blindness.

Based on the color test results, the camera interface will adapt to the conflicting colors. The camera interface uses the OpenCV library and instead of applying a filter to the entire image, we highlight those regions that present conflicting colors (red tones for protanopia and green tones for deuteranopia), allowing for the user to identify the objects. There are two working modes: (1) the filter mode which highlights the conflicting color regions merged with a personalized color (see Fig. 2) or (2) the gray-scale mode that modifies all the image to a gray-scale but the conflicting colors which will remain unmodified. The camera interface applies directly explainable algorithms, allowing the developers to analyze the correctness of the functioning [8, 16].

Both functioning modes work with the HSV color space (see Fig: 3 (a)) and start by selecting the conflicting colors to create a mask based on a threshold computed empirically (see Fig. 3 (b)). We then invert this mask to achieve a second mask with the non-conflicting colors (see Fig. 3 (c)). These two masks are applied on the original image to select conflicting pixels and non-conflicting pixels. Based on the mask with the conflicting colors, the filter mode applies the personalized color selected by the user (in the example the pink color is selected) and merges the colored mask with the original colors on that region (75% mask and 25% original image) (see Fig. 4 (a)). The resulting image is the combination of the original image with the conflicting colors modified with the user's selected color (see Fig. 4 (b)). In the gray-scale mode, adding the conflicting pixels mask over the gray-scale image (see Fig. 4 (c)), allows to keep the original colors only within the conflicting color regions (see Fig. 4 (d)).

The design of the app includes a limited color palette (white for the background, black for the text and lilac for the buttons) avoiding conflicting colors. Only two screens include extra colors but do not interfere with others: to highlight the errors in the color test, and to select the color filter for the conflicting colors (see Fig. 2), which includes also text.

In order to assess the app, we conduct a usability evaluation both with individuals with and without color blindness. The aim is to compile a global view of subjective assessments of usability (effectiveness, efficiency and satisfaction) [10] and usefullness.

We conducted a comparison with another app that applies a filter to the entire image. The aim of this comparison was to study which strategy was more effective: apply image processing to all the image or only to the regions with conflicting colors. Therefore, we chose Spectra as the app to compare because it most closely resembles the structure and functionality of the application proposed in this work.

4.2 Material

The material used for the evaluation was the app presented in this work and the Spectra app. Spectra is a commercial Android app which includes an adaption of the Farnsworth-Munsell test to identify the type of color blindness: protanopia, deuteranopia o tritanopia, and the degree. Then, according to the type and degree of color blindness, the camera interface corrects the colors. There is also the option to switch to normal mode and the option to freeze the image (see Fig. 5).

4.4 Results

Observing Table 1, participants present color blindness as they do not answer correctly to at least 9 plates (column Orig.). In the third column, we observe the outcomes achieved on the test when using the Spectra app. It is evident that the results improve, with the average number of correct answers increasing by 4.85 points per user (average). When using the app proposed in this work to conduct the color test, in all instances, the number of correct responses further increases (see Table 1, column App), generally by an additional 2 points (see Table 1, column Dif.).

Table 1: Number of correct answers to the Ishihara color test. id: participant; Orig.: with no app; Dif. difference of correct answers

id	Orig.	Spectra	App	Dif.
1	7	10	12	2
2	5	11	13	2
3	3	9	11	2
4	3	10	12	2
5	6	9	12	3
6	7	10	12	2
7	3	9	11	2

This enhancement in results can be attributed to the fact that using the applications makes it easier to distinguish between some of the color combinations. However, this improvement does not occur uniformly across all the plates. With the Spectra app, it is easier for users to read plates 1, 2, 3, 6, and 7. On the other hand, the application developed in this project significantly aids in the identification of plates 1, 6, 7, 10, 11, 12, 13, 16, and 17. As we can see, there are some plates where improvement is noted with both applications, but we next examine cases where one application proves more effective than the other.

For example, in the case of the third plate, which displays the number 29, the majority of users correctly identified the number using the Spectra application for assistance, whereas with the proposed application, a considerable number of incorrect responses were noted. As can be seen in the image (See Fig. 6), the Spectra application's filter enhances the red hues, which aids in visualizing the contours of both numbers. In the case of the application presented, the lower stroke of the 2 and the central part of the 9 are not sufficiently identified as conflicting colors, leading to outcomes such as 70 or 20 during the tests. The opposite scenario occurs with the plate containing the number 73. In this instance, none of the participants managed to correctly identify the number using the Spectra app. However, all participants were able to clearly see the number when using the application developed in this work. As depicted in Fig. 7, while using the proposed app, the number itself is not highlighted, but the background is very clearly emphasized, thus allowing for perfect readability. This same situation occurred with the plates containing the numbers 5, 7, and 16, corresponding to plates 10, 11, and 12 of the test.

We developed a vision based interface to help enhancing conflicting colors for people with color blindness. The approach identifies the confusing colors for red-green color blindness, and highlights them to the user in two different ways: applying a colored filter on the conflicting colors or removing color information of all zones (converting them to gray-scale) but the conflicting regions. There exist diverse types and degrees of color blindness, therefore, the existence of a broad spectrum of tools and solutions may effectively accommodate different profiles and needs, allowing for the user to choose their optimal solution.

Future lines of work include redesigning the app following Android design guidelines, the comparison of different algorithms with the proposed approach, designing for blue-yellow color blindness or developing a third “contour mode” where the contours of the conflicting regions would be highlighted.