An Integrated Variable-Magnitude Approach for Accessibility Evaluation of Healthcare Institute Web Pages

Abstract

The World Wide Web has become an important platform for sharing a wide array of information within the world community. In the post-COVID-19 scenario, the web become a primary source of information in the context of healthcare information dissemination. Healthcare institutions, such as hospitals and clinics, utilize this platform to provide services to reach their target users. It is essential to evaluate the web pages of healthcare institutions and compute their accessibility score for people with disabilities or special needs. This paper presents a variable-magnitude approach to compute the accessibility score of healthcare web pages, considering several requirements of people with disabilities. To compute the accessibility score through the proposed approach, we considered two different components and integrated them to compute the accessibility score through the proposed algorithm. The proposed approach was experimentally applied to sixteen healthcare institutes’ web pages in Hungary. Based on the experiment’s results and the received feedback from an accessibility specialist, a set of suggestions is provided to minimize the accessibility barrier and improve the accessibility score for people with disabilities to access web resources without difficulty. The main contribution of this work is in enhancing awareness of web platform accessibility for web practitioners to improve accessibility, so that people with disabilities can effectively access web resources.

1. Introduction

The World Wide Web (WWW) is an advanced platform for instantly sharing information among wide arrays of people [1]. Nowadays, the WWW provides various services (information sharing, communication, serving administrative tasks, etc.) for sectors such as the education, healthcare, and e-commerce sectors. The variety of services provided on the web is unprecedented. This global platform has enormous functionalities and allows of a wide spectrum of devices, highlighting the uniqueness of the web [2].

Although the WWW has the potential to improve global standards, it might not be accessible for people with disabilities if its usage is not designed properly. Therefore, according to our universal accessibility specialist’s opinion, the use of the WWW should be designed with concern for all requirements of people with disabilities to keep this platform accessible for them [3].

In 2019, the global COVID-19 pandemic pushed the global scenario into a new dimension. Previously, the WWW was used as a secondary option to provide services such as healthcare, banking, and education. However, in the post-COVID-19 scenario, the WWW has become the primary option for providing these services. For example, people are now comfortable with online consultation with doctors; students prefer to join online classes; and people are obtaining banking services (receiving information, transferring money, checking balances, etc.) online. In this scenario, it is important to maintain the web platform as accessible and barrier-free for all groups of users, especially for people with disabilities.

In the post-COVID-19 scenario, hospitals across the world have started to deliver their services through the web platform (e.g., websites), bringing a new dimension to accessing healthcare information and services [4]. Thus, the official web pages of hospitals are becoming the primary source to obtain healthcare resources, such as doctors’ information, face-to-face consultation schedules, online consultation appointments, accessing test reports, emergency contact numbers, and patient status. According to the World Health Organization, 15% of the world’s population has some sort of disability, which amounts to 1 billion people or one in seven persons in the total population [5]. In addition, according to EU statistics, one in six people who suffer from some disability are active on the Internet [6], while the United Nations reported that more than 10% of people have disabilities in the Asia region [7]. Due to inaccessible web platforms, these significant numbers of people should not be hindered in accessing healthcare information, as obtaining consistent healthcare services is a basic right of everyone.

In the Hungarian context, the official census statistics of 2011 show that 500,000 people, or 5% of the total population, experience some type of disability, and this ratio has increased in recent times [8]. Thus, the government of Hungary started several projects, such as “Supporting people with disabilities” and “Human Dignity without Barriers”. The primary objective of these projects was to facilitate access to digital services by introducing a barrier-free environment (i.e., accessible web pages). In addition, Hungary has a significant elderly population (19% of the total population) whose members have one or more disabilities [9]. Ensuring accessibility in both the physical and the digital infrastructure is a crucial factor in assessing the accessible digital healthcare platform and services for elderly people and others who have disabilities. Therefore, in this study, our primary focus is web-page accessibility, especially the accessibility provided by hospital web pages, which are the entry point of healthcare services for people with disabilities.

To serve the healthcare needs of so many people, it is compulsory that healthcare web pages are barrier-free. In recent times, information technology serves several facilities to make digital platforms/resources (i.e., web, software, mobile) accessible for people with disabilities [10]. In that context, hospital web pages should be accessible for elderly people and people with disabilities. To ensure the accessibility of hospital web pages, the emerging need is to focus on the proper design and development of the accessibility feature. Past research and researcher opinions concluded that a portion of the population of a country must be independent in terms of obtaining healthcare services [11,12]. Although the earlier-mentioned statistics regarding disabilities and elderly people are from the Hungarian context, the requirements of people with disabilities across the globe are similar. However, the most important advantage of the web is its ability to spread information to a wide spectrum of people. With this prolific expansion of the web platform, it is crucial to ensure that web resources are accessible to every group of people.

The initial and most important requirement in achieving an accessible web platform is to measure the accessibility score of the current web platform (i.e., the web pages). Measuring accessibility status and identifying the most frequent problems are crucial in increasing the web’s accessibility [13]. From that perspective, the computation of the web accessibility score of hospital web pages gains significant importance.

In this work, measuring the accessibility scores of web pages is the primary objective. To do so, the proposed variable-magnitude approach considers the evaluation results of two automated tools as input variables and calculates the accessibility score by altering the weight of the input variables, based on their importance, and integrating multiple variables’ results to compute the final score. To classify the computed score in assessing the accessibility of the evaluated web pages, mathematical statistics, such as threshold measurement, are employed. Furthermore, expert evaluation has been incorporated to evaluate the selected web pages to justify the computed score. The quantitative and qualitative results depict that the proposed approach is effective in identifying the accessibility of the selected web pages, which might be helpful for web practitioners in improving the accessibility of their web pages in the future. Our application of the proposed approach is the key to focusing on accessibility improvement to make every user feel comfortable and independent. The objectives of this research are the following:

• Proposing an accessibility-score-computation approach that focuses on a variable-magnitude method;
• Evaluating the accessibility status of hospitals’ and clinics’ web pages, specifically homepages in Hungary, with the computed accessibility scores;
• Providing an array of suggestions to improve the accessibility of the tested web pages to make them fully accessible for people with disabilities.

The remainder of this paper is organized as follows. A detailed discussion of related works is presented in Section 2. The methodology of the proposed work is presented in Section 3, with the following subsections: details of the selection of accessibility testing tools and their descriptions, and the mathematical and algorithmic accessibility computation process. The experiment’s result and the accessibility analysis of the web pages are presented in Section 4. In Section 5, we provide suggestions for enhancing the web accessibility of healthcare institution web pages, according to the analysis results and experts’ opinions. Finally, the conclusion and future directions are addressed in Section 6.

2. Past Research

Accessibility computation and validation are important research areas that are aligned with the human–computer interaction (HCI) domain [14]. The HCI domain focuses on effective interaction between humans and computers, whereas accessibility computation focuses on effective service to people with disabilities, to empower them through computer technology [15]. The objective of accessibility computation is to incorporate computer technology into human activity, as computer technology can reduce people’s dependency on human assistance that might be expensive, complex, or not available on demand. In particular, computer technology can be a great advantage for people with disabilities, by providing a wide array of advanced services that could replicate human interaction. This paper focuses on an analysis of the accessibility of digital platforms, particularly hospital web-page accessibility, to enhance their accessibility status for people with disabilities. In the last few years, accessibility analysis has been carried out globally by various research groups [16,17,18,19].

The evaluation of healthcare web pages has been recently conducted by several research studies, considering several aspects [19,20]. In the context of accessibility conformance, Aizpurua et al. [14] evaluated hospital web pages in Poland, focusing on website performance, availability, SEO quality, and mobile friendliness. Raufi et al. [15] investigated healthcare web-page accessibility in terms of readability. Alismail et al. [21] evaluated COVID-19 vaccine registration web pages through automated accessibility-testing tools. These studies focused on web pages from a particular country or geographical region using several automated accessibility-testing tools.

Other studies focused on other domains and techniques, such as a longitudinal comparison of the tested results of educational institutions’ web pages. This technique is effective for the analysis of accessibility results [22]. In addition, Hungary-specific web accessibility analysis has been carried out in the educational domain through automated accessibility-testing tools. In addition to the educational domain, there were studies of e-commerce and tourism web pages, including some specific applications associated with assisting people with disabilities [23,24,25].

In addition, most of the past studies considered an evaluation of the accessibility of desktop and laptop browsers [26,27]. Few studies focused on mobile browser accessibility evaluation, as mobile devices are easy to use and are increasingly popular among people with disabilities [28,29]. These studies addressed several requirements that can enhance the accessibility of the mobile platform.

Web accessibility research has become an important topic for researchers across the globe. From different parts of the world, researchers are contributing by focusing their efforts on the particular research issue of accessibility. However, there are several issues associated with accessibility that remain unsolved. Furthermore, to improve inaccessibility research, Kuppusamy and Balaji [30] suggested that the variable-magnitude approach is effective in identifying the accessibility of web pages in depth. They computed the accessibility barrier of several web pages through a variable-magnitude approach, considering AChecker and WAVE reports. Their proposed approach was effective; however, we found some limitations that could reduce the effectiveness of their evaluated results. For example, for severity-factor selection, they chose a constant value of 0.1, which should be applicable for all the variables; however, they did not incorporate the severity factor for the initial variable (likely error, error). In other cases, the dimension of the input set of two algorithms was different (3:2), which might lead to an inappropriate calculation of the overall score. In addition, for different assessment parameters, Kuppusamy and Balaji did not classify the dependent and independent parameters and their potentiality. For example, among known errors, likely errors, and potential errors, Kuppusamy and Balaji did not address which error potentially reduces the accessibility or prime factor for the inaccessible scenario. Treating the three errors collectively might not be effective in identifying the actual accessibility scenario of web pages.

Addressing all of these aspects in this paper, we attempt to contribute to that earlier research through our proposed variable-magnitude approach in computing accessibility scores and to provide suggestions to develop a barrier-free accessible web platform for people with disabilities or special needs.

3. Methodology

This study presents the evaluation and computation of the accessibility scores of healthcare web pages in Hungary to determine the accessibility status of these web pages for people with disabilities. Healthcare web pages provide an array of information to people, including doctor information, appointment information, emergency numbers, doctor availability, and test information. Focusing on the importance of healthcare we-page accessibility, research considering several healthcare web pages in different countries has been carried out [19,20]. Following such previous research, in this work we tried to add a contribution to this particular area. This section describes the proposed approach in detail, focusing on evaluation-tool selection and accessibility-score computation, incorporating the proposed variable-magnitude approach. All aspects of the process are discussed in the following subsections.

3.1. Automated Accessibility Tools

To evaluate the accessibility of web pages, there is an array of automated accessibility-testing tools that have been introduced by accessibility-testing groups and scholars [31,32]. In this work, two automated accessibility-testing tools have been incorporated, as listed in

Table 1. Selected Tools.

Tools Name	URLs	Open Source	Browser Plugin
Mauve++	https://mauve.isti.cnr.it/ (accessed on 6 January 2023)	√	√
TAW	https://www.tawdis.net (accessed on 6 January 2023)	√	√

, to evaluate the selected hospital web pages and compute the integrated accessibility score.

Mauve++ is an open-source accessibility-evaluation environment that allows users to test a particular website through their URLs [33]. It validates websites against the Web Content Accessibility Guidelines (WCAG), providing various platforms (such as desktop, iPad, mobile phone, and tablet) to conduct the evaluation process. It evaluates websites against various checkpoints of the WCAG and classifies the evaluation report in terms of “Passed”, “Failed”, “Not Tested”, and “Not Decided” evaluation matrices. Figure 1 shows the accessibility-evaluation result of the SOPRONI Gyogykozpont website via Mauve++, which shows the evaluation result in terms of four assessment terminologies. The details of the Mauve++ tool can be found in [34].

TAW is another open-source accessibility evaluation framework that allows the evaluation of a website’s accessibility status without charging costs. It generates an evaluation report that mentions the total number of issues on the web page that are related to accessibility aspects in terms of the number of “Passed”, “Failed”, “Not Tested”, and “Not Decided” success criteria [33]. Figure 2 shows the accessibility-evaluation result of the SOPRONI Gyogykozpont website via TAW, in terms of their assessment terminologies.

Recently, the web initiative World Wide Web Consortium (W3C) published several accessibility-evaluation tools that can be found at https://www.w3.org/WAI/ER/tools/ (accessed on 6 January 2023). In addition, some research classified these tools in terms of particular domains [35,36]. Many aspects could be considered during appropriate accessibility-testing tool selection. In our case, we wanted to select tools that allow analyzing the computed result via our proposed approach. Therefore, we set two criteria that allowed us to identify the appropriate tools to incorporate into this work: (i) the result should specify which success criteria or guidelines have passed, failed, not been tested, or not been decided, and (ii) the report should highlight all the investigated guidelines. By taking these two criteria into consideration, we evaluated six tools: WAVE, Nibbler, MAUVE++, Web accessibility, AChecker, and TAW, according to their popularity and capability in performing the validation process. Unexpectedly, among these six tools, Mauve++ and TAW fulfilled the selected criteria, and we decided to incorporate them into the proposed approach. The Mauve++ and TAW tools highlight all the tested guidelines and represent their results in terms of four terminologies: “Pass”, “Fail”, “Not Tested”, and “Not Decided”, where “Pass” refers to the guidelines that were properly implemented in the website, following the WCAG; “Fail” refers to the guidelines that were not properly implemented in the website, following the WCAG; “Not Tested” refers to the guidelines that were not considered for implementation in the website, according to the WCAG; and “Not Decided” refers to the guidelines that were undetected, according to the WCAG. The details of these terminologies could be found in [36]. In addition, the secondary reason to consider these two tools is their ability to provide the evaluation results structurally, which allows us to integrate the results via our proposed approach. In addition, these two tools allow interaction with the web through programmatic interfaces (API) that make the testing process unbiased and reduce complexity. In addition, the prototypes of the generated results are aligned with our integrated accessibility-computation algorithmic objective. The API provides interaction through the browser interface and fetches the tested result by entering the URL of the tested web page. Both tested tools provide a browser plugin.

3.2. Accessibility Score Computation

In this paper, we propose a variable-magnitude approach to compute accessibility scores for web pages. The proposed approach is an integration of two automatic accessibility-testing tools’ results. The first result is the evaluation result of several components computed by Mauve++ and the second result is the evaluation result of multiple components computed by TAW. Then, the computed results of Mauve++ and TAW are integrated via the accessibility computation algorithm. The accessibility score represents the magnitude of the accessibility status of a web page for people with disabilities.

3.2.1. Mauve++ Accessibility Score

To compute the accessibility score of the selected web page, the URL of the selected web page is passed as input to Mauve++. The rule set is selected as Web Content Accessibility Guideline 2.1, considering level AAA. The output of Mauve++ is generated according to four components, as shown in Equation (1).

$$\mathrm{Evaluation} \mathrm{components}, \kappa \kappa =\left\{\beta , \phi , \gamma , \delta \right\}$$

(1)

In Equation (1), β represents the number of passed guidelines, φ indicates the number of failed guidelines, γ indicates the number of not-tested guidelines, and δ refers to the number of not-decided guidelines. Based on the severity of the tested guidelines, Equation (2) shows the guidelines in descending order of severity.

$$\mathrm{Severity} \mathrm{order}=\mathsf{\Phi }\left[\beta \right]>\mathsf{\Phi }\left[\phi \right]>\mathsf{\Phi }\left[\gamma \right]>\mathsf{\Phi }\left[\delta \right]$$

(2)

Following the severity order, four different weight coefficients ω_β, ω_φ, ω_γ, and ω_δ, together with a severity factor ϵ, are introduced to compute the accessibility score. β is multiplied by its weight coefficient ω_β and severity factor ϵ, as shown in Equation (3), to compute the ratio of the passed guidelines, $\rho 1$.

$$\mathrm{Pass} \mathrm{ratio}, \rho 1=\left[\beta \ast {\omega }_{\beta }\ast ϵ\right]$$

(3)

The weight coefficient of the failed guidelines ω_φ is calculated by scaling down the weight coefficient of the number of pass guidelines ω_β by multiplying the severity factor ϵ, as shown in Equation (4).

$$\mathrm{Weight} \mathrm{coffeciecnt}, {\omega }_{\phi }=\left[{\omega }_{\beta }\ast ϵ\right]$$

(4)

φ is multiplied by its weight coefficient ω_φ to compute the ratio of the failed guidelines, $\rho 2$, as shown in Equation (5).

$$\mathrm{Fail} \mathrm{ratio}, \rho 2=\left[\phi \ast {\omega }_{\phi }\right]$$

(5)

The weight coefficient of the not-tested guideline ω_γ is calculated by scaling down the weight coefficient of the number of failed guidelines ω_φ by multiplying the severity factor ϵ, as shown in Equation (6).

$$\mathrm{Weight} \mathrm{coffecient}, {\omega }_{\gamma }=[{\omega }_{\phi }\ast ϵ]$$

(6)

γ is multiplied by its weight coefficient ω_γ to compute the not tested ratio, $\rho 3$, as shown in Equation (7).

$$\mathrm{Not}-\mathrm{tested} \mathrm{ratio}, \rho 3=\left[\gamma \ast {\omega }_{\gamma }\right]$$

(7)

The weight coefficient of the not-decided guidelines ω_δ is calculated by scaling down the weight coefficient of the number of not-decided guidelines ω_γ by multiplying the severity factor ϵ, as shown in Equation (8).

$$\mathrm{Weight} \mathrm{coffecient}, {\omega }_{\delta }=\left[{\omega }_{\gamma }\ast ϵ\right]$$

(8)

δ is multiplied by its weight coefficient ω_δ to compute the not-decided ratio, $\rho 4$, as shown in Equation (9).

$$\mathrm{Not}-\mathrm{decided} \mathrm{ratio}, \rho 4=\left[\delta \ast {\omega }_{\delta }\right]$$

(9)

After computing each component’s score (ρ1, ρ2, ρ3, ρ4), the score of Mauve, ρ_(Mauve) is computed through Equation (10). The ComputeAccessibility.Mauve function represents the full procedure as shown in Algorithm 1.

$$\mathrm{Mauve} \mathrm{score}, {\rho }_{\left(Mauve\right)}=\rho 1-\sum \left(\rho 2+\rho 3+\rho 4\right)$$

(10)

Algorithm 1: ComputeAccessibility.Mauve

3.2.2. TAW Accessibility Score

The accessibility score computation by TAW is based on four components, κ, λ, μ and χ. These four components refer to the number of passed, failed, not-tested, and not-decided guidelines, similar to the returned component by Mauve. Like the accessibility-barrier computation via Mauve, TAW-based accessibility-barrier computation considers variable weight and severity factors. The four variable weights are ω_κ, ω_λ, ω_μ, ω_χ, and their severity factor is ϵ. The weight coefficients ω_κ, ω_λ, ω_μ, and ω_χ are scaled by their severity factors ϵ, as shown in Algorithm 2. The pass, fail, not-tested, and not-decided ratio computations are shown in Equations (11)–(14).

$$\rho 1=\left[\kappa \ast {\omega }_k\ast ϵ\right]$$

(11)

$$\rho 2=\left[\lambda \ast {\omega }_{\lambda }\right]$$

(12)

$$\rho 3=\left[\mu \ast {\omega }_{\mu }\right]$$

(13)

$$\rho 4=\left[\chi \ast {\omega }_{\chi }\right]$$

(14)

The score of TAW, ρ_(Taw), is computed through Equation (15). The ComputeAccessibility.TAW function represents the full procedure, as shown in Algorithm 2.

$$\mathrm{Taw} \mathrm{score}, {\rho }_{\left(Taw\right)}=\rho 1-\sum \left(\rho 2+\rho 3+\rho 4\right)$$

(15)

Algorithm 2: ComputeAccessibility.TAW

3.2.3. Overall Accessibility Score

The overall accessibility score is computed on the basis of the mathematical summation of the accessibility counts vias Mauve++ and TAW, as shown in Equation (16). The complete procedure for the overall accessibility score is described in Algorithm 3 under the ComputeScore function.

$$\mathrm{Overall} \mathrm{accessibility} \mathrm{score} \mathrm{count}, \rho =\frac{\sum {\rho }_{\left(Mauve\right)}+{\rho }_{\left(Taw\right)}}{n}$$

(16)

Algorithm 3: Compute Accessibility

4. Computed Score Analysis

The primary focus of this work is on the healthcare sector, to measure its accessibility status and to compute its accessibility score via the proposed variable-magnitude approach by incorporating two automated accessibility-testing results. The initial application was performed by considering the home pages of hospital and clinical websites in Hungary. A consideration of the entire website is beyond the scope of this work. The hospitals and clinics were chosen from the capital city, other large cities, and small towns in Hungary. The URLs of the considered hospitals’ and clinics’ web pages are listed in

Table 2. Sixteen Healthcare Institute Website Homepage URLs.

Website ID	Hospital/Clinical Point Name	Web Page URLs
ID-1	Országos Onkológiai Intézet	https://onkol.hu/ (accessed on 6 January 2023)
ID-2	Uzsoki Kórház	https://www.uzsoki.hu/ (accessed on 6 January 2023)
ID-3	Szegedi Közlekedési Társaság	https://szkt.hu/szeged-megyei-jogu-varos-ii-korhaza (accessed on 6 January 2023)
ID-4	Buda Health Center Budai Egészségközpont	https://bhc.hu/ (accessed on 6 January 2023)
ID-5	Heim Pál Children’s Hospital	http://heimpalkorhaz.hu/ (accessed on 6 January 2023)
ID-6	Jahn Ferenc Dél-Pesti Kórház	https://www.delpestikorhaz.hu/ (accessed on 6 January 2023)
ID-7	Klinikai Központ Semmelweis Egyetem	https://semmelweis.hu/klinikaikozpont/ (accessed on 6 January 2023)
ID-8	Klinikai Központ Debreceni Egyetem	https://klinikaikozpont.unideb.hu/ (accessed on 6 January 2023)
ID-9	Szegedi Tudományegyetem	http://www.med.u-szeged.hu/karunkrol/klinikak/klinikak?folderID=24366&objectParentFolderId=24366 (accessed on 6 January 2023)
ID-10	Klinikai Központ Pécsi Tudományegyetem	https://kk.pte.hu/klinikak-intezetek (accessed on 6 January 2023)
ID-11	Tormay Károly Egészségügyi Központ Tüdőszűrő	https://tormay.hu/ (accessed on 6 January 2023)
ID-12	B.A.Z Megyei Kórház és Oktató Kórház	https://www.bazmkorhaz.hu/ (accessed on 6 January 2023)
ID-13	Soproni Gyógyközpont	http://www.sopronkorhaz.hu/ (accessed on 6 January 2023)
ID-14	Petz Aladár Megyei Oktató Kórház	https://www.petz.gyor.hu/ (accessed on 6 January 2023)
ID-15	Veszprém Megyei Csolnoky Ferenc Kórház Nonprofit Zrt.	https://csfk.hu/ (accessed on 6 January 2023)
ID-16	Markhot Ferenc Teaching Hospital and Clinic	https://www.mfkh.hu/ (accessed on 6 January 2023)

.

To compute the integrated accessibility score, first, the accessibility scores were computed by incorporating Mauve ρ_(Mauve) via the ComputeAccessibility.Mauve process (Algorithm 1). The computed results of 16 healthcare institutions’ web pages are shown in

Table 3. Computed score of ρ_(Mauve).

Website ID	Pass	Fail	Not Tested	Not Decided	ρ(Mauve)
ID-1	21	11	42	8	13.43
ID-2	20	12	42	8	7.19
ID-3	25	13	35	9	49.36
ID-4	22	15	37	8	7.04
ID-5	20	13	38	11	9.45
ID-6	24	12	37	9	45.22
ID-7	21	19	32	10	40.32
ID-8	25	11	36	10	69.56
ID-9	22	12	38	10	20
ID-10	19	12	43	8	18.70
ID-11	23	12	38	9	31.78
ID-12	25	11	37	9	69.33
ID-13	27	8	41	6	139.08
ID-14	20	11	41	10	3.90
ID-15	23	10	41	8	50.26
ID-16	23	15	35	9	6.26

. As a visual illustration of the computed accessibility score, Figure 3 shows a bar graph, with the corresponding website ID as a reference.

Second, the accessibility scores were computed through TAW, ρ_(Taw), via the ComputeAccessibility.TAW process (Algorithm 2). The computed results are shown in

Table 4. Computed score of ρ_(Taw).

Website ID	Pass	Fail	Not Tested	Not Decided	ρ(Taw)
ID-1	14	12	6	18	0.44
ID-2	17	7	7	19	53.31
ID-3	14	12	5	19	1.34
ID-4	16	7	9	18	41.47
ID-5	16	6	9	19	47.61
ID-6	14	9	10	17	12.99
ID-7	16	5	9	20	53.76
ID-8	25	11	36	10	16.32
ID-9	15	7	9	19	32.64
ID-10	14	6	10	20	29.12
ID-11	15	6	9	18	40.5
ID-12	15	11	5	19	13.43
ID-13	13	7	9	21	17.47
ID-14	14	7	10	19	23.74
ID-15	13	9	8	20	8.32
ID-16	14	5	14	17	30.91

. Following the visual representation of the computed accessibility score by Mauve, Figure 4 shows the visual representation of the computed accessibility score by TAW, considering a bar graph.

Moreover, the overall accessibility score (ρ) is computed through the ComputeScore process, as shown in Algorithm 3. The overall computed scores are listed in

Table 5. Computed Accessibility Score, ρ.

Website ID	Website URLs	Accessibility Score	Status
ID-1	https://onkol.hu/ (accessed on 6 January 2023)	6.94	Not accessible
ID-2	https://www.uzsoki.hu/ (accessed on 6 January 2023)	23.05	Not accessible
ID-3	https://szkt.hu/szeged-megyei-jogu-varos-ii-korhaza (accessed on 6 January 2023)	25.35	Not accessible
ID-4	https://bhc.hu/ (accessed on 6 January 2023)	17.21	Not accessible
ID-5	http://heimpalkorhaz.hu/ (accessed on 6 January 2023)	19.08	Not accessible
ID-6	https://www.delpestikorhaz.hu/ (accessed on 6 January 2023)	29.10	Not accessible
ID-7	https://semmelweis.hu/klinikaikozpont/ (accessed on 6 January 2023)	6.71	Not accessible
ID-8	https://klinikaikozpont.unideb.hu/ (accessed on 6 January 2023)	42.94	Likely accessible
ID-9	http://www.med.u-szeged.hu/karunkrol/klinikak/klinikak?folderID=24366&objectParentFolderId=24366 (accessed on 6 January 2023)	26.32	Not accessible
ID-10	https://kk.pte.hu/klinikak-intezetek (accessed on 6 January 2023)	5.20	Not accessible
ID-11	https://tormay.hu/ (accessed on 6 January 2023)	36.14	Not accessible
ID-12	https://www.bazmkorhaz.hu/ (accessed on 6 January 2023)	41.38	Likely accessible
ID-13	http://www.sopronkorhaz.hu/ (accessed on 6 January 2023)	78.27	Potentially accessible
ID-14	https://www.petz.gyor.hu/ (accessed on 6 January 2023)	13.82	Not accessible
ID-15	https://csfk.hu/ (accessed on 6 January 2023)	29.29	Not accessible
ID-16	https://www.mfkh.hu/ (accessed on 6 January 2023)	18.58	Not accessible

, with their URLs and individual IDs. Figure 5 shows the graphical representation of the computed overall accessibility score by a bar graph. Every graph is represented with its corresponding reference ID.

To evaluate the aforementioned computed overall accessibility scores, we set a threshold value in the ranges of α = 0 to <40, 40 to <70, and 70 to =<100. If the computed accessibility score was in the range of α = 0 to <40, it was considered as a not-accessible web page. If the score was in the range of α = 40 to <70, the web page was considered to be likely accessible. If the computed score was within the range of α = 70 to =<100, it was considered that the web page was potentially accessible.

According to the computed accessibility ratio (Table 5/Figure 5), our observation is that the majority of the web pages’ accessibility scores were in the range of α = 0 to <40, which depicts that they have low accessibility scores and are not accessible for people with disabilities. This represents a poor concern in the context of universal accessibility. In addition, two web pages had accessibility scores in the range of α = 40 to <70, which were determined to be likely accessible (ID-8, ID-12), and one web page had an accessible score in the range of α = 70 to =<100, which was determined to be potentially accessible (ID-13). The analysis results show the emerging need concerning universal accessibility.

Additionally, expert testing was incorporated into this work to validate the evaluated results of the proposed approach. Expert testing is the process of identifying several accessibility issues by incorporating human judgment [37]. The expert testing was conducted through ten questionnaires with three assessment parameters: “Accessible”, “Partially Accessible”, and “Not Accessible”. If an expert found a website partially accessible or not accessible, then we asked the expert to provide a descriptive opinion about what aspects made the website inaccessible or partially accessible. In addition, the expert was asked to submit valuable suggestions to improve such inaccessible scenarios for future consideration. To carry out this task, four experts were invited to evaluate the selected websites and present their reports in terms of the three assessment criteria. All of the experts were working at the Department of Electrical Engineering and Information Systems and the Department of Computer Science, University of Pannonia, Hungary. Each of the experts had more than 6 years of experience in the field of “web accessibility”, “human–computer interaction”, “multimedia design”, and “web informatics”.

Table 6. Questionnaire for expert evaluation with assessment parameter.

Questionnaire	Assessment Parameter
Q1. Does the website accessible to low-vision people?	Accessible/Partially accessible/Not accessible
Q2. Does the website accessible to color-blind people?	Accessible/Partially accessible/Not accessible
Q3. Does the website accessible to a person with a motion disability?	Accessible/Partially accessible/Not accessible
Q4. Does the website accessible to a person with a hearing disability?	Accessible/Partially accessible/Not accessible
Q5. Does the website accessible to a person with a cognitive disability?	Accessible/Partially accessible/Not accessible
Q6. Does the website require user information?	Accessible/Partially accessible/Not accessible
Q7. Does the website have keyboard functionality?	Accessible/Partially accessible/Not accessible
Q8. Does the website have manual font size adjustment functionality?	Accessible/Partially accessible/Not accessible
Q9. Does the website have inaccessible features?	Accessible/Partially accessible/Not accessible
Q10. Is it possible to distinguish between links and buttons?	Accessible/Partially accessible/Not accessible
N.B. If ‘Partially accessible/Not accessible’, then please provide descriptive opinion on what criteria make the website partially accessible or not accessible and provide valuable suggestions to improve such inaccessible scenario.

shows the questionnaire that we established for the expert evaluations. Each of the questions was associated with five disability types—vision impairment, color-blind issues, motion difficulty, hearing difficulty, and cognitive difficulty.

The expert evaluation results were concluded in terms of ten questions set out in

Table 7. Expert assessment/evaluation results.

Website ID	Website URLs	Expert Opinion
ID-1	https://onkol.hu/ (accessed on 6 January 2023)	Not accessible
ID-2	https://www.uzsoki.hu/ (accessed on 6 January 2023)	Not accessible
ID-3	https://szkt.hu/szeged-megyei-jogu-varos-ii-korhaza (accessed on 6 January 2023)	Not accessible
ID-4	https://bhc.hu/ (accessed on 6 January 2023)	Partially accessible
ID-5	http://heimpalkorhaz.hu/ (accessed on 6 January 2023)	Not accessible
ID-6	https://www.delpestikorhaz.hu/ (accessed on 6 January 2023)	Not accessible
ID-7	https://semmelweis.hu/klinikaikozpont/ (accessed on 6 January 2023)	Partially accessible
ID-8	https://klinikaikozpont.unideb.hu/ (accessed on 6 January 2023)	Partially accessible
ID-9	http://www.med.u-szeged.hu/karunkrol/klinikak/klinikak?folderID=24366&objectParentFolderId=24366 (accessed on 6 January 2023)	Not accessible
ID-10	https://kk.pte.hu/klinikak-intezetek (accessed on 6 January 2023)	Not accessible
ID-11	https://tormay.hu/ (accessed on 6 January 2023)	Not accessible
ID-12	https://www.bazmkorhaz.hu/ (accessed on 6 January 2023)	Accessible
ID-13	http://www.sopronkorhaz.hu/ (accessed on 6 January 2023)	Accessible
ID-14	https://www.petz.gyor.hu/ (accessed on 6 January 2023)	Not accessible
ID-15	https://csfk.hu/ (accessed on 6 January 2023)	Not accessible
ID-16	https://www.mfkh.hu/ (accessed on 6 January 2023)	Not accessible

. The evaluation results were categorized into three criteria: (a) if a website was categorized as accessible by the expert for at least nine questions (evaluation criteria (accessible) >= nine questions), then the website was considered to be accessible; (b) if a website was categorized as accessible by the expert for less than nine questions but equal to or more than seven questions (evaluation criteria (accessible) < nine questions to >= seven questions), then the website was considered to be partially accessible; (c) if a website was categorized as accessible by the expert for less than seven questions (evaluation criteria (accessible) < seven questions), then the website was considered not to be accessible. The results show that majority of the websites were not accessible, and very few were accessible for people with disabilities (Table 7). From the descriptive opinions of the expert, it could be concluded that not-accessible websites have several issues in relation to people with disabilities; more specifically, provided information was not organized, there were absences of a manual translation and a manual font-size-adjustment option, the search functions were not responsive, buttons were not visible or understandable, no navigation direction was provided, the font size of the content was very small and hard to read even for normal people, very dark colors were used, and additional user information was required in some cases. All of these issues limited the acceptability of these websites for people with disabilities. In terms of disability types, the majority of the websites were not accessible to people with vision impairment or cognitive disabilities. Although automated evaluation and analytical statistics were useful in predicting the overall accessibility score, it is quite a difficult task to identify particular issues associated with disability. Thus, the expert evaluation was effective in identifying the unreported accessibility issues that are associated with people with disabilities.

From the analysis results, it can be concluded that the web page designers and developers did not properly comply with the web content accessibility guidelines. Frequently, web designers and developers ignored the accessibility guidelines during development, even though the associated accessibility issues could have been easily fixed. Our objective was to emphasize that every disability type and its requirements should be taken into consideration throughout the design and development phase, as disabled people contribute to the economic development of a country. Therefore, we encourage all designers, developers, and practitioners who are working with any development aspects (e.g., game development, web development, software development, and mobile application development) to follow the Web Content Accessibility Guideline (WCAG) standards, issued by the World Wide Web Consortium (W3C), throughout their web-page development process.

5. Accessibility Improvement Suggestions

This section illustrates an array of suggestions that might help to improve the accessibility of web pages. These suggestions are provided based on the accessibility analysis results and the opinions of our accessibility specialists. We invited four specialists (each with more than five years of expertise in accessibility and digital platforms) from Hungary to observe the selected websites, provide their feedback about the accessibility status, and share their expert opinions about other aspects that could be considered for further accessibility improvement. The details of the questions asked and the assessment criteria were discussed in Section 4. Based on the specialists’ valuable feedback, the suggestions are provided below.

Multilingual web pages: Ensuring multilingual accessibility reflects the standards for developed web pages and is a requirement of the accessibility guidelines. To ensure multilingual accessibility, web pages should focus on the translation of the web pages from one language to another language. In addition, when web pages are translated from one language to another, inconsistencies may be introduced that need to be resolved. Furthermore, all the information should be translated, provide enough focus on the text’s alternative (alt-text function) properties, focus on the correct meaning of the translated version based on the original context, and focus on the precision of the translated context.

Audio/Video Accessibility: Websites have several non-text components, such as audio, video, and images. To ensure proper accessibility to these components, a proper caption should be added. According to the accessibility guidelines, captions for video and audio are compulsory requirements to make multimedia components accessible. In addition, images should have an appropriate title or description. Ensuring proper captions in video and audio components and proper descriptions for images makes the content accessible for people with disabilities. For example, captions in video content help people with hearing impairments to understand the content, and captions in audio content helps people with visual impairments to understand the content. Similarly, ensuring a proper description helps screen readers understand the content.

Color Contrast: Color contrast is an important aspect in making any digital platform (e.g., websites, software, and games) accessible for people with disabilities. The significance of different colors might vary from user to user, based on their respective disabilities and requirements. For example, partially blind people might have requirements that might not be acceptable for color-blind people. Accordingly, adjusting color contrast based on user preference is a crucial requirement. Although a few web pages currently follow this essential requirement, this issue is not being considered for the development of the majority of web pages. Web page designers who have not followed this requirement should add this feature in their development so that persons with disabilities can access the web pages according to their specific requirements. In addition, color-contrast requirements vary for different devices. For example, the color-contrast requirements for small-size displays, such as those of a mobile phone or an iPad, are different from those for bigger displays such those of as laptops or desktops. Therefore, during the development of digital platform devices, these factors need consideration as to whether the assigned color-contrast ratios are suitable/accessible for people with disabilities.

Text Resize: As disability types and requirements are different from one person to another, sometimes text size is also varied according to different disability types and their specific requirements. None of our scored web pages focused on this crucial requirement. Failure to provide this service causes serious inaccessibility to the content. Ensuring this requirement might help users—especially people with visual impairments—understand the content

Inadequacy of Provided Information: One of the crucial aspects of web accessibility is to provide sufficient information that is required by the user. Insufficient information limits the accessibility of web pages. For example, in healthcare web pages, all information related to healthcare services, including doctor information, online consultations, appointment opportunities, appropriate addresses, and contact numbers, should be added. Ensuring access to adequate and appropriate information might help people feel comfortable and confident.

Text Alternatives: Text alternatives for non-textual components, such as images, videos, animation, and icons, are among the important aspects of web accessibility. The absence of textual description of non-textual components causes difficulty in understanding the content and information of non-textual resources. Hence, all non-text elements should have proper descriptions so that they are accessible to people with disabilities, especially those people who are using screen-reader software to access the information. Sometimes, web pages have a few additional images that are used purely as an attraction, but these components introduce some difficulty in distinguish them from important information. These unnecessary non-textual components should be avoided, in order to improve the accessibility of web pages.

Accessible Links: The purpose of using several links in web pages is to provide additional information that is important in understanding the content. In a few cases, such as broken links, links that are not properly distinguishable from text, invisible links, and links that are not validated, the added links are not accessible. To avoid unexpected accessibility problems, these issues should be considered carefully. In addition, proper hyperlinks should be inserted.

Readability: To ensure the readability of the text content of a web page, requirements for every type of disability should be considered, as they vary. For example, the readability level of people with cognitive disabilities is different from that of people with other disabilities. Hence, readability needs to be maintained according to different readability levels so that content may be easily understandable by people with different disability types.

Some important additional guidelines: A few issues with the headings and titles of web pages could hamper screen-reader users in navigating the page in an accessible manner. All headers should be maintained systematically to organize the content. Additionally, before finalizing and making web pages available for users, accessibility testing of all components should be compulsory, taking into account the requirements of people with disabilities, to validate accessibility. Maintaining the aforementioned suggestions might enhance the accessibility of web pages for people with disabilities. All of the provided suggestions in this article are based on the investigated healthcare web pages, but they also apply to any domain’s web page for ensuring universal accessibility.

From the analysis result of the computed scores, the expert evaluations and suggestions show that the proposed approach is effective in presenting the current accessibility situation of healthcare web pages in Hungary. This inaccessibility applies not only in Hungary, as similar scenarios have been observed in other countries in different parts of the world, as we discussed in our previous work [32]. However, this research is not free from limitations. The major limitation of this work is the consistency of the computed results. For example, websites are continuously updating; therefore, the same experimental procedure for a future updated version of a website might alter the results. Furthermore, we only considered four experts’ opinions; considering more experts’ opinions might influence the overall results. In addition, user evaluations have not been included in this work; thus, the incorporation of user evaluations might help in identifying other accessibility issues.

6. Conclusion and Future Directions

In this paper, we presented a web accessibility analysis of healthcare institution web pages in Hungary, adopting a variable-magnitude approach to compute accessibility scores. Two different approaches have been presented in this work—two different algorithms focusing on the variable-magnitude and variable-weight techniques. These two algorithms have been developed to compute accessibility scores based on the four components or terminologies (pass, fail, not tested, and not decided) obtained from two automated accessibility-testing tools, Mauve++ and TAW. The accessibility scores, ρ_(Mauve) and ρ_(Taw), were computed considering four components together with a severity factor to scale down the weight of the components, based on their importance. To generate the overall score, the computed scores of ρ _(Mauve) and ρ_(Taw) were integrated and computed into a final score. Through the introduced integrated system, the web pages of sixteen hospital and clinic were analyzed and scored. Based on the expert opinions of our accessibility specialists, we added a few suggestions to enhance the accessibility of these web pages to make them accessible for people with disabilities. Although considering expert suggestions is appropriate, it is also crucial to justify expert suggestions by considering scientists’ opinions of this field. Therefore, our future direction involves the incorporation of people with several disabilities to validate the computed results. In this work, we focused on the homepage only; a focus on the entire website is a prime objective for future work. In additions’, to justify the expert suggestion, scientist opinions will be incorporated into our future work. This presented work aimed to increase awareness about web accessibility among web developers, web designers, and other associated practitioners, so that shared information is accessible without any barrier for people with disabilities and they can take the advantage of the web in an effective manner.