Enhancing quality of life for the hearing-impaired: a holistic approach through the MoSIoT framework

Abstract

This article unveils MoSIoT (Modeling Scenarios of the Internet of Things), an innovative framework meticulously designed to simulate IoT systems, addressing the nuanced needs of individuals with hearing impairments. Grounded in the quality-of-life framework proposed by Verdugo et al., MoSIoT has been enriched and evolved to integrate and manage a spectrum of deafness technologies, thereby marking a significant enhancement in the overall quality of life for the affected individuals. The focal point of this work is the advocacy for accessibility and well-being through technology-driven and inclusive solutions. The presented case study provides a comprehensive insight into the transformative impact of MoSIoT, highlighting improvements in key quality of life dimensions such as “Interpersonal Relationships”, “Emotional Well-Being”, and “Personal Development”. These enhancements underscore the pivotal role of continuous innovation and inclusive design in the evolution of assistive technologies. The article delves into the intricacies of the system’s implementation, exploring its service-based architecture, which ensures modularity, scalability, and adaptability to diverse user needs and technological advancements. Furthermore, the potential for future research and development in this field is discussed, emphasizing the endless possibilities and avenues for enhancing assistive technology and promoting an inclusive and accessible future for individuals with disabilities.

1 Introduction

In recent years, assistive technology has emerged as a pivotal player in addressing the needs of individuals with disabilities, enabling them to maintain independence and enhance overall functioning. The significance of assistive technology is underscored by the global demand, with over 1 billion personspersons requiring at least one assistive product-a number projected to surpass 2 billion by 2030. Regrettably, access to these essential products is limited to only a fraction of those in need. The World Health Organization (WHO) has identified several challenges impeding access to assistive technology, including policy barriers, product availability, provision of services, and personnel support [1]. Addressing these barriers necessitates a focus on universal health coverage to ensure equitable access to assistive products and services for all.

Research by authors such as Daniel et al. [2] underscores the significance of assistive technologies (ATs) in fostering a conducive work environment and developing accessible IT devices. The study elucidates that ATs align with user expectations and extend benefits beyond individuals with disabilities. There is a highlighted necessity for integrating assistive technologies into the system requirements during the IT product development process, ensuring that accessibility features are integral.

However, it is imperative to acknowledge that while assistive technology offers invaluable support to individuals with disabilities, the solutions provided do not comprehensively address all aspects of quality-of-life improvement. This gap is acknowledged by authors like Schalock and Verdugo [3, 4], who advocate for a holistic approach to assessing and enhancing the quality of life for individuals with disabilities, considering dimensions such as emotional well-being, interpersonal relationships, personal development, material well-being, physical well-being, self-determination, social inclusion, and rights.

A notable illustration of the need for advancements in assistive technology solutions is the scenario faced by the deaf community. Despite technological progress, comprehensive solutions addressing the multifaceted challenges experienced by deaf individuals in their daily lives are still being developed. Technologies such as hearing aids and cochlear implants, while beneficial, do not fully address communication barriers, access to information and services, and social inclusion. In instances where deaf individuals interact with those unfamiliar with sign language, alternative solutions like text-to-speech converters can facilitate communication.

Conversely, for deaf individuals with limited literacy skills, relying solely on text-to-speech converters is not viable. An alternative approach involves utilizing real-time sign language interpreters through video recognition systems, enabling effective communication in diverse environments. Additionally, addressing device accessibility challenges, such as interacting with voice-controlled features on an Alexa speaker, requires innovative solutions like sign language to audio converters.

Moreover, from a technical standpoint, the assistive technology landscape is characterized by a lack of standardization and interoperability among different ATs. This fragmentation poses challenges for deaf individuals in integrating multiple devices and systems seamlessly into their daily routines, leading to inefficiencies and unmet needs. The prohibitive cost of many advanced assistive technologies further exacerbates access limitations, perpetuating the digital divide and marginalizing individuals with hearing impairments.

This work introduces a holistic solution, grounded in the MoSIoT (Modeling Scenarios of the Internet of Things) framework [5], aimed at enhancing the quality of life for hearing-impaired individuals in a personalized manner. The MoSIoT framework facilitates technology integration with various devices and evaluates the individual's quality of life, proposing tailored solutions for improvement.

The core of this proposal employs an IoT architecture of models based on Model-Driven Engineering [6] that is specialized for IoT systems for persons with disabilities and consists of two elements: (i) a domain model, in which a domain expert specifies a knowledge base that gathers the invariant information of most recognized standards relating to three fields, namely, accessibility (AFA) [7], IoT systems (Web of Things), and healthcare (FHIR) [8]; and (ii) a scenario model, in which a domain expert introduces the static and dynamic behavior of the entities specific to a concrete IoT scenario of persons with disabilities. The two models are linked, allowing the invariant concepts of the domain model to feed the variable entities defined in each scenario.

One of the most important contributions of this work is the extension of the MoSIoT framework to integrate Schalock and Verdugo's levels of quality of life into its knowledge base, which allows directing the actions to be taken towards achieving a higher level of quality of life. To this end, the MoSIoT metamodel has been extended with the dimensions of quality of life (emotional well-being, interpersonal relationships, etc.), and a relationship has been established with the actions with the devices that help to improve them to a greater or lesser extent.

In order to evaluate the solution, we have focused on the case study of hearing-impaired persons in this article. To do so, we apply the MoSIoT implementation process that starts with defining those elements in the domain model in each dimension (devices, accessibility, or treatment), adding the dimension of quality of life.

The MoSIoT process continues with the domain expert (social worker or health worker) who interviews the person with a disability to provide the IoT means to improve their quality of life. At that moment, the expert models an IoT scenario in which the different ways in which the person interacts with the devices are collected. Significantly, MoSIoT is based on the AccessForAll (AFA) standard adapted to IoT, which allows a particular level of accessibility customization for each device, where the source and destination communication is set precisely for each person and situation.

A case study has demonstrated how this system can positively impact the well-being of individuals with hearing impairments by promoting accessibility and inclusivity through IoT solutions. This highlights the importance of designing inclusive technologies and leveraging IoT to enhance individuals with hearing impairments' overall quality of life.

In the following sections, we delve into the simulation framework and examine the integration of IoT technologies to showcase the transformative potential of assistive technologies for individuals with hearing impairments. By embracing inclusive design principles and technology-driven solutions, we can pave the way for a more accessible and inclusive future. The following structure is adopted for the paper: Sect. 2 provides the background of our study, including the obstacles faced by the deaf and hard of hearing, communication barriers, and safety concerns. Additionally, we discuss the quality-of-life models, specifically the model proposed by Schalock and Verdugo and its definition, and how our work could improve some of its dimensions. Furthermore, we evaluate various assistive life approaches and their connections to quality-of-life models and assess them according to the model proposed by Verdugo and Schalock. Moving on to Sect. 3, we provide an overview of the MoSIoT framework, designed to cater to the needs of persons with disabilities within IoT Healthcare Management Systems (HMS). This section focuses on the MoSIoT framework's application for deaf and hard-of-hearing persons and explains the MoSIoT application to the Quality-of-life dimensions. In Sect. 4, we present a case study illustrating the impact of the MoSIoT architecture on the quality of life of a hearing-impaired individual. Section 5 concludes the case study and discusses the potential future applications of IoT for individuals with hearing impairments, emphasizing the importance of continuous innovation and inclusive design to create a more accessible and inclusive future for individuals with disabilities.

2 Background

This section provides a knowledge base on assistive technologies for hearing-impaired persons, setting the context and rationale for the research. We begin by discussing the daily challenges faced by hearing-impaired persons, providing a clear picture of the areas in which assistive technologies can be most beneficial. This is followed by an introduction to quality-of-life models, essential to understanding how these technologies can improve the lives of persons with hearing impairments. Finally, we explore existing assistive technologies, providing an overview of current solutions and highlighting areas for improvement and innovation.

To extract data from various sources, we developed a comprehensive set of keywords aligned with our research questions and objectives. The keywords included terms related to disabilities ("Disable" and "Physical disability"), elderly ("Elderly"), IoT ("Internet of things," "Web of things," and "Internet of medical things"), and AR/XR ("Augmented reality", "AR", and "Extended Reality"). Additionally, keywords related to accessibility ("Accessibility" and "Assistive Technology" and "AT") and hearing impairments ("deaf," "hearing impairment," "aural rehabilitation," and "hard of hearing") were included. Moreover, the keywords "smart building" and "Smart house" were added to capture papers on these topics. In addition to the keywords mentioned above, we also incorporated the terms "Case study," "Case report," and "Case analysis" to ensure a more targeted and comprehensive search, capturing literature that explicitly examines case studies or reports related to the field of assistive technology for individuals with hearing impairments.

Once the keyword set was established, we expanded it by including alternative spellings and synonyms identified from the literature review. The final list of keywords was optimized and consolidated.

The search string derived from these keywords was as follows:

(("Disab*") OR ("Physical disab*") OR ("Elderly")) AND (("internet of things") OR ("web of things") OR ("internet of medical things") OR ("Assistive Technolog*") OR (at))) OR ((("Augmented reality") OR (Ar) OR ("Extended Reality"))) AND ("Accessibility") AND (("deaf") OR ("hearing impairment ") OR ("aural rehabilitation") OR ("hard of hearing")) AND (("smart building*") OR ("Smart hous*"))) OR (("Case study") OR ("Case report") OR ("Case analysis")).

The search strategy mentioned earlier was applied to the following search engines: SpringerLink, SCOPUS, IEEE Xplore, PubMed, IOPscience, and Science Direct. These search engines were selected as part of the search space to retrieve relevant papers.

2.1 Obstacles confronted by the hearing impaired

Researchers, including Dhanjal et al. [9], Mpofu et al. [10], Pamela Luft [11], Harris et al. [12], the BDA (British Deaf Association) [13], and the WHO (World Health Organization) [14], have extensively documented the multifaceted challenges confronting the hard-of-hearing community. These challenges span several dimensions of their daily experiences:

Public announcements and navigation: A primary concern for this community is the inability to perceive essential public announcements at transportation hubs such as railway stations, bus stops, and airports [9]. This limitation considerably impedes their navigation and journey-planning capabilities. While text-based communication offers a potential solution, it is not always effective, especially in low visibility or nocturnal scenarios, thereby restricting their interaction and ability to solicit assistance [9].

Road safety: The safety of deaf and hard-of-hearing individuals on roads is compromised due to their inability to detect auditory warning signals like car horns [15]. This auditory limitation heightens their vulnerability to potential accidents and hazardous situations. Communication barriers further compound these challenges, often necessitating the intervention of interpreters to facilitate communication between deaf and hearing individuals [15]. However, there remains a pressing need to enhance the availability and proficiency of trained interpreters to ensure effective communication support [16].

Employment challenges: The deaf and hard-of-hearing community faces significant obstacles in securing employment opportunities and navigating job interviews, adversely impacting their financial stability and professional advancement prospects [9]. Pamela Luft [11] underscores the employment-related challenges confronting deaf individuals, noting their elevated unemployment and underemployment rates. Consequently, they often experience reduced lifetime earnings compared to their hearing peers.

Entertainment accessibility: The entertainment industry, encompassing public theaters and cinemas, must make concerted efforts to cater to the specific needs of the hearing-impaired audience [9].

Technological implementations in public establishments: There is a discernible lack of technological adaptations, such as Teletypewriters, interpreters, and visual alert systems, in public establishments like restaurants, banks, hospitals, and shopping centers. This deficiency hampers effective communication and overall accessibility for the deaf and hard-of-hearing community [9, 12, 13].

Besides, the systematic literature review by Peñeñory et al. [17] analyzes 13 relevant studies covering fundamental and perceptual motor skills. The study examines technology-based interventions to improve psychomotor skills in hearing-impaired children, including interactive floors and virtual reality devices, targeting challenges related to posture, coordination, balance, spatial awareness, timing, and rhythm perception. The review recognizes limitations and the need for additional research. Moreover, it emphasizes the significance of customized approaches in utilizing technology to aid the psychomotor development of hearing-impaired children. Considering these challenges, comprehensive assistive technologies tailored to the distinct needs of the hearing-impaired community must be developed and implemented.

2.2 Conceptualizing quality of life: a multidimensional approach

While the discourse surrounding quality of life is not new, scholars like Verdugo et al. [3, 4] have provided a nuanced understanding of the term. They conceptualize quality of life as both a "process and an organizing principle aimed at enhancing the lives of individuals with disabilities and evaluating the outcomes and social validity of prevailing rehabilitation practices."

Verdugo et al. propose a comprehensive model anchored in eight pivotal dimensions collectively contributing to an individual's quality of life. These dimensions encompass emotional well-being, interpersonal relationships, personal development, material well-being, physical well-being, self-determination, social inclusion, and rights. While universally applicable, each dimension holds distinct significance and potential for enhancement, irrespective of an individual's disability status or the extent of support required.

The subsequent table (Table 1) delineates each dimension and elucidates its inherent importance.

Table 1 Dimensions of Quality-of-Life Framework by Verdugo et al. [3]

Our research aims to expand the applicability of the quality-of-life framework beyond its initial focus on individuals with intellectual disabilities. By including individuals with all major types of disabilities, for that purpose, we based on the Web Accessibility Initiative (WAI) W3C classification [18], which defines the following types: auditory, cognitive, physical, speech, and visual. Our proposal is based on integrating Internet of Things (IoT) technologies, potentially enhancing the quality of life for individuals with disabilities.

2.3 State of the art in assistive technologies for persons with hearing impairment

This section focuses on how these technologies have played an important role in meeting the needs of persons with hearing loss, enabling them to maintain their independence and improve their overall functioning. Despite significant advances, it is recognized that the solutions offered do not directly address the improvement of all aspects of quality of life. The 14 proposals for assistive technologies for persons with hearing loss are analyzed next, followed by a table showing which dimensions of the Quality of Life model [3] are addressed.

Nakul Nagpal et al. [19] introduce an intelligent communication module using image-processing techniques to translate deaf individuals' signs into text. The system comprises functional blocks like image matching, noise removal, and gesture recognition. It utilizes background and informative images to identify hand gestures, focusing on real-time motion recognition. The output block displays results as text, aiding comprehension for those unable to speak or hear. Moreover, it fosters interpersonal connections by bridging communication gaps and empowers individuals with hearing and speech impairments to express themselves independently.

Gumay et al. [20] developed a panic button app for individuals with hearing impairments, focusing on visual communication and simple designs for ease of use. They employed a conceptual model and wireframes, using the USE Questionnaire method to evaluate the app. The study cited previous research emphasizing the importance of audio, video, animation, and text in such apps. Addressing self-determination and social inclusion dimensions, the work aims to enhance accessibility during emergencies, empower individuals to engage in response efforts and foster community inclusion.

Nugroho et al. [21] developed a mobile application using the Ionic framework to provide an alternative means of digital learning for persons who are deaf or hard of hearing. This application enables teachers and parents to help deaf students recognize words and object shapes, making learning more engaging and interactive. The authors used a prototype model consisting of stages such as Communication, Quick Plan, Modelling Quick Design, Construction of Prototype, and Deployment Delivery and Feedback. By helping deaf students learn word recognition and object shapes, the application directly addresses the personal development dimension of quality of life for persons with hearing disabilities. It enhances their communication skills and educational opportunities, thus contributing to their personal growth.

Mohammdi et al. [22] introduced a system for translating Arabic Sign Language (ArSL) to facilitate communication between persons who are deaf or hard of hearing and the general public. The system enables speech-to-ArSL and ArSL-to-speech translation and outlines its architectural design. It addresses speech recognition and tokenization challenges while presenting sign images or sequences with text values as output. The educational application facilitates ArSL learning in academic settings, aiming to bridge communication gaps between deaf and hearing individuals and enhance interpersonal relationships.

Agrawal et al. [23] developed a sign language recognition system using CNN to interpret hand gestures from a live stream captured by a webcam. They modified and augmented the Sign Language MNIST dataset from Kaggle for improved variety. Two models were trained: one without a base model and the other using transfer learning with Inception V3. Achieving high accuracy, the system recognizes 13 letters of the sign language alphabet in real-time, addressing the dimension of interpersonal relationships in the quality-of-life model by Schalock and Verdugo. This facilitates communication between deaf and mute individuals and others, enhancing their interpersonal relationships.

Samonte et al. [24] present the development of BridgeApp, an offline mobile application that enhances everyday communication for individuals with diverse communication abilities. The app incorporates features like text-to-speech, Speech-to-Text, Text-to-Image, Haptic Feedback, and American Sign Language (ASL) and Filipino Sign Language (FSL) to cater to the needs of persons who are deaf, hard of hearing, mute, and persons without disabilities. The study encompasses user acceptance testing and covers essential FSL signs, validating the app's usability. Additionally, the article highlights the prevalence of disabilities, hearing loss, and assistive technology. The app directly improves the quality of users' interpersonal relationships by facilitating communication and interaction.

Chuckun et al. [25] developed the mobile application HandyApps to improve the lives of individuals who are visual, speech, or deaf and hard of hearing. This application incorporates text and face recognition, sign language, speech-to-text, and text-to-speech capabilities, following a comprehensive analysis of existing mobile applications for the visually and hearing impaired. The app promotes material well-being by fostering independence and efficiency in daily activities, offering functionalities like text, object recognition, and face detection. Additionally, its features, including detecting alarms, doorbells, and sounds, enhance the physical well-being of hearing-impaired individuals by ensuring safety and facilitating appropriate responses to their environment.

Syahriza et al. [26] discuss the development of a sign language interpreter tool emphasizing the use of Kansei Engineering. They propose using the Data Glove concept for accurate finger movement detection, addressing the limitations of existing interpreters. The hardware design includes an Arduino master, flex sensor units, DFplayer Mini, LCD, and a Buzzer Speaker. This innovative approach aims to accurately detect finger movements for sign language interpretation, promoting personal development and social inclusion. By bridging communication gaps, the tool enhances interaction, reduces barriers, and enriches socialization opportunities for deaf and mute individuals.

Reza Naqvi et al. [27] highlight the challenges deaf users face with audio and video interfaces, stressing affordable, accessible designs. They emphasize user feedback for developer improvement and adherence to sign language rules. Design principles like learnability and satisfaction aim to enhance communication and social interaction skills, aiding deaf individuals in overcoming barriers.

Alireza Darvishy [28] led a special thematic session on IoT applications for persons with disabilities and the elderly, showcasing innovative solutions and successful case studies. The session featured a brain-computer interface (BCI) driven domotic control system, BUZZBAND vibrating wristband for sound alerts, alternative input methods for hands-free interaction in augmented reality, and Ontenna, a tactile feedback device for hearing-impaired individuals. These technologies target improving personal development, independence, and engagement in daily activities for their users.

Jindal et al. [29] propose a method to convert sign language to text, utilizing MATLAB and AlexNet to identify seven movements with 70% accuracy. They also introduce a Python-based Convolutional Neural Network (CNN) model recognizing 20 actions with 98% training accuracy and 94% testing accuracy. Both models use real-time webcam image capture and pre-processing techniques, including thresholding, segmentation, and edge detection. Feature extraction involves contour, angle, color, and histogram methods, with MATLAB employing k-means clustering and Python utilizing skin color filtering and canny edge detection. Morphological operations like dilation, erosion, and Gaussian smoothing are employed to remove noise. This research directly addresses the communicative aspect of quality of life for persons with hearing/speech impairments, aiming to improve the communication capabilities of deaf and dumb persons.

Mubin et al. [30] proposed a framework for enhancing the learning experiences of deaf and mute children using Extended Reality (XR) technology. The framework involves three phases: Analysis, Design Development, and Evaluation. It gathers learners' concerns, utilizes XR components like VR and AR, and evaluates effectiveness. The aim is to optimize capabilities in a classroom by creating an immersive, user-friendly virtual environment. The research directly targets personal development, aiming to enhance learning experiences for the target group through customizable XR technologies.

Cano et al. [31] investigate Internet of Tangible Things (IoTT) integration with Tangible User Interfaces (TUIs) for children with hearing impairments, assessing its impact on social, emotional, cognitive, and visual skill development. The article examines how IoTT can foster the development of crucial social, emotional, cognitive, and visual skills through four case studies involving children with cochlear implants and those using sign language. Each case study delves into specific IoTT applications, from cognitive rehabilitation mini-games to interactive toys and educational tools. The study underscores considering social interaction and emotional feedback in IoTT design, highlighting IoTT's educational benefits, including personalized learning and real-time progress tracking.

Although some works do not directly address specific dimensions of quality of life as the Schalock and Verdugo model proposes, they indirectly address some dimensions. Firstly, in terms of personal development, the researchers seek to create a technology that bridges the language gap, potentially opening avenues for personal growth within the deaf community. Secondly, in terms of empowerment, the proposed technologies aim to reduce the reliance on intermediaries for communication, thus enabling hearing-impaired persons to express their needs and wishes more independently. Finally, improved communication accessibility can improve interpersonal relationships between the deaf community and hearing individuals, fostering better understanding, empathy, and social inclusion.

Regarding the dimensions of quality of life outlined in Verdugo's model, we can analyze the above works as follows. To better understand the specific dimensions targeted and indirectly impacted by each work, we categorized them as directly addressed, indirectly addressed, or not addressed. Table 2 summarizes the study results based on their direct and indirect impact or lack thereof. The table highlights how MoSIoT fills the gaps left by other proposals and offers significant advances and improvements in addressing the quality of life of individuals with hearing loss. While the studies above have made valuable contributions to the field, our work surpasses them in terms of comprehensiveness and coverage of different dimensions of QoL. The MoSIoT framework goes beyond this by integrating specific health monitoring systems that address the physical well-being dimension of QoL. It includes features such as tracking vital signs, monitoring medication adherence, providing alerts for medical emergencies, and empowering persons with hearing loss to take control of their health and well-being. This comprehensive approach ensures that persons with hearing loss access comprehensive healthcare support. Sections three and four of the present paper discuss how our system enhances interaction and improves accessibility for persons with hearing loss. By offering a complete solution, we address multiple dimensions of quality of life and provide a more holistic and enriched experience for persons with hearing loss. In addition, our work goes beyond the scope of a single application or system. We present a simulation framework for IoT systems that highlights the potential of IoT technologies to positively impact the lives of persons with hearing loss. This framework integrates and manages different assistive technologies, facilitating personalized and tailored solutions for individuals with specific needs.

Table 2 Comparison of Research Studies Based on Coverage of Quality-of-Life Dimensions

In summary, our work goes beyond the existing studies discussed in the previous findings by providing a comprehensive and integrated approach to addressing the quality of life of individuals with hearing loss. Our complete MoSIoT system incorporates specific health monitoring systems, improves interaction, and enhances accessibility, thereby covering multiple dimensions of quality of life in more detail. This is further discussed in section four of our article.

3 A MoSIoT overview

MoSIoT is a framework that focuses on the representation of different scenarios faced by an IoT Health Monitoring System (HMS) for persons with disabilities. It uses a Model Driven Engineering (MDE) approach to generate a data-intensive simulator, allowing it to inspect the complex behavior of the IoT HMS under different hypotheses or scenarios. The framework comprises a Domain Model, where a domain expert specifies a knowledge base that gathers invariant information from recognized standards, and a Scenario Model allows a medical expert to introduce the static and dynamic behavior of specific entities for a given scenario. These interconnected models allow the invariant concepts of the domain model to feed the variable entities defined in each scenario. In addition, MoSIoT incorporates a Trigger-Action Programming paradigm, allowing the creation of rules and recipes that further enhance the accessibility and usability of the HMS IoT for individuals with disabilities.

The MoSIoT framework comprises several packages within the Domain Model: (1) The Patient profile package takes center stage in addressing the unique requirements of individuals with disabilities. It draws inspiration from accessibility standards such as AccessForAll (AFA) [7] to ensure that users' accessibility needs and preferences are adequately represented. This package encompasses access modes and diverse adaptations tailored to different disabilities. The Patient profile also gathers detailed information such as preferred language, region, conditions, and disabilities, which define the person's Access Mode elements within the IoT system. This customization ensures precise communication methods and language preferences. For example, devices can adapt to various cultural norms, gender-specific choices, and age-related usability issues, utilizing features like voice interpreters and smartwatches. During the Scenario Model definition phase, domain experts collaborate directly with individuals with disabilities, such as social workers or healthcare professionals, tailoring IoT scenarios to their specific needs. This includes considering cultural nuances, gender identities, and age-related factors to customize IoT devices and services accordingly. For instance, a Spanish-born individual who is deaf might use sign language with the Spanish version, while a young American woman with acquired deafness might prefer textual language in American English; (2) The Device package focuses on defining IoT device templates that are essential to the operation of the framework. These templates describe the characteristics of the devices, including behaviors, commands, properties, and telemetry. The idea is to have a standardized representation of devices used within the health monitoring system. This package considers established standards such as the W3C Web of Things to ensure interoperability and adoption of standard practices. In practice, this means that a domain expert can configure and define new devices within the system, ensuring they are tailored to specific patient needs and align with industry best practices and standards; (3) The Healthcare package within the MoSIoT framework focuses on providing comprehensive care for persons with disabilities. This package leverages several health information standards, including ISO13606, OpenEHR, and HL7 CDA, with a particular emphasis on the HL7 Fast Healthcare Interoperability Resources (FHIR) [8]. Using the FHIR-based CarePlanTemplate, healthcare providers can define clinical or administrative activities, set goals, and design communication strategies tailored to the specific needs of patients with disabilities. The idea is to have a standardized and personalized approach to care, ensuring that interventions and treatments are appropriate and effective for each individual. This means that care plans can seamlessly integrate with health information systems, ensuring efficient coordination and patient-centered care.

One of the most important components of MoSIoT is the Scenario Model. This multifaceted tool simulates various situations in IoT Health Monitoring Systems (HMS) and communication environments involving SAAC (Augmentative and Alternative Communication Systems). This simulation capability is essential to anticipate and adapt interventions in various scenarios. For example, it is possible to simulate how a deaf person uses an SAAC to interact and communicate in a building, using augmented reality tools that provide visual and contextual information. In addition, the intuitive design and high level of abstraction of the Scenario Model make it easy for medical experts and professionals, such as social workers and educators working with persons with disabilities, to define and customize these scenarios. By raising the level of abstraction, MoSIoT ensures that these professionals focus on patient's needs and well-being according to their cultural norms, gender dynamics, and age-related preferences. At the same time, the system manages the technical complexities, offering tailored and personalized solutions for each situation.

Generally, MoSIoT stores the expert's knowledge within its domain model. Within this model, a feedback loop and initial expert insights lead to creating various templates. These templates encompass the User profile, which archives information from past patients; the Device Template, housing data on previously defined devices; and the CarePlan Template, which catalogs care plans for individuals with disabilities and other conditions, including their activities. This framework enables experts to utilize previous information when defining new scenarios, streamlining the process by leveraging insights from different experts and past experiences.

The MoSIoT framework transforms IoT connectivity using an IoT Hub. The Hub connects IoT devices with cloud services as a centralized bridge while managing and securing communication. The Hub facilitates interoperability by abstracting protocols, enabling devices to communicate across diverse networks like MQTT, AMQP, and HTTP. Moreover, it ensures compatibility by converting messages into various formats for efficient data processing by cloud applications. Scalable and adaptable, the framework accommodates deployments of any scale, from thousands to millions of devices, while integrating with cloud services like data analytics and AI for comprehensive insights. Robust security measures, including per-device authentication and encrypted data transmission, safeguard against potential threats, ensuring the integrity and confidentiality of IoT data.

3.1 Applying MoSIoT to the quality-of-life dimensions

Based on our research, we establish that MoSIoT can improve four dimensions of Verdugo’s quality of life directly:

• 1. Personal development: We explore how integrating IoT technologies and a health monitoring system can empower individuals with disabilities to enhance their skills and knowledge [32] and overall personal growth. By providing accessible tools and resources, we aim to enable individuals to expand their capabilities and achieve their full potential. In this sense, emerging technologies such as virtual reality (VR) or augmented reality (AR) are tools with enormous potential in the field of education due to their motivational aspect and because they make teaching content possible that would otherwise be unfeasible. To further support their development, we propose using a personalized time-tracking system. This system would encourage consistent learning by tracking reading and writing practice hours; this tracking system would offer insights into their progress. We can better understand their personal development journey by recording and analyzing time spent on these activities – manually or automatically, for example, by smart pens, smartphones, or smart watches;

• 2. Physical well-being: indicators include health condition, nutrition, mobility, leisure, activities of daily living, and Health monitoring system. MoSIoT provides scenario modeling, where not only the wearable devices that collect health information are set up but also a care plan where the care activities that the person with a disability must perform are planned, be it nutrition, medication, communication with the doctor, or physical activities. For instance, we use smartwatches to measure sleep duration, enabling us to calculate and enhance the dimension of physical well-being;

• 3. Interpersonal relationships: By leveraging IoT solutions, we develop communication platforms and devices that facilitate seamless communication between individuals with hearing impairments, friends, family, and the broader community. In addition, we recognize the importance of positive relationships, including friendships, family bonds, and romantic partnerships, in promoting well-being and overall quality of life [33, 34]. At this point, MoSIoT prioritizes the implementation of a SAAC (Systems of Augmentative and Alternative Communication); SAACs [35] refer to the set of strategies, including technological devices, that promote the autonomy of individuals with communication disorders, enhance their social abilities, adaptive behavior, and family relationships. SAAC might include real-time translation capabilities, enabling seamless interactions between individuals with hearing impairments and their hearing friends, family, and the broader community. This fosters closer relationships, reduces social isolation, and promotes overall well-being. The MoSIoT platform is a channel for instant conversations, showing ongoing dialogues and counting message exchanges. With the total messages exchanged, we match this data with a set care plan target to get a proper measure. This measure helps us understand how deeply the person with a hearing disability interacts with persons, supporting our aim to assess interpersonal relationship levels;

• 4. Emotional well-being is another dimension we address through our research. There are two ways to help emotional well-being: by monitoring stress values and through health monitoring systems, which monitor blood pressure, pulse, and temperature, they can predict whether their stress level is altered. On the other hand, using recreational activities such as reading, games, or medical apps based on mobile phones or augmented reality will improve the emotional level of persons with disabilities [36, 37]. Our platform encompasses a range of care activities, all contributing to enhancing and improving emotional well-being by adding appropriate activities, which we monitor through smartwatches. Stabilizing these indicators signifies a tangible enhancement in the quality of life.

In the following, we indicate how the use of MoSIoT can indirectly help the other dimensions:

• 5. Social inclusion: By leveraging accessibility for persons with disabilities in IoT systems, we aim to break down barriers and promote equal participation in various social settings. We facilitate individual engagement in community activities through assistive technologies, enabling active involvement and fostering a sense of belonging [38, 39]. For example, it might facilitate participation in group activities by providing real-time sign language or speech-to-text interpretation, ensuring individuals can actively engage in discussions, meetings, and community events. Additionally, it could enable deaf individuals to access public announcements and information through IoT-connected devices, fostering a more inclusive and integrated environment.

• 6. Material well-being: The IoT-enabled smart home security systems with cameras, motion detectors, door/window sensors, real-time monitoring, and alerts can enhance safety [40]. Additionally, smart communication devices facilitate seamless participation in remote work, virtual meetings, and collaborative tasks, promoting employment opportunities. These systems also enable access to essential information and services through speech-to-text-equipped devices, enhancing accessibility to employment, finances, and rights-related resources. Furthermore, IoT systems facilitate supportive social and employment environments by enabling real-time sign language interpretation and enhancing interactions at work.

• 7. Self-determination: IoT systems offer a pathway to autonomy, choice-making, and personal control, allowing individuals with hearing impairments to express preferences with clarity through communication-enhancing devices, such as real-time translation-capable smart communication tools. Smart home automation furnishes environmental management, while wearable devices facilitate well-informed decisions. Tasks like reminders and routines become self-managed through IoT devices, and customizable settings in these systems harmonize with personal values. Meanwhile, platforms elevate social interactions and financial independence, and IoT-enabled event information fosters community engagement. This seamless integration emboldens individuals with hearing impairments, empowering them to assert agency, nurture independence, and align with their aspirations for self-determined growth.

4 A case study: improving the quality of life of a hearing-impaired person

Hearing impairment is a widespread disease affecting millions of persons worldwide [41]; it is very important to implement effective interventions to improve the quality of life of those affected. As explored in the study by Holman et al. [42], individuals with hearing impairment encounter various challenges that significantly impact their daily lives. One major issue is pervasive fatigue, which stems from various sources. The heightened mental effort required during conversations leads to cognitive fatigue, where individuals must concentrate intensely and focus on non-verbal cues to understand spoken words. Emotion-driven fatigue adds to the burden, as negative emotions like frustration, stress, and anxiety arise from the struggle to hear and actively participate in social interactions. Even without explicit verbal expressions, the necessity to take breaks and recover in social situations is evidence of the underlying mental fatigue experienced during conversations. Sleep disruption, linked mainly to tinnitus, contributes to ongoing tiredness. The perceived relationship between hearing impairment and fatigue varies among individuals, suggesting a complex interplay. Considering the intricate balance of these challenges, the study underscores the need for tailored strategies and support to enhance the well-being of individuals with hearing impairment.

In an extensive examination undertaken by Maila Trujillo [43], the focus was placed on assessing the quality of life of 50 persons with hearing disabilities. This in-depth study produced results concerning this demographic's perceived quality of life. Interestingly, the findings disclosed that 40% of the participants reported a notably low quality of life. Moreover, 34% of the subjects considered their quality of life less than satisfactory. Furthermore, 24% of them described their quality of life as acceptable, while only 2% reported experiencing a positive, good quality of life (Table 3).

Table 3 The quality of life of hearing-impaired persons on the GENCAT scale [44]

The aim is to detail which aspects of quality of life significantly impact the lives of deaf individuals with low levels of hearing. In the study of Maila Trujillo [43], the different dimensions of quality of life provided by the GENCAT scale [44] were measured. This scale facilitates greater objectivity in measuring users' quality of life. The results of the participants with disabilities are presented below. According to the study by Maila Trujillo [43], it is essential to recognize that the low quality of life experienced by individuals with hearing impairments is closely tied to factors such as inadequate research, the influence of coping strategies, the presence of limited social support, and the imperative need for heightened awareness and targeted interventions to address these complex challenges.

A simple way to visualize a person's quality of life is to use a radar chart (Fig. 1). This graph visually portrays the values extracted from different dimensions of the table, depicting a polygon with vertices aligned to the weights assigned to each dimension. It highlights areas characterized by varying low, medium, and high satisfaction levels across different quality-of-life dimensions.

Fig. 1

The Quality-of-Life Radar Chart of Hearing-Impaired Persons with Initiative Values

4.1 Applying MoSIoT for improving the quality of life of persons with hearing impairment

Following the use case proposed in this work, we apply the MoSIoT framework to model and obtain IoT scenarios for persons with hearing impairment. To do so, we first start by indicating the process that is followed from the introduction of the generic templates in the domain model through the definition of the simulated scenario and, finally, the real scenario. Next, we discuss one of the main contributions of this work, which is the introduction of the quality-of-life framework within MoSIoT itself, thus allowing well-defined objectives to be set for the improvement of different dimensions of quality of life thanks to the proposed IoT system.

We started with the MoSIoT process (see Fig. 2) to define the use case for the hearing impaired. On the left side of Fig. 2, we can observe the configurations previously made by the MoSIoT Admin. At this point, the admin introduces the concepts that can be used in different cases of persons with disabilities based on previous cases and the literature. This is the case of the Alexa voice interpreter device, used to activate or deactivate devices; we start from the fact that it introduces a screen to be able to send text-to-speech messages to it and that it is also in charge of transcribing the voice messages to text on that screen.

Fig. 2

MoSIoT Process Cycle with Admin, Domain Expert, and User, illustrating the progression from generic template introduction to real scenarios, incorporating personalized IoT solutions

Moving to the middle section, attention shifts to the Domain Expert, who has persistently pursued the completion of the system by integrating additional devices and fine-tuning the accessibility, care plan, and care activity configurations.

In addition to configuring the voice interpreter device for accessibility, the Domain Expert has introduced another device: the smartwatch. The access mode for the smartwatch is tailored to the user's preferences, including textual accessibility as an 'AdaptationRequest,' vibration as an 'AdaptationTypeRequired,' and synthesis as an 'AdaptationTypeDetail'.

Two care plans have also been implemented to address stress and fatigue. These plans include specific goals, such as walking for two hours daily and ensuring seven hours of sleep each night.

Regarding care activities, the Domain Expert relies on the appointment with the therapist, which the MoSIoT administrator created, but a different type of care activity has been added, focusing on medication, with a daily teaspoon of magnesium to promote well-being.

One of the main contributions of this work is the ability of MoSIoT to assess the quality of life of a person with or without a disability. To this end, we proceed to introduce in the MoSIoT domain model, specifically in the Healthcare package, the elements derived from Verdugo's framework [3, 4], i.e., the different dimensions (Physical Well-being, etc..); however, we will focus only on those that we have indicated in Sect. 3.1, which are directly improvable through the use of IoT technologies, i.e., Personal Development, Physical Well-being, Interpersonal relationship, and Emotional well-being. The remaining dimensions may benefit indirectly, as indicated above.

To apply the MoSIoT assessment of a person's quality of life, we rely on one of the most important domain model packages of the proposal, namely the Healthcare package, which contains the care plans from the FHIR standard. In them, on the one hand, the Domain expert introduces a care plan based on a series of activities, including physical activities, nutrition, medication, and communication. The care plan is linked to the person's condition, and in this condition, in addition to indicating the person's illness/s and disability/s, the specification of the dimensions of quality of life is introduced as a novelty. However, in order to establish a measurement of the improvement in the quality of life, we need a Goal, or the entity where the objective to be achieved with the care plan is specified, to be directly linked to the dimension or dimensions of quality of life to be achieved. This goal could be expressed as "improve physical activity by 40%", thus linking the goal to the physical well-being dimension of quality of life. On the other hand, the Target, the entity where the objective is specified in a measurable form, would indicate that the measure is to be linked to the telemetry "a number of steps," thus indicating an "increase by 40% the number of steps". Finally, such telemetry is linked to the smartwatch device within the IoT system.

4.2 Domain model for persons with hearing impairment

Before defining the scenarios for hearing-impaired patients, it is necessary to introduce generic concepts or templates in the MoSIoT domain model, their responsibilities, and relationships that apply to any case of a deaf person and are also independent of any technology used. As stated in [5], the MoSIoT domain model is divided into three distinct packages: (1) the patient profile package, which allows for the definition of the adaptation profiles of patients based on the types of disabilities and conditions; (2) the device package, which defines the device templates or types of devices used in these systems with their characteristics and the types of telemetries; (3) The healthcare package, which proposes different care plan templates with activities, goals, and communications for patients with specific conditions and disability types.

In this way, we will focus on the most significant elements of the hearing-impaired person’s case study. The first step is to enter the data corresponding to the patient profile package because it will collect the common characteristics of disability and accessibility that often occur with hearing-impaired persons.

In Fig. 3, the image is divided into three distinct sections. On the left side, the admin has provided patient profile details, including a general situation overview and the patient's preferred language and region. In the central portion of the image, Diabetes and Depression are introduced as diseases afflicting the patient, each accompanied by their respective descriptions. Finally, the right side of the image is dedicated to the classification of disabilities based on the Web Accessibility Initiative (WAI) W3C guidelines [18]. In this context, the admin has selected the "auditory" disability and described its characteristics. Additionally, the severity of this auditory disability has been specified as "severe."

Fig. 3

The domain model for Patient Profile detailing general essential patient information with specific details of diseases and disabilities

Although no standards to address disability for IoT currently exist, MoSIoT relies on initiatives such as the ISO/IEC 24751 AccessForAll (AFA) specification, which proposes a reference model that represents users’ personal accessibility needs and preferences and controls the presentation of the information.

Focusing on the accessibility side, the admin creates different configurations for different hearing levels or degrees of knowledge presented by these persons. For example, two traditional ways to access a device that emits some kind of sound, either a voice interpreter or alarm, are a speech-to-text transcription for deaf persons who have adequate reading skills or a real-time transcription to a sign language for those who only know that form of communication.

According to Caitlin McKeown et al. [45], the typical graduation level of deaf students from high school corresponds to a fourth-grade reading proficiency, with around 20% concluding their education at a second-grade reading level or below.

The main element of the Domain Model that defines how a person interacts with a device is the access mode. Within each AccessMode, provide the framework with general adaptation elements for any hearing-impaired person.

Figure 4 displays the three key elements of AccessMode values in the Detail access mode tab.

Fig. 4

The domain model Access mode configures preferred access modes for deaf individuals, including textual and sign language options, in real-time

In the left section of Fig. 4, we can observe the AdaptationRequest, configuring the preferred access mode for devices relying on auditory or voice interactions with deaf individuals. The admin introduces the Textual mode in this section, enabling users to receive text-based information instead of relying on auditory cues. This not only eliminates the reliance on auditory cues but also ensures inclusivity.

In the center section of Fig. 4, we find AdaptationTypeRequired, based on deaf accessibility. This element configures the required adaptation type. In this scenario, the admin assigns the required adaptation type as Sign language, allowing users to receive information both textually and through advanced sign language interpretation.

Finally, in the right section of Fig. 4, we have AdaptationTypeDetail. This element optimizes the settings for real-time interpretation within the Sign language adaptation, ensuring real-time communication for users with hearing disabilities.

By leveraging the AFA-based MoSIoT framework, the admin ensures that devices adequately meet the needs of deaf accessibility, enabling them to interact with the devices using text-based information complemented by real-time sign language interpretation.

The next step is to include the elements in the device package that allow the definition of IoT device templates for persons with hearing disabilities. Figure 5 shows that a version of Amazon Alexa is introduced that features a display on which a deaf person can visualize text information that is also provided by voice. After configuring the new device template and introducing its data (such as Amazon Alexa, etc.), the Simulator admin assigns the access mode to the device, which has previously been created. It allows the person to enter messages to the Alexa display in textual mode to give orders to the devices or receive information from them in real time by sign language.

Fig. 5

The domain model device views with Amazon Alexa device with a display for deaf individuals, enabling real-time interaction through text and sign language

The last element we are going to create is a care plan template for hearing hearing-impaired disability. The health package’s core element is the CarePlanTemplate, which is based on the HL7 FHIR CarePlan [8] to define all the elements (Activities, Communications, and Goals) that will enable the establishment of an IoT-assisted care plan. The CarePlanTemplate allows you to define a generic treatment plan for a specific disability; in this example, we define a care plan for persons with severe hearing impairment (see Fig. 6).

Fig. 6

A Care Plan template for severe hearing impairment detailing the treatment of severe deafness strategies, targets, and goals linked to the quality-of-life dimensions

As can be seen in Fig. 6, an audiological assessment is defined as detailing audiograms and specific hearing thresholds or levels of hearing loss.

For instance, Fig. 6 shows a care plan to enhance communication strategies training. The primary focus is integrating communication devices to facilitate interactions across diverse settings. This Goal element (see Fig. 6. b) is linked to the dimension of quality of life and interpersonal Relationships, and the objective is that if we start from 67% satisfaction, taking the average value, we want to increase satisfaction by 20%. This objective involves initiating at least one hour of daily conversation through the MoSIoT messenger. The care plan emphasizes utilizing the MoSIoT e-book feature as an additional goal, which is the way the MoSIoT framework can measure the time that persons communicate over the platform. Incorporating this resource encourages individuals to explore and engage with content in an accessible format, fostering greater understanding and knowledge acquisition.

Ultimately, the Care plan template also includes a set of care activities. Specifically, Fig. 7 presents a care activity of type appointment that establishes a set of appointments with a speech therapist to be scheduled twice monthly. It is located in the Hospital of San Vicente in Spain. With support from speech therapists, patients can receive personalized strategies and exercises that enable hearing-impaired persons to navigate diverse communication situations more effectively. In the following section, a scenario model will be defined, which allows the establishment of a specific IoT scenario for a person.

Fig. 7

Scheduler Simulator admin Care Activity view of appointments with speech therapists, who provide personalized strategies for effective communication for hearing-impaired individuals

4.3 A scenario model of a hearing-impaired person

Once the domain model contains the generic templates for hearing-impaired persons, the next step is to model the scenario model to define the IoT elements that address a person's specific case. In this case, it is the domain expert who will be a caregiver or medical expert with knowledge of hearing-impaired persons who know a patient with special needs, either through an interview or by observation, will be in charge of defining the scenario based on existing templates or defining them from scratch.

In Fig. 8, we present a data flow diagram that encompasses the IoT elements within the context of the persons with hearing disability scenario. The scenario includes the patient equipped with a smartwatch and a smartphone exhibiting accessible interfaces. The system also incorporates an e-book, equipped with an adaptive user interface, into the user's mobile device. This adaptation captures the user's duration with reading and writing materials, which we collect to generate telemetry. On the other hand, the voice assistant collects ambient audio data and relays it to the user’s mobile device. This not only aids the individual in comprehending their surroundings but also enables Domain experts to understand the percentage of social interaction between the patient and their environment (work, relatives, and friends). Various data streams encompass the quartet of devices-the e-book, the smartwatch, the voice assistant, and the Smartphone. These devices collaboratively generate multiple data, including Heart Rate Variability (HRV) and movement data, sleep-related metrics, and the count of read-and-write operations.

Fig. 8

Data flow diagram illustrating IoT elements in a scenario for persons with hearing disabilities, showcasing device interactions and data generation to enhance user experience and quality of life

The Domain Expert plays a crucial role in interpreting the data and providing insights into the patient's condition. This interdependent relationship between data generation and interpretation highlights the clinical significance of this framework. Furthermore, the diagram captures the dynamic interaction between the Domain Expert and the patient. Communication occurs through messages, serving as a bridge for exchanging information.

Within this scenario, we define the trigger-action receipt element dictating how their devices should interact based on specific events or conditions. For instance, they represent a rule: "When I wake up in the morning, activate the coffee machine, display the most important news visually, and deliver a message about the weather forecast through vibration." Similarly, they establish another rule: "When I leave home for work, deactivate all devices, secure the door, and send a smartphone message detailing the traffic situation." Utilizing MoSIoT, they simulate these rules to validate correctness and feasibility before deploying them onto their devices. By incorporating machine learning into MoSIoT, predictive capabilities can be harnessed to anticipate events in an IoT scenario catering to deaf individuals, considering their specific attributes and contexts. For instance, the system could forecast potential risks or emergencies impacting individuals with hearing disabilities and provide visual or vibrating alerts. It might also predict communication requirements and furnish sign language translation or interpretation tools. Moreover, artificial intelligence could predict entertainment or educational preferences and offer tailored content catering to their comprehension and accessibility needs. The seamless integration of these IoT devices into their daily life significantly elevates their overall quality of life. Effective communication, convenient information access, and assured safety become tangible benefits. These devices not only assist them in overcoming the challenges posed by hearing disabilities but also enrich their everyday experiences, enabling them to lead a more fulfilling life.

4.4 Results analysis of MoSIoT case study for hearing-impaired persons

In addition to instantiating all IoT elements, the scenario model has associated goals that are either derived from the care plan template or predefined by the domain expert. As indicated in Fig. 6, we can measure through the Target elements if, indeed, by the telemetry received, in this case, the hours dedicated to communication, the person is fulfilling a Goal associated with improving a dimension of the quality of life. For this purpose, the application updates the progress quantitatively and represents it in a quality-of-life assessment report. This evaluation report involves assigning weights to different dimensions of quality of life, which are then visualized through a radar chart. This visualization aids the Domain Expert in identifying the dimensions that have experienced positive changes and the areas that still require further enhancement.

In Fig. 9, an initial evaluation report primarily tailored to individuals with hearing disabilities has shown a limited overall impact on the quality of life due to its broad applicability.

Fig. 9

The initial quality-of-life evaluation report, primarily tailored to individuals with hearing disabilities, shows a limited overall impact on the quality of life

Finally, in Fig. 10, we can see the progress of a patient who has improved in several quality-of-life dimensions by following the activities established in the scenario model's care plan.

Fig. 10

The progress of a patient who has improved in several quality-of-life dimensions by following the activities established in the care plan of the scenario model

Table 4 compares the initial values of various quality-of-life dimensions and the improvements observed in a patient's condition following intervention. Each dimension is evaluated based on specific criteria, with higher values indicating better quality of life. The patient's value improvements represent the positive changes achieved in each dimension after the intervention.

Table 4 Comparison of Initial Quality of Life Values and Patient Value Improvements

Highlighted cases include the dimensions of interpersonal relationships, emotional well-being, and personal development, where notable improvements were observed. Interpersonal relationships are assessed through the message counter of the MoSIoT messenger. Emotional well-being is evaluated using biometric values such as HRV (Heart Rate Variability) and hours of sleep.

Furthermore, personal development is significantly improved, as evidenced by the increased reading hours suggested in the care plan.

4.5 MoSIoT architecture design for deafness case study scenario

Figure 11 illustrates the application of the MoSIoT framework, leveraging Model-Driven Engineering (MDE) principles, within an enterprise architecture that relies on web services. This figure, describes the critical components of this architecture and their roles in the system.

Fig. 11

MoSIoT Architecture Design for Deafness Case Study Scenario

In the center of the image, we have the core Web services, representing the heart of the MoSIoT architecture. These services have object-oriented business logic that controls the domain model's functionalities [46]. They also offer a secure REST façade for external access and ensure database persistence for storing and mapping domain-related data. To facilitate data management in the database, we employ an Object-Relational Mapping Framework (e.g., Hibernate). Each web server scenario encapsulates the system's behavior and includes a prediction module that feeds data to the IoT Hub.

To enhance the system's capabilities, we integrate Azure IoT Central, a potent IoT accelerator that provides a REST API for sending and retrieving IoT-related data. This integration facilitates remote communication and seamless access to the IoT system's information. To manage the backend implementation of the domain model efficiently and promote rapid updates, we apply the OOH4RIA [47] approach.

Moving to the left side of the image, we find the "Simulator or MoSIoT Admin," a web app component designed for managing information within the simulator, implemented according to the specifications outlined in Reference [48]. Developed using Angular, a Progressive Web Application framework, this tool enables the simulator admin to input generic information through provided forms, efficiently constructing the MoSIoT domain model.

We have a mobile app tailored to domain experts, which has been successfully implemented as described in Reference [49] and built using Ionic on the bottom left of the image. The app connects to the central core of the simulator and retrieves default information related to user profiles, connected devices, and care plan templates, specifically for Healthcare Monitoring Systems (HMS). Model-driven web Engineering (MDWE) transformations in adjusting entities within the IoT scenario, resulting in the dynamic conversion of the model into a MoSIoT simulator.

On the right side is the design and implementation of a mobile application intended for users with disabilities, which has been realized based on the details provided in Reference [50], specifically in this case study focused on persons with deafness impairment. The application allows users to interact effectively with the system, taking their disabilities into account. Additionally, for the HMS scenario targeting persons with deafness, users receive information about their latest medical conditions through caregivers and connected devices with variable accessibility solutions.

As part of the simulator's output, all the dynamic elements and data within a specific IoT scenario are injected into an IoT Cloud platform (such as Azure IoT Central) [51] using REST APIs.

To further enrich the system's capabilities, consider that integrating technologies such as Google Cloud Speech-to-Text [52], Google Cloud Text-to-Speech [53], Google MediaPipe library [54], and SignalR [55] can be highly beneficial to develop an IoT system specifically designed for persons with hearing impairments. These technologies open avenues for advanced functionalities.

Google Cloud Speech-to-Text provides real-time speech recognition capabilities, converting spoken language into written text. This allows individuals with hearing impairments to communicate through speech, which can be converted into text for further processing or display. Google Cloud Text-to-Speech can convert any sign language or text into audible speech. This service enables the system to relay messages from a person with hearing impairments to a third person by voice, providing an alternative channel for communication. The Google MediaPipe library offers tools and components for building perception-based applications, including computer vision capabilities. By utilizing MediaPipe, visual-based tasks such as sign language recognition can be implemented, allowing the interpretation and translation of sign language gestures into meaningful actions or textual representations. SignalR, a real-time web application framework, facilitates bi-directional communication between clients and servers. It can deliver real-time notifications, alerts, or updates to various devices or user interfaces within the IoT system. This ensures that users with hearing impairments receive timely information and stay informed about relevant events or changes.

5 Conclusion and future work

This work has elucidated a pioneering approach by simulating Internet of Things (IoT) systems, focusing on enhancing the quality of life for individuals with hearing impairments. The MoSIoT (Modeling Scenarios of the Internet of Things) system, evaluated and extended through the quality-of-life framework proposed by Schalock and Verdugo, has showcased the transformative potential of integrating and managing existing deafness technologies. The presented case study underscores the profound impact of assistive technologies and champions the imperative of inclusive design and technology-driven solutions in promoting accessibility and holistic well-being.

Significant improvements were observed in key quality-of-life dimensions, namely "Interpersonal Relationships," "Emotional Well-Being," and "Personal Development." These enhancements were attributed to the MoSIoT system’s ability to facilitate enriched social interactions, foster emotional stability, and contribute to the personal growth of individuals with hearing impairments. The system’s integration of advanced communication technologies and personalized interaction mechanisms played a pivotal role in breaking down barriers and fostering a sense of community and belonging among users. Furthermore, the emphasis on user-centric design and adaptive learning enhanced emotional well-being and personal development, thereby illuminating the transformative potential of IoT systems in enriching lives.

Our future work involves expanding the MoSIoT framework to cater to a broader range of disabilities, including visual, mobility, and cognitive impairments and their combinations, necessitating the integration of technologies tailored to address these specific needs. In the foreseeable future, we plan to integrate auditory augmented reality (AAR) and artificial intelligence (AI) technologies into the MoSIoT framework to enhance individuals' visual impairments' interaction and navigation abilities. We are developing components like Object Detection, Scene Recognition, Object Measurement, and Augmented Reality Geolocation to deliver detailed auditory cues about the surroundings, bolster spatial awareness, and facilitate outdoor navigation. These technologies rely on AI vision capabilities for advanced image processing and analysis, enabling accurate recognition of objects and scenes.

We will also explore how AI technologies can be further utilized to develop innovative solutions for a wide range of disabilities. Additionally, leveraging the synergy between IoT systems and intelligent environments, such as smart homes or buildings, can significantly enhance the accessibility and usability of assistive technologies. By seamlessly integrating with these environments, assistive devices can offer improved communication, environmental control, and safety features.

Evaluations with real-world case studies are essential for understanding assistive technologies' efficacy and tangible benefits. Empirical studies can provide valuable insights into the effectiveness of IoT solutions in improving the quality of life for individuals with disabilities and help identify areas for refinement and improvement. Addressing the lack of standardization and interoperability among different assistive technologies remains a crucial challenge. Therefore, our future efforts should establish common standards, protocols, and interfaces to ensure seamless integration and compatibility between IoT devices and systems.

In conclusion, the advancements made in this work signify a notable progression in addressing the multifaceted needs of individuals with hearing impairments through the innovative application of IoT systems. The ongoing developments in assistive technology harbor the promise of significantly elevating the quality of life and fostering full societal participation for individuals with disabilities.