Original Article
doi: https://doi.org/10.1590/1983-1447.2022.20220108.en
☑ Knowledge translation and advances in health and nursing practices
Revista Gaúcha
de Enfermagem
Educational hypermedia on embracement and
obstetric risk classification: content
validation and usability
Hipermídia educativa em acolhimento e classificação de risco
obstétrico: validação de conteúdo e usabilidade
Hipermedios educativos sobre la adopción y clasificación del riesgo
obstétrico: validación del contenido y la usabilidad
Francisco Mayron Morais Soaresa,b
Kirley Kethellen Batista Mesquitac
Igor Cordeiro Mendesd
Rodrigo da Silva Nunesb
João de Deus de Araújo Filhoa
Luciana Mara Monti Fonsecae
Gilson Vasconcelos Torresa
Francisco Arnoldo Nunes de Mirandaa
How to cite this article:
Soares FMM, Mesquita KKB, Nunes
RS, Araújo Filho JD, Fonseca LMM,
Torres GV, Miranda FAN. Educational
hypermedia on embracement and
obstetric risk classification: content
validation and usability. Rev Gaúcha
Enferm. 2022;43(spe):e20220108.
doi: https://doi.org/10.1590/19831447.2022.20220108.en
a
Universidade Federal do Rio Grande do Norte
(UFRN), Departamento de Enfermagem. Natal, Rio
Grande do Norte, Brasil.
b
Faculdade Uninta Itapipoca, Departamento de
Enfermagem. Itapipoca, Ceará, Brasil.
c
Universidade Federal do Ceará (UFC), Departamento
de Enfermagem. Fortaleza, Ceará, Brasil.
d
Universidade Federal do Maranhão (UFMA),
Departamento de Enfermagem. Pinheiro,
Maranhão, Brasil.
e
Universidade de São Paulo (USP), Escola de
Enfermagem de Ribeirão Preto. Ribeiro Preto, São
Paulo, Brasil.
ABSTRACT
Objective: To validate the content and usability of educational hypermedia about embracement and obstetric risk classification.
Methods: Methodological study, development by the basic instructional design model, conducted in five stages. Twenty-two judges
participated to validate the content and usability. For the analysis, the Content Validity Index, the System Usability Scale, and the
binomial test were used.
Results: In the content it was obtained a Content Validity Index of 0.96 and for and usability it was obtained 91.9. In the overall
evaluation, all requirements obtained an index of 0.98.
Conclusion: The educational hypermedia developed presents evidence of validity and constitutes an innovative resource for the
teaching and learning process in Nursing.
Keywords: Pregnancy complications. Education, nursing. Educational technology. Validation study.
RESUMO
Objetivo: Validar o conteúdo e usabilidade de hipermídia educativa sobre acolhimento e classificação de risco obstétrico.
Métodos: Estudo metodológico, desenvolvimento pelo modelo de design instrucional básico, realizado em cinco etapas. Participaram
22 juízes para validar conteúdo e usabilidade. Para a análise, foram utilizados o Índice de Validade de Conteúdo a System Usability
Scale e o teste binomial.
Resultados: No conteúdo obtiveram um Índice de Validade de Conteúdo de 0,96 e para e usabilidade obteve-se 91,9. Na avaliação
global, todos os requisitos obtiveram 0,98 de índice. Conclusão: A hipermídia educativa desenvolvida apresenta evidência de validade
e se constitui como recurso inovador para o processo de ensino e aprendizagem em Enfermagem.
Palavras-chave: Complicações na gravidez. Educação em enfermagem. Tecnologia educacional. Estudo de validação.
RESUMEN
Objetivo: Validar el contenido y la utilidad de la hipermedia educativa sobre el acolchado y la clasificación del riesgo obstétrico.
Métodos: Estudio metodológico, desarrollo del modelo de diseño instruccional básico, realizado en cinco etapas. Participaron 22
jueces para validar el contenido y la usabilidad. Para el análisis se utilizó el Índice de Validez de Contenido, la Escala de Usabilidad del
Sistema y la prueba binomial.
Resultados: En el contenido se obtuvo un Índice de Validez de Contenido de 0,96 y para la usabilidad se obtuvo un 91,9. En la
evaluación global, todos los requisitos obtuvieron un 0,98 de índice.
Conclusión: La hipermedia educativa desarrollada presenta evidencia de validez y se constituye como recurso innovador para el
proceso de enseñanza y aprendizaje en Enfermería.
Palabras clave: Complicaciones del embarazo. Educación en enfermería. Tecnología educacional. Estudio de validación.
Online Version Portuguese/English: www.scielo.br/rgenf
www.seer.ufrgs.br/revistagauchadeenfermagem
Rev Gaúcha Enferm. 2022;43(spe):e20220108
1
Soares FMM, Mesquita KKB, Nunes RS, Araújo Filho JD, Fonseca LMM, Torres GV, Miranda FAN
INTRODUCTION
During pregnancy, the woman undergoes several changes and adaptations inherent to the pregnancy state. This
peculiar event can result in unfavorable conditions resulting
from pre-existing clinical/obstetric conditions and require
emergency care(1). In scenarios of health complications related to pregnancy, it is important that clinical management
is conducted to avoid obstetric complications and improve
the quality of care(1,2).
The coming of clinical/obstetric changes requires that
the nursing professional use risk stratification based on severity criteria, with additional tests and services(2). Studies
demonstrate barriers, such as lack of specific knowledge,
which makes it difficult to implement this process and directly
imply the embracement and obstetric risk classification(3,4).
In Brazil, despite the decline in maternal mortality in
recent decades, rates remain at high levels, with an average
of 107 deaths per 100,000 live births in 2021(5). In this regard,
the Ministry of Health, from Stork Network (Rede Cegonha),
released the Manual for Embracement and Obstetric Risk
Classification (Manual de Acolhimento e Classificação de Risco
Obstétrico – ACRO), implemented in 2014 and updated in
2017, with the instrumentalization of access to the obstetric emergency service, with improvement of indicators of
maternal and perinatal morbidity and mortality(6).
The ACRO protocol is a tool that supports clinical decisions
in universal language for obstetric emergencies. Its purpose
is the prompt identification of critical cases and enabling fast
and safety care according to the potential risk, supported
by scientific evidence, besides to establishing a simple and
systematic analysis of life-threatening situations(1).
In this new epidemiological scenario, there is a need for
qualified professionals to apply the ACRO in obstetric emergency services in the light of the best scientific evidence.
Nurses are one of them, as, according to Decree/Law 94,046
of 1987, they can provide nursing care to pregnant women
with emergencies and delivery without dystocia(7).
Thus, it is important to provide nurses with education
that fosters an accurate clinical vision for comprehensive care
for pregnant women, in addition to emphasize the importance of periodic and updated training. Therefore, Nursing
education must have quality in the teaching and learning
process and in Nursing care in the global era.
Therefore, it is necessary to structure forms of education
supported by the results of scientific research together with
Information and Communication Technologies (ICT), from
2
Rev Gaúcha Enferm. 2022;43(spe):e20220108
the perspective of use of resources capable of assisting in
the educational and innovative scenario for the apprehension of content and motivation in learning, as the example
of hypermedia(8).
Hypermedia is an educational resource that allows interactive learning with the use of multiple media, dynamic
and/or static, with the example of hypertexts, images, figures, videos, clinical cases and links, presenting itself as an
enriching didactic strategy in the teaching-learning process,
which connect during learning(9). Among the hypermedia, it
stands out the virtual learning environment (VLE), a teaching modality mediated by digital technology in the health
area, evidenced as an effective approach and adding value
to student performance, as well as in perceptions of the
learning experience.
With the advance and use of ICT in nursing education,
the development and evidence of validity of educational
technologies for nursing education becomes inexorable as
an additional resource to assist in skills and competences
with the training of students and professionals to provide
effective and safe care(8).
Nevertheless, professors and students in Brazil had a
more active participation in the teaching of digital nursing, which made more active the participation and use
of technologies that interact virtually in the activities proposed in the curricular subjects. Nursing schools have been
using digital education for years both for courses and for
undergraduate subjects(10,11,12).
Associating care with educational actions aims to share
practices and knowledge in a horizontal relationship. Although relevant, when verifying the literature, there was a
reduced number of studies about educational technologies
in obstetrics. Similar studies validated educational hypermedia for teaching labor(13), a booklet to guide professionals in
conducting prenatal care and delivery(14), and validation of
an educational manual for birth companions(15) and a video
case study for safe delivery(16).
Thus, to fill gaps in the literature on educational technologies in embracement and obstetric risk classification, this
study contributes to the advancement of knowledge by
providing hypermedia with focusing on teaching students
and nursing professionals.
When considering the importance of developing educational technologies in the field of obstetric nursing and
the relevance of the theme, the study aimed to validate
the content and usability of educational hypermedia on
embracement and obstetric risk classification.
Educational hypermedia on embracement and obstetric risk classification: content validation and usability
METHOD
This is an applied and methodological research, that
involved the technologic production for teaching in nursing,
on the embracement and obstetric risk classification(8).
For the development and validation of educational hypermedia, it was used the classic instructional design model
ADDIE(17), an acronym for analysis, design, development,
implementation, and evaluation, presented in Figure 1.
In the stage of instructional design analysis, it was sought
to understand the educational gap and the possible solutions
to solve the study problem. To clarify the situational diagnosis,
it was performed a narrative review of the literature, whose
databases used to survey the content were the following:
Latin American and Caribbean Literature on Health Sciences
(LILACS), Medical Literature Analysis and Retrieval System
Online (MEDLINE) via PubMed, Cumulative Index to Nursing
and Allied Health Literature (CINHAL), Scopus and Cochrane,
as well as Ministry of Health websites that contained the
ACRO manual.
The following terms were used to survey published studies: Education in Nursing; Educational technology; Validation
Studies; Risk to Human Health; Nursing; Risk Classification;
Obstetrics; Obstetric Nursing; Pregnancy Complications.
Then, it was defined the planning of the modules addressed
in the hypermedia.
In the content survey, which expanded the understanding of embracement and obstetric risk classification and the
use of educational technologies, allowed to subsidize the
development of the teaching plan and learning objectives,
with digital resources that supported the formative matrix
for learning by the educational technologies.
The design consisted in the planning in which the basic
requirements for hypermedia were defined, the target audience, the workload needed for the educational action, the
learning objectives, the necessary resources, evaluation of
learning through the VLE and the references that provided
theoretical, methodological and applicability support(4,11,13,18).
In the development stage, learning objects and media
production were constructed. It was opted to use both static
(texts, images, flowcharts, books, and articles) and dynamic
(videos and website links) media. Criteria were established
for the selection of works in the public domain, with open
access, available in Portuguese, such as videos, images
and texts produced by the Research Group on Advanced
Practices and Nursing Technologies (Grupo de Pesquisa em
Práticas Avançadas e Tecnologias em Enfermagem – GEPATE)
in partnership with the Department of communication at
Faculdade Uninta Itapipoca.
The implementation of the ACRO contents was divided
into two stages, the first on the elaboration of the material for
the internet and the second, the availability of hypermedia
in a publicly hosted e-mail address. It is emphasized that
the materials made available were implemented by the
researchers and their scientific initiation scholarships from
April to June 2021.
Figure 1 – Instructional design phases that were covered and the actions performed in this study. Itapipoca, Ceará, Brazil, 2021
Source: Designed by the authors.
Rev Gaúcha Enferm. 2022;43(spe):e20220108
3
Soares FMM, Mesquita KKB, Nunes RS, Araújo Filho JD, Fonseca LMM, Torres GV, Miranda FAN
The evaluation stage of the study refers to the validity of
the educational material, the hypermedia from its analysis
by judges, its content, and its functionality. The literature
recommends that the validation process should be performed by a professional specialized in the subject area of
the material(10).
To calculate the number of experts, a non-probabilistic
and intentional sample was used. The calculation estimate
was based on the infinite population formula, with the statistical criteria in a minimum proportion of 85% agreement
with the pertinence of each item evaluated. It was assumed
a difference of 15% in this agreement. Thus, the sample size
was defined according to the following calculation: n= Za².
P.(1-P)/d², where Zα is the confidence coefficient (95% –
1,96), P is the proportion of individuals (85%) and d is the
difference to be detected(8,19).
As for the final calculation, it was determined by n=
1,962x085x0,1/0,152 and, therefore, the sample consisted of
22 experts(20). From these, eleven were experts in the area of
nursing (content validation) and, the same number, eleven
for the area of computing (usability validation).
The judges’ selection criteria were those used in a study(20)
about the impact on teaching, care and research activities
involving the following areas of obstetric nursing, emergency
and technology validation, based on the database of the
Directory of the National Council for Scientific and Technological Development (Conselho Nacional de Desenvolvimento
Científico e Tecnológico – CNPq).
It is emphasized that the judges were contacted via
e-mail, after contact, the expert professors were recruited,
who were also asked to indicate other professionals with an
eligible profile for participation (non-probabilistic sampling,
snowball type). After the indication, the Lattes Curriculum
was consulted to verify the judge’s suitability for the selection
criteria for this study.
Twenty-five (25) nurses and twenty-two (22) IT professionals were invited to participate in the study, of which sixteen
(16) nurses and thirteen (13) IT professionals responded, after
accepting the invitation, a link to the Google Forms form
was sent containing the Free and Informed Consent Form
and the validation instruments of the ACRO hypermedia. To
set the sample of twenty-two expert judges, the first eleven
of each group who answered the questionnaire within the
requested period of 30 days were considered.
During data collection, two instruments were used(13,15)
validated and available in the literature and of the Likert
type, ranging from 1 – inappropriate to 5 – appropriate,
which included overall impressions; goals; structure and
presentation, relevance, usability and efficiency, in addition
to a space for suggestions for improving hypermedia, besides
these, to evaluate the usability of hypermedia, it was used
4
Rev Gaúcha Enferm. 2022;43(spe):e20220108
the System Usability Scale (SUS), translated and validated
into Portuguese, which demonstrates acceptable levels of
internal consistency (Cronbach’s alpha 0.76)(21).
The data filled in the forms by the judges were tabulated
and statistically analyzed using the computer software, SPSS,
version 24.0. For analysis, the Content Validity Index (CVI) was
considered, which measures the proportion of judges who
agree on certain aspects of the instrument. The index score
for each item was calculated by adding the items marked as
4 or 5 (agree and strongly agree, respectively) by the judges,
divided by the total number of responses.
For the analysis of the global validation of educational
hypermedia, the sum of all CVI calculated independently was used, dividing them by the number of items of
the instrument(22).
Usability was calculated from the SUS scale scores from
the sum of the individual collaboration of each statement.
The odd items were subtracted one from the scale marked
by the user, however for the even items the score was five
minus the scale marked by the user. Next, the individual
scores of the participants were summed and the value was
multiplied by 2.5 to obtain the total usability scale (SUS Score)
with the satisfaction index ranging from 0 to 100 points(23).
A proportion of agreement greater than or equal to 0.80
was considered to determine validation. In order to verify
whether the proportion of agreement between the judges
was statistically equal to or greater than the value 0.80, the
binomial test was used, and the significance level adopted
was 5%(19,20).
The study was approved by the Research Ethics Committee
of Faculdade Uninta Itapipoca with a favorable opinion under
Protocol nº 5.042.351/2021 and CAAE 46117521.3.0000.8133.
The research was conducted in compliance with resolution
466/2012, following the required ethical standards, with a
previous request for the signing of the Free and Informed
Consent Form.
RESULTS
An educational hypermedia was designed, in the form of
a website, entitled: “Embracement and Obstetric Risk Classification”, or simply, ACRO, available at: www.crobstetrico.
online. The guidelines of the Revised Standards for Quality
Improvement Reporting Excellence (SQUIRE 2.0) were adopted, as it is a study aimed at improving the service.
The theoretical content developed for the educational
hypermedia/ACRO was organized into the following topics:
conceptual approach, Module I: Approach to the theme of
Embracement and Obstetric Risk Classification; Module II:
Emergency Obstetric Embracement; Module III: Obstetric
Risk Classification; Module IV: Decision Making Process in
Educational hypermedia on embracement and obstetric risk classification: content validation and usability
Embracement and Obstetric Risk Classification; Module V:
Classifying the Risk – Classification Exercises, References, Complementary Material (virtual library) and Authors’ presentation.
In the complementary material, there are submenus
that have the organization of the educational hypermedia
content, the checklist for the embracement and obstetric
risk classification, the support materials with links to the main
technical manuals related to the subject used nationally
and internationally. A forum is available for any questions
and doubts to be discussed asynchronously, in addition to
a communication resource such as a chat for synchronous
interaction between tutor-student and complementing a
glossary of terms and acronyms used in hypermedia.
The multimedia created and used in hypermedia (videos,
photos, forums, texts, among others) were displayed in each
module, as shown in Figure 2.
Each hypermedia module built has a submenu, which, in
addition to having content, performs a pre-test and a posttest to capture the users’ level of knowledge and thus enable
an evaluation and self-evaluation of the teaching-learning
process and operational functionality.
The evaluation of general learning at the end was focused on clinical cases related to the possible reality that
the student faces in their practices in obstetric emergencies.
It is believed that this type of evaluation will stimulate the
student’s clinical and critical reasoning for possible situations
in their daily practice.
Regarding the hypermedia content validation process,
the eleven expert judges with expertise in obstetric emergency had care experience, with management in obstetric
emergencies, where: one (9%) had a master’s degree and ten
(91.0%) had a doctoral degree in nursing and related areas,
of these, nine were professors in the health area.
According to Table 1, the agreement between the judges
was evidenced by p>0.05, ensuring a satisfactory sample proportion in agreement and reaffirming the content validation
of the judges about the topics: overall impressions, objectives,
structure and presentation, layout, usability, and efficiency.
It is emphasized that none of the items obtained the
CVI below the cut-off score. The result shows the excellent
internal consistency of the validation and, at the same time,
reaffirms the content validation of the educational hypermedia, called ACRO. Except for the only items (1.3 and 2.7),
in which there was slight disagreement. In the justification,
the judge reported that the item could only be evaluated if
there was an intervention study to measure behavior change
and learning.
Regarding the process of validating the usability of hypermedia, according to the data in Table 2, of the eleven expert
judges with expertise in IT had professional experience, in
addition to software and technology validation, where: one
(9%) was a doctor and ten (91 .0%) master’s in computing,
education or telecommunications related areas, of these, five
were professors of higher education in IT and technologies.
Figure 2 – Instructional design phases that were covered and the actions performed in this study. Itapipoca, Ceará, Brazil, 2022
Source: Designed by the authors.
Rev Gaúcha Enferm. 2022;43(spe):e20220108
5
Soares FMM, Mesquita KKB, Nunes RS, Araújo Filho JD, Fonseca LMM, Torres GV, Miranda FAN
Table 1 – Validation of educational hypermedia items by nursing professionals. Itapipoca, Ceará, Brazil, 2022
Evaluated Items
n (%)
CVI
p*
Overall impressions
11(100.0)
1.00
1.000
BL1 – It is easy to use.
11(100.0)
1.00
1.000
BL2 – It is didactic.
11(100.0)
1.00
1.000
BL3 – I recommend hypermedia for teaching the embracement
of obstetric risk to Nursing students and professionals.
11(100.0)
1.00
1.000
Objective
10.8 (8)
0.98
0.96
1.1 They are coherent with Nursing practice.
11(100.0)
1.00
1.000
1.2 They are coherent from the point of view of the
teaching-learning process.
11(100.0)
1.00
1.000
1.3 Promotes behavior change and develops critical thinking.
10(90.9)
0.90
0.832
1.4 It can circulate in the scientific environment, because the
information is correct.
11(100.0)
1.00
1.000
1.5 Meets the objectives of hospital institutions that work with
obstetric emergencies.
11(100.0)
1.00
1.000
Structure and presentation
10.8 (98)
0.98
0.97
2.1 The hypermedia script is appropriate for nursing students.
11(100.0)
1.00
1.000
2.2 Data are presented in an organized, clear, and
objective manner.
11(100.0)
1.00
1.000
2.3 The material is appropriate for the level of nursing students.
11(100.0)
1.00
1.000
2.4 It follows the logical sequence of the content proposed for
student learning.
11(100.0)
1.00
1.000
2.5 The way of presenting hypermedia contributes to
student learning
11(100.0)
1.00
1.000
2.6 The patient profile provides sufficient data to make a
clinical judgment
11(100.0)
1.00
1.000
2.7 The writing style corresponds to the level of knowledge of
the target audience
10(90.9)
0.90
0.832
6
Rev Gaúcha Enferm. 2022;43(spe):e20220108
Educational hypermedia on embracement and obstetric risk classification: content validation and usability
Table 1 – Cont.
Evaluated Items
n (%)
CVI
p*
Relevance
11(100.0)
1.00
1.000
5.1 The number and the characters meet the
proposed objective.
11(100.0)
1.00
1.000
5.2 The speech of the text is used efficiently and comprehensible
to the clientele.
11(100.0)
1.00
1.000
5.3 Visual resources are used appropriately.
11(100.0)
1.00
1.000
5.4 The characters are presented as visual resources and reflect
an approach to reality.
11(100.0)
1.00
1.000
Usability
11(100.0)
1.00
1.000
4.1 The concepts presented are understandable and easy
to apply.
11(100.0)
1.00
1.000
4.2 Provides guidance clearly.
11(100.0)
1.00
1.000
4.3 Provides guidance effectively.
11(100.0)
1.00
1.000
4.4 Provides guidance quickly and reading is not exhaustive.
11(100.0)
1.00
1.000
Efficiency
11(100.0)
1.00
1.000
5.1 The number and the characters meet the
proposed objective.
11(100.0)
1.00
1.000
5.2 The speech of the text is used efficiently and comprehensible
to the clientele.
11(100.0)
1.00
1.000
5.3 Visual resources are used appropriately.
11(100.0)
1.00
1.000
5.4 The characters are presented as visual resources and reflect
an approach to reality.
11(100.0)
1.00
1.000
Environment
11(100.0)
1.00
1.000
6.1 The website is appropriate for content presentation.
11(100.0)
1.00
1.000
6.2 Resources are appropriate for learning the subject.
11(100.0)
1.00
1.000
6.3 Resources provide learning situations.
11(100.0)
1.00
1.000
10.9
0.98
0.98
TOTAL
Source: Research data, 2022.
*p>0.05 Agreement Binomial Test; BL: Baseline; CVI: Content Validity Index
Rev Gaúcha Enferm. 2022;43(spe):e20220108
7
Soares FMM, Mesquita KKB, Nunes RS, Araújo Filho JD, Fonseca LMM, Torres GV, Miranda FAN
Table 2 – Validation of educational hypermedia items by IT professionals. Itapipoca, Ceará, Brazil, 2022
Evaluated Items
n (%)
CVI
p*
Overall impressions
11(100.0)
1.00
1.000
BL1 – It is easy to use.
11(100.0)
1.00
1.000
BL2 – It is didactic.
11(100.0)
1.00
1.000
BL3 – I recommend hypermedia for teaching the embracement of obstetric
risk to Nursing students and professionals.
11(100.0)
1.00
1.000
Functionalities
11(100.0)
1.00
1.000
1.1 Hypermedia presents as an appropriate tool for the proposal for which
it is intended.
11(100.0)
1.00
1.000
1.2 Hypermedia allows to generate positive results.
11(100.0)
1.00
1.000
Usability
11(100.0)
1.00
1.000
2.1 Hypermedia is easy to use.
11(100.0)
1.00
1.000
2.2 It is easy to learn the concepts used and their applications.
11(100.0)
1.00
1.000
2.3 Allows control of the activities presented in it, being easy to apply.
11(100.0)
1.00
1.000
2.4 Allows the user to easily apply the worked concepts.
11(100.0)
1.00
1.000
2.5 The way of presenting hypermedia contributes to student learning.
11(100.0)
1.00
1.000
2.6 Provides help clearly and completely.
11(100.0)
1.00
1.000
2.7 Provides help quickly, not being tiring.
11(100.0)
1.00
1.000
Efficiency
11(100.0)
1.00
1.000
5.1 The hypermedia design is appropriate for its use.
11(100.0)
1.00
1,000
5.2 The number of classes is consistent with the proposed time.
11(100.0)
1.00
1.000
5.3 The organization of thematic topics is appropriate for a good
understanding of the content, as well as the easy location of the
desired topic.
11(100.0)
1.00
1.000
5.4 Resources are used efficiently and understandably.
11(100.0)
1.00
1.000
Environment
11(100.0)
1.00
1.000
6.1 The website is appropriate for content presentation.
11(100.0)
1.00
1.000
6.2 Resources are appropriate for learning the subject.
11(100.0)
1.00
1.000
6.3 Resources provide learning situations.
11(100.0)
1.00
1.000
TOTAL
11(100.0)
1.00
1.000
Source: Research data, 2022.
p*>0.05 Agreement Binomial Test; BL: Baseline;| CVI: Content Validity Index
8
Rev Gaúcha Enferm. 2022;43(spe):e20220108
Educational hypermedia on embracement and obstetric risk classification: content validation and usability
The Overall CVI of educational hypermedia was 0.98,
therefore, the content and usability were validated with
expert judges in nursing and IT.
Finally, after the content validation process, it was performed the usability evaluation of the educational hypermedia according to the data in Table 3. Hypermedia was
considered easy to use and to learn the concepts used,
providing help in the teaching-learning process.
The evaluation score of the total SUS obtained from the
judges’ evaluation was 91.9, demonstrating excellent usability
of educational hypermedia in ACRO.
Table 3 – Distribution of the score of the professors’ answers to the items of the System Usability Scale (SUS). Itapipoca,
Ceará, Brazil, 2022
Usability Assessment Items
Mean
p*
1 – I think I would like to use this hypermedia often.
5.0
0.01
2 – I think the hypermedia is unnecessarily complex.
2.0
0.01
3 – I think the hypermedia system is easy to use.
5.0
0.01
4 – I think I would need help from a person with technical knowledge to
use hypermedia.
1.5
0.001
5 – I think that the various functions of hypermedia are very
well integrated.
4.0
0.003
6 – I think that hypermedia has a lot of inconsistency.
1.5
0.02
7 – I imagine people will learn how to use this hypermedia quickly.
5.0
0.007
8 – I think that the hypermedia is difficult to use.
1.0
0.01
9 – I felt confident using hypermedia.
5.0
10 – I had to learn a lot of new things before I could use hypermedia.
1.0
0.001
91.9
0.595
Total Usability Score
Source: Research data, 2022.
* Shapiro-Wilk data normality test.
DISCUSSION
The teaching hypermedia about ACRO reached content
validity and usability of all eligible nursing and IT judges and
obtained excellent levels with an overall CVI of 0.98 and a
result of the SUS score of 91.9.
The content judges acted as professors and doctors with
experience in validating at least one educational material.
While the usability ones acted as professors and programmers, which resulted in an accurate sample to validate the
proposed teaching hypermedia. These findings are similar
to other studies that validated the content and usability of
a teaching program for airway suctioning, with an expert
agreement of 0.89, therefore, almost perfect, and of a software for teaching diagnostic reasoning with a high validation
index 4-5/5(18,24).
The legislation that supports good practices in the obstetric context mentions the nurse and the team as responsible
for the ACRO process and the monitoring of the pregnant
woman. Within this context, it is inferred that this is the first
study that associated the theme of ACRO with an educational
technology, of hypermedia type, for the purpose of improve
the care provided.
Rev Gaúcha Enferm. 2022;43(spe):e20220108
9
Soares FMM, Mesquita KKB, Nunes RS, Araújo Filho JD, Fonseca LMM, Torres GV, Miranda FAN
It stands out the idea that teaching mediated by educational technologies represents an advance in producing
nursing knowledge and the empowerment of students
and professionals in interactive learning. Thus, the use of
technologies should not compete with or replace traditional
teaching but should be incorporated as a strategic resource
for education in nursing.
In the area of urgency and emergency in the obstetric
context, there is a need for teaching at the undergraduate
level, which brings great benefits for students and future
nursing professionals. The focus of using this hypermedia
is to enable and qualify them for effective and safe care,
which demands theoretical and practical training, making
them empowered to work in obstetric care(25).
It is highlighted the importance of an evaluation process
of different educational technologies that support and reinforce the use of clear, targeted, and concise information,
respecting ethical, moral and cybernetic principles, especially
fighting in the era of hyper information, against fake news.
Thus, studies that underwent the process of evaluating
educational hypermedia, with diverse themes, obtained
satisfactory indexes of evidence of validity. Examples are
one on childbirth care, one on prevention of bloodstream
infection and another on cervical cancer screening, validated through the Delphi technique, with rates of over 80%
of evaluation(26,27).
Validation research, such as the present study, favors the
proper selection of Learning objects and effective methods
to obtain satisfactory results in the teaching-learning process.
The development of educational hypermedia fosters thinking and clinical reasoning skills and is provided by learning
environments that ease the transfer of knowledge(9,10,16,23,24).
Hybrid teaching, with the reduction of rigid schedules,
direct orders and mandatory classroom teaching, make
the in-person moment better used when the theoretical is
synchronously taught through digital learning(9,24,28).
The purpose of content evaluation is to determine what
learners will acquire from the knowledge and techniques
taught through the available content, besides to verifying
the quality of the teaching provided. In view of this, the
content presented/available previously and concomitantly
in the VLE requires planning with clear, objective instructions,
in a logical and organized sequence, with a self-explanatory
and dynamic teaching method(18).
The teaching-learning process, promoter of skills and
competences, when combined with the use of digital technologies, helps the development of clinical competences
in a virtual environment. Based on the literature, learning
modeled through VLE already presents superior gains when
compared to the teaching of lectures/dialogued classes(18).
10
Rev Gaúcha Enferm. 2022;43(spe):e20220108
A study conducted in Spain that analyzed the effects
of interactivity of students with digital media revealed that
they prefer higher levels of interactivity and understand
that digital learning technologies provide better positive
subjective experiences(18). Such evidence prospect important
results that must be presented to educational institutions
to integrate their learning activities in virtual environments.
Corroborating this aspect, during the pandemic, Brazil, in
order to respond quickly and dynamically to the educational
condition imposed by COVID-19, implemented several forms
of hybrid learning, with synchronous and asynchronous
aspects(26). Thus, it is important to check that the strategy
present in this study, in addition to following a strict methodological standard of development, is suitable for the teaching-learning process, both to narrow the educational losses
due to the pandemic, as for strengthen learning mediated
by digital technologies.
The usability of a technology consists of verifying how the
product can be used by users to achieve the specific objectives of gaining knowledge, attractiveness and functionality
in its use, whether synchronously or asynchronously. In this
sense, usability studies propose to evaluate the development,
functioning and adequacy centered on the user to develop
better results of the increase of knowledge. This evaluative
issue becomes important as it verifies how viable hypermedia
will be in the teaching-learning context where the student
is inserted(18,29).
CONCLUSION
The educational hypermedia of embracement and obstetric risk classification achieved the proposed objective
and presents evidence of content validity and usability as a
teaching tool. The use of this material with nursing students
will ease the learning of ACRO, considering that it is an educational and interactive technology capable of favoring
the learning process.
As limitations of this study, it was observed the summarized number of studies on embracement and obstetric risk
classification. The absence of usability evaluation by the target
audience, in addition to the difficulty of obtaining timely
responses from the judges contacted for the validation of
the hypermedia.
This study presents benefits and advances in knowledge,
as it provides an educational hypermedia with free and
universal access. A digital and educational resource that can
be used in teaching and health institutions, as a permanent
education strategy. In addition, the results of the present
study may support the development of future research
aimed at evaluating the effects on gaining knowledge
Educational hypermedia on embracement and obstetric risk classification: content validation and usability
with the use of hypermedia for undergraduate nursing
and health students.
REFERENCES
1. Freitas VCA, Quirino GS, Giesta RP, Pinheiro AKB. Clinical and obstetric situation
of pregnant women who require prehospital emergency care. Rev Bras Enferm.
2020;73(Suppl 4):e20190058. doi: http://doi.org/10.1590/0034-7167-2019-0058.
2. Amorim TV, Souza IEO, Moura MAV, Queiroz ABA, Salimena AMO. Nursing
care perspectives in high-risk pregnancy: integrative review. Enferm Glob.
2017;16(2):530-43. doi: https://doi.org/10.6018/eglobal.16.2.238861.
3. Correia RA, Rodrigues ARM, Araújo PF, Monte AS. Análise do acolhimento com
classificação de risco em uma maternidade pública terciária de Fortaleza. Enferm
Foco. 2019;10(1):105-10. doi: https://doi.org/10.21675/2357-707X.2019.v10.
n1.1448.
4. Angadi NB, Kavi A, Shetty K, Hashilkar NK. Effectiveness of flipped classroom
as a teaching-learning method among undergraduate medical students –
An interventional study. J Educ Health Promot. 2019;8:211. doi: https://doi.
org/10.4103/jehp.jehp_163_19.
5. Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Departamento de
Análise Epidemiológica e Vigilância de Doenças Não Transmissíveis [Internet].
Painel de Monitoramento da Mortalidade Materna. Brasília, DF: Ministério da
Saúde; 2022 [cited 2022 May 05]. Available from: http://svs.aids.gov.br/dantps/
centrais-de-conteudos/paineis-de-monitoramento/mortalidade/materna/.
6. Ministério da Saúde (BR). Manual de acolhimento e classificação de risco em
obstetrícia [Internet]. Brasília, DF: Ministério da Saúde; 2014 [cited 2022 May
05]. Available from: https://bvsms.saude.gov.br/bvs/publicacoes/manual_
acolhimento_classificacao_risco_obstetricia_2017.pdf.
7. Ferreira FM, Venâncio KCMP, Narchi NZ. Network care: relationship
between prenatal care adequacy and hospital obstetric care in a crosssectional study. Rev Esc Enferm USP. 2022;56:e20220011. doi: https://doi.
org/10.1590/1980-220X-REEUSP-2022-0011en.
8. Hulley SB, Cummings SR, Browner WS, Grady DG, Newman TB. Delineando a
pesquisa clínica. 4. ed. Porto Alegre: Artmed; 2015.
9. Salvador PTCO, Rodrigues CCFM, Ferreira Júnior MA, Fernandes MID, Martins JCA,
Santos VEP. Construction of hypermedia to support the systematization of the
nursing care education. Rev Gaúcha Enferm. 2019;40:e20180035. doi: https://
doi.org/10.1590/1983-1447.2019.20180035.
10. Costa CC, Gomes LFS, Teles LMR, Mendes IC, Oriá MOB, Damascen AKC. Construction
and validation of an educational technology for the prevention of congenital
syphilis. Acta Paul Enferm. 2020;33:eAPE20190028. doi: https://doi.org/10.37689/
acta-ape/2020AO00286.
11. Sousa VEC, Lopes MVO, Keenan GM, Lopez KD. Developing and testing of a software
prototype to support diagnostic reasoning of nursing students. Int J Nurs Knowl.
2018;29(2):124-32. doi: https://doi.org/10.1111/2047-3095.12145.
12. Cordeiro ALPC, Braga FTMM, Mata LRF, Mendes KDS, Fófano RC, Dalri MCB.
Blended learning program for the development of skills in the aspiration of
artificial airways. Rev Latino Am Enfermagem. 2021;29:e3462. doi: http://doi.
org/10.1590/1518-8345.4539.3462.
13. Oliveira LL, Mendes IC, Balsells MMD, Bernardo EBR, Castro RCMB, Aquino PS, et
al. Educational hypermedia in nursing assistance at birth: building and validation
of content and appearance. Rev Bras Enferm. 2019;72(6):1471-8. doi: http://doi.
org/10.1590/0034-7167/2018-0163.
14. Santos ALM, Souza MHT. Elaboration of new technologies in nursing: use of a
prevention booklet. J Nurs UFPE on line. 2017;11(10):3893-8. doi: http://doi.
org/10.5205/reuol.12834-30982-1-SM.1110201725.
15. Teles LMR, Oliveira AS, Campos FC, Lima TM, Costa CC, Gomes LFS, et al.
Development and validating an educational booklet for childbirth companions.
Rev Esc Enferm USP. 2014;48(6):997-84. doi: http://doi.org/10.1590/
S0080-623420140000700003.
16. Costa ARC, Imoto AM, Gottems LBD. Videocase sobre a lista de verificação do
parto seguro: sensibilização dos profissionais da saúde. Enferm Foco. 2019 [citado
2022 maio 05];10(5):13-9. Disponível em: http://revista.cofen.gov.br/index.php/
enfermagem/article/view/2355/627.
17. Filatro A. Design instrucional na prática. São Paulo: Pearson; 2008.
18. Lopez KD, Chin CL, Azevedo RFL, Kaushik V, Roy B, Schuh W, et al. Electronic health
record usability and workload changes over time for provider and nursing staff
following transition to new EHR. Appl Ergon. 2021;93:103359. doi: https://doi.
org/10.1016/j.apergo.2021.103359.
19. Arango HG. Bioestatística teórica e computacional. 3. ed. Rio de Janeiro: Guanabara
Koogan; 2009.
20. Galindo Neto NM, Caetano JA, Barros LM, Silva TM, Vasconcelos EMR. First aid in
schools: construction and validation of an educational booklet for teachers. Acta Paul
Enferm. 2017;30(1):87-93. doi: http://doi.org/10.1590/1982-0194201700013.
21. Lourenço DF, Carmona EV, Lopes MHBM. Translation and cross-cultural adaptation
of the System Usability Scale to Brazilian Portuguese. Aquichan. 2022;22(2):e2228.
doi: https://doi.org/10.5294/aqui.2022.22.2.8.
22. Polit DF, Beck CT. The content validity index: are you sure you know what’s being
reported? critique and recommendations. Res Nurs Health. 2006;29(5):489-97.
doi: https://doi.org/10.1002/nur.20147.
23. Cavalcanti HGO, Bushatsky M, Barros MBSC, Melo CMCS, Delgado Filho
AJF. Evaluation of the usability of a mobile application in early detection of
pediatric cancer. Rev Gaúcha Enferm. 2021;42:e20190384. doi: https://doi.
org/10.1590/1983-1447.2021.20190384.
24. Costa IKF, Tibúrcio MP, Melo GSM, Leite JEL, Dantas RAN, Torres GV. Construction and
validation of a distance Basic Life Support Course. Rev Bras Enferm. 2018;71(Suppl
6):2698-705. doi: https://doi.org/10.1590/0034-7167-2018-0122.
25. Campos BL, Góes FGB, Silva LF, Silva ACSS, Silva MA, Silva LJ. Elaboração e
validação de vídeo educativo sobre o banho domiciliar do recém-nascido a
termo. Enferm Foco. 2021;12(5):1033-9. doi: https://doi.org/10.21675/2357707X.2021.v12.n5.4684.
26. Silva MMJ, Panobianco MS, Clapis MJ. Information and communication technologies
in postgraduate Nursing education during the COVID-19 pandemic. Rev Min Enferm.
2021;25:e-1368. doi: https://doi.org/10.5935/1415.2762.20210016.
27. Santos LA, Rodrigues NM, Stacciarini TSG, Cordeiro ALPC, Engel RH, Melo AF,
et al. Ensino híbrido em educação permanente em enfermagem: experiência
de um curso sobre prevenção de lesão por pressão. In: Castro LHA, Pereira TT,
Oesterreich SA, organizadores. Ciências da Saúde: campo promissor em pesquisa
6. Ponta Grossa, PR: Atena; 2020. p. 87-95. doi: https://doi.org/10.22533/
at.ed.88220110210.
28. Kavanagh JM, Szweda C. A crisis in competency: the strategic and ethical
imperative to assessing new graduate nurses’clinical reasoning. Nurs Educ Perspect.
2017;38(2):57-62. doi: https://doi.org/10.1097/01.NEP.0000000000000112.
29. Dias ICB, Lima EFA, Fioresi M, Carrara D, Oliveira IS, Primo CC. Construction
of hypermedia for prevention of bloodstream infection. Rev Bras Enferm.
2020;73(6):e20190593. doi: https://doi.org/10.1590/0034-7167-2019-0593.
Rev Gaúcha Enferm. 2022;43(spe):e20220108
11
Soares FMM, Mesquita KKB, Nunes RS, Araújo Filho JD, Fonseca LMM, Torres GV, Miranda FAN
Authorship contribution:
Project administration: Francisco Mayron Morais Soares,
Francisco Arnoldo Nunes de Miranda.
Formal analysis: Francisco Mayron Morais Soares,
Gilson Vasconcelos Torres, Francisco Arnoldo Nunes de
Miranda.
Conceptualization: Francisco Mayron Morais Soares,
Francisco Arnoldo Nunes de Miranda.
Data curation: Francisco Mayron Morais Soares,
Francisco Arnoldo Nunes de Miranda.
Writing-original draft: Francisco Mayron Morais
Soares, Igor Cordeiro Mendes, João de Deus de
Araújo Filho, Rodrigo da Silva Nunes, Kirley Kethellen
Batista Mesquita, Luciana Mara Monti Fonseca, Gilson
Vasconcelos Torres, Francisco Arnoldo Nunes de
Miranda.
Writing-review & editing: Francisco Mayron Morais
Soares, Luciana Mara Monti Fonseca, Gilson
Vasconcelos Torres, Francisco Arnoldo Nunes de
Miranda.
Investigation: Francisco Mayron Morais Soares, João
de Deus de Araújo Filho, Rodrigo da Silva Nunes, Kirley
Kethellen Batista Mesquita, Francisco Arnoldo Nunes
de Miranda.
Methodology: Francisco Mayron Morais Soares, Igor
Cordeiro Mendes, Kirley Kethellen Batista Mesquita,
Luciana Mara Monti Fonseca, Gilson Vasconcelos Torres,
Francisco Arnoldo Nunes de Miranda.
Software: Francisco Mayron Morais Soares, Gilson
Vasconcelos Torres, Francisco Arnoldo Nunes de
Miranda.
Supervision: Francisco Mayron Morais Soares, Francisco
Arnoldo Nunes de Miranda.
Validation: Francisco Mayron Morais Soares, Igor
Cordeiro Mendes, Luciana Mara Monti Fonseca, Gilson
Vasconcelos Torres, Francisco Arnoldo Nunes de
Miranda.
The authors declare that there is no conflict of interest.
Corresponding author:
Francisco Mayron Morais Soares
E-mail: mayronenfo@gmail.com
Associate editor:
Rosana Maffacciolli
Received: 05.09.2022
Approved: 09.02.2022
12
Rev Gaúcha Enferm. 2022;43(spe):e20220108
Editor-in-chief:
João Lucas Campos de Oliveira