Keywords

1 Introduction

In recent years, e-learning has evolved by giant steps in terms of educational content, technological resources and possibilities of interaction. Today, many institutions add this form of learning as a complement to traditional teaching. Also, provides new possibilities for professors, students, and university staff through new forms of training and learning innovations [1]. Among them are educational content management, activities, assessment and communication between students and teachers [2].

When evaluating the quality of the teaching-learning process, usability and User eXperience (UX) is of great importance. But institutions, today, seek not only to evaluate the teaching-learning process, but also the institution as a whole, and that is where we focus on Customer Experience (CX).

ISO 9126-1 defines “Usability refers to the ability of software to be understood, learned, used and attractive to the user, under specific conditions of use” [3]. Meanwhile, Dilon A. defines the User Experience as “the sum of three levels: action, what the user does; result, what the user gets; and emotion, what the user feels” [4].

Evaluation methods should be able to identify the emotional, functional and rational factors that come into play in each interaction, in a specific cultural context of the service. The aim of CX is to enhance relationships with customers and build customer loyalty [5].

The objective of this document is to describe a systematic mapping study related to CX in e-Learning, in its form of interaction, and to identify the evaluation techniques of usability, UX, CX in this context. In this paper we present some conclusions about the state of art in this field and we hope to contribute to further research.

This document is organized by: Sect. 2 presents background. Section 3 describes the method applied to carry out this investigation. Section 4 presents the results. Section 5 presents the discussion of our results. Finally, Sect. 6 concludes the document.

2 Backgrounds

2.1 User eXperience and Usability: Their Relationship with e-Learning

User experience (UX) plays the role to identify the users feeling which can result in either positive or negative results. [6, 7] ISO 9241 standard defines how “user’s perceptions and responses that result from the use and/or anticipated use of a system, product or service”. [8], taking an entire view of a person’s emotions, attitudes, and expectations. Whereas, Usability is the facility, satisfaction with which they can use a product; [8] defined as “The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use”.

E-Learning is “A teaching and learning modality, which may represent all or part of the educational model in which it is applied, which exploits electronic means and devices to facilitate access, evolution and improvement of the quality of education and training” [9]. These concepts, Usability and UX, in e-Learning are very important, because if the system is not usable, it obstructs learning.

2.2 Customer eXperience

CX is increasingly discussed but not yet clearly defined. The Customer eXperience (CX) expresses the feelings customers have about the extent to which their experiences with a company have met their needs [10]. CX is considered as “the set of physical and emotional experiences that occur in the interactions with products and/or services offered by a certain brand and/or company from the first contact with the consumer, including the entire “journey” of the consumer, up to the final post-consumer stage” [11].

CX influences customers’ perceptions and opinions of value and service quality and therefore affects customer loyalty [12]. CX focusses on new dimensions of products and services, besides the traditional marketing.

Human Computer Interaction (HCI) and particularly User eXperience (UX) design and research focuses on the interactions that take place during the customer lifecycle. UX analyzes customer’s sensations resulting from any interaction with an interactive software system.

CX has a holistic approach, focusing on any interaction between the costumer and the company, in so-called “touchpoints” [13]. In each touchpoint CX should be assessed using both quantitative and qualitative methods; this is the only way to understand the “journey” of the costumer.

3 Research: Applied Method

A systematic mapping study is a defined method to build classifications in order to perform thematic analysis to obtain a visual map of the existing knowledge on a particular topic [14]. With the information obtained, it allows us to answer more specific research questions, following a strategy or protocol based on a systematic review process of scientific production. With the use of this tool, it also allows us to identify topics where there are sufficient primary studies to conduct systematic reviews and topics where more primary studies need to be generated. [15].

We have done a systematic mapping on CX in e-Learning, based on the method proposed by Petersen [16], following a procedure that includes 5 stages, A) Define research questions, B) Perform literature search, C) Select studies, D) Classify articles and E) Extract and aggregate data.

3.1 Researh Questions

We oriented the systematic mapping to understand that motivates the Customer eXperience in the e-learning domain, for that we defined three concrete research questions (see Table 1) to set the scope of the research, more detailed knowledge and a comprehensive view of the subject.

Table 1. Research questions defined in this work
Full size table

3.2 Data Source and Search Strategy

For the searches, we selected five different academic repositories: ACM Digital Library, Springer Link, ScienceDirect, IEEE Xplore Digital Library and Google Scholar to determine the main results of the application of this procedure, indicating the main findings obtained from this analysis. They are specialized in the field of Software Engineering and Human-Computer Interaction and in them we will find hundreds of scientific articles.

The search strings (see Table 2) were defined attending to the research questions proposed aiming to identify studies related to CX, UX, Usability and educational computing or approaches in general concerning Virtual Learning environment or e-learning.

Table 2. Search string
Full size table

These strings were applied to the specified search sources and the number of articles that were found was too high to be analyzed.

3.3 Select Studies

The search for articles for study selection we used the inclusion criteria stated in Table 3. And we also considered the exclusion criteria described in Table 4, for refining the list of primary studies with high contribution.

Table 3. Inclusion criteria for the Systematic Mapping Study
Full size table
Table 4. Exclusion criteria for the Systematic Mapping Study
Full size table

3.4 Classify Articles

The selection of the primary studies was done in two stages: First, applying the inclusion and exclusion criteria, reading the title, abstracts and keywords of the studies found in the search.

Then, a validation process was carried out by reading the full text of each of the studies selected in the first stage, among which there were studies included as well as those discarded from this mapping.

3.5 Extract and Aggregate Data

We developed data extraction sheets for each of the search strings considered in the study where the findings are recorded along with all the corresponding bibliographic information. Then, all the forms are joined in a single form by removing the duplicated articles.

For the analysis, tables and graphs with the collected data have been used as a basis for answering each research question. In the case of the graphs, the numbers presented are equivalent to the Quantity/Percentage of the selected studies.

4 Results

4.1 Study Selection

The search strings applied in all the specified sources gave a total of 1264 articles between the years 2015 and 2019.

The Table 5 presents the summary of the number of studies obtained as a result of applying the search chain in each library.

Table 5. Summary of primary studies obtained
Full size table

From the total of results in the searches of digital libraries, we found that 324 articles were duplicated, leaving 873 for analysis.

After applying the inclusion and exclusion criteria and when the results of the different search chains are combined, a total of 43 works are obtained that apply to the research topic (see Table 6). The results of the systematic mapping of the literature are presented as answers to each of the questions posed in the planning.

Table 6. Selected studies
Full size table

Figure 1 shows the distribution of the studies according to the year of publication. The number of studies published in 2016 doubled that of 2015, while in 2017, 2018 there were fewer than the year before. In 2019 they are higher than 2017 and 2018 but the same as in 2016.

Fig. 1.
figure 1

Distribution of primary studies by year.

Full size image

The Fig. 2 shows the distribution of primary studies by the type of forum in which they have been published. Table 7 shows the primary studies for each type of forum. The classification shows that 18% of them were published as conference or workshop papers; the 77% of them were more complete papers published in journals and 5% in Others.

Fig. 2.
figure 2

Distribution of primary studies by type of forum.

Full size image
Table 7. Distribution of primary studies by type of forum.
Full size table

4.2 What Is the CX in e-Learning?

Specifically two articles are related to CX and E-learning [A25] [A32] and another 8 are related to CX in education [A1] [A2] [A3] [A5] [A15] [A27] [A30] [A33].

We consider that the definition of CX in education is the total of the products, systems and/or services offered in the educational institution. The students pass a prolonged time in contact with institution, where CX involves customer participation at different levels, such as rational, emotional, sensory and physical [5, 13]. Customers expect to move seamlessly between each point of contact [17], that is, through the channels according to their preferences, needs and behavior. To do this, it is necessary to trace the customer’s journey through the different processes to determine which channels are involved and how they are involved [18].

The literature suggests that whatever the service, system or indeed product, a customer is buying or receiving, that customer will have an experience; good, bad or indifferent.

E-learning is the way to carry out distance education, distributing learning material and processes over the Internet [19]. Also, making data and tools available to users requires considering their different characteristics cultural background, technical experience, technological equipment, and physical/cognitive abilities, that will allow creating a more pleasurable experience in the user [20].

Actually, e-Learning, indicates online experience is affected by perceptual fluency [21]. Earlier barriers to education were limited access to study material and course content. High reliability of connectivity solves this problem as digital modes of education provide all the necessary information, where learning is accelerated. Becoming in one of the optimistic as well as a fulfilling experience of an individual’s life [17, 22, 23].

4.3 What Kinds of Studies Were Published About the CX, UX and Usability in e-Learning?

In terms of this research question, our aim was to investigate what types of studies were made (see Table 8) we analyze what kind of contexts.

Table 8. Type of research and methodology used
Full size table

4.4 What Evaluation Methods Are Used in These Studies?

Concerning this research question, we analyzed the included studies in terms of methods that are used to evaluated. (see Table 9). We found 39 studies that provide some method of evaluation.

Table 9. Evaluation methods
Full size table

5 Discussion

This systematic mapping aimed to identify the Customer eXperience in e-Learning. Also evaluation techniques of CX, UX and Usability in the context of education, specifically in e-learning.

The results showed that there are several studies regarding the techniques used to evaluate the usability and UX. However, there are still some gaps that can be explored in Customer eXperience, by further studies:

  • Few studies were conducted in the context customer experience. Among the studies that were found, none presented a new technique. Altogether, 7 contained evaluation.

  • Few techniques performs the usability/UX evaluation process in an automated way;

  • There were no studies relating the influence of the usability/UX improvement in the learning process.

These identified gaps may contribute to give an insight to further research in order to improve the quality of the evaluation techniques of Customer eXperience in e-Learning.

6 Conclusions

In this Systematic Mapping Study (SMS), we analyzed the publications regarding the CX in e-Learning, as well as evaluation techniques of CX, UX and Usability based on research published during the last 5 years and available on the relevant scientific databases. From a starting set of 1264 publications, a total of 43 publications were selected.

These researches were published mainly in journals, achieving 77% of the total.

Through the research questions it was possible to specify the relationship between the CX concept and e-Learning. Also what kind of studies were published, in which 58% use qualitative methods, 23% quantitative methods and 19% mixed methods. And what methods are being used to evaluate Usability, UX and CX, where we found 39 studies that provide some method.

This SMS revealed that there is still a need for more research in this area.

The literature indicates that there are several techniques to evaluate UX and Usability in e-learning, but there is no one technique that is more adequate in CX. This identified gap in SMS may be the starting point for further research.