Delone and McLean information systems success model in the

public sector: A systematic review

Elhoussin ELAZZAOUI
Phd student at the Faculty of Legal, Economic and Social Sciences Ain Sebaa, Hassan 2 University
Research Laboratory in Management, Information and Governance (LARMIG)
houssin11.azzaoui@gmail.com

Siham LAMARI
Professor at Faculty of Legal, Economic and Social Sciences Ain Sebaa, Hassan 2 University
Research Laboratory in Management, Information and Governance (LARMIG)
siham.lamari@gmail.com

ABSTRACT
Delone and McLean(D&M) had launched in 1992 one of the most popular models for evaluating the
success factors of information systems (IS), ten years later, they suggested an updated model that
incorporated certain elements that were not taken into the first model. In this paper, we have based the
search on several articles, which meet a set of criteria. In fact, we found that the model of Delone and
McLean has been adopted in different countries and contexts, yet in some cases, it is necessary to integrate
the variables of other models or theories in order to better understand the subject and fill its voids,
especially in the public sector and its specificities. The most frequently used variable in these papers was
"trust", this reflects the importance of this concept when studying information systems in the public sector.
Finally, we list the main items and dimensions used in these articles for each of the seven variables as
well as their specificities in the public sector.

K e y w o r d s : Delone and McLean information systems success model, system quality, information
quality, service quality, user satisfaction, trust.

Page 1 of 24
1. Introduction
With the appearance of the New Public Management (NPM) trend, governments and public
administrations have touched the need to modernize their working methods and techniques, as a
consequence of that, several countries have started to introduce private sector management methods to
the public sector. One of the radical elements which helps in this process of modernization is the
implantation of information systems. In order to understand the success of this process of implantation,
of use, of continue to use and of its impact on the performance and value of the organization, Delone and
McLean (1992) presented a model for evaluating the success of IS. Based on a review of the literature
covering the period 1981-1987 -totalling one hundred and eighty articles- and the works of Shannon and
Weaver (1949) and Mason (1978), they suggested a model that focuses on three levels, namely, technical
level (system quality), semantic level (information quality) and effectiveness or influence level (use, user
satisfaction, individual impact and organizational impact). Delone and McLean information systems
success model (1992) is presented in the figure 1.

System
quality Use

Individual organizational
impact impact
Information User
quality satisfaction

Figure 1. Delone and McLean (1992)

This model was considered as a reference in the area of information systems. Ten years later, Delone and
McLean (2003) launched an update taking into account the criticisms addressed to the first model, the
principal modifications were:
• An independent variable was added, which was “service quality”.
• The “use” variable was split into two variables, namely, “intention to use” and “use”.
• The third and last change is the aggregation of the “individual impact” and “organizational impact”
into one construct, namely “net benefits”.

System quality
Intention to
Use
use
Information quality Net benefits

User satisfaction
Service quality
quality
Figure 2. Delone and McLean (2003)

Page 2 of 24
Through this systematic review, we seek to understand the specificities and the particularities of Delone
and McLean model (2003) in the public sector, by studying and analysing forty articles covering the period
between 2007-2019. We will answer the following questions:
1. What are the theories and models mobilized in parallel with the IS success theory of Delone and
McLean?
2. What are the main variables that are specific to the public sector that were not taken into account
by the model of D&M (2003)?

3. What are the different dimensions of the seven variables of the 2003 model? And which the main
items are used to measure its variables?
4. What do the seven variables refer to in the public sector?

2. Research Methodology
1.1. Data collection
In this paper, data collection was based on four main phases: identification stage, screening stage,
eligibility stage and the inclusion stage.

Identification stage

We conducted a search through Scopus, Web of Science and JSTOR databases, the parameter used was,
the different combinations of the following key words: Delone and McLean model, e-government, public
sector, public value, government, system quality, information quality, service quality, use, intention to
use, user satisfaction, net benefits, information systems success model.

Screening stage

In this stage, duplicate articles were removed, as well as articles written in languages other than English.

Eligibility stage

Based on the keywords, the abstracts and review ranking offered by the SCImago Journal Rank website,
we defined the included and excluded criteria as shown in the table below:

Table 1: eligibility and illegibility criteria

eligible criteria illegible criteria
- Review ranking: Q1,Q2,Q3,Q4. - Private sector domain paper.
- Covering 2007 – 2019. - Non integration of at least one or more
- Article include one or more variables of the variables of the Delone and McLean
Delone and McLean model (2003). model(2003).

Inclusion stage

Finally, we excluded Book chapters, conference papers, literature reviews, systematic and meta-analysis
review, and only empirical studies were included.

Page 3 of 24
Identification

Articles identified through database searching (N = 940)

Duplicate articles removed (N = 640)

Articles after duplicates removed (N = 300)

Screening

Articles written in languages other than

English removed (N = 60)

Total articles (N = 240

Eligibility

Illegible criteria (N = 173)

Total eligible articles (N = 67)

Articles other than empirical studies

Inclusion

were excluded (N = 27)

Total papers included (N = 40)

Figure 3. Systematic literature review (SLR) flow diagram

1.2. Data Processing

The first selection of articles allowed us to have 940 articles initially, and after taking into account the
criteria already quoted, we ended up choosing forty papers, to be analysed using Nvivo software. The
figure 4 shows the evolution of these forty articles selected by the year of publication.

5
Number of publications

0
2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019
Year
Figure 4 : Number of publications by the year

The table below shows the distribution of articles by journal and publication date:
Page 4 of 24
Table 2 : Papers by journal and date of publication
Journals 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019
Technology in SocietyTiN 1
Government Information Quarterly GIQ 1 1 1 1 1
Decision Support Systems 1
International journal of information management 1
Computers & Education 1
Telematics and Informatics 1
International Journal of Medical Informatics 1 1
Electronic Commerce Research and Applications 1
Information & Management 1 1
Journal of Taibah University Medical Sciences 1
Asia Pacific Management Review 1
European Journal of Information Systems 1
Information Systems Management 1
Journal of Management Information Systems 1
Journal of Computer Information Systems 1
Healthcare Informatics Research 1
International Journal of Information Systems and Project Management 1
Transforming Government: People, Process and Policy 2 1 1
The International Journal of Information and Learning Technology 1 1
Business Process Management Journal 1
Library Hi Tech 1
Internet Research 1
EuroMed Journal of Business 1
Electronic Library 1
Information Systems Frontiers 1
International Journal of Technological Learning, Innovation and Development 1
International Journal of Business Information Systems 1
International Journal of Electronic Governance 1
International Journal of Electronic Government Research 1
The International Journal of Digital Accounting Research 1
Page 1 of 24
 3. Results
3.1. Location, context and data collection methods
The table below shows the number of papers by country and context of the study, as well as the data
collection method adopted. In fact, for the data analysis methods, we have noticed that the most technique
used on the forty papers was Structural Equation Modeling (SEM).
Table 3: Location, context, data collection methods of forty papers
Dimensions Number of publications Percentage
Bangladesh 1 2,5%
USA 3 7,5%
South Africa 1 2,5%
Saudi Arabia 2 05%
Dutch–German 1 2,5%
Greek 1 2,5%
Yemen 6 15%
Denmark 2 05%
Denmark/Norway 1 2,5%
Jordan 1 2,5%
Malaysia 2 05%
Hungary 1 2,5%
Serbia 1 2,5%
Location Mauritius 1 2,5%
Taiwan 1 2,5%
Norwegia 1 2,5%
United Arab Emirates 1 2,5%
Tanzania 1 2,5%
Indonesia 1 2,5%
Pakistan 1 2,5%
China 1 2,5%
India 4 10%
USA and Spain 1 2,5%
Nepal 1 2,5%
Nigeria 1 2,5%
Thailand 1 2,5%
Singapore 1 2,5%
Total 40 100
Other1 18 45%
Education 5 05%
Medical and health 6 15%
Taxation 3 7,5%
Context Webmastering 2 05%
Grievance 3 7,5%
Corruption 1 2,5%
Elections 1 2,5%
Disaster management 1 2,5%
Total 40 100
Questionnaire 38 95%
Data collection Interview 3 7,5%
methods (2) Focus group 2 05%
Delphi technique 1 2,5%

1
Can cover many contexts
2
A study can use several methods for collecting data
Page 2 of 24
3.2. The variables studied in the articles
Through the table below, we seek in the first place to determine which of the seven variables was studied in each of the forty articles. These variables, must be
explicitly reported exactly as they were mentioned in the 2003 model of D&M (S.Q: System Quality, I.Q: Information Quality, S.Q: Service Quality, Us: Use,
I.U: Intention to Use, N.B: Net Benefits), not their synonyms or dimensions. Furthermore, to list the other non-D&M variables studied, some of these variables
may be synonyms or dimensions of the seven variables of the Delone and McLean model. Then, to list the other theories and models mobilized in parallel with
the theory of success of the IS of Delone and McLean (2003). Finally, find out the sample size for each of the data collection techniques.

Table 4 : D&M (2003) variables studied, other variables, theories mobilised and sample size for each paper
D&M (2003) variables studied Sample size
Other variables Theories /
N~ Article Delphi Focus
S.Q I.Q S.Q Us I.U U.S N.B studied Models Questionnaire Interview
techniques group
-Performance Unified Theory
expectancy of Acceptance
Determinants of user
-Effort expectancy and Use of
acceptance and use of open
-Behavioral Technology
1 government data (OGD): An * * 285
intention (UTAUT)
empirical investigation in
-Social influence
Bangladesh IS success model
-Facilitating
(D&M)
conditions
The e-Government evaluation
challenge: A South African IS success model
2 * 106 6
Batho Pele-aligned service (D&M)
quality approach
The role of trust in e- IS success model
Government effectiveness, -Trust (D&M)
3 operational effectiveness and * * * * -Operational Theory of 200
user satisfaction: Lessons effectiveness Reasoned Action
from Saudi Arabia in e-G2B (TRA)
Group value and intention to IS success model
4 * * -Relevance to task 46 2
use — A study of multi- (D&M)

Page 1 of 24
 D&M (2003) variables studied Sample size
Other variables Theories /
N~ Article Delphi Focus
S.Q I.Q S.Q Us I.U U.S N.B studied Models Questionnaire Interview
techniques group
agency disaster management -Response timeliness The Technology
information systems for -Information Acceptance
public safety accessibility Model (TAM)
-Perceived task Theory of
support Reasoned Action
-Expected group (TRA)
value Theory of
Planned
Behavior (TPB)
IS success model
Measuring the success of the
(D&M)
5 Greek Taxation Information * * * * Perceived usefulness 340
IS success
System
(Seddon)
Online learning usage within IS success model
Yemeni higher education: The -Overall quality (D&M)
role of compatibility and task- -Compatibility
6 * * * * * 448
technology fit as mediating -Task-technology fit Task-Technology
variables in the IS success -Performance impact Fit (TTF)
model
The role of transformational
leadership as a mediating
-Overall quality
variable in DeLone and
-Transformational IS success model
7 McLean information system * * * * * 448
leadership (D&M)
success model: The context of
-Performance impact
online learning usage in
Yemen
Evaluation of a
comprehensive EHR based on IS success model
8 * * * * 244 13 7
the DeLone and McLean (D&M)
model for IS success:

Page 2 of 24
 D&M (2003) variables studied Sample size
Other variables Theories /
N~ Article Delphi Focus
S.Q I.Q S.Q Us I.U U.S N.B studied Models Questionnaire Interview
techniques group
Approach, results, and
success factors
Physician satisfaction with
electronic medical records in IS success model
9 * * * 115
a major Saudi Government (D&M)
hospital
Consumer
Customer empowerment: -Customer
Behaviour
Does it influence electronic empowerment
10 * Theory 176
government success? A -Trust
IS success model
citizen-centric perspective -Personalization
(D&M)
-Performance IS success model
Could on-line voting boost
expectancy (D&M)
desire to vote? – Technology
11 * -Effort expectancy Technology 608
acceptance perceptions of
-Internet trust Acceptance
young Hungarian citizens
-Government trust Model (TAM)
Evaluating the success of an -Individual Impact
IS success model
12 emergency response medical * * * * * * -Organizational 64
(D&M)
information system Impact
IS success model
-IT quality triad (D&M)
Evaluating leadership, IT -Leadership triad Malcolm
13 quality, and net benefits in an * * * * - Leadership Baldrige 178
e-government environment -Strategic planning National Quality
-Customer market Award
(MBNQA)
Assessing the Success of E-
Demographic IS success model
14 Government Systems: An * * * * * * * 154
conditions (D&M)
Employee Perspective

Page 3 of 24
 D&M (2003) variables studied Sample size
Other variables Theories /
N~ Article Delphi Focus
S.Q I.Q S.Q Us I.U U.S N.B studied Models Questionnaire Interview
techniques group
What determines success of IS success model
an e-government service? -Perceived risks (D&M)
15 Validation of an integrative * * * * * -Confirmation Expectation 645
model of e-filing continuance -Habit Confirmation
usage Theory (ECT)
Assessing eGovernment
systems success: A validation
IS success model
16 of the DeLone and McLean * * * * * * 119
(D&M)
model of information systems
success
Perceptions of information
system success in the public IS success model
17 * * * * * 541
sector: Webmasters at the (D&M)
steering wheel?
Website quality in
government: Exploring the IS success model
18 * * * 8
webmaster’s perception and (D&M)
explanation of website quality
IS success model
(D&M)
E-government adoption and -Social influence Unified Theory
19 user’s satisfaction: an * * * * -Perceived of Acceptance 1800
empirical investigation effectiveness and Use of
Technology
(UTAUT)
An extension of Delone and -Self-efficacy
McLean IS success model -Performance impact IS success model
20 * * * * * 448
with self-efficacy: Online -Overall quality (D&M)
learning usage in Yemen

Page 4 of 24
 D&M (2003) variables studied Sample size
Other variables Theories /
N~ Article Delphi Focus
S.Q I.Q S.Q Us I.U U.S N.B studied Models Questionnaire Interview
techniques group
-IS Facilities
The impact of technological -IS Competency
factors on information -IS Integration
IS success model
21 systems success in the * * * -User Support 201
(D&M)
electronic-government -IS Structure
context -Perceived
Usefulness
Internet Usage, User Task-Technology
Satisfaction, Task- Fit (TTF)
Technology Fit, and -Performance Impact
22 * * 508
Performance Impact among -Task-technology fit IS success model
Public Sector Employees in (D&M)
Yemen
Impacts of quality antecedents
on faculty members’ IS success model
23 * * * * 204
acceptance of electronic (D&M)
resources
-Electronic Word of
The relation of e-government Mouth (eWoM)
quality on public trust and its intention IS success model
24 * * * * -Dispositional trust 293
impact towards public (D&M)
participation -Institutional trust
-Interpersonal trust

Factors influencing e- -Service quality

IS success model
25 government adoption in * * -Information quality 138
(D&M)
Pakistan -Perceived risk

Page 5 of 24
 D&M (2003) variables studied Sample size
Other variables Theories /
N~ Article Delphi Focus
S.Q I.Q S.Q Us I.U U.S N.B studied Models Questionnaire Interview
techniques group
- Information
Security Technology
- Transaction Acceptance
Model (TAM)
Security
-Transaction
services
-Intention to adopt
e-government
services
-Perceived
usefulness Diffusion Of
- Perceived ease of Innovation (DOI)
use
- Paralingual web
support
-ICT infrastructure
-Awareness
-Perceived
usefulness
-Perceived ease of
use
-Assurance
User acceptance of a G2B -Responsiveness
system: a case of electronic -Facilitating IS success model
26 * 358
procurement system in conditions (D&M)
Malaysia -Perceived risk
-Trust
-Actual transaction
behavior
-Web design quality
information
Page 6 of 24
 D&M (2003) variables studied Sample size
Other variables Theories /
N~ Article Delphi Focus
S.Q I.Q S.Q Us I.U U.S N.B studied Models Questionnaire Interview
techniques group
-Web design quality
system
-Web design quality
service
Effectiveness of information
technology in reducing
corruption in China: a IS success model
27 * * * * * 387
validation of the Delone and (D&M)
McLean information systems
success model
-Trust in e- IS success model
Assessing E-Excise Sucess government (D&M)
28 with Delone and McLean's * * * * * * websites Attribution 77
Model -Individual Theory
characteristics
-Trust in
Government
Trust and Electronic
-Trust in IS success model
29 Government Success: An * * * * * 214
Technology (D&M)
Empirical Study
-Trust in e-
government website
Examining the Success of the -Attitude
Online Public Grievance -Perceived trust
IS success model
30 Redressal Systems: An * * * * -Subjective norms 419
(D&M)
Extension of the IS Success
Model
Measuring eGovernment -Cost+
IS success model
31 success: a public value * -Time 347
(D&M)
approach -Convenience

Page 7 of 24
 D&M (2003) variables studied Sample size
Other variables Theories /
N~ Article Delphi Focus
S.Q I.Q S.Q Us I.U U.S N.B studied Models Questionnaire Interview
techniques group
-Personalization
-Communication
-Information
retrieval Public value
-Trust
-Well-informedness
-Participation
A cross-national comparison
of e-government success E-government IS success model
32 * * * * 280++/86+++
measures: A theory-based success (D&M)
empirical research
-Task quality IS success model
-Top management (D&M)
quality Task-Technology
Factors determining user -Individual quality Fit (TTF)
satisfaction of internet usage -Social quality The Unified
33 * * * * 530
among public sector Theory of
employees in Yemen Acceptance and
Use of
Technology
(UTAUT)
Success factors of online
IS success model
34 services in Kathmandu, * * * * * * 104
(D&M)
Nepal: an empirical analysis
Validation of the DeLone and
IS success model
35 McLean Information Systems * * * * * * 442
(D&M)
Success Model
Investigating success of an e- -Perceived IS success model
36 * * * * 419
government initiative: usefulness (D&M)

Page 8 of 24
 D&M (2003) variables studied Sample size
Other variables Theories /
N~ Article Delphi Focus
S.Q I.Q S.Q Us I.U U.S N.B studied Models Questionnaire Interview
techniques group
Validation of an integrated IS -Perceived ease of
success model use
-Behavioural
intention
-Perceived risk
-Perceived
satisfaction
-Complexity
Evaluating the Validity of IS
-Facilitating
Success Models for the
condition IS success model
37 Electronic Government * * * * 419
-Perceived trust (D&M)
Research: An Empirical Test
-Perceived
and Integrated Model
usefulness
An assessment of business
IS success model
38 intelligence in public * * * * Individual impact 746
(D&M)
hospitals
-Performance IS success model
expectancy (D&M)
-Effort expectancy Unified Theory
Antecedents and outcomes of
-Social influence of Acceptance
internet usage within
-Facilitating and Use of
organisations in Yemen: An
conditions Technology
39 extension of the Unified * * 520
-Task-technology fit (UTAUT)
Theory of Acceptance and
-Knowledge Task-Technology
Use of Technology (UTAUT)
acquisition Fit(TTF)
model
-Communication
quality
-Decision quality

Page 9 of 24
 D&M (2003) variables studied Sample size
Other variables Theories /
N~ Article Delphi Focus
S.Q I.Q S.Q Us I.U U.S N.B studied Models Questionnaire Interview
techniques group
-Convenience of Theory of
service Planned
-Web service Behavior (TPB)
The Influence of Theory of
content Technology
Planned Behavior,
-Perceived ease of Acceptance
Technology Acceptance
use Model (TAM)
Model, and Information
40 * * -Perceived risk 201
Systems Success Model on
-Computer&web
the Acceptance of Electronic
skills
Tax Filing System in an IS success model
-Perceived
Emerging Economy (D&M)
usefulness
-Behavioral
intentions
Total 30 31 25 14 13 27 10 38 3 1 2
+
Public Value Net Benefits construct: Cost, time, convenience, personalization, communication, Information retrieval, trust, well-informedness,
participation.
++
USA, +++Spain

Page 10 of 24
3.3. The main items used to measure the seven variables of D&M (2003)
We have found in some papers that the same measurement item has been used for two different variables, so we try to assign them while taking into
account the frequency of their use for each of the seven variables.

Table 5: The most items used on the forty papers

Variables Items
The system is user-friendly; the system is easy to use; the system is clear and understandable; the system is easy to navigate through; it has
been easy to learn how to use the system; the system is always accessible; the system provides access to all the information needed; response
time for login is satisfactory; the system provides fast information access; the system makes information readily accessible; the system
provides datasets, reports in different usable formats; the system design and features are frequently updated to meet users demands; the system
helps to reduce time needed for finding a service or a product; the system has all of the needed features; the system provides helpful instructions
System
for filing demands; the steps to complete a task in the system follow a logical sequence; the software reacts quickly enough to commands; the
quality
workflow in the organization was improved through the use of the system; the system is integrated with the workflow; business critical
functions are supported by the system; performing an operation in the system leads to a predicted result; the system responds on time to the
information needs, the system responds rapidly and satisfactorily when shifting between screens; the system is available when users need it;
the system is fast (minimal wait times); the system is updated; using The system is enjoyable; the use of the system is intuitive and clear; the
system is flexible to interact with; the system is usable at any time; the users could use the system anytime and anywhere.
Up-to-date information; the information is current; the information is on time; the information is accurate; the information is trustable; the
information is precise; the information is adapted; the information is detailed; the information is sufficient; the information is concise; the
information is convenient; the information is easy to understand; the information is reliable; the information is relevant; the information is
Information
necessary; the information is complete and comprehensive; the information is valid; the information is important; decisions are made based
quality
on the information of the system; the information in the system appears orderly; the information is well organized; the format of information
is acceptable; the information is in a form that is readily usable; the resolution of the images captured and stored meets the standards; the
information contained on the system is easy to read and understand.
Response to questions (answered adequately, in a timely manner); language (colloquialism, language choice); interactive communication;
assurance of protection of the data provided by the system; security of personal information; design is simple and attractive; no private and
Service commercial marketing or pop-ups; direct links to other government systems are available; search function is available and able to present any
quality government resource whether it is local, provincial, or national; web pages load quickly; downloadable documents; the sizes of downloadable
documents are kept as small as possible; online forms are to the point and easy to complete; application forms for services can be completed
and submitted online; application forms for services are available online for downloading; satisfaction with the quality of service provided by
Page 11 of 24
 Variables Items
the system support staff, the system support staff does understand the needs of users; the quality of the service reflects the quality of the
training given to the users of the organization; the level of on-going support is acceptable; the level of training is acceptable; the support staff
for the system has the knowledge to do their job well; the technical support staff for the system is dependable; the technical support staff for
the system gives a prompt service to users; the technical support staff for the system understands the specific needs of its users; the system
support staff shows a sincere interest in solving a problems; the staff is promptly available for assistance; the staff is willing to help users; the
staff is available to respond to users’ requests; the system offers various multimedia content(audio, video, text); the system provides an
improved quality of services; the processes are simplified and standardized; the staff expresses empathy for users, helpfulness and willingness
to solve users’ problems; the organisation provides the necessary software for users, the organisation provides the necessary hardware for
users; the services provided to users are prompt.
Frequency of using the system; the time spent on using the system; the number of users visiting the system; the use of the system made the
Use
work easier; the system entails new tasks to users; the system allows to hand over tasks to others; the user is dependent on the system.
The system will definitely be used in similar situations in the future; users will use the system; users will not discontinue use of the system; the
Intention to user would not hesitate to use the system; users intend to continue using the system rather than discontinue it; using the system is the first choice
use to achieve a task; users would like to recommend the system to their relatives and friends; users intend to continue using the system rather than
use any alternative means.
Users are satisfied with the services provided by the system; users are happy because the services offered by a system are of high quality; users
User
have a pleasant experience using a system; the decision to use the system was a wise one; the system meets the expectations of users; the system
satisfaction
is convenient and helpful.
The system allows to save time; the system ensures reduced time for completion of tasks; the performance was improved with the help of the
system; the system features allow to perform work well; the system makes the job easier; the system is useful for the job; the system helps to
reduce corruption; the system reduces the cost of providing a service; the system allows users to save money; The system is useful; the system
provides a quicker response to a question or request compared to other means; the system helps to accomplish things quicker; the system enables
users to avoid having to deal directly with government staff; the system allows users to complete tasks at any time; users can personalise the
Net benefits services offered by the system; the system is an efficient way of communicating with government departments; the system is a valuable way of
communicating with government departments; the system contains a lot of useful information about government services; the system helps
users to understand more about government services; the system answers any queries users might have about government services; users feel
comfortable interacting with the system because it generally fulfils its duties efficiently; the system allows users to give their opinions about
things; the system enhances users feeling of being part of an active democracy; the system lets users feel that decision-makers listen to them;
the system makes users feel that they are being consulted about important issues; transparency (Equity, accountability and participation);
modernisation; efficiency (Service delay reduction, reuse of data and greater coordination among departments).

Page 12 of 24
4. Analysis and Discussion
In this study, we first found that the term "trust" was studied in ten papers (quarter of the articles) as a
variable with one of the seven D&M model variables or more, so it is important to taking it into account
when studying the success of IS in public administrations. Trust is considered as an antecedent of adoption
or satisfaction with the services offered by systems, and it doesn’t depend on proprieties attributed to the
institution -trust between citizen and government institutions- only, but by the system mediating transaction
-technology and internet trust- too. In this case, building trust is a process that must be pursued slowly and
steadily as users continuously interact with the system. And this, by avoiding to offer incomplete
information-for instance-, as it will create doubts and increase the sources of risks especially the privacy
and security risks. At the same time the term "risk" is studied in five papers, it can be a braking factor that
prevents the adoption, use or intention to use of information systems, especially at the beginning before
"trust" is built.
"Perceived usefulness" was mentioned in seven papers, it refers to the benefits and services provided by
the system, that allow users to appreciate and perceive the value and usefulness of its use. we found through
this study that “perceived usefulness”, “perceived effectiveness”, “Performance expectancy”,
“performance impact”, “operational effectiveness”, “net benefits” have several points of convergence, they
are the same final output sought expressed differently.
"Facilitating conditions" is mentioned in four papers, and it can be defined as the degree to which the user
believes that an organizational and technical infrastructure exists to support the system. Also, "effort
expectancy" and "social influence" were studied in many papers. These three variables are part of the model
of The Unified Theory of Acceptance and Use of Technology. Moreover, the variable "task-technology fit"
was studied many times and it refers to a theory of Task-Technology Fit(TTF).
We found that several articles also mobilized other theories or models in order to consider certain
specificities and attributions that do not exist in the Delone & McLean model, namely, Attribution
Theory(AT), The Unified Theory of Acceptance and Use of Technology (UTAUT), Reasoned Action
Theory (TRA), Task-technology fit (TTF), Theory of Planned Behavior (TPB), Malcolm Baldrige National
Quality Award (MBNQA), Technology Acceptance Model (TAM), Public Value(PV), Diffusion of
Innovation (DOI), the Expectation Confirmation Theory (ECT) and Consumer Behaviour Theory (CBT).
These theories and models will help to better understand the issues and to fill the gap by integrating other
variables with one or more of the seven D&M variables.
“Overall quality” or “triad quality” were used in more than three articles to designate the merging and
aggregation of the characteristics of system quality, information quality and service quality into a single
variable.
Most of the papers adopted a quantitative approach, and rare were those that adopted a qualitative or hybrid
methodology, this means that most studies are confirmatory. What the authors seek by adopting the hybrid
methodology, is to take into account the characteristics of the study context, in order to feed the D&M
model or the integrated model (D&M model plus the variables of the other models or theories mobilized).
The table below details the number of times the variables have been studied in the forty papers and their
main specificities in the public sector.

Table 6: Frequency of the seven variables studied and their main specificities on the public sector
Papers
Variables Observations / specificities for the public sector
Number Percentage
• The system quality and information quality were used in more
than twenty-nine papers, this explains the importance of
System
30 77,50% considering those variables when studying the success of IS in the
quality
public sector, and their role as crucial factors in their success in

Page 1 of 24
 Papers
Variables Observations / specificities for the public sector
Number Percentage
the implementation, adoption and intent to use. On the other hand,
the “service quality” was studied in twenty-five papers.
Information • Potential users’ first impression is influenced by system
31 80% quality.
quality
• Simplicity is one of the most important qualities of the system
as it will be used by a large number of citizens that have different
study levels.
• In the public sector, the information is shared and disseminated
for the welfare of people instead of profit, as opposed the private
sector.
• Some studies have found that the most common reason citizens
visit information systems in the public sector is to obtain
information, therefore, its quality is essential for accurate
decision-making.
• The information delivered should be simple in order to be
Service understood by all citizens.
25 65%
quality • Service quality can give a nation a competitive advantage for
international business.
• Citizens expect higher quality services at the lowest cost when
they make the decision to use an IS.
• In many cases, service quality refers to the quality of support
delivered by the IS department of the public administration.
• Service quality offer by the administration depends on the
quality of processes, more simplified and standardized services
and improved decision taking processes in the administration.
• “User satisfaction” is used in more than twenty-six papers,
while “intention to use” has been studied thirteen times and
Use 14 35% fourteen times for “use”.
• Citizens may prefer to continue to use systems that they
already know well, instead of using new technologies, especially
if they require more knowledge and effort.
• The citizen’s acceptance and adherence can be determined by
Intention to the number of hours spent on the system and the frequency of its
13 32,5%
use use.
• The satisfaction of citizens doesn’t concern just a system but
also the services provided by the government on the system.
• In the public sector, we should take into consideration the
User difference between a citizen (outside) and civil servants (inside)
27 67,5% when we treat the specificities of “use”, “intention to use” and
satisfaction
“user satisfaction”. This is also true for the other variables (system
quality, service quality, information quality and net benefits).
• “Net benefits” was mentioned in ten papers.
• Net benefits in the public sector not only conceives the gain in
terms of cost or time but also, other abstract net benefits such as
Net benefits 10 27,5
the improvement of the degree of trust between the citizen and the
government, and through the improvement of transparency and
other dimensions .
Page 2 of 24
5. Conclusion
Through this systematic review, we conclude that the Delone and McLean model has been adopted in
several countries around the world, as well as in different contexts, and this indicates its effectiveness
in explaining the success factors of the implementation, the use, and future reuse of IS. We underline
that, most studies adopt a quantitative methodology, and therefore, are confirmatory ones.
The D&M model is considered as one of the most common model used to explain and clarify the success
factors in the field of IS, because it does not only take into account the technical part of the system but
also the semantic level (quality information), the social level (use, user satisfaction and intention to use)
and at the end, the output of the relationship between the technical and the social levels, namely the net
benefits. However, in the public sector, and in some cases, the D&M model is not sufficient, other
theories or models can be used to fill its voids and take into account the characteristics of the
public administration’s context, while creating a new integrated model.
In the end and after explaining the different items to measure the seven variables, we categorized these
items into homogeneous groups that gave us a dimension (several items) each. For example, the items:
the system is user-friendly, the system is easy to use, system is clear and understandable, refer to the
same dimension which is simplicity. By grouping and aggregating several items, we tried to offer for
researchers some different possible dimensions combinations of each of the seven Delone and McLean
variables (2003). The following diagram explains these different dimensions:

Page 3 of 24
Interoperability Cost

Accessibility willingness Similar situation Frequency Time of use Time

Simplicity Decision
making
Functionalities
Trust

Currency Communication

Accuracy System Intention Transparency

Use
quality to use
Format Net
Effort
Information benefits expectancy
Order or quality
organization User
satisfaction Participation
Service
quality
Support staff Modernization
quality
Navigation Efficiency

System design
Reuse of the
Expectations Experiences
Training system
Variable Dimension

Figure 5: Main dimensions of the seven variables of D&M model (2003)

Page 1 of 24
References

Leopoldo A. Gemoets and others, ‘A Cross-National Comparison of e-Government Success Measures: A

Theory-Based Empirical Research’, International Journal of Electronic Governance, 4.3 (2011), 184
Rikke Gaardboe, Niels Sandalgaard, and Tom Nyvang, ‘An Assessment of Business Intelligence in Public
Hospitals’, IJISPM - International Journal of Information Systems and Project Management, 3, 2017, 5–18
Adnan Aldholay and others, ‘An Extension of Delone and McLean IS Success Model with Self-Efficacy:
Online Learning Usage in Yemen’, The International Journal of Information and Learning Technology, 35.4
(2018), 285–304
Osama Isaac, Zaini Abdullah, Adnan H. Aldholay, and Ali Abdulbaqi Ameen, ‘Antecedents and Outcomes of
Internet Usage within Organisations in Yemen: An Extension of the Unified Theory of Acceptance and Use of
Technology (UTAUT) Model’, Asia Pacific Management Review, 24.4 (2019), 335–54
Vachiraporn Khayun, Peter Ractham, and Daniel Firpo, ‘ASSESSING E-EXCISE SUCESS WITH DeLONE
AND McLean’S MODEL’, Journal of Computer Information Systems, 2012, 11.
Yi-Shun Wang and Yi-Wen Liao, ‘Assessing EGovernment Systems Success: A Validation of the DeLone
and McLean Model of Information Systems Success’, Government Information Quarterly, 25.4 (2008), 717–33
Darko Stefanovic and others, ‘Assessing the Success of E-Government Systems: An Employee Perspective’,
Information & Management, 53.6 (2016), 717–26
András Nemeslaki, Márta Aranyossy, and Péter Sasvári, ‘Could On-Line Voting Boost Desire to Vote? –
Technology Acceptance Perceptions of Young Hungarian Citizens’, Government Information Quarterly, 33.4
(2016), 705–14
Haitham Alshibly and Raymond Chiong, ‘Customer Empowerment: Does It Influence Electronic Government
Success? A Citizen-Centric Perspective’, Electronic Commerce Research and Applications, 14.6 (2015), 393–
404
Md Shamim Talukder and others, ‘Determinants of User Acceptance and Use of Open Government Data
(OGD): An Empirical Investigation in Bangladesh’, Technology in Society, 56 (2019), 147–56
ALaa Aldin A AL Athmay, Kamel Fantazy, and Vinod Kumar, ‘E-Government Adoption and User’s
Satisfaction: An Empirical Investigation’, EuroMed Journal of Business, 11.1 (2016), 57–83
Hu Xinli, ‘Effectiveness of Information Technology in Reducing Corruption in China: A Validation of the
DeLone and McLean Information Systems Success Model’, The Electronic Library, 33.1 (2015), 52–64
Victor R. Prybutok, Xiaoni Zhang, and Sherry D. Ryan, ‘Evaluating Leadership, IT Quality, and Net Benefits
in an e-Government Environment’, Information & Management, 45.3 (2008), 143–52
Stacie Petter and Ann Fruhling, ‘Evaluating the Success of an Emergency Response Medical Information
System’, International Journal of Medical Informatics, 80.7 (2011), 480–89
Nripendra P. Rana, Yogesh K. Dwivedi, and Michael D. Williams, ‘Evaluating the Validity of IS Success
Models for the Electronic Government Research: An Empirical Test and Integrated Model’, International Journal
of Electronic Government Research, 9.3 (2013), 1–22
Claus Bossen, Lotte Groth Jensen, and Flemming Witt Udsen, ‘Evaluation of a Comprehensive EHR Based
on the DeLone and McLean Model for IS Success: Approach, Results, and Success Factors’, International Journal
of Medical Informatics, 82.10 (2013), 940–53
Nripendra P. Rana, Yogesh K. Dwivedi, Michael D. Williams, and Banita Lal, ‘Examining the Success of the
Online Public Grievance Redressal Systems: An Extension of the IS Success Model’, Information Systems
Management, 32.1 (2015), 39–59
Osama Isaac, Zaini Abdullah, T Ramayah, and Ahmed M Mutahar, ‘Factors Determining User Satisfaction of
Internet Usage among Public Sector Employees in Yemen’, 32.
Mariam Rehman, Vatcharaporn Esichaikul, and Muhammad Kamal, ‘Factors Influencing E‐government
Adoption in Pakistan’, Transforming Government: People, Process and Policy, 6.3 (2012), 258–82
JinKyu Lee and others, ‘Group Value and Intention to Use — A Study of Multi-Agency Disaster Management
Information Systems for Public Safety’, Decision Support Systems, 50.2 (2011), 404–14
Edda Tandi Lwoga and Alfred Said Sife, ‘Impacts of Quality Antecedents on Faculty Members’ Acceptance
of Electronic Resources’, Library Hi Tech, 36.2 (2018), 289–305

Page 2 of 24
 Osama Isaac, Zaini Abdullah, T. Ramayah, and Ahmed M. Mutahar, ‘Internet Usage, User Satisfaction, Task-
Technology Fit, and Performance Impact among Public Sector Employees in Yemen’, The International Journal
of Information and Learning Technology, 34.3 (2017), 210–41
Nripendra P. Rana, Yogesh K. Dwivedi, Michael D. Williams, and Vishanth Weerakkody, ‘Investigating
Success of an E-Government Initiative: Validation of an Integrated IS Success Model’, Information Systems
Frontiers, 17.1 (2015), 127–42
Murray Scott, William DeLone, and William Golden, ‘Measuring EGovernment Success: A Public Value
Approach’, European Journal of Information Systems, 25.3 (2016), 187–208
Jordan Floropoulos and others, ‘Measuring the Success of the Greek Taxation Information System’,
International Journal of Information Management, 30.1 (2010), 47–56
Osama Isaac, Adnan Aldholay, Zaini Abdullah, and T. Ramayah, ‘Online Learning Usage within Yemeni
Higher Education: The Role of Compatibility and Task-Technology Fit as Mediating Variables in the IS Success
Model’, Computers & Education, 136 (2019), 113–29
Hanne Sørum and others, ‘Perceptions of Information System Success in the Public Sector: Webmasters at the
Steering Wheel?’, Transforming Government: People, Process and Policy, 6.3 (2012), 239–57
Hana Alharthi and others, ‘Physician Satisfaction with Electronic Medical Records in a Major Saudi
Government Hospital’, Journal of Taibah University Medical Sciences, 9.3 (2014), 213–18
Shisir Prasad Manandhar, Seunghyun Kim, and Junseok Hwang, ‘Success Factors of Online Services in
Kathmandu, Nepal: An Empirical Analysis’, International Journal of Business Information Systems, 18.4 (2015),
422
Godwin Kaisara and Shaun Pather, ‘The E-Government Evaluation Challenge: A South African Batho Pele-
Aligned Service Quality Approach’, Government Information Quarterly, 28.2 (2011), 211–21
Ramlah Hussein, Nor Shahriza Abdul Karim, and Mohd Hasan Selamat, ‘The Impact of Technological Factors
on Information Systems Success in the Electronic‐government Context’, Business Process Management Journal,
13.5 (2007), 613–27
Gaurav Gupta and others, ‘The Influence of Theory of Planned Behavior, Technology Acceptance Model, and
Information System Success Model on the Acceptance of Electronic Tax Filing System in an Emerging
Economy’, The International Journal of Digital Accounting Research, 15 (2015)
Rizqa Nulhusna and others, ‘The Relation of E-Government Quality on Public Trust and Its Impact on Public
Participation’, Transforming Government: People, Process and Policy, 11.3 (2017), 393–418
Adnan H. Aldholay and others, ‘The Role of Transformational Leadership as a Mediating Variable in DeLone
and McLean Information System Success Model: The Context of Online Learning Usage in Yemen’, Telematics
and Informatics, 35.5 (2018), 1421–37
Ricardo Santa, Jason B. MacDonald, and Mario Ferrer, ‘The Role of Trust in E-Government Effectiveness,
Operational Effectiveness and User Satisfaction: Lessons from Saudi Arabia in e-G2B’, Government Information
Quarterly, 36.1 (2019), 39–50
Thompson S. H. Teo, Shirish C. Srivastava, and Li Jiang, ‘Trust and Electronic Government Success: An
Empirical Study’, Journal of Management Information Systems, 25.3 (2008), 99–132
Murali Sambasivan, George Patrick Wemyss, and Raduan Che Rose, ‘User Acceptance of a G2B System: A
Case of Electronic Procurement System in Malaysia’, Internet Research, 20.2 (2010), 169–87
Adebowale I. Ojo, ‘Validation of the DeLone and McLean Information Systems Success Model’, Healthcare
Informatics Research, 23.1 (2017), 60
Hanne Sørum, Kim Normann Andersen, and Torkil Clemmensen, ‘Website Quality in Government: Exploring
the Webmaster’s Perception and Explanation of Website Quality’, Transforming Government: People, Process
and Policy, 7.3 (2013), 322–41
Narvadha Veeramootoo, Robin Nunkoo, and Yogesh K. Dwivedi, ‘What Determines Success of an E-
Government Service? Validation of an Integrative Model of e-Filing Continuance Usage’, Government
Information Quarterly, 35.2 (2018), 161–74
‘The DeLone and McLean Model of Information Systems Success: A Ten-Year Update’, Journal of
Management Information Systems, 19.4 (2003), 9–30
William H. DeLone and Ephraim R. McLean, ‘Information Systems Success: The Quest for the Dependent
Variable’, Information Systems Research, 3.1 (1992), 60–95

Page 3 of 24