International Journal of Ambient Systems and Applications (IJASA) Vol.1, No.

1, March 2013

Evaluating the Quality of Software in ERP Systems Using the ISO 9126 Model
Thamer A. Alrawashdeh, Mohammad Muhairat and Ahmad Althunibat
Department of software Engineering, Alzaytoonah University of Jordan, Amman, Jordan Abstract This paper presents the quality model of enterprise resource planning (ERP) systems by
adapting the ISO9126 standard. This model is used to verify that whether the implementation of ERP systems will succeed or fail in higher educational institutions. Six quality characteristics are suggested to be minimum requirements for creating the quality model of ERP systems, including functionality, reliability, usability, efficiency, maintainability, and portability of ERP systems. The quality characteristics could not be measured directly. Thus in this study, they are divided into twenty seven subcharacteristics.

Keyword Software Quality models, Software Quality Characteristics and ERP systems
Quality Model.

1 INTRODUCTION
The growth of Information Systems (IS) has an important role in improving the operations of higher education institutions. In this respect, ERP (Enterprise Resource Planning) systems have integrated information systems, in order to control business functions in an organization. Many studies on the ERP systems in different domains found that the information that provided by the ERP systems have a positive effect on the decision making, since they can provide decision makers with valuable information from different functional areas (Madapusi, 2008, Holsapple and Sena, 2005 and Bendoly, 2003). By implementing such system, institutions and organizations expect to improve quality and productivity of business operations. Therefore, higher education institutions have spent millions of dollars and the time taken for ERP systems implementation, sometimes takes more than two years (Swartz and Orgill, 2001). Thus, the institutions have moved to use the ERP systems for better quality. However, many studies have shown a rather high failure rate in the implementation of ERP systems (Zornada and Velkavrh, 2005). In the educational environment, although a number of research activities have been conducted on the quality of education institutions information systems, most of these studies have been conducted to assess the quality of e-learning websites (Abdellatief et al. 2011 and Padayachee et al. 2010). Therefore, the quality of ERP systems is a complex concept, due the lack of studies in this field. As well as, the ERP systems in the education institutions compose technological, organizational, administrative, usage and instructional risk. Hence, how to measure a quality of such systems is still not clear. Therefore, this paper aims to develop a quality model to provide a framework for evaluating the quality of education institutions ERP systems.

International Journal of Ambient Systems and Applications (IJASA) Vol.1, No.1, March 2013

2 LITERATURE REVIEW
In order to propose an appropriate software quality model for ERP systems, this section highlights the most popular software quality models in the literature, their contributions and disadvantages. These models are McCalls software quality model, Boehms software product quality model, Dromeys quality model, FURPS quality model and ISO\IEC 9126.

A. McCalls Quality model

McCalls model is one of the most commonly used software quality models (Panovski, 2008). This model provides a framework to assess the software quality through three levels. The highest level consists eleven quality factors that represent the external view of the software (customers view), while the middle level provides twenty three quality criteria for the quality factors. Such criteria represent the internal view of the software (developers view). Finally, on the lowest level, a set of matrices is provided to measure the quality criteria (McCall et al. 1977). According to Fahmy et al. (2012) the contribution of the McCall Model is assessing the relationships between external quality factors and product quality criteria. However, the disadvantages of this model are the functionality of a software product is not present and not all matrices are objectives, many of them are subjective (Behkamal et al. 2009).

B. Boehms Quality Model

In order to evaluate the quality of software products, Boehm proposed quality model based on the McCalls model. The proposed model has presented hierarchical structure similar to the McCalls model (Boehm et al. 1978). Many advantages are provided by the Boehms model namely taking the utility of a program into account and extending the McCall model by adding characteristics to explain the maintainability factor of software products (Fahmy et al. 2012). However, it does not present an approach to assess its quality characteristics (Panovski, 2008).

C. FURPS Quality Model

The FURPS model was introduced by Robert Grady in 1992. Its worth mentioning that, the name of this model comes from five quality characteristics including Functionality, Usability, Reliability, Performance and Supportability. These quality characteristics have been decomposed into two categories: functional and nonfunctional requirements (Grady, 1992). The functional requirements defined by inputs and expected outputs (functionality), while nonfunctional requirement composes reliability, performance, usability and supportability. However, the one disadvantage of this model is the software portability has not been considered (Al-Qutaish, 2010).

D. Dromeys Quality Model

Dromys model extends the ISO 9126: 1991 by adding two high-level quality characteristics to introduce a framework for evaluating the quality of software products. Therefore, this model comprehends eight high-level characteristics. Such characteristics are organized into three quality models including requirement quality model, design quality model and implementation quality model (Dromey, 1996). According to Behkamal et al. (2009), the main idea behind Dromeys model reveals that, formulating a quality model that is broad enough for different systems and assessing the relationships between characteristics and sub-characteristics of software product quality.
2

International Journal of Ambient Systems and Applications (IJASA) Vol.1, No.1, March 2013

The One disadvantage of Droemys model is the reliability and maintainability characteristics could not be judged before a product actually implemented (Fahmy et al. 2012).

E. ISO 9126 Model

ISO 9126 is an international standard for software quality evaluation. It was originally presented in 1991; then it had been extended in 2004. The ISO 9126 quality model presents three aspects of software quality which address the internal quality, external quality and quality in use (ISO, 2004). Therefore, this model evaluates the quality of software in term the external and internal software quality and their connection to quality attributes. In this respect, the model presents such quality attributes as a hierarchical structure of characteristics and subcharacteristics. The highest level composes six characteristics that are further divided into twenty one sub-characteristics on the lowest level. The main advantage of this model is the model could be applied to the quality of any software product (Fahmy et al. 2012). As a summary of this section, Table 1 compares the quality characteristics of different quality models. It is visible that some characteristics are not much considered (have less effect on the software product quality) namely, correctness, human engineering, process maturity, performance, supportability and changeability. Therefore, this paper will not pay attention for such characteristics. On the other hand, two reasons to adapt the ISO 9126 quality model, in order to develop ERP systems quality model. These reasons include generality of the ISO 9126 model, since it could be applied to measure the quality of various systems; and it has taken the common quality characteristics into consideration.

TALE 1 COMPARISON BETWEEN THE SOFTWARE QUALITY MODELS

Quality characteristics Functionality Usability Testability Correctness Efficiency Understandability Reliability Flexibility Human Engineering Integrity Interoperability X X 3 X X X X X X X X X X X X X X X X X X X McCall Boehm FURPS X X Dromey X X ISO 9126 X X X

International Journal of Ambient Systems and Applications (IJASA) Vol.1, No.1, March 2013 Process maturity Performance Supportability Maintainability Changeability Portability Reusability X X X X X X X X X X X X X

ERP SYSTEM QUALITY MODEL

Existing software quality models, especially the ISO 9126 model provide quality characteristics that are general and common for evaluating the quality of every type of software product. Despite that, there are many types of software products have its own characteristics. Thus, in order to evaluate the quality of such software products, existing software quality models should be modified and extended. In this respect meaning, starting from a specific quality model, the characteristics and sub-characteristics of such a model should be adjusted according to the nature of a new system being evaluated. Such adapting involves eliminating some characteristics; redefine others, and introducing new characteristics. Although, the research on the quality software products based on ISO 9126 for the education domain is not newly approach ( Chua and Dyson, 2004; Padayachee et al. 2010; and Fahmy et al. 2012), the studies on adapting ISO 9126 to evaluate the quality of ERP system in education domain are very limited, leading to the novelty of this research. So, as previously mentioned, the focus of this work is on evaluating the quality of ERP systems in higher educational institutions, by adapting the ISO 9126 quality model. It is worth mentioning that, although the ISO 9126 quality model does not provide specific quality requirements, it however presents a general framework to evaluate the quality of software products. This is the main advantages and strength of such model, since it can be used across a variety of systems among of them education institutions system i.e. ERP systems. Many scholars have adapted The ISO 1926 quality model, in order to evaluate a variety of systems. Among such systems are e-book system (Fahmy et al. 2012), web site e-learning systems (Padayachee et al. 2010), computer-based systems (Valenti et al. 2002) and egovernment systems (Quirchmayr et al. 2007). The generality of ISO 9126 quality model requires further analysis of characteristics, before it is fully adjusted for evaluating the quality of the ERP system. The ISO 1926 standard defines a quality model with six characteristics including functionality, reliability, usability, efficiency, portability, and maintainability which are further divided into twenty seven sub-characteristics (ISO, 2001). The following includes how these characteristics and sub-characteristics are adapted for this research.

International Journal of Ambient Systems and Applications (IJASA) Vol.1, No.1, March 2013

The functionality has been defined by (ISO 2001) as the capability of the software to provide functions which meet the stated and implied needs of users under specified conditions of usage. In order to evaluate such characteristics, it has been divided into five sub-characteristics namely accuracy, suitability, interoperability, security, and functionality compliance (Kumar et al. 2009). Adapting the functionality to the ERP systems in the higher education institutions reveals that the systems software should provide functions and services of higher education institutions as per the requirements when it is used under specific conditions. The reliability is the capability of the software to maintain its level of performance under stated conditions for a stated period of time. Reliability has four sub-characteristics consist maturity, fault tolerance, recoverability, and reliability compliance (Fahmy et al. 2012). In terms of ERP systems, the reliability refers to the capability of the systems to maintain its service provision under specific conditions for a specific period of time. In other words, the probability of the ERP system fails in a problem within a given period of time. The usability is the capability of the software to be understood learned, used, and attractive to the users, when used under specified conditions. The usability has set of sub-characteristics including understandability, learnability, operability, and attractiveness (Kalaimagal and Srinivasan, 2008). This characteristic is employed in this study to suggest that the ERP systems should be understood, learned, used and executed under specific conditions. The efficiency refers to the capability of a system to provide performance relative to the amount of the used resources, under stated conditions. It has also been divided into three subcharacteristics namely time behavior, resource utilization an efficiency compliance (ISO, 2001). Adapting this characteristic to the ERP systems in the higher education institutions suggests that the systems should be concerned with the used resources when providing the required functionality. The maintainability is the capability of the software to be modified. The maintainability consists five sub-characteristics including analyzability, changeability, stability, testability, and maintainability compliance (Al-Qutaish, 2010, ISO, 2001). In this research, any feature or part of the ERP system should be modifiable. As well as identifying a feature or part to be modified, modifying, diagnosing causes of failures, and validating the modified ERP system should not require much effort. Finally, the portability of software refers to the capability of the software to be transferred from one environment to one another (ISO, 2001). Therefore, the ERP systems in the higher education institutions should be applied using different operating systems; be applied at different organizations or departments; and be applied using a variety of hardware. Similar to the previous quality characteristics, the portability has set of sub-characteristics namely adaptability, installability, coexistence, replaceability, and portability compliance (Fahmy et al. 2012). Based on the previous argument, table 2 presents the ERP systems quality model which is based on the ISO 9126. This model includes quality characteristics and sub-characteristics. Additionally, it shows how these characteristics and sub-characteristics influence ERP systems in the higher education institutions.
5

International Journal of Ambient Systems and Applications (IJASA) Vol.1, No.1, March 2013

TABLE 2 ERP SYSTEM OF THE HIGHER EDUCTION INSTITUTIONS

Characteristic Functionality Sub-characteristic Suitability Description Can the ERP systems software perform the required functions? Are the results of ERP systems software as anticipated? Can the ERP systems software interact with other systems Can the ERP system prevent unauthorized access? Does the ERP system adhere to the applications standards and regulations of the law? Have the faults in the ERP systems software and hardware devices been eliminated over time? Is the ERP system capable to maintain a specified level of performance in case of software and hardware errors? Can the ERP system resume working and recover affected data in case of a failure? Does the ERP systems software adhere to the existing reliability standards? Does the ERP systems user recognize how to use the system easily? Can the ERP system be learnt easily? Can the ERP system work with a minimal effort? Does the ERP systems interface Look good? Does the ERP systems software meet the existing usability standards? How quickly does the ERP system respond? 6

Accurateness

Interoperability

Security

Functionality Compliance

Reliability

Maturity

Fault tolerance

Recoverability

Reliability compliance Usability Understandability

Learnability Operability

Attractiveness

Usability Compliance Efficiency Time behavior

International Journal of Ambient Systems and Applications (IJASA) Vol.1, No.1, March 2013 Resource utilization Efficiency compliance Maintainability Analyzability Does the ERP system utilize resources efficiently? Does the ERP systems software adhere to the existing efficiently standards? Do diagnose faults or identification a part to be modified within the ERP system, require a minimal effort? Can the ERP system be modified easily? Can the ERP system continue functioning after the change? Can the modified ERP system be easily validated? Can the ERP system be moved easily to the other environment? Can the ERP systems software be installed easily? Does the ERP system adhere to the portability standards? Can the ERP system be replaced easily with similar system?

Changeability Stability

Testability

Portability

Adaptability

Installability

Portability compliance Replaceability

CONCLUSION

This study proposes a model for evaluating the quality of ERP systems in the higher education institutions, while its quality characteristics and sub-characteristics have been proposed based on the ISO 9126. There are two contributions that are provided by this work including offering comparison between existing quality models and identifying the quality characteristics of ERP systems. The extension of this study will be conducted, in order to rank the main quality characteristics of the proposed model. The provided results will enable a greater understanding of the interrelation and the impact these sub-characteristics have on the quality characteristics.

REFERENCES
1) Al-Qutaish, R. E. (2010). Quality Models in Software Engineering Literature: An Analytical and Comparative Study. Journal of American Science Volume 6, Number 3. 2) Behkamal, B., Kahani. M., and Akbari, M. K. (2009). "Customizing ISO 9126 Quality Model For Evaluation Of B2B Applications", Journal Information and Software Technology, vol. 51, Issue 3. 7

International Journal of Ambient Systems and Applications (IJASA) Vol.1, No.1, March 2013 3) Bendoly, E. (2003). Theory and Support for Process Frameworks of Knowledge Discovery and Data Mining from ERP Systems. Information & Management 40(7), 639-647. 4) Boehm, B. W., Brown, J. R., Kaspar, H., Lipow, M., McLeod G. and Merritt, M. (1978). Characteristics Of Software Quality, North Holland Publishing, Amsterdam, The Netherlands. 5) Chua, B. B. and Dyson, L. E. (2004). Applying The ISO 9126 Model To The Evaluation Of An E-Learning. in Proceedings of the 21st ASCILITE Conference, pp. 184-190. 6) Dromey, R. G. (1996). Concerning The Chimera [Software Quality]. IEEE Software, vol. 13, pp. 33-43 7) Fahmy, S. Haslinda, N. Roslina, W and Fariha, Z. (2012). Evaluating the Quality of Software in e-Book Using the ISO 9126 Model. International Journal of Control and Automation Vol. 5, No`. 2. 8) Quirchmayr, G., Funilkul S. and Chutimaskul, W. (2007). A Quality Model Of E-Government Services Based On The ISO/IEC 9126 Standard. on The Proceedings of International Legal Informatics Symposium IRIS. 9) Holsapple, C.W. and Sena, M.P. (2005). ERP plans and decision-support benefits. Decision Support Systems, 38, 575-590. 10) Padayachee, I., Kotze P. and van der Merwe, A. (2010). ISO 9126 External Systems Quality Characteristics, Sub-Characteristics And Domain Specific Criteria For Evaluating E-Learning Systems. In Proceedings of SACLA Conference. 11) ISO. ISO/IEC 9126-1: Software Engineering - Product Quality - Part 1: Quality Model. International Organization for Standardization,Geneva, Switzerland, 2001. 12) ISO. ISO/IEC TR 9126-4: Software Engineerin- Product Quality - Part 4: Quality in Use Metrics. International Organization for Standardization, Geneva, Switzerland. Switzerland, 2004. 13) Kalaimagal, S., Srinivasan, R. (2008). A retrospective on software component quality models. SIGSOFT Software Engineering 33, 110. 14) Kumar, A., Grover, P.S., and Kumar, R. (2009). A quantitative evaluation of aspect-oriented software quality model (AOSQUAMO). ACM SIGSOFT Software Engineering Notes 34, 19. 15) Madapusi, A. (2008). ERP Information Quality and Information Presentation Effects on Decision Making. SWDSI 2008 Proceedings, pp. 628-633 16) McCall, J. A., Richards P. K. and Walters, G. F. (1977). Factors In Software Quality, US Rome Air Development Center Reports, US Department of Commerce, USA. 17) Padayachee, I., Kotze, P. and van Der Merwe, A. (2010). ISO 9126 external systems quality characteristics, sub-characteristics and domain specific criteria for evaluating e-Learning systems. In: The Southern African Computer Lecturers Association, University of Pretoria, South Africa. 18) Panovski, G (2008). Product Software Quality. Technishe University Eindhoven. Department of Mathematics and Computing Science 19) R. B. Grady, "Practical Software Metrics for Project Management and Process Improvement", Prentice Hall, Englewood Cliffs, NJ, USA, (1992). 20) Valenti, S., Cucchiarelli, A. and Panti, M. (2002). Computer Based Assessment Systems Evaluation via the ISO9126 Quality Model. Journal of Information Technology Education, vol. 1, no. 3, pp. 157-175. 8

International Journal of Ambient Systems and Applications (IJASA) Vol.1, No.1, March 2013 21) Swartz, D. and Orgill, K. (2001). Higher Education ERP: Lessons Learned. EDUCAUSE Quarterly, (2), pp. 20-27. 22) Zornada, L. and Velkavrh, T.B. (2005). Implementing ERP systems in higher education institutions, Proceeding of the 27th International Conference on Information Technology, ICTI, Cavtat, Croatia. 23) Saini, R., Dubey S. K. and Rana, A. (2011). Analytical Study Of Maintainability Models For Quality Evaluation. Indian Journal of Computer Science and Engineering, vol. 2, Issue 3.