Evaluation of Saudi Educational Websites

The International Journal of E-Learning and Educational Technologies in the Digital Media (IJEETDM) 2(4): 141-147 The Society of Digital Information and Wireless Communications (SDIWC), 2016 ISSN: 2410-0439 (Online) Evaluation of Saudi Educational Websites Sahar A. El_Rahman1,2 1 Electrical Department, Faculty of Engineering-Shoubra, Benha University, Cairo, Egypt 2 Computer Science Department, College of Computer and Information Sciences, Princess Nourah Bint Abdulrahman University, Riyadh, Saudi Arabia Sahr_ar@yahoo.com, sahar.hassan@feng.bu.edu.eg ABSTRACT Website is a software product used by different organizations for marketing and information exchange. It is one of the best technologies for information system applications. Generally, universities have complex websites, which include a collection of many sub-websites related to the different sections of universities. This work employed software tools-based evaluation method and evaluator-based evaluation method to comprehensively evaluate five big university websites in Saudi Arabia that are King Saud University (KSU), King Faisal University (KFU), Princess Nourah Bint Abdulrahman University (PNU), Prince Sultan University (PSU) and Dar Al Uloom University (DAU). The evaluation involves testing sample pages related to the selected universities. This study provides an overview regarding the weakness and strengths of the five Saudi university websites. Where it aims to comprehensively evaluate the five Saudi university websites, using the software tools (WebCHECK and Sitebeam), and point out their weakness and strengths. KEYWORDS Evaluation, Website Evaluation, University Website, Usability, Human Computer Interaction. 1 INTRODUCTION University sites are both informational and promotional. It tells students, parents, academic and administrative staffs about courses, timetables, and other relevant information, and it tells prospective students and prospective employees about the university and its programs [1][2][3]. websites aiming simply to have a presence on the web. Today, academic websites have become a critical component of academic organizations, and unity of their most visible aspects. Thus, the aim of the websites of the academic institutions has changed [2]. As the importance of academic institutional websites has increased with the increasing number of academic websites, and the number of Internet users, the importance of university ranking websites, which review, and rank university websites, has increased as well. In fact, university ranking systems (eg. Eduroute) provide the educated seeker with all the information they need about the universities in terms of quality of education, accreditation, and reputation of the universities [4]. Various evaluation methods have been produced to evaluate the websites. The methods could be sorted out as three categories, which are users, evaluators, and tools. Evaluator based methods are directed at finding usability problems that users might encounter while interacting with an interface, from the evaluator’s point of perspective. It requires accepting a number of evaluators assess the user interface, and judge whether it adjusts to a set of usability principles. The other evaluation methods, which involve users in the process of identifying usability problems, include observations, questionnaires, and interviews [2]. Evaluation instruments are software tools or online services that help determine if a website is usable and accessible [5]. The university websites goal differed over time due to technological advances, and the increasing number of Internet users. For instance, in early 1990, university websites started as informational 141 The International Journal of E-Learning and Educational Technologies in the Digital Media (IJEETDM) 2(4): 141-147 The Society of Digital Information and Wireless Communications (SDIWC), 2016 ISSN: 2410-0439 (Online) 2 MANUAL AND AUTOMATED EVALUATION Manual evaluation or evaluator-based method can determine the availability of a web page or any other evaluation measures. The accuracy of the results solely depends on the evaluator knowledge. Manual evaluation allows finding accessibility problems that cannot be found programmatically. For example, an evaluation instrument can specify if an image has descriptive text associated with it, but during manual evaluation, it is possible to determine if the description provides enough information about the image. It needs an accurate overview, especially in case of complex and larger websites to ensure that all elements and pages are covered, that of course consumes longer time [6]. Automatic evaluation can significantly decrease the effort and time needed to carry out evaluations. It offers an initial assessment much faster, and give a good estimation of the accessibility of the website on a larger scale. Nevertheless, on that point are certain issues which automated evaluation cannot detect, in some events, evaluation tools are likely to create false or misleading results such as not identifying or signal incorrect code. In summation, some pages could be lacking if the website pages are not linked up very easily. Automated evaluation requires human judgment and must be measured manually using different techniques. While it delivers a substantial amount of time, it is important to double check the results and rule out other issues [5][7]. Table 1. Evaluation Methods Evaluation methods Accessibility Accuracy Authority Coverage Metadata Search- ability Orientation Currency Interactivity Navigability 3 WEB EVALUATION CRITERIA The evaluation methods have been used are relying on the following criteria factors as indicated in Table 1. Usability Readability Description It basically means that people with disabilities can use the Web. More specifically, web accessibility means that people with disabilities can perceive, understand, navigate, and interact with the Web, regardless of any internet browser that they are using. Web accessibility encompasses all disabilities that affect access to the Web, including visual, auditory, physical, speech, cognitive, and neurological disabilities [7]. It refers to the up to date information. It is the degree to which the information and materials available on the web site are correct and trustworthy [8]. It refers to the credibility and expertise of the authorship of the information on the website. There are two levels of authority of the website: the authority of the author (the author of the information) and the authority of the web site (publisher) which may or may not be the same [9]. It is the degree to which information and contents are presented according to various topics through the site. Good contents and coverage should be engaging, relevant, concise, clear, and appropriate for the audience [8][9]. It refers to tags added to the HTML document containing descriptive information that does not appear in the document body. Metadata can be used by resource discovery tools such as search engines to increase the relevance of the information retrieved in searches [7]. It is the ability to browse, search, and acquire data within a website [7][8]. It includes Website purpose and scope, origins and status of the types of information, and services provided. Orientation information should be easily located [9][10]. It refers to the timeliness of information, documents, materials and services available on the web site. Websites should be seen as a way of providing very recent information. This criterion is extremely important to people who rely on web resources for up-to-date information [10]. It is the way that a site allows the user to do something. It allows the user to give and receive. Interactive elements allow users to control what elements are to be delivered and when they are to be delivered through the interface[8][9][10]. It evaluates the organization of information on the site and how easily users can move through sections of the website. Sites with good Navigation are consistent and effective as they offer easy access to the breadth and depth of the site's content [11]. It is a quality attribute that assesses how easy user interfaces are to use. The word "usability" also refers to methods for improving ease-of-use during the design process. Usability is defined by five quality components includes learnability, efficiency, memorability, error and satisfaction [12]. It refers to all the factors that affect reading and understanding a piece of text. These factors include: the interest and motivation, page layout (e.g., foreground/background color, spacing between lines and objects), text effects (e.g., font typefaces, size and styles), among others, the quality of the user's monitor as well as the actual composition of the website content [13]. 142 The International Journal of E-Learning and Educational Technologies in the Digital Media (IJEETDM) 2(4): 141-147 The Society of Digital Information and Wireless Communications (SDIWC), 2016 ISSN: 2410-0439 (Online) 3 METHODOLOGY An automated web testing tools or software tools are used. Where Evaluator-based method considered old. It is a manual evaluation using a browser, a text editor and the evaluator knowledge. The software tools are Sitebeam and WebCHECK which are illustrated in the following subsections. 3.2.1 WebCHECK Analysis of Contributing Factors The analysis breaks down scores on each of the WebCHECK Professional's two dimensions to four contributing factors: Stimulating and Meaningful (Value (V)) and Organized and Easy-to-Use (Expectation of Success (XV)) [16] as shown in formulas 1 and 2: S+M=V O + E = XS 3.1 Sitebeam It is a Silktide's main product, Silktide is a British software developer founded in July 2001 by Oliver Emberton. Sitebeam is a web-based reporting and testing tool. Testing is based on best practice guidelines and independent public standards. It includes SEO, accessibility, content, social marketing and technology to provide insights into websites [15]. Sitebeam evaluate whole websites or multiple pages with no or little user interaction [5]. 3.2 WebCHECK The Web Site Evaluation Instrument© (aka WebCHECK) funded by an Institute of Museum and Library Services SPARKS! Ignition Grant. WebCHECK provides checklists to assess various features that are easy-to-use instruments that motivate individuals to explore, visit and return to a website. These instruments were planned for use by students, educators and Website developers. Users simply snap on a response for each item on the instrument, and, once all items are loads, their scores are automatically compiled and a printable, full report is brought forth. These reports include graphical and text represented results, with a compiled score explanation. The scores are broken down into two main categories: whether the user has an expectancy for success in using the site and the evaluator's perceptions of the site's value. It breaks down these two main categories further into four subcategories: Stimulating (S), Meaningful (M), Organized (O) and Easy-to-Use (E) [14]. (1) (2) The highest possible score for each factor is 36; the lowest possible score for each factor is 0. The scores for each factor are the sum of that factor's corresponding item score (see Table 2) [14][16].     A stimulating Web site is one that arouses curiosity, attracts, and sustains attention and involvement. A meaningful Web site contains credible, relevant, and accessible data. An organized Web site is one that submits data in a readable, consistent, and orderly structure. An easy-to-use Web site is well navigated and searchable. Table 2. Item Scores by Factors [16] A score 27 or higher on one or more of the four factors All four factors score 27 or higher A score between 18 and 26 on any factor If any factor scores between 9 and 17 A score of 8 or below on any factor A high score, but still may require modest revision. The Website is considered an overall "Awesome!" Website. Means that factor is above average, but could be improved with some modification(s). The website is considered below average, requiring substantial revision. The website is considered low and requires the most comprehensive improvements. 3.2.1.1 Item Scores by Factors The items are grouped according to their corresponding factor. Items are listed in descending order, from highest (3) to lowest (0) score. This permits you to pinpoint specifically in 143 The International Journal of E-Learning and Educational Technologies in the Digital Media (IJEETDM) 2(4): 141-147 The Society of Digital Information and Wireless Communications (SDIWC), 2016 ISSN: 2410-0439 (Online) Sitebeam result PSU DAU 5.8 2.5 0 2.6 3.5 8.6 9.5 8.7 8 2.8 2.9 4.6 4.6 6.6 6.5 6.8 3.4 3.2 2.4 Web site evaluation instrument results of the five Saudi university websites using Sitebeam and WebCHECK are shown in Figure 1 and Figure 2. WebCHECK Analysis of Contributing Factors of five Saudi university websites as shown in Fig. 3 and Item Scores by Factors of five Saudi university websites as shown in Table 3. KFU 5.3 KSU 7.7 7.5 PNU 4 RESULTS 4.3  Items with a score of 3 are highly rated. Items with a score of 2 may need some revision or alteration. Items with a score of 1 or 0 are in serious need of revision or modification. 4.8   Analysis of Contributing Factors of five Saudi university websites as shown in Figure 3 and Item Scores by Factors of five Saudi university websites as shown in Table 3. 5.7 what ways this website may be changed or revised [14] [16]. 4.1 Sitebeam Results This tool evaluates five web pages from each website and generates a summary report evaluation. Overall score indicated that KFU website scored the highest one among the university websites, breaking down the overall into different evaluation factors such as accessibility, content, marketing and technology. KFU website scored the highest one in content and technology while the KSU website scored the highest in accessibility and marketing (see Figure 1). Figure 1. Sitebeam results of five Saudi university websites 4.2 WebCHECK Results Expectation for success refers to a how organized and easy to use a web site or resource is. KFU and KSU websites scored high xs=54 and xs=58 respectively, while other university websites scored above average, ranging from 46 to 51 points yet there still may be ways to improve this dimension. The value score refers to how stimulating and meaningful this web resource or site is. KFU website scored high v= 54. While other university websites scored above average, ranging from 36 to 49 degrees, so far there still may be ways to increase value to users (see Figure 2). For more details, see WebCHECK Figure 2. WebCHECK Results of Five Saudi University Websites 144 The International Journal of E-Learning and Educational Technologies in the Digital Media (IJEETDM) 2(4): 141-147 The Society of Digital Information and Wireless Communications (SDIWC), 2016 ISSN: 2410-0439 (Online) V = Value; XS = Expectation for Success S=Stimulating, M=Meaningful, O=Organized, E=Easy-to-use (a) http://ksu.edu.sa (b) http://www.kfu.edu.sa (c) http://www.psu.edu.sa (d) http://www.pnu.edu.sa (e) http://dau.edu.sa/en/home Figure 3. WebCHECK Analysis of Contributing Factors of Five Saudi University Websites 145 The International Journal of E-Learning and Educational Technologies in the Digital Media (IJEETDM) 2(4): 141-147 The Society of Digital Information and Wireless Communications (SDIWC), 2016 ISSN: 2410-0439 (Online) Table 3. Item Scores by Factors of Five Saudi University Websites WebCHECK Contributing Factors [16] Stimulating [in descending order from 3-0] There is nothing on this Web site that distracts attention from the content. The visual layout of this Website attracts attention. The content on this Web site is fresh and engaging. I would re-visit this Web site. This Web site's content is current and up-to-date. This Web site provides a list of resources that may be accessed to obtain additional information. Functional hyperlinks within and outside of this Web site stimulate further exploration of content. This Web site stimulates curiosity and exploration. This Web site has a novel or unique features that make it more interesting. This Web site provides opportunities for interactivity through participatory features (e.g. Social networking, games, polls, commenting, etc.) A variety of formats for presenting information (e.g. Text, images, sounds) helps maintain attention without limiting persons with disabilities from access to that information. There are opportunities to read and/or share different ideas and viewpoints that make this Web site interesting. Meaningful [in descending order from 3-0] The authority of this Web site author and/or publisher is credible for the content. This Web site provides links to other related or useful Web sites. This Web site appears to contain credible information. The Information on this Web site appears to be accurate. The information contained in this Web site is current and up-to-date. The authority of this Web site author(s) or creator(s) is readily discernible. The author and/or publisher of this Web site is explicitly stated. This Web site provides opportunities to communicate with its creator(s) or author(s). This Web site's content, either provides an objective perspective or makes its bias known. This Web site provides adequate coverage of topics presented. This Web site provides accessible opportunities for all (including those with visual, hearing and mobility impairments) to actively participate and contribute content. This Web site contains little or no redundant or irrelevant information. Organized [in descending order from 3-0] The text on this Web site is well-written without grammatical, spelling or other errors. The information on this Web site is presented in a clear and consistent manner. The purpose of this Web site is clear. The organization of this Web site is simple and clear. When clicking hyperlinks, the ability to revisit the selected path (i.e. via a "breadcrumb trail" or the Web browser's back button) is available. No matter where I am in this Web site, I can return directly to the home page. The information on this Web site is well-organized. This Web site provides adequate coverage of topic(s) presented. This Web site's design uses a navigation system that enables efficient access to any Web site section from any page on the site. Video or multimedia content may be launched in a new window or frame so as not to get lost when accessing this content. Visual (e.g. Videos, photographs) or audio content included in this Web site helps to clarify or describe the topic(s) presented. This Web site works well whether or not pop-up functionality is enabled on a Web browser. Easy-to-use [in descending order from 3-0] This Web site makes it easy to search or query for information. The features on this Web site are active and fully functioning. Buttons, links and other navigation mechanisms work the way they should on this Web site. There is little or no delay in accessing media content from this Web site. Features of this Web site are easy-to-use. Navigating this Web site does not require any special skills or experience. This Web site is optimized for mobile access (i.e. Smart Phones, tablets, etc.). At this Web site, I can control what information I wish to access. The information on this Web site is accessible to all, including those with hearing impairments, by offering closed-captioning and/or transcripts of audio content. The information on this Web site is accessible to all, including those with sight impairments, by providing content that is screen reader-enabled, employing descriptive audio and offering a simple design to assist those using magnification tools. This Web site provides an easy-to-use help function. The information on this Web site is accessible to all, including those with mobility challenges, by offering an uncluttered screen design that requires limited dexterity to navigate. KSU 20 3 3 2 2 2 2 2 1 1 KFU 26 3 3 3 3 3 3 2 2 2 PSU 33 3 3 3 2 2 2 2 2 2 PNU 21 3 3 2 2 2 2 2 2 2 DAU 21 3 3 3 2 2 2 2 2 1 1 2 1 1 1 1 0 1 0 0 0 29 3 3 3 3 3 3 3 2 2 2 0 26 3 3 3 3 3 3 3 3 2 2 0 18 3 3 2 2 2 2 1 1 1 1 0 24 3 3 3 3 2 2 2 2 2 2 0 15 3 3 3 2 2 1 1 0 0 0 2 0 0 0 0 0 34 3 3 3 3 0 33 3 3 3 3 0 28 3 3 3 3 0 31 3 3 3 3 0 24 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 2 2 3 3 3 3 2 2 3 3 2 3 1 3 2 2 2 1 3 2 1 1 0 1 24 3 3 3 3 3 3 2 2 2 23 3 3 3 3 3 3 3 2 1 18 3 2 2 2 2 2 2 2 1 20 3 3 3 3 2 2 2 1 0 22 3 3 3 3 3 2 2 1 1 0 1 1 1 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 146 The International Journal of E-Learning and Educational Technologies in the Digital Media (IJEETDM) 2(4): 141-147 The Society of Digital Information and Wireless Communications (SDIWC), 2016 ISSN: 2410-0439 (Online) 5 CONCLUSION This work comprehensively aims to evaluate five large university websites in Saudi Arabia, namely: King Saud University, King Faisal University, Princess Noura bint Abdulrahman University, Prince Sultan University and Dar Al Uloom University, including sample pages related to their staffs and departments, using a web site evaluation instrument and software evaluation tool methods. Both methods target different website evaluation criteria. The results pointed out the weakness and forces of each website, also the comparison between them. 8. 9. 10. 11. 12. ACKNOWLEDGMENTS Aside from the efforts of me, the success of work depends largely on the encouragement guidelines of many others. Our thanks to people who have been instrumental in successful completion of this work. any and the the 6 REFERENCES 1. 2. 3. 4. 5. 6. 7. M. S. Khalid, A. Mustafa and I. Haque (2008): "Application of Kano’s Model for Evaluating Information Quality," in the 2008 International Conference on Semantic Web & Web Services, Las Vegas, Nevada, 2008. L. HASAN, "Heuristic Evaluation of Three Jordanian University Websites," Informatics in Education, vol. 12, no. 2, pp. 231-151, 2013. H. Djajadikerta and T. Trireksani (2006): "Measuring University Web Site Quality: A Development of a UserPerceived Instrument and its Initial Implementation to Web sites of Accounting Departments in New Zealand’s Universities," Australia, 2006. L. HASAN (2013): "Using University Ranking System to Predict Usability of University Websites," Journal of Information Systems and Technology Management : JISTEM, vol. 10, no. 2, pp. 235-250, 2013. "Selecting Web Accessibility Evaluation Tools," [Online]. Available: http://www.w3.org/WAI/eval/selectingtools.html Even Grounds Inc., Accessibility Consulting, "Manual And Automated Web Accessibility Testing," [Online]. Available: http://evengrounds.com/articles/manual-andautomated-web-accessibility-testing. R. Rutter, P. H. Lauke, C. Waddell, J. Thatcher, S. L. Henry, B. Lawson, A. Kirkpatrick, C. Heilmann, M. R. Burks, B. Regan and M. Urban (2006): Web Accessibility: Web Standards and Regulatory 13. 14. 15. 16. Compliance, NewYork: Springer-Verlag NewYork Inc., 2006. J. Evener, "Criteria for Evaluating the Accuracy/Validity of a Website," University of ST.Augustine for health science. The EETAP Resource Library, "Evaluating the Content of Websites," Environmental Education and Training Partnership, Ohio. H. M. Selim (2012): "Content Evaluation Criteria for General Websites: Analysis and Comparison," International Journal of Online Marketing, vol. 2, no. 3, pp. 21-38, 2012. R. Sinha, M. Hearst and M. Ivory, "Content or Graphics? An Empirical Analysis of Criteria for AwardWinning Websites," University of California, Berkeley, California. J. Nielsen, "Usability 101: Introduction to Usabilit," NN/g Nielsen Norman Group, 4 1 2012. [Online]. Available: http://www.nngroup.com/articles/usability101-introduction-to-usability/ A. Al-Badi, S. Ali and T. Al-Balushi (2012): "Ergonomics of usability/accessibility-ready websites: Tools and guidelines," Webology, vol. 9, no. 2, pp. 130, 2012. Institute of Museum and Library Services SPARKS! Ignition Grant, "WEBCHECK: The Websites Evaluation Instrument," Knowledge Quest, vol. 42, no. 3, pp. 58-63, 2014. Silktide , [Online]. Available: http://sitebeam.net/ Ruth V. Small and Marilyn P. Arnone (2013): WebCHECK Professional (formerly WebMAC Professional, copyright 1997, revised 2010), Center for Digital Literacy, Syracuse, NY, © 2013. 147