Functionality Assessment Checklist for Evaluating Geoportals Useful in Planning Sustainable Tourism

Abstract

Sustainable tourism minimises the adverse impact of tourism on the natural environment and local culture while stimulating the socioeconomic development of regions or even countries. Geoportals and (geo)informational mashup portals significantly affect sustainable tourism planning through modern computer solutions for more sustainable planning of tourist activities on the demand and supply sides. This study had two research aims: (1) to develop a checklist for assessing the functionality of tourist geoportals and evaluate it and (2) to assess the inventory of functions useful for sustainable tourism planning available at selected geoportals with the checklist. The aims were pursued with an original research tool, the Functionality Assessment Checklist (FAC). The FAC is a set of original criteria useful for assessing the quality of tourist geoportals. This study investigated the following research questions: (Q1) What functions should be included on a checklist for assessing the functionality of geoportals useful for sustainable tourism planning? (Q2) What functions should be included in a geoportal to facilitate sustainable tourism planning? The original contributions of this article are (1) the checklist for assessing the functionality of tourist geoportals and (2) the assessment of the impact of geoportal’s functionality on the possibility of planning sustainable tourism. The functionality assessment revealed that the tested geoportals have most of the tourism, informational, educational, and use-related functions. This suggests they can be useful for sustainable tourism planning.

1. Introduction

Geoportals are Internet gateways that organise such types of content and services as directories, search tools, community information, auxiliary resources, data, and applications [1]. Map services provide users with web services offering geospatial data. Through these services, spatial data can be displayed but also filtered and queried for specific attribute values or spatial characteristics [2]. This makes the geoportal a versatile tool. It is used, for example, in crisis management, spatial planning, navigation, or resource management in general [3]. They also come in handy for tourism stakeholders on the supply and demand side alike. Geoportals allow tourists to create custom travel itineraries [4] and plan trips effectively by simultaneously taking into consideration the place, time, and cost enriched with unique local conditions, be it natural, cultural, legal, or related to personal safety [5]. Their value is elevated with integrated systems of suggestions, comments, recommendations, and booking and payment. On the road, geoportals provide detailed maps for effective navigation in unfamiliar areas, including routing with alternative routes or means of transport [6]. Therefore, geoportals promote the effective use of time and resources when planning travel, covering—often automatically—tourist attractions, hospitality locations, and other components of a successful journey [7].

Map portals are a component of IT systems for managing geospatial databases. Apart from providing (geo)data, geoportals provide educational content related to regional history, culture, and heritage [8]. On the flip side, the growing number of various (geo)information portals and user-generated content can be overwhelming and turn trip planning into a tedious task. The most pronounced issues are the mass of information potentially causing confusion [9] and uncertainty about the reliability of the content [10].

Geoportals provide cultural heritage information on a macro level—for the entire country or region—and on a micro scale when a place or location with their local tourist attractions, such as museums, heritage sites, national parks, or tourist trails, are presented [8]. Map services often go beyond spatial information related to the location of the feature and characteristics of its surroundings to offer practical insights, which are valuable when planning a trip, such as opening hours, entry fees, parking amenities, public transport access, or other circumstances and attributes typical of the place [6]. All this is often enhanced with user-generated accounts and comments, which can be useful for purchasing decisions [4]. Additionally, ‘static’ content is often supported by real-time information such as current weather, traffic alerts, and other local conditions. All this makes geoportals increasingly useful for planning trips.

1.1. Geoportal Quality Assessment, Including Tourist Geoportals

The practical potential of geoportals for tourism has been studied extensively. Singh et al. [11] identified a link between geospatial platforms and tourism planning. They demonstrated that geoportals can have multiple applications in tourism management and can be useful in trip planning. Singh et al. [11] employed a questionnaire survey, collected data, and analysed it statistically. They discussed the architecture and development techniques for providing travel-related information on geoportal websites that can be useful in decision-making. In their desk research, Vishnevskaya et al. [12] demonstrated that WebGIS technologies can collaborate to generate spatial data visualisations potentially useful in tourism. These visualisations come as interactive map applications, multimedia tourist guides, digital models, dynamic charts, geoinfographics, and other mashup applications [13]. Borràs et al. [14] noted that advances in online technology opened new horizons for generating customised content and service management in tourism. They presented and discussed the functionalities of a map-based tool employing geospatial technologies and AI for tourism. Its main purpose is to facilitate sustainable destination management, which can improve the profitability of tourist operators.

Their research emphasises the benefits of geoportals and map applications that support tourist activity and operations of tourist sector entities. For this reason, geoportal quality is investigated increasingly more often, compared against standards of the Open Geospatial Consortium (OGC) or International Organization for Standardization (ISO) and tested for conformity with Infrastructure for Spatial Information in the European Community (INSPIRE) and World Wide Web Consortium (W3C) standards [15]. Such technical aspects as performance and responsiveness are tested along with accessibility to people with disabilities, ergonomics, and design [16]. Auditors have looked into the usefulness and reliability of geoportals [17], comfort and frequency of use [18], and the scope of data and services [19]. The authors have employed questionnaire surveys and statistical methods [11], desk research analyses [12], web applications [20], checklists and scoring [21], with the latter being most often used in usability studies [22].

1.2. Aim and Research Gap

Checklists are widely used in usability studies [23]. For example, the Usability Metric for User Experience (UMUX) and its shorter form variant, UMUX-LITE, complement standardised usability questionnaires [22]. UMUX is designed to yield results similar to outcomes of the 10-item System Usability Scale (SUS). It is founded on the ISO 9241-11 definition of usability [24]. The Standardised User Experience Percentile Rank Questionnaire (SUPR-Q) consists of eight items to measure four website factors: usability, trust, appearance, and loyalty [25]. Another metric, the System Usability Scale (SUS), was created in 1986 by John Brooke and has been used extensively in various industries in application and system tests [21]. Still, checklists are criticised as well. Meola [26] concluded that it is hard to implement the checklist model in practice, and it encourages a mechanistic way of evaluating that is at odds with critical thinking.

The literature suggests that usability studies are conducted more readily than functionality studies [16]. This may be due to the availability of diverse, relevant techniques and tools, including checklists for evaluating the perceived, subjective comfort of use [23], while Functionality Assessment Checklists are nowhere to be found. This offers a research gap worth filling in. Therefore, the present study provides new functionality assessment tools, focusing on thematic geoportals in this case, and starts a discussion on the scope of the assessment.

This study had two research aims: (1) to develop a checklist for assessing the functionality of tourist geoportals and evaluate it and (2) to assess the inventory of functions useful for sustainable tourism planning available at selected geoportals with the checklist. This study investigated the following research questions:

• Q1: What functions should be included on a checklist for assessing the functionality of geoportals useful for sustainable tourism planning?
• Q2: What functions should be included in a geoportal to facilitate sustainable tourism planning?

The tangible contributions of this article are the following:

• A checklist for assessing tourist geoportal functionality;
• The assessment of selected tourist geoportals focused on sustainable tourism planning.

This article is further organised as follows: Section 2 defines the concepts of usefulness, functionality, and usability and pinpoints differences between these concepts, establishing their relevance to website quality studies. Next, we describe how checklists are applied in website quality research and how geoportals differ from other websites. Section 3 introduces the methodology and details the functionality checklist development process—how it was evaluated—and the object of study. Section 4 presents the structure and content of the checklist with the case study results. Section 5 discusses the results and selected attributes of the checklist. The last component is a summary of practical implications and limitations.

2. Background

2.1. Usefulness of Geoportals in Sustainable Tourism Planning

Geoportals come in many flavours. They can present information on the natural environment, infrastructure, legal, cultural, or social conditions as real-time or historical data and more or less static in form. Tourist geoportals provide spatially referenced information useful in such domains as tourism or the promotion of regions and cultural heritage. The websites differ in terms of development techniques, purpose (role), and business model. Many geoportals present ‘static’ georeferenced information, which is updated relatively seldom, such as lists of environment protection sites or tourist trails [27]. Such directories are rarely modified compared with information on road traffic or weather conditions, so the geoportals are updated infrequently.

Geoportals usually present interactive maps where users can find various layers (thematic maps), including tourist attractions and tourist trails, hospitality locations, and other useful information. Their primary functions are informative and educational [28]. Certain stability (insignificant variability) of the (geo)information is an important attribute of these map services. These geoportals can be classified as informational and educational geoportals (acronym IE). Interactive tourist guides are a special type of IE geoportals. They are usually mashup multimedia applications combining various functions with a base map [29]. The applications can provide detailed historical descriptions, practical tips, or suggestions regarding landmarks (acronym TG). Both IE and TG geoportals are one-way communication tools. Information is presented to the audience, who are unable to contribute, for example, through comments. Such a feature is implemented in social geoportals for travellers [30]. These online mashup platforms are called geocollaborative portals. They are places where travellers can share insights, reviews, and tips related to travel accompanied by information on a map, i.e., spatially referenced information (acronym GP). Navigation applications for tourists provide real-time information on a map (acronym NA). They provide navigation services, hints concerning tourist routes, location, and other information useful on the go, such as road incidents or weather conditions [31]. Many geoportals present tourist information, such as cultural events, festivals, sporting events, information on walking, biking, and other routes, and e-commerce map and travel planner applications (acronym ECG). They facilitate route planning, finding accommodation, ticket booking, car rental, etc. Another important group is regional geoportals (acronym RG). These websites or web applications focus on specific regions and offer detailed tourist information on local attractions, cuisine, history, and other matters specific to the geographic area [11,32]. Note that map services usually provide various combinations of the functions listed above, which hinders clear-cut classification. A synergistic approach seems to be the most reasonable when planning tourist trips. Simultaneous use of various geoportals can yield the best results for sustainable tourism planning. Nevertheless, regardless of its type, a geoportal should have certain basic functions that define its functionality.

2.2. How Are Usefulness, Functionality, and Usability Different?

Usefulness refers to the general value, effectiveness, or efficiency of an object, tool, skill, or piece of information to meet specific needs, solve specific problems, or attain the expected objectives. Usefulness assessment can be subjective and most often depends on the individual abilities of the auditor, their expectations, requirements, and circumstances. Usefulness is sometimes considered a metric of quality, value, or effectiveness for a specific purpose [33]. Usefulness is followed by usability. It is a more specific variation of the former focused on ‘user friendliness’, i.e., the comfort of use. This means that usability concerns the ease and comfort of using a product or system [34].

This quality triad is complemented by functionality, which is the ability of a product, system, program, or service to perform specific tasks [30]. Functionality answers the question of what can be done using a specific product, system, or program [14]. Generally, functionality is a set of abilities of a program or system, a set of software functions constituting a coherent whole in terms of use, or a set of specific tools of a product, system, or software package. On a micro level, functionality is actualised through functions, i.e., the unique capabilities of an object to perform individual tasks. The right set of high-usability functions is essential for an exhaustive and satisfactory user experience. All these qualities are analysed when website quality is evaluated. Such evaluations often employ checklists.

2.3. Checklists in Website Quality Assessments

The checklist is a tool for organising the work of the auditor, tester, or whoever performs the evaluation. It is usually a formal document, such as a hard copy or digital form. The form contains a list of items, questions, or issues (criteria), usually in a specific order, that should, must, or might be performed, met, or achieved by a specific process, project, or job. Checklists can improve the objectivity, credibility, and repeatability of website quality assessment [35].

Checklists are most often employed to assess the quality of websites, particularly in usability studies [22], but also to assess website information content [36]. In this case, they can identify potential gaps in the website’s information content. Still, checklists are employed in more contexts than simply website quality evaluation. Other applications include business development planning, operational monitoring and governance, and outcome assessment, both under formative (in-process evaluation) and summative (post-factum, summary) evaluation [37].

Checklists are often used to evaluate website quality. Raward [35] proposed a concept of a checklist aimed to ensure and/or verify the quality of academic library websites. He used the Human-Computer Interface (HCI) literature when compiling the checklist. Still, Raward’s [35] checklist actually assesses the use attributes of the websites because, in his article, Raward [35] (p. 123) suggested that ‘the design will be most successful when a user-centred design model is included in the development and implementation of academic library web pages.’ Linares and Romero [38] employed a checklist to evaluate the functionality of foreign language teaching websites. The authors demonstrated that the checklist helps foreign language teachers choose resources that best meet their communicative teaching purposes. Items on Linares and Romero’s [38] checklist are divided into technical aspects (related to the ease of use and reliability criteria), pedagogical considerations (referring to the content quality criterion and the material potentiality criterion), and language approach principles (that reflected on criteria such as autonomy, language contextualisation, language skills integration, interculturality, and interaction). In short, Linares and Romero [38] included aspects of usability and functionality in their checklist (

Table 1. Examples of checklist applications in website and web application quality assessment.

Checklist Purpose	How It Was Developed	Keywords	Reference
Checklists to detect potential predatory biomedical journals	Comparison of content of multiple checklists and their metric characteristics	checklists, checklist tool	Cukier et al. [39]
Academic library website quality assurance and/or verification	Developed based on principles derived from the Human-Computer Interface (HCI) literature	Usability Index Checklist, user acceptance and usability, user-centred design model	Raward [35]
Development of an e-commerce website that can contribute to trust and purchase intention of consumers	The methodology was adapted from previous lab-based usability testing studies and Kansei engineering	e-commerce design checklist, usability testing, usability requirements	Goh et al. [40]
Assessment of functionality of websites for foreign language teaching	A checklist based on a literature review and expert opinion	checklist for evaluating language websites, website evaluation, evaluation criteria	Linares and Romero [38]
Presentation of a designer’s checklist and an evaluative instrument constructed based on the library website model	The library website model was developed through a Delphi study by two panels of experts. Then, a designer’s checklist and an evaluative instrument were developed from the proposed model through a series of brainstorming sessions.	designer’s checklist, evaluative instrument, web design features	Wijayaratne and Singh [41]
Mobile application functionality assessment	Assessment criteria based on a review of literature on functional requirements.	score assessment, criteria (factor) list, mobile technologies, mobile applications	Manczak and Bajak [42]
Checklist for a vision of the entire system	Own experience and interdepartmental and expert consultations. A checklist developed by experts in website design.	guidelines, checklist of issues	Cummings et al. [43]

).

Abdinnour-Helm and Chaparro [44] developed the Balanced Usability Checklist (BUC) and applied it to assess the usability of ten Palestinian hotel websites. Then, the authors ranked the websites by their total score. Cummings et al. [43] created a checklist covering issues critical for designing knowledge Internet portals: what to consider before starting, during the design phase, and during implementation, as well as technical standards and specifications. Their checklist is broad and encompasses components of the general system vision, including the purpose of the website, its objectives, and its target. At the same time, it considers technical matters related to usability and development techniques.

According to Goh et al. [40], for websites to represent high usability and functionality, their design must also ensure appropriate emotional impact. Hence, the approach proposed by Goh et al. [40] complements the functionality and usability of e-commerce websites with a design in line with Kansei Engineering, a design method that considers users’ emotions and feelings to be able to guide them. Kansei Engineering aims to present the product or service in a way that fuels the desirable emotional response to improve conversion. This approach sheds new light on checklist design, which may expand into new quality assessment planes.

Wijayaratne and Singh [41] proposed a library website model and an ensuing designer’s checklist and evaluative instrument. The model spans 60 Web content elements and 80 Web design features. The designer’s checklist has 140 items, while the evaluative instrument comprises 60 content elements and 57 design features. The tools by Wijayaratne and Singh [41] can verify whether a website has specific functions and assess its design, which is part of usability assessment.

Checklists are also used beyond the context of website quality evaluation. For example, Hosie et al. [45] proposed a framework and checklists for evaluating online learning in higher education. Manczak and Bajak [42] scored the functionality of the mobile application VisitMalopolska. Regrettably, the scoring focused on functions typical of the application. Their checklist is not versatile. It was designed for the specific application to evaluate a subjective ‘potential usefulness of specific functions to users’. Cukier et al. [39] analysed checklists for identifying predatory journals. They compared the content of the checklists and evaluated their metric characteristics. Most of the checklists they investigated were in English, could be completed in under five minutes, and contained less than twenty criteria (items, questions, points, etc.), which were not weighted. The checklists had instructions in point form, bullet form, tabular format, or listed items. Their comparative analysis offers design suggestions regarding such matters as the size of the checklist and methods for collecting responses.

Regarding website quality assessment, checklists are used during the design phase and after it is deployed. This is because the online ecosystem is dynamic. Frequent versioning and changes in the website’s environment mean that high quality is not set in stone. As a result, the quality of every website should be monitored to optimise and ensure the most stable level of conversion [46]. Auditors use checklists to test websites regardless of their type or special characteristics, which means they are applied indiscriminately to e-commerce websites and (geo)information portals, including geoportals. The websites can vary significantly, which may affect the assessment process.

3. Materials and Methods

Functionality tests aim to determine whether a specific website performs the tasks expected by its publishers and whether it has specific functions required by relevant standards. The specific objective of the functionality test can be to verify the operability of individual functions. It can be performed with functional and unit tests (functional verification of individual components of a system), integration tests (detecting issues with interaction between components, modules, or systems), or acceptance tests (verification of system completeness and its correct functioning) [47]. As a result, functionality tests are often binary (yes/no/not applicable) and employ checklists. Functionality tests evolve into usability tests, which focus on user experience and assessment of the comfort of use of program functions [22].

This study employed black-box functional tests with a checklist [48]. This means that we evaluated the functions of a system (geoportal) against a list of requirements (checklist) without considering the software architecture or being unaware of its source code.

3.1. Developing the Checklist

The checklist [Appendix A] is based on (1) a literature review and (2) an analysis of checklists by Travis [49], Raward [35], and Keevil [50]. When building the checklist, we considered attributes mentioned in research reports, including (1) attributes of geographic content: presentation of detailed geographic (thematic) maps with various map views (satellite, terrain, road) [51]; (2) search functions: whether the geoportal has effective search tools for searching the map directly and for searching the entire website and whether the search results can be filtered [52]; (3) personalisation and interactivity: whether users can create personalised content on the base map and/or add custom content [13,53]. We employed Raward’s [35] methods when developing the checklist.

Raward [35] used his checklist to evaluate the quality of Australian academic library websites (AALW). His checklist is based on the Keevil Web Usability Index Checklist [50]. He chose only those questions from Keevil’s checklist [50] that he and a panel of experts deemed relevant for evaluating the quality of AALW. Then, Raward expanded this modified Keevil’s list [50] with attributes relevant to accessibility to people with disabilities. As a result, Raward’s list [35] consists of 100 questions in four main sections: finding information, understanding information, supporting user tasks, and presenting information. Raward believes [35] (p. 128) that checklists contribute numerous benefits to website quality assessments, including that (1) the checklist is cost-effective and easily implemented, (2) one auditor is enough to complete the checklist, and (3) the checklist can be amended in accordance with advances in usability and technical communication techniques. Although Raward’s research [35] and Keevil’s checklist [50] concerned usability, their checklists were binary, with the main predefined answers being YES (1 point) or NO (0 points) with optional comments [46].

When developing the checklist, we analysed questionnaires for usability studies based on UMUX, UMUX-LITE, SUPR-Q, and SUS [21,22,25]. Questionnaire items were completed based on exploratory research during a cognitive walkthrough and then verified against CDC guidelines (Checklist Development Checklist). CDC is useful for planning and proceeding through the steps to develop and employ a good-quality checklist [37]. The exploratory research consisted of browsing through tourist geoportals and recording functions useful for planning trips or holidays [36].

The main assumption for the checklist was that it would be binary in nature. Presence of specific functions would be assessed as ‘conforms’—‘does not conform’, ‘is’—‘is not’, ‘done’—‘not done’, coded as 1 = is, 0 = is not. This approach helps with summarising and drawing conclusions [37]. Checklists can be more complex. For example, they can include qualitative scales, comments, or open-ended questions [22]. We refrained from weighting the functions, assuming that they were all equally important for functionality, as did Manczak and Bajak [42].

The checklist contains no items concerning user experience attributes, such as the perceived user-friendliness of the interface, service stability, and performance, including time to load and compatibility with software, mostly web browsers. Another assumption is that being ‘up to date’ is not a functional attribute while presenting news in real time is.

3.2. Object of Analysis

The selection of the tested geoportals was purposeful. The assessment covered five geoportals useful for tourism planning: three Polish, one Czech (Geoportal Prague), and a Lithuanian geoportal (

Table 2. Selected attributes of the assessed geoportals.

Acronym	Type	Geoportal’s Name	Description and Publisher
G1	IE, TG, RG	Mapa REGIA	REGIA: regional geoinformational environment service. State Enterprise Centre of Registers, Vilnius, Lithuania.
G2	IE, TG, RG	Geoportal Prague	Geographic data and maps of the capital city of Prague. Prague Institute of Planning and Development, Prague, Czech Republic.
G3	IE, TG	Geoportal ‘Zabytek’	Data on monuments and heritage objects National Institute of Cultural Heritage, Warsaw, Poland
G4	IE, TG, NA	Geoportal Tatra	Map with the latest tourist information by the Tatra National Park. Tatra National Park, Zakopane, Poland.
G5	IE, TG, NA, GP	Mapa Turystyczna	Map of mountain tourist trails. Routing calculator and planner. Kraków, Poland.

IE—informational and educational map portal; TG—tourist map portal; GP—geocollaborative portals; NA—navigation tourist portal; RG—regional geoportal. G1: https://regia.lt/ (accessed on 1 May 2024); G2: https://geoportalpraha.cz/ (accessed on 1 May 2024); G3: https://zabytek.pl/pl/mapa/ (accessed on 1 May 2024); G4: http://www.geoportaltatry.pl/ (accessed on 1 May 2024); G5: https://mapa-turystyczna.pl/ (accessed on 1 May 2024).

). These geoportals were selected because they present regional and local information useful for planning tourist trips (demand side) and providing auxiliary services to public administration (supply side). Moreover, the selected geoportals had to conform to technical and subject-matter criteria, including W3C and WCAG (Web Content Accessibility Guidelines). These geoportals are interoperational, interactive, responsive, and accessible. They also ensure current and accurate data.

Mapa REGIA (G1), or the regional geoinformational environment service, is a map tool developed for municipalities: their residents, authorities, and local businesses. REGIA aims to facilitate the making of geographically based decisions and the exchange of information. The website additionally publishes information about recreation and tourism, including accommodation, catering, entertainment, bike and walking paths, museums, and such public locations as religious buildings, cultural centres, or commercial facilities. REGIA also publishes other information useful for planning trips related to infrastructure and transport, including railway and bus stations, transit stops, service stations, car parks, or garages.

Geoportal Prague (G2) provides geographical data and thematic maps of Prague, the capital of the Czech Republic and a popular tourist destination. The service publishes a 3D model of Prague in a virtual landscape, an interactive map of cultural heritage conservation with various listed heritage objects, including buildings and archaeological sites, old maps of the city centre, etc. The data help comprehend the qualities of the unique urbanscape and structure of Prague. Zabytek.pl (G3) is kept by the National Institute of Cultural Heritage in Poland. It provides multimedia information about Polish monuments and heritage objects. It offers past and current photographs, digitalised documentation from the state heritage register, 3D models of heritage objects, point clouds, panoramic images, and documentaries. The geoportal presents spatially referenced heritage.

Geoportal Tatra (G4) presents maps with current tourist information from the Tatra National Park (TNP). The website provides up-to-date information about trail closures, some tourist facilities, and trail obstacles. It also offers information about TNP’s cultural heritage, including herding, religious objects, and mining and metallurgy (drifts and mines in Polish Tatra). ‘Mapa Turystyczna’ (Tourist Map) (G5) offers tourist trail hiking planning capabilities.

4. Results

4.1. Structure and Content of the FAC

The checklist (Functionality Assessment Checklist, FAC) can verify whether a geoportal has functions that are useful for planning a journey, tourist trip, or everyday commuting. It covers 60 unique functions of seven types in three categories (

Table 3. Summary of domains and functions on the checklist.

Functional Category	Domain	Function Type	No. of Functions
Spatial presentation of data	Tourism	Tourist functions	6
	Transport	Logistic functions	10
	Information and education	Informational and educational functions	13
Transactions and networking	Transactions	Transactional functions	3
	Social networks and networking	Networking functions	9
Development technique	Versatility	Use functions	16
	Data sharing	Data functions	3
Total			60

). The spreadsheet with the checklist is provided as Appendix A.

During the desk research, we noted that a checklist that is too extensive and too detailed hinders the audit process, including increasing the time necessary. Long and detailed checklists are also less flexible, which can decrease their versatility and practical value when assessing the quality of websites. Therefore, FAC combines selected functions into groups and describes them unambiguously and broadly. Note that the proposed functional division is subjective. The FAC can be modified and adapted to special-purpose geoportals and map services that have dedicated functions.

The set of tourist functions in the ‘spatial presentation of data’ category holds functions related to the presentation of information useful for tourist trip planning, including information about public and agricultural service points, local businesses relevant to tourism, tourist trails, accommodation, and tourist attractions. This set includes functions for sharing maps with information about tangible and intangible cultural heritage, including movable and immovable heritage objects and natural heritage. It contains six unique functions.

The ‘spatial presentation of data’ category’s logistic functions are related to trip and journey planning, point-to-point route optimisation, finding alternative routes and means of transport, and real-time traffic alerts. This set includes such functions as geolocation and presentation of thematic maps, including service stations, electromobility, and charging stations for electric vehicles. This set contains ten unique functions.

The set of informational and educational functions of the ‘spatial data presentation category’ covers functions linked to educating and informing about environmental, socioeconomic, and cultural conditions. These functions focus on thematic maps with informational and educational content, such as places and areas that are protected and valuable in terms of natural environment and culture. This function set also includes the presentation of various maps, including orthophotos and informational and educational thematic maps, as well as old and current maps. This set contains thirteen different functions.

The transactional set in the ‘transactions and networking’ category contains functions relevant to the purchasing process, such as booking, buying, presentation, and sale of tourist services, as well as tickets to tourist attractions and public transport fees. This set contains three unique functions. The same category contains networking functions related to building a geoportal community. To this end, the set contains functions such as capabilities to add user-generated content, including multimedia and commenting. This set also includes functions to share a view of (a piece of) a map and interact with other services or present content in other languages (internationalisation). This set contains eight different functions.

The set of use functions of the ‘development technique’ category has functions that improve the service’s versatility and facilitate its use, including responsiveness to view the geoportal on any device, availability of the help system, including a FAQ, and accessibility functions for people with disabilities. This set contains sixteen different functions. Data functions in the same category contain data download (export) functions (such as WMS, WMTS, and NTM) and user data import. This set contains three different functions.

4.2. Evaluation of the Checklist—A Case Study

The functional tests aim to evaluate the checklist and assess the inventory of functions facilitating tourist trip planning offered by the geoportals. The object of the tests is the scope of useful data presented by geoportals and how it is presented. We determined that the test environment does not affect functionality test outcomes because they assess components of client–server tourist geoportals. The tests were conducted on a desktop in thin client mode, i.e., on a client machine independent of the server application.

The maximum score was 60 points, reflecting the sixty functions on the FAC. The highest absolute level of functionality under the employed research design represents G2, while the lowest is G3 (

Table 4. Results of the quantitative FAC functionality tests.

Function Type	Total No. of Functions	No. of Identified Functions
		G1	G2	G3	G4	G5	Total	Percentage (%)
Tourist functions	6	6	6	3	6	6	27	90.0
Logistic functions	10	3	5	2	3	6	19	38.0
Informational and educational functions	13	10	12	5	8	5	40	61.5
Transactional functions	3	0	0	0	0	0	0	0.0
Networking functions	9	3	3	1	0	6	13	28.9
Use functions	16	13	14	10	6	13	56	70.0
Data functions	3	0	1	0	0	0	1	6.7
Total	60	35	41	21	23	36	156	52.0
Percentage (%)	100.0	58.3	68.3	35.0	38.3	60.0	N/A	N/A

G1: https://regia.lt/ (accessed on 1 May 2024); G2: https://geoportalpraha.cz/ (accessed on 1 May 2024); G3: https://zabytek.pl/pl/mapa/ (accessed on 1 May 2024); G4: http://www.geoportaltatry.pl/ (accessed on 1 May 2024); G5: https://mapa-turystyczna.pl/ (accessed on 1 May 2024).

). Most of the geoportals have tourist (90%), use (70%), and informational and educational functions (61.5%). The least popular type of function was data functions (6.7%). Transactional functions were not recorded at all. The tested portals had 156 out of 300 potential functions in total (52%).

The tested websites are updated from time to time (at a set interval or after a data package is available) rather than providing information in real time. The services have informational and educational functions but do not provide interactive planning of trips, point-to-point routes (POI), etc. The geoportals are mostly one-way communication tools: they provide—rather static—information but have no functions to post user-generated content. Usually, only the home page of the geoportals was available in foreign languages. Such descriptive components of the map viewer, such as the legend, layer browser, or navigation buttons, are translated less often. This fact poses a certain difficulty in assessing the presence of these functions.

Geoportals focused on a narrow topic have an inherently narrower range of functions necessary to deliver the content. In such a case, a lower functionality score does not entail that the geoportal is of poorer quality but rather that its functions are less numerous. This means that functionality is one of many attributes that should be evaluated as part of the general geoportal quality assessment.

None of the geoportals had transactional functions; users cannot book accommodation or buy entry or transport tickets there. Transactional functions can be found on mashup booking and sale portals today. These websites use the map component to present the object, place, or service in space. Tourist geoportals could potentially soon have transactional functions.

The tested geoportals do not have data functions. Today, these functions are typical of geoportals that belong to state spatial information infrastructure, local government geoportals, and geoportals of entities engaging in spatial planning, geodesy, spatial management, land cover analysis, or crisis management [54]. The functions are vital to ensuring access to geoinformation and might become standard in any geoportal regardless of its purpose as technology advances.

5. Discussion

New applications are not always tested for usability, user usefulness, or functionality. Engelmo-Moriche et al. [55] proposed an architecture for a tourist geoportal of Estremadura (Spain). According to the authors, the geoportal is useful for planning and completing trips and gives access to information on local conditions before arrival. Engelmo-Moriche et al. [55] developed the tourist geoportal with free software to present the tourist offer, cultural heritage, and natural assets of Estremadura and offer accommodation and tourist trail information. The authors proposed components of the geoportal and prototyped it. No usability or functionality tests were performed. Luppichini et al. [56] employed the same approach. They built a map service using free and open programming libraries. According to the authors, their research was a step forward compared with the state of the art. They created a triple link between real and virtual tours and made them available on a platform that allowed users to consult geographic information, as well as other types of information, such as information panels, tabulated data, or images. Still, just like in the case of the Estremadura application [55], these authors did not evaluate the usability or functionality of their map application.

Important research questions that we had not anticipated when planning this study emerged during the process: which geoportal functions can be considered critical for sustainable tourism planning, and which are of limited relevance in this regard? Note that the overall number of geoportal’s functions is a quantitative, not qualitative, measure. Some functions may be more relevant to (sustainable) tourism planning than others. Therefore, one could consider weighing the functions, which would nevertheless complicate the evaluation model and has to be investigated in depth. Functions that are particularly useful for tourism planning (also sustainable tourism) are tourist and logistic functions, especially those that provide information in real time. Transactional and data functions will become more relevant in the future. Informational and educational use and networking functions may be of lesser importance, although it may depend on individual user preferences and skills. The question of whether geoportal currency is a functionality attribute emerged during the research process. If so, how can it be described in the FAC form, considering that functionality is a set of capabilities (tools) of a system or program, and function is a specific capability (tool) that the system or software has [14]. In the broad sense, currency seems to be a usability characteristic or usefulness attribute rather than a system function. In a way, real-time presentation functions and transactional and networking functions attest to geoportal currency in the FAC.

A website can have specific functions, which could use stale data. In practical terms, it means that currency should be considered a separate category, which is ambiguous and does not wholly and directly fit either functionality or usability. Currency should be part of a broader website quality assessment, covering both functionality and usability.

Selected Attributes of the FAC

Preparing a checklist for the comprehensive evaluation of geoportal’s functionality consists of multiple stages. First, one has to define such concepts as usability, functionality, function, and usefulness as precisely as possible. These categories encompass specific quality attributes. This division is fundamental for identifying characteristics for functionality assessment.

Functionality consists of a website’s functions that constitute a coherent whole in terms of use. ‘Functional’ means serving its role well, having a use-related function, and being useful [57]. Considering this definition and the significant diversity of geoportals as both geoinformation and geocomputer systems, it is a challenge to develop an exhaustive list of functions required of the service. It seems even harder to compose a list of functions required of a tourist geoportal to be considered functional in sustainable tourism planning.

Geoportals usually have a set of default functions and are founded on a base map. Nevertheless, each geoportal should be assessed (including in terms of functionality, SEO, UX, content, performance, and other audits) as an individual, in-depth case study. It is because each geoportal is unique and has a specific purpose. As a result, it can have specific functions, while others may be unavailable because of the geoportal’s role and vision of the publisher.

Functionality is complex and, therefore, difficult to assess. The checklist is the spine for a consolidated and coherent research design. Its purpose is to facilitate as objective and exhaustive as possible an assessment of the functionality of an individual service to enable the comparison of multiple portals. To complete these tasks properly, the properties of checklists should not be taken lightly. Checklists employed by auditors (assessors) must conform to certain requirements. Hence, our FAC meets the following original conditions:

• The requirement of non-negative assessment result and constant extreme values—the result of the evaluation is always non-negative and assumes one of three values: 1, 0, or N/A;
• The requirement of additivity—the checklist’s results can be aggregated, and partial scores can be totalled;
• The requirement of scorability—the checklist can describe functionalities with a synthetic aggregate score;
• The requirement of comparable results—the investigated websites can be ranked based on the functionality assessment and aggregate final score;
• The requirement of automatic or semiautomatic testing—the digital checklist form automatically totals test results in the summary table to streamline the process;
• The requirement of scalability—the number of evaluated services is unlimited, or the results can be interpreted based on a conventional quality scale;
• The requirement of assessment flexibility—the checklist can be modified (customised);
• The requirement of potential evaluation—the checklist can calculate the realised potential of the design, i.e., the score compared to the maximum score under a specific research design.

Checklists should be easy to use, and the result should be easy to interpret. The following scoring scale is proposed for the FAC evaluation of geoportals, which covers all functions listed on the FAC:

• 0–20—basic functionality. The geoportal is static and offers no networking or transactional functions. It is an educational and informational geoportal with basic use and information functions;
• 21–40—limited functionality. The geoportal has basic use functions and serves limited informational and educational functions. Its usefulness for tourism planning is limited, and it may lack some basic functions, especially tourism-related ones;
• 41–60—satisfactory functionality. The geoportal has most of the use functions. It fulfils its informational and educational role and is highly useful in tourism planning and everyday travelling; it may lack some transactional functions.

Note that there is no single universal model of a website audit. An audit’s structure emerges from the auditor’s expertise, skills, and experience. At the end of the day, the auditor assembles the test toolkit and interprets the results. It means that the auditor makes the final (evidence-based) decision whether a website—such as a geoportal—is or is not functional. Therefore, the proposed FAC scale is not set in stone. It is a proposal for a synthetic interpretation of the results, which may be of particular use in quantitative and comparative research.

The FAC can be useful at the development stage. It improves control over the inventory of geoportal’s components that are useful for users. The checklist can also be used in comparative and competitive analyses of tourist portals.

6. Conclusions

This study sheds new light on testing tourist geoportal quality and provides new tools for assessing their functionality. One can better assess the potential of tourist geoportals for sustainable tourism planning after understanding the role of functionality and when aware of the number and types of functions they have.

This article presents a new approach for testing geoportal quality. It does not include user-friendliness or geodata quality in the assessment of geoportal quality. Here, high quality of a tourist geoportal means functionality considered as the number of functions it has that improve its usefulness in sustainable tourism planning. Note that the functionality assessment revealed that the tested geoportals have most of the tourist, informational and educational, and use functions. This suggests they can be useful for sustainable tourism planning (under the employed research design).

A geoportal that is useful for sustainable tourism planning should have tourist, logistic, and informational and educational functions categorised as ‘spatial presentation of data’, such as viewing maps with information about accommodation, cultural heritage objects, tourist trails, and tourist information centres, but also functions for optimised point to point route planning. Use functions are also important. They make it easier to use tourist and logistic functions by converting outcomes of spatial analyses into raster files and then hardcopy documents (printouts), for example. Moreover, networking functions can reduce various barriers—such as language and disability—and help tap into the knowledge and experience contributed by the geoportal community. Transactional, sale, and data functions can be expected to appear in more geoportals in the future. These functions will allow tourists and organisations managing tourist areas to better manage their processes and resources, leading to better decisions. Full geoportal functionality can streamline sustainable tourism planning by promoting local entrepreneurship and economy, responsible travelling, and minimising tourist impact on the natural environment and local communities.

Practical Implications and Limitations

The subjective selection of a limited number of tested geoportals is a limitation of this study. Moreover, any functionality test is conducted by an expert (auditor, tester) and bears the risk of ‘human error’. Inattention during exploratory research and cognitive walkthrough can result in missing a function from the questionnaire and one point fewer for the geoportal (there may be various reasons for the auditor to miss a function when browsing a geoportal). As a result, the assessment score can be understated. It is recommended that at least two independent auditors carry out the assessment and compare the results afterwards. Note that there are basically three ways expert judgement objectivism can be improved: (1) It can be improved through a greater number of evaluators in the expert assessment, such as experts or users. A functionality assessment by a larger number of evaluators improves the results’ reliability. (2) It can be improved with two independent groups of evaluators, experts and ‘regular users’, conducting the functionality tests and then confronting their outputs. This approach corresponds with the first one and improves the results’ reliability. (3) It can be improved by using automatic testing tools, such as web applications. Still, according to our best knowledge, no tools for evaluating website functionality, such as geoportals, are available today.

Therefore, in order to reduce the risk of human error to a minimum and improve objectivity, further conceptual and design work should focus on a web application for automatic or partly automatic functionality auditing. Such a web application for evaluating functionality, including the functionality of tourist geoportals, would address a certain research gap and market niche as a useful audit tool. The design concept of this tool could be modelled after web applications that automatically evaluate search engine optimisation of websites or assign websites a synthetic aggregate quality score. The synergy between expert and algorithmic audits could improve testing quality and streamline the evaluation of functionality.

Functionality assessment can follow one of two approaches. The first one makes no distinction among geoportal types and allows only two audit response categories: (1) function found (1 point) or (2) function not found (0 points). Note, however, that this approach ‘discriminates’ against informational and educational geoportals, for example, because they have inherently fewer transactional functions. So, it is obvious from the beginning that they will score lower. The other approach involves predetermining the type of geoportal before assessing the functionality. The typological choice leads to disregarding selected functions during the assessment process. For example, transactional functions are not evaluated for typical informational and educational geoportals. In this case, the auditor chooses between three response categories: (1) function found (1 point), (2) function not found (0 points), and (3) not applicable. This reduces the maximum total number of points available for the particular functionality assessment. This approach has a downside: the geoportal type is chosen arbitrarily, and the set of functions typical of the type is defined subjectively.