Evaluation of search engine results pages in various languages and contents: the changes in strategies and criteria

This study aims to discover how domain familiarity and language affect the cognitive load and the strategies applied for the evaluation of search engine results pages (SERP).

This study used an experimental research design. The pattern of the experiment was based upon repeated measures design. Each student was given four SERPs varying in two dimensions: language and content. The criteria of students to decide on the three best links within the SERP, the reasoning behind their selection, and their perceived cognitive load of the given task were the repeated measures collected from each participant.

The evaluation criteria changed according to the language and task type. The cognitive load was reported higher when the content was presented in English or when the content was academic. Regarding the search strategies, a majority of students trusted familiar sources or relied on keywords they found in the short description of the links. A qualitative analysis showed that students can be grouped into different types according to the reasons they stated for their choices. Source seeker, keyword seeker and specific information seeker were the most common types observed.

This study has an international scope with regard to data collection. Moreover, the tasks and findings contribute to the literature on information literacy.