The effect of task domain on search

Reviewer: Donald Harris Kraft

An interesting study of Web searching behavior, exploring the effects of domain on searching, is presented in this paper. A relatively small sample of 48 Canadian adults-all relatively young, well educated, and experienced, but with moderate experience as Web searchers-was selected. The members of the sample group were given a set of queries to search on, using a slightly modified Google interface. The queries were in the domains of consumer health, general research, shopping, and travel, with specific problem statements coming from the interactive track of the Tenth Annual Text Retrieval Conference (TREC). Some of the queries, assigned statistically, had portions that allowed users to personalize the problem statement. Users could search with keywords or use the Google categories. The authors collected data on the demographics of the searchers, and on pre- and post-search evaluations based on questionnaires. They also used audiotape to record semistructured interviews with the searchers. In addition, they used transaction log analysis to record the titles and uniform resource locators (URLs) of visited sites, the keystrokes entered, and time stamps. Finally, they used screen capture software to record video of the search process. The authors' analysis revealed some interesting results, including modest but statistically significant differences across domains, with regard to efficiency (time in search states). For example, queries in the research domain caused more printing than in the shopping domain. Moreover, the searchers spent less time using categories in the research domain than in the travel domain, and more time looking at site contents in the shopping and travel domains than in the other two domains. There were similar findings for effectiveness measures. Pages found in the travel domain tended to be less about the topic than those found in the research domain, and searchers seemed less certain about the quality of the answers in the shopping domain. In addition, the searchers seemed to find searching the shopping domain to be more difficult and less satisfying. In terms of search strategy, categories were used less in the consumer health and research domains. The authors note that this might be because of an inherent bias in the interface toward the use of queries rather than categories. The authors provide a detailed discussion of the search processes in the various domains. The bottom line is that domain differences exist, and should be the subject of further analysis. The importance of this finding is noted by the authors, who discuss improvements in the interface for each domain. For example, in consumer health, entries on the hit list could indicate the level (medical professional or layperson) and scope (brief overview or detailed discussion) of a site, and the source of the information (research center or pharmaceutical company) could connect medical terms to their everyday equivalents, and could find means to limit the search results based on these factors. All in all, this is an important, exploratory, user-oriented study that deserves attention. Online Computing Reviews Service