Using Metrics of Curation to Evaluate Information-Based Ideation

Evaluating creativity support environments is challenging. Some approaches address people’s experiences of creativity. The present method measures creativity, across conditions, in the products that people make.

This research introduces information-based ideation (IBI), a paradigm for investigating open-ended tasks and activities in which users develop new ideas. IBI tasks span imagining, planning, and reflecting on a weekend, vacation, outfit, makeover, paper, internship, thesis, design, campaign, crisis response, career, or invention. What products do people create through engagement in IBI? Curation of digital media incorporates conceptualization, finding and choosing information objects, annotation, and synthesis. Through engagement in IBI tasks, people create curation products. This article formulates a quantitative methodology for evaluating IBI support tools, building on prior creative cognition research in engineering design to derive a battery of ideation metrics of curation. Elemental ideation metrics evaluate creativity within curated found objects. Holistic ideation metrics evaluate how a curation puts elements together.

IBI support environments are characterized by their underlying medium of curation. Curation media include lists, such as listicles, and grids, such as the boards of Pinterest.

An in-depth case study investigates information composition, an art-based medium representing a curation as a freeform visual semantic connected whole. We raise two creative cognition challenges for IBI. One challenge is overcoming fixation—for instance, when a person gets stuck in a counterproductive mental set. The other challenge is to bridge information visualization’s synthesis gap, by providing support for connecting findings. To address the challenges, we develop mixed-initiative information composition (MI²C), integrating human curation of information composition with automated agents of information retrieval and visualization.

We hypothesize that MI²C generates provocative stimuli that help users overcome fixation to become more creative on IBI tasks. We hypothesize that MI²C’s integration of curation and visualization bridges the synthesis gap to help users become more creative. To investigate these hypotheses, we apply ideation metrics of curation to interpret results from experiments with 44 and 49 participants.