Glyphs for exploring crowd-sourced subjective survey classification
Authors: 
A. Kachkaev, J. Wood, J. DykesAuthors Info & Claims
A. Kachkaev, J. Wood, J. DykesAuthors Info & ClaimsAbstract
The findings drawn from opinion survey responses are usually made by producing summary charts or conducting statistical analysis. Both involve data aggregation and filtering as exploring the unaggregated data has traditionally been impractical or error-prone for large numbers of responses. We propose the use of glyphs with parallel coordinate plots to show all survey responses in a single view and design an interactive visual analytics tool around the representation to explore the data. We use this software for a 'photo content assessment' survey, where 359 participants classify 900 images by seven criteria. The proposed approach allows all 8,434 responses (49,285 answers to questions in total) to be represented in a single view and helps analysts to both clean the data and understand the nature of the survey responses. We describe the construction of the survey response glyphs and the interface to the interactive visual analytics software and generalise the design principles that arise from the approach. We apply the tool to two other datasets to evaluate the technique and to confirm its wider applicability for surveys with Likert scale responses.
References
[1]
{Ame07} Amegashie J. A.: Intentions, Insincerity, and Prosocial Behavior. Working Papers 0703, University of Guelph, Department of Economics and Finance, 2007. 2
[2]
{AS05} Abramovich S., Sugden S.: Spreadsheet conditional formatting: An untapped resource for mathematics education. Spreadsheets in Education (eJSiE) 1, 2 (2005), 3. 2
[3]
{BKC*12} Borgo R., Kehrer J., Chung D. H., Maguire E., Laramee R. S., Hauser H., Ward M., Chen M.: Glyph-based visualization: Foundations, design guidelines, techniques and applications. In Eurographics 2013-State of the Art Reports (2012), The Eurographics Association, pp. 39--63. 2
[4]
{BN11} Brandes U., Nick B.: Asymmetric relations in longitudinal social networks. Visualization and Computer Graphics, IEEE Transactions on 17, 12 (2011), 2283--2290. 2
[5]
{CM84} Cleveland W. S., McGill R.: Graphical perception: Theory, experimentation, and application to the development of graphical methods. Journal of the American Statistical Association 79, 387 (1984), 531--554. 5
[6]
{CM07} Cawthon N., Moere A. V.: The effect of aesthetic on the usability of data visualization. In Information Visualization, IV'07. 11th International Conference (2007), p. 637--648. 4
[7]
{Dat12} Datahub: Wikipedia article ratings. http://bit.ly/wikipedia_article_ratings, 2012. 8
[8]
{EvKSS13} Eppstein D., van Kreveld D., Speckmann B., Staals F.: Improved grid map layout by point set matching. In Visualization Symposium (PacificVis), 2013 IEEE Pacific (2013), IEEE, pp. 25--32. 6
[9]
{FC73} Fleiss J. L., Cohen J.: The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and Psychological Measurement 33, 3 (Oct. 1973), 613--619. 6
[10]
{Few07} Few S.: Save the pies for dessert. Visual business intelligence newsletter. http://www.perceptualedge.com/articles/08-21-07.pdf, 2007. 2
[11]
{Fry08} Fry B.: Visualizing data: exploring and explaining data with the Processing environment. O'Reilly Media, Inc., 2008. 3
[12]
{GAW*11} Gleicher M., Albers D., Walker R., Jusufi I., Hansen C. D., Roberts J. C.: Visual comparison for information visualization. Information Visualization 10, 4 (oct 2011), 289--309. 2
[13]
{HR07} Heer J., Robertson G. G.: Animated transitions in statistical data graphics. Visualization and Computer Graphics, IEEE Transactions on 13, 6 (2007), 1240--1247. 4, 6
[14]
{ID90} Inselberg A., Dimsdale B.: Parallel coordinates: a tool for visualizing multi-dimensional geometry. In Proceedings of the 1st conference on Visualization '90 (Los Alamitos, CA, USA, 1990), VIS '90, IEEE Computer Society Press, p. 361--378. 2
[15]
{LKH09} Lie A. E., Kehrer J., Hauser H.: Critical design and realization aspects of glyph-based 3d data visualization. In Proceedings of the 25th Spring Conference on Computer Graphics (2009), ACM, pp. 19--26. 2
[16]
{LSR10} LSR Online: Place survey. http://www.lsronline.org/placesurvey.html, 2010. 2
[17]
{Med12} MediaWiki: Article feedback / version 4. http://www.mediawiki.org/wiki/Article_feedback/Version_4, 2012. 9
[18]
{Mic10} Microsoft: Quick start: Apply conditional formatting. http://office.microsoft.com/en-001/excel-help/quick-start-apply-conditional-formatting-HA010370614.aspx, 2010. 2
[19]
{MRSS*12} Maguire E., Rocca-Serra P., Sansone S.-A., Davies J., Chen M.: Taxonomy-based glyph design---with a case study on visualizing workflows of biological experiments. Visualization and Computer Graphics, IEEE Transactions on 18, 12 (2012), 2603--2612. 2
[20]
{Nag12} Nagel T.: Gum and weighted connections. http://tillnagel.com/2012/11/gum-and-weighted-connections/, 2012. 9
[21]
{O'B13} O'Brien, O.: This place, visualisation of 2011 census data for england and wales. http://casa.oobrien.com/thisplace/, 2013. 2
[22]
{Off11} Office for National Statistics: 2011 census interactive. http://www.ons.gov.uk/ons/guide-method/census/2011/census-data/2011-census-interactive-content/index.html, 2011. 2
[23]
{PD84} Porter T., Duff T.: Compositing digital images. ACM Siggraph Computer Graphics 18, 3 (1984), 253--259. 4
[24]
{Pow12} Powers D. M.: The problem with kappa. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics (2012), p. 345--355. 6
[25]
{RC94} Rao R., Card S. K.: The table lens: merging graphical and symbolic representations in an interactive focus+ context visualization for tabular information. In Proceedings of the SIGCHI conference on Human factors in computing systems (1994), ACM, pp. 318--322. 2
[26]
{RW00} Rose S. J., Wong P. C.: Driftweed: a visual metaphor for interactive analysis of multivariate data. In Electronic Imaging (2000), International Society for Optics and Photonics, pp. 114--121. 5
[27]
{Sap99} Sapsford R.: Survey research. Sage Publications, London; Thousand Oaks, Calif, 1999. 2
[28]
{Tuf83} Tufte E.: The visual display of quantitative information. No. v. 914 in The Visual Display of Quantitative Information. Graphics Press, 1983. 2, 3, 8
[29]
{TW06} Tastle W., Wierman M.: An information theoretic measure for the evaluation of ordinal scale data. Behaviour Research Methods 38, 3 (2006), 487--494. 2, 6
[30]
{Uni13} United States Census Bureau: Data visualization gallery. http://www.census.gov/dataviz/, 2013. 2
[31]
{War02} Ward M. O.: A taxonomy of glyph placement strategies for multidimensional data visualization. Information Visualization 1, 3-4 (2002), 194--210. 2, 6
[32]
{War08} Ward M. O.: Multivariate data glyphs: Principles and practice. In Handbook of data visualization. Springer, 2008, pp. 179--198. 2
[33]
{WHWC12} Wickham H., Hofmann H., Wickham C., Cook D.: Glyph-maps for visually exploring temporal patterns in climate data and models. Environmetrics 23, 5 (Aug. 2012), 382--393. 2
[34]
{WS12} Wickham H., Stryjewski L.: 40 years of boxplots. Tech. rep., had.co.nz, 2012. 2
[35]
{WSD*13} Walker R., Slingsby A., Dykes J., Xu K., Wood J., Nguyen P. H., Stephens D., Wong B., Zheng Y.: An extensible framework for provenance in human terrain visual analytics. Visualization and Computer Graphics, IEEE Transactions on 19, 12 (2013), 2139--2148. 3
[36]
{WW05} Wilkinson L., Wills G.: The Grammar of Graphics. Statistics and Computing. Springer, 2005. 2
[37]
{Zih13} Zihler O. F. M.: Implementation of a Spatially-Aware Image Search Engine and Its Evaluation using Crowdsourced Relevance Judgements. Master's thesis, Department of Geography, University of Zurich, 2013. 8
Index Terms
- Glyphs for exploring crowd-sourced subjective survey classification
Recommendations
Pathline glyphs
Visualization of pathlines is common and highly relevant for the analysis of unsteady flow. However, pathlines can intersect, leading to visual clutter and perceptual issues. This makes it intrinsically difficult to provide expressive visualizations of ...
Visualizing web search results using glyphs: Design and evaluation of a flower metaphor
While the Web provides a lot of useful information to managers and decision makers in organizations for decision support, it requires a lot of time and cognitive effort for users to sift through a search result list returned by search engines to find ...
Ross-Chernoff Glyphs Or: How Do We Kill Bad Ideas in Visualization?
CHI EA '18: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing SystemsAs data increases in dimensionality or complexity, it becomes difficult to graphically represent data items or series in a straightforward way. Chernoff faces encode data values as features of a human face, but afford only a handful of dimensions, and ...
Comments
Information & Contributors
Information
Published In
Sponsors
- KAUST: King Abdullah University of Science and Technology
- Winton Capital Management: Winton Capital Management
- tableau: Tableau Software
- Grid-Tools: Grid-Tools
- KTN: Knowledge Transfer Network
- We Predict: We Predict
- STFC: Science & Technology Facilities Council
Publisher
Eurographics Association
Goslar, Germany
Publication History
Published: 09 June 2014
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 14 Jan 2025