Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3411764.3445325acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

MetaMap: Supporting Visual Metaphor Ideation through Multi-dimensional Example-based Exploration

Published: 07 May 2021 Publication History

Abstract

Visual metaphors, which are widely used in graphic design, can deliver messages in creative ways by fusing different objects. The keys to creating visual metaphors are diverse exploration and creative combinations, which is challenging with conventional methods like image searching. To streamline this ideation process, we propose to use a mind-map-like structure to recommend and assist users to explore materials. We present MetaMap, a supporting tool which inspires visual metaphor ideation through multi-dimensional example-based exploration. To facilitate the divergence and convergence of the ideation process, MetaMap provides 1) sample images based on keyword association and color filtering; 2) example-based exploration in semantics, color, and shape dimensions; and 3) thinking path tracking and idea recording. We conduct a within-subject study with 24 design enthusiasts by taking a Pinterest-like interface as the baseline. Our evaluation results suggest that MetaMap provides an engaging ideation process and helps participants create diverse and creative ideas.

Supplementary Material

MP4 File (3411764.3445325_videopreview.mp4)
Preview video

References

[1]
Virgil C. Aldrich. 1968. Visual Metaphor. Journal of Aesthetic Education 2, 1 (1968), 73–86. http://www.jstor.org/stable/3331241
[2]
Suyun Sandra Bae, Oh-Hyun Kwon, Senthil Chandrasegaran, and Kwan-Liu Ma. 2020. Spinneret: Aiding Creative Ideation through Non-Obvious Concept Associations. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376746
[3]
Aaron Bangor, Philip Kortum, and James Miller. 2009. Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of usability studies 4, 3 (2009), 114–123.
[4]
Nathalie Bonnardel and Evelyne Marmèche. 2005. Towards supporting evocation processes in creative design: A cognitive approach. International Journal of Human-Computer Studies 63, 4 (2005), 422 – 435. https://doi.org/10.1016/j.ijhcs.2005.04.006 Computer support for creativity.
[5]
John Brooke. 1996. SUS: a “quick and dirty” usability scale. CRC press, Florida, USA, 189.
[6]
David C Brown. 2008. Guiding computational design creativity research.
[7]
Noel Carroll. 1994. Visual Metaphor. Springer Netherlands, Dordrecht, 189–218. https://doi.org/10.1007/978-94-015-8315-2_6
[8]
Noël Carroll. 2001. Visual Metaphor. Cambridge University Press, Cambridge, UK, 347–368. https://doi.org/10.1017/CBO9780511605970.023
[9]
Andrew Cattle and Xiaojuan Ma. 2017. Predicting Word Association Strengths. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Copenhagen, Denmark, 1283–1288. https://doi.org/10.18653/v1/D17-1132
[10]
Joel Chan, Pao Siangliulue, Denisa Qori McDonald, Ruixue Liu, Reza Moradinezhad, Safa Aman, Erin T. Solovey, Krzysztof Z. Gajos, and Steven P. Dow. 2017. Semantically Far Inspirations Considered Harmful? Accounting for Cognitive States in Collaborative Ideation. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition (Singapore, Singapore) (C&C ’17). Association for Computing Machinery, New York, NY, USA, 93–105. https://doi.org/10.1145/3059454.3059455
[11]
Lydia B. Chilton, Savvas Petridis, and Maneesh Agrawala. 2019. VisiBlends: A Flexible Workflow for Visual Blends. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3290605.3300402
[12]
Mihaly Csikszentmihalyi. 1990. Flow: The Psychology of Optimal Experience. Vol. 1990. Harper & Row, New York, USA.
[13]
Marcel Danesi. 1990. Thinking is seeing: Visual metaphors and the nature of abstract thought. Semiotica 80, 3-4 (1990), 221–238. https://doi.org/10.1515/semi.1990.80.3-4.221
[14]
Simon De Deyne, Danielle J Navarro, Amy Perfors, Marc Brysbaert, and Gert Storms. 2019. The “Small World of Words” English word association norms for over 12,000 cue words. Behavior research methods 51, 3 (2019), 987–1006.
[15]
Claudia Eckert and Martin Stacey. 2000. Sources of Inspiration: A Language of Design. Design Studies 21 (09 2000). https://doi.org/10.1016/S0142-694X(00)00022-3
[16]
Hermine Feinstein. 1985. Art as Visual Metaphor. Art Education 38, 4 (1985), 26–29. http://www.jstor.org/stable/3192822
[17]
Jonas Frich, Lindsay MacDonald Vermeulen, Christian Remy, Michael Mose Biskjaer, and Peter Dalsgaard. 2019. Mapping the Landscape of Creativity Support Tools in HCI. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–18. https://doi.org/10.1145/3290605.3300619
[18]
Milene Gonçalves, Carlos Cardoso, and Petra Badke-Schaub. 2014. What inspires designers? Preferences on inspirational approaches during idea generation. Design Studies 35, 1 (2014), 29 – 53. https://doi.org/10.1016/j.destud.2013.09.001
[19]
Nancy C Goodwin. 1987. Functionality and usability. Commun. ACM 30, 3 (1987), 229–233.
[20]
H. Gough. 1976. Studying creativity by means of word association test.The Journal of applied psychology 61 3 (1976), 348–53.
[21]
Tom Hewett, Mary Czerwinski, Michael Terry, Jay Nunamaker, Linda Candy, Bill Kules, and Elisabeth Sylvan. 2005. Creativity support tool evaluation methods and metrics., 10–24 pages.
[22]
Forrest Huang, John F. Canny, and Jeffrey Nichols. 2019. Swire: Sketch-Based User Interface Retrieval. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–10. https://doi.org/10.1145/3290605.3300334
[23]
Ali Jahanian, Shaiyan Keshvari, S. V. N. Vishwanathan, and Jan P. Allebach. 2017. Colors – Messengers of Concepts: Visual Design Mining for Learning Color Semantics. ACM Trans. Comput.-Hum. Interact. 24, 1, Article 2 (Jan. 2017), 39 pages. https://doi.org/10.1145/3009924
[24]
David G. Jansson and Steven M. Smith. 1991. Design fixation. Design Studies 12, 1 (1991), 3 – 11. https://doi.org/10.1016/0142-694X(91)90003-F
[25]
EunJin Kim and Hyeon-Jeong Suk. 2017. Thoughts and Tools for Crafting Colors: Implications from Designers’ Behavior. In Proceedings of the 2017 Conference on Designing Interactive Systems (Edinburgh, United Kingdom) (DIS ’17). Association for Computing Machinery, New York, NY, USA, 321–331. https://doi.org/10.1145/3064663.3064769
[26]
Janin Koch, Andrés Lucero, Lena Hegemann, and Antti Oulasvirta. 2019. May AI? Design Ideation with Cooperative Contextual Bandits. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300863
[27]
Ranjitha Kumar, Jerry O. Talton, Salman Ahmad, and Scott R. Klemmer. 2011. Bricolage: Example-Based Retargeting for Web Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI ’11). Association for Computing Machinery, New York, NY, USA, 2197–2206. https://doi.org/10.1145/1978942.1979262
[28]
Mackenzie Leake, Hijung Valentina Shin, Joy O. Kim, and Maneesh Agrawala. 2020. Generating Audio-Visual Slideshows from Text Articles Using Word Concreteness. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3313831.3376519
[29]
Jiunde Lee. 2007. The Effects of Visual Metaphor and Cognitive Style for Mental Modeling in a Hypermedia-Based Environment. Interact. Comput. 19, 5–6 (Dec. 2007), 614–629. https://doi.org/10.1016/j.intcom.2007.05.005
[30]
Ting Liu, Kit Cho, G. Aaron Broadwell, Samira Shaikh, Tomek Strzalkowski, John Lien, Sarah Taylor, Laurie Feldman, Boris Yamrom, Nick Webb, Umit Boz, Ignacio Cases, and Ching-sheng Lin. 2014. Automatic Expansion of the MRC Psycholinguistic Database Imageability Ratings. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14). European Language Resources Association (ELRA), Reykjavik, Iceland, 2800–2805. http://www.lrec-conf.org/proceedings/lrec2014/pdf/178_Paper.pdf
[31]
Nick Logler, Daisy Yoo, and Batya Friedman. 2018. Metaphor Cards: A How-to-Guide for Making and Using a Generative Metaphorical Design Toolkit. In Proceedings of the 2018 Designing Interactive Systems Conference (Hong Kong, China) (DIS ’18). Association for Computing Machinery, New York, NY, USA, 1373–1386. https://doi.org/10.1145/3196709.3196811
[32]
Lucero, AA (Andrés). 2009. Co-designing interactive spaces for and with designers:supporting mood-board making. https://doi.org/10.6100/IR641288
[33]
Xiaojuan Ma. 2013. Evocation: analyzing and propagating a semantic link based on free word association. Language Resources and Evaluation 47, 3 (01 Sep 2013), 819–837. https://doi.org/10.1007/s10579-013-9219-2
[34]
Thomas Merten and Ines Fischer. 1999. Creativity, personality and word association responses: associative behaviour in forty supposedly creative persons. Personality and Individual Differences 27, 5 (1999), 933 – 942. https://doi.org/10.1016/S0191-8869(99)00042-2
[35]
Donald A. Norman. 2002. The Design of Everyday Things. Basic Books, Inc., USA.
[36]
Heather O’Brien. 2016. Theoretical Perspectives on User Engagement. Springer International Publishing, Cham, 1–26. https://doi.org/10.1007/978-3-319-27446-1_1
[37]
Matti Perttula and Pekka Sipilä. 2007. The idea exposure paradigm in design idea generation. Journal of Engineering Design 18, 1 (2007), 93–102. https://doi.org/10.1080/09544820600679679 arXiv:https://doi.org/10.1080/09544820600679679
[38]
Savvas Petridis and Lydia B. Chilton. 2019. Human Errors in Interpreting Visual Metaphor. In Proceedings of the 2019 on Creativity and Cognition (San Diego, CA, USA) (C&C ’19). Association for Computing Machinery, New York, NY, USA, 187–197. https://doi.org/10.1145/3325480.3325503
[39]
Barbara J Phillips and Edward F McQuarrie. 2004. Beyond visual metaphor: A new typology of visual rhetoric in advertising. Marketing theory 4, 1-2 (2004), 113–136.
[40]
Elisabeth El Refaie. 2003. Understanding visual metaphor: the example of newspaper cartoons. Visual Communication 2, 1 (2003), 75–95. https://doi.org/10.1177/1470357203002001755 arXiv:https://doi.org/10.1177/1470357203002001755
[41]
A. Reid and Ian Solomonides. 2007. Design students’ experience of engagement and creativity. Art, Design and Communication in Higher Education 6 (2007), 25–39.
[42]
John T. E. Richardson. 1975. Concreteness and Imageability. Quarterly Journal of Experimental Psychology 27, 2(1975), 235–249. https://doi.org/10.1080/14640747508400483 arXiv:https://doi.org/10.1080/14640747508400483
[43]
Daniel Ritchie, Ankita Arvind Kejriwal, and Scott R. Klemmer. 2011. D.Tour: Style-Based Exploration of Design Example Galleries. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (Santa Barbara, California, USA) (UIST ’11). Association for Computing Machinery, New York, NY, USA, 165–174. https://doi.org/10.1145/2047196.2047216
[44]
Sam H. Ross, Ecenaz Jen Ozmen, Maria V. Kogan, and Lydia B. Chilton. 2020. WordBlender: Principles and Tools for Generating Word Blends. In Proceedings of the 25th International Conference on Intelligent User Interfaces (Cagliari, Italy) (IUI ’20). Association for Computing Machinery, New York, NY, USA, 38–42. https://doi.org/10.1145/3377325.3377527
[45]
Albert Rothenberg. 1973. Word Association and Creativity. Psychological Reports 33, 1 (1973), 3–12. https://doi.org/10.2466/pr0.1973.33.1.3 arXiv:https://doi.org/10.2466/pr0.1973.33.1.3PMID: 4728480.
[46]
Yang Shi, Yang Wang, Ye Qi, John Chen, Xiaoyao Xu, and Kwan-Liu Ma. 2017. IdeaWall: Improving Creative Collaboration through Combinatorial Visual Stimuli. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSCW ’17). Association for Computing Machinery, New York, NY, USA, 594–603. https://doi.org/10.1145/2998181.2998208
[47]
Ben Shneiderman. 2007. Creativity Support Tools: Accelerating Discovery and Innovation. Commun. ACM 50, 12 (Dec. 2007), 20–32. https://doi.org/10.1145/1323688.1323689
[48]
Pao Siangliulue, Kenneth C. Arnold, Krzysztof Z. Gajos, and Steven P. Dow. 2015. Toward Collaborative Ideation at Scale: Leveraging Ideas from Others to Generate More Creative and Diverse Ideas. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (Vancouver, BC, Canada) (CSCW ’15). Association for Computing Machinery, New York, NY, USA, 937–945. https://doi.org/10.1145/2675133.2675239
[49]
Pao Siangliulue, Joel Chan, Krzysztof Z. Gajos, and Steven P. Dow. 2015. Providing Timely Examples Improves the Quantity and Quality of Generated Ideas. In Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition (Glasgow, United Kingdom) (C&C ’15). Association for Computing Machinery, New York, NY, USA, 83–92. https://doi.org/10.1145/2757226.2757230
[50]
R. Smith. 2007. An Overview of the Tesseract OCR Engine. In Proceedings of the Ninth International Conference on Document Analysis and Recognition - Volume 02(ICDAR ’07). IEEE Computer Society, USA, 629–633.
[51]
Lisanne Weelden, Alfons Maes, Joost Schilperoord, and Marc Swerts. 2012. How Object Shape Affects Visual Metaphor Processing. Experimental psychology 59 (07 2012), 1–8. https://doi.org/10.1027/1618-3169/a000165
[52]
T Wiegers, Y Song, JSM Vergeest, 2010. Shape Ideation Through Shape Metaphors. In DS 60: Proceedings of DESIGN 2010, the 11th International Design Conference. The Design Society, Dubrovnik, Croatia, 481–490.
[53]
Merryl J. Wilkenfeld and Thomas B. Ward. 2001. Similarity and Emergence in Conceptual Combination. Journal of Memory and Language 45, 1 (2001), 21 – 38. https://doi.org/10.1006/jmla.2000.2772
[54]
Ziming Wu, Zhida Sun, Taewook Kim, Manuele Reani, Caroline Jay, and Xiaojuan Ma. 2018. Mediating Color Filter Exploration with Color Theme Semantics Derived from Social Curation Data. Proc. ACM Hum.-Comput. Interact. 2, CSCW, Article 187 (Nov. 2018), 24 pages. https://doi.org/10.1145/3274456
[55]
Nanxuan Zhao, Nam Wook Kim, Laura Mariah Herman, Hanspeter Pfister, Rynson W.H. Lau, Jose Echevarria, and Zoya Bylinskii. 2020. ICONATE: Automatic Compound Icon Generation and Ideation. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376618

Cited By

View all
  • (2024)A New Optimized Hybridization Approach for in silico High Throughput Molecular Docking on FPGA PlatformCurrent Computer-Aided Drug Design10.2174/157340991966623050309441120:3(236-247)Online publication date: Jun-2024
  • (2024)DesignPrompt: Using Multimodal Interaction for Design Exploration with Generative AIProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661588(804-818)Online publication date: 1-Jul-2024
  • (2024)Smart "Error"! Exploring Imperfect AI to Support Creative IdeationProceedings of the ACM on Human-Computer Interaction10.1145/36373988:CSCW1(1-28)Online publication date: 26-Apr-2024
  • Show More Cited By

Index Terms

  1. MetaMap: Supporting Visual Metaphor Ideation through Multi-dimensional Example-based Exploration
      Index terms have been assigned to the content through auto-classification.

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
      May 2021
      10862 pages
      ISBN:9781450380966
      DOI:10.1145/3411764
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 07 May 2021

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Creativity Support Tool
      2. Design
      3. Visual Metaphor

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Funding Sources

      • The HKUST-WeBank Joint Laboratory Project Grant

      Conference

      CHI '21
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)420
      • Downloads (Last 6 weeks)42
      Reflects downloads up to 04 Oct 2024

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)A New Optimized Hybridization Approach for in silico High Throughput Molecular Docking on FPGA PlatformCurrent Computer-Aided Drug Design10.2174/157340991966623050309441120:3(236-247)Online publication date: Jun-2024
      • (2024)DesignPrompt: Using Multimodal Interaction for Design Exploration with Generative AIProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661588(804-818)Online publication date: 1-Jul-2024
      • (2024)Smart "Error"! Exploring Imperfect AI to Support Creative IdeationProceedings of the ACM on Human-Computer Interaction10.1145/36373988:CSCW1(1-28)Online publication date: 26-Apr-2024
      • (2024)DesignQuizzer: A Community-Powered Conversational Agent for Learning Visual DesignProceedings of the ACM on Human-Computer Interaction10.1145/36373218:CSCW1(1-40)Online publication date: 26-Apr-2024
      • (2024)Demystifying Tacit Knowledge in Graphic Design: Characteristics, Instances, Approaches, and GuidelinesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642886(1-18)Online publication date: 11-May-2024
      • (2024)GenQuery: Supporting Expressive Visual Search with Generative ModelsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642847(1-19)Online publication date: 11-May-2024
      • (2024)CreativeConnect: Supporting Reference Recombination for Graphic Design Ideation with Generative AIProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642794(1-25)Online publication date: 11-May-2024
      • (2024)How AI Processing Delays Foster Creativity: Exploring Research Question Co-Creation with an LLM-based AgentProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642698(1-25)Online publication date: 11-May-2024
      • (2024)Formulating or Fixating: Effects of Examples on Problem Solving Vary as a Function of Example Presentation Interface DesignProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642653(1-16)Online publication date: 11-May-2024
      • (2024)Beyond Numbers: Creating Analogies to Enhance Data Comprehension and Communication with Generative AIProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642480(1-14)Online publication date: 11-May-2024
      • Show More Cited By

      View Options

      Get Access

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media