Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3078971.3078986acmconferencesArticle/Chapter ViewAbstractPublication PagesicmrConference Proceedingsconference-collections
research-article

How to Make an Image More Memorable?: A Deep Style Transfer Approach

Published: 06 June 2017 Publication History

Abstract

Recent works have shown that it is possible to automatically predict intrinsic image properties like memorability. In this paper, we take a step forward addressing the question: "Can we make an image more memorable?". Methods for automatically increasing image memorability would have an impact in many application fields like education, gaming or advertising. Our work is inspired by the popular editing-by-applying-filters paradigm adopted in photo editing applications, like Instagram and Prisma. In this context, the problem of increasing image memorability maps to that of retrieving ``memorabilizing'' filters or style ``seeds''. Still, users generally have to go through most of the available filters before finding the desired solution, thus turning the editing process into a resource and time consuming task. In this work, we show that it is possible to automatically retrieve the best style seeds for a given image, thus remarkably reducing the number of human attempts needed to find a good match. Our approach leverages from recent advances in the field of image synthesis and adopts a deep architecture for generating a memorable picture from a given input image and a style seed. Importantly, to automatically select the best style a novel learning-based solution, also relying on deep models, is proposed. Our experimental evaluation, conducted on publicly available benchmarks, demonstrates the effectiveness of the proposed approach for generating memorable images through automatic style seed selection.

References

[1]
X. Alameda-Pineda, A. Pilzer, D. Xu, E. Ricci, and N. Sebe. Viraliency: Pooling local virality. In CVPR, 2017.
[2]
X. Alameda-Pineda, E. Ricci, Y. Yan, and N. Sebe. Recognizing emotions from abstract paintings using non-linear matrix completion. In CVPR, 2016.
[3]
A. K. Anderson, P. E. Wais, and J. D. Gabrieli. Emotion enhances remembrance of neutral events past. PNAS, 103(5):1599--1604, 2006.
[4]
A. F. Blackwell. Correction: A picture is worth 84.1 words. In ESP Student Workshop, 1997.
[5]
M. A. Borkin, A. A. Vo, Z. Bylinskii, P. Isola, S. Sunkavalli, A. Oliva, and H. Pfister. What makes a visualization memorable? IEEE TVCG, 19(12):2306--2315, 2013.
[6]
T. F. Brady, T. Konkle, G. A. Alvarez, and A. Oliva. Visual long-term memory has a massive storage capacity for object details. PNAS, 105(38):14325--14329, 2008.
[7]
Z. Bylinskii, P. Isola, C. Bainbridge, A. Torralba, and A. Oliva. Intrinsic and extrinsic effects on image memorability. Vision research, 116:165--178, 2015.
[8]
L. Cahill and J. L. McGaugh. A novel demonstration of enhanced memory associated with emotional arousal. Consciousness and cognition, 4(4):410--421, 1995.
[9]
L. A. Gatys, A. S. Ecker, and M. Bethge. A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576, 2015.
[10]
L. A. Gatys, A. S. Ecker, and M. Bethge. Image style transfer using convolutional neural networks. In CVPR, 2016.
[11]
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In NIPS, 2014.
[12]
M. Gygli, H. Grabner, H. Riemenschneider, F. Nater, and L. Gool. The interestingness of images. In ICCV, 2013.
[13]
M. Hilbert. How much information is there in the "information society"? Significance, 9(4):8--12, 2012.
[14]
R. R. Hunt and J. B. Worthen. Distinctiveness and memory. Oxford University Press, 2006.
[15]
P. Isola, D. Parikh, A. Torralba, and A. Oliva. Understanding the intrinsic memorability of images. In NIPS, 2011.
[16]
P. Isola, J. Xiao, T. Antonio, and A. Oliva. What makes an image memorable? In CVPR, 2011.
[17]
P. Isola, J. Xiao, D. Parikh, A. Torralba, and A. Oliva. What makes a photograph memorable? IEEE TPAMI, 36(7):1469--1482, 2014.
[18]
A. Khosla, W. Bainbridge, A. Torralba, and A. Oliva. Modifying the memorability of face photographs. In ICCV, 2013.
[19]
A. Khosla, A. S. Raju, A. Torralba, and A. Oliva. Understanding and predicting image memorability at a large scale. In ICCV, 2015.
[20]
A. Khosla, A. D. Sarma, and R. Hamid. What makes an image popular? In WWW, 2014.
[21]
A. Khosla, J. Xiao, P. Isola, A. Torralba, and A. Oliva. Image memorability and visual inception. In SIGGRAPH Asia, 2012.
[22]
A. Khosla, J. Xiao, A. Torralba, and A. Oliva. Memorability of image regions. In NIPS, 2012.
[23]
C. Li and M. Wand. Precomputed real-time texture synthesis with markovian generative adversarial networks. In ECCV, 2016.
[24]
J. Machajdik and A. Hanbury. Affective image classification using features inspired by psychology and art theory. In ACM Multimedia, 2010.
[25]
S. Maren. Long-term potentiation in the amygdala: a mechanism for emotional learning and memory. Trends in neurosciences, 22(12):561--567, 1999.
[26]
J. L. McGaugh. Make mild moments memorable: add a little arousal. Trends in cognitive sciences, 10(8):345--347, 2006.
[27]
K.-C. Peng, T. Chen, A. Sadovnik, and A. C. Gallagher. A mixed bag of emotions: Model, predict, and transfer emotion distributions. In CVPR, 2015.
[28]
E. A. Phelps. Human emotion and memory: interactions of the amygdala and hippocampal complex. Current opinion in neurobiology, 14(2):198--202, 2004.
[29]
A. Sartori, V. Yanulevskaya, A. A. Salah, J. Uijlings, E. Bruni, and N. Sebe. Affective analysis of professional and amateur abstract paintings using statistical analysis and art theory. ACM Transactions on Interactive Intelligent Systems, 5(2):8, 2015.
[30]
D. Ulyanov, V. Lebedev, A. Vedaldi, and V. Lempitsky. Texture networks: Feed-forward synthesis of textures and stylized images. ICML, 2016.
[31]
B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva. Learning deep features for scene recognition using places database. In NIPS. 2014.

Cited By

View all
  • (2023)Memorability of line drawings of scenes: the role of contour propertiesMemory & Cognition10.3758/s13421-023-01478-4Online publication date: 30-Oct-2023
  • (2023)A large-scale television advertising dataset for detailed impression analysisMultimedia Tools and Applications10.1007/s11042-023-14704-783:7(18779-18802)Online publication date: 1-Aug-2023
  • (2022)Material Translation Based on Neural Style Transfer with Ideal Style Image RetrievalSensors10.3390/s2219731722:19(7317)Online publication date: 27-Sep-2022
  • Show More Cited By

Index Terms

  1. How to Make an Image More Memorable?: A Deep Style Transfer Approach

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ICMR '17: Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval
    June 2017
    524 pages
    ISBN:9781450347013
    DOI:10.1145/3078971
    • General Chairs:
    • Bogdan Ionescu,
    • Nicu Sebe,
    • Program Chairs:
    • Jiashi Feng,
    • Martha Larson,
    • Rainer Lienhart,
    • Cees Snoek
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 06 June 2017

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. deep style transfer
    2. image memorability
    3. memorability
    4. photo enhancement
    5. retrieval

    Qualifiers

    • Research-article

    Conference

    ICMR '17
    Sponsor:

    Acceptance Rates

    ICMR '17 Paper Acceptance Rate 33 of 95 submissions, 35%;
    Overall Acceptance Rate 254 of 830 submissions, 31%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)20
    • Downloads (Last 6 weeks)2
    Reflects downloads up to 16 Oct 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2023)Memorability of line drawings of scenes: the role of contour propertiesMemory & Cognition10.3758/s13421-023-01478-4Online publication date: 30-Oct-2023
    • (2023)A large-scale television advertising dataset for detailed impression analysisMultimedia Tools and Applications10.1007/s11042-023-14704-783:7(18779-18802)Online publication date: 1-Aug-2023
    • (2022)Material Translation Based on Neural Style Transfer with Ideal Style Image RetrievalSensors10.3390/s2219731722:19(7317)Online publication date: 27-Sep-2022
    • (2022)Modulating human memory for complex scenes with artificially generated imagesScientific Reports10.1038/s41598-022-05623-y12:1Online publication date: 28-Jan-2022
    • (2021)Usability Evaluation of Food Wastage Mobile Application: A Case of PakistanSustainability10.3390/su13241402713:24(14027)Online publication date: 20-Dec-2021
    • (2021)Memorability: An Image-Computable Measure of Information UtilityHuman Perception of Visual Information10.1007/978-3-030-81465-6_8(207-239)Online publication date: 22-Jul-2021
    • (2019)Collecting, Analyzing and Predicting Socially-Driven Image Interestingness2019 27th European Signal Processing Conference (EUSIPCO)10.23919/EUSIPCO.2019.8902803(1-5)Online publication date: Sep-2019
    • (2019)Review of Image Memorability PredictionProceedings of the 3rd International Conference on Graphics and Signal Processing10.1145/3338472.3338479(24-28)Online publication date: 1-Jun-2019
    • (2019)Assist Users' Interactions in Font Search with Unexpected but Useful Concepts Generated by Multimodal LearningProceedings of the 2019 on International Conference on Multimedia Retrieval10.1145/3323873.3325037(235-243)Online publication date: 5-Jun-2019
    • (2019)Increasing Image Memorability with Neural Style TransferACM Transactions on Multimedia Computing, Communications, and Applications10.1145/331178115:2(1-22)Online publication date: 5-Jun-2019
    • Show More Cited By

    View Options

    Get Access

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media