research-article

Image2Scene: Transforming Style of 3D Room

Authors:

Qinping ZhaoAuthors Info & Claims

MM '15: Proceedings of the 23rd ACM international conference on Multimedia

Pages 321 - 330

https://doi.org/10.1145/2733373.2806274

Published: 13 October 2015 Publication History

Abstract

We propose a style transformation system to transform a 3D room into one that resembles the style of a photograph. We focus on two major components of interior scene style: layout and color. Using an interior image database, we learn the related style guidelines. Given a reference image and a 3D room of two different interior rooms, we first establish semantic correspondence between the two scenes. The styles of the reference image are then extracted in the form of layout constraints and color schemes. Finally, our framework performs layout rearrangement followed by recoloring of the scene to match the learned style of the reference image. We show style transformation results on numerous examples to demonstrate the effectiveness and efficiency of our system.

References

[1]

ARTSPARX. http://www.artsparx.com/.

[2]

J. Chen, Y. Han, X. Cao, and Q. Tian. Object coding on the semantic graph for scene classification. In ACM Multimedia, pages 493--496, 2013.

Digital Library

[3]

K. Chen, Y.-K. Lai, Y.-X. Wu, R. Martin, and S.-M. Hu. Automatic semantic modeling of indoor scenes from low-quality RGB-D data using contextual information. ACM Trans. Graph., 33(6):208:1--208:12, 2014.

Digital Library

[4]

X. Chen, D. Zou, J. Li, X. Cao, Q. Zhao, and H. Zhang. Sparse dictionary learning for edit propagation of high-resolution images. In CVPR, pages 2854--2861, 2014.

Digital Library

[5]

X. Chen, D. Zou, Q. Zhao, and P. Tan. Manifold preserving edit propagation. ACM Trans. Graph., 31(6):132:1--7, 2012.

Digital Library

[6]

W. Choi, Y.-W. Chao, C. Pantofaru, and S. Savarese. Understanding indoor scenes using 3d geometric phrases. In CVPR, pages 33--40, 2013.

Digital Library

[7]

L. Chu, S. Jiang, S. Wang, Y. Zhang, and Q. Huang. Robust spatial consistency graph model for partial duplicate image retrieval. IEEE TMM, 15(8):1982--1996, 2013.

Digital Library

[8]

D. Cohen-Or, O. Sorkine, R. Gal, T. Leyvand, and Y.-Q. Xu. Color harmonization. ACM Trans. Graph., 25(3):624--630, 2006.

Digital Library

[9]

M. Fisher, D. Ritchie, M. Savva, T. Funkhouser, and P. Hanrahan. Example-based synthesis of 3d object arrangements. ACM Trans. Graph., 31(6):135:1--11, 2012.

Digital Library

[10]

M. Fisher, M. Savva, and P. Hanrahan. Characterizing structural relationships in scenes using graph kernels. ACM Trans. Graph., 30(4):34, 2011.

Digital Library

[11]

T. Germer and M. Schwarz. Procedural arrangement of furniture for Real-Time walkthroughs. Computer Graphics Forum, 28(8):2068--2078, 2009.

[12]

Y. Han, F. Wu, J. Shao, Q. Tian, and Y. Zhuang. Graph-guided sparse reconstruction for region tagging. In CVPR, pages 2981--2988, 2012.

Digital Library

[13]

V. Hedau, D. Hoiem, and D. Forsyth. Recovering the spatial layout of cluttered rooms. In ICCV, pages 1849--1856, 2009.

[14]

V. Hedau, D. Hoiem, and D. A. Forsyth. Recovering free space of indoor scenes from a single image. In CVPR, pages 2807--2814, 2012.

Digital Library

[15]

A. Hertzmann, C. E. Jacobs, N. Oliver, B. Curless, and D. H. Salesin. Image analogies. In SIGGRAPH, pages 327--340, 2001.

Digital Library

[16]

Y. M. Kim, N. J. Mitra, D.-M. Yan, and L. Guibas. Acquiring 3d indoor environments with variability and repetition. ACM Trans. Graph., 31(6):138:1--138:11, 2012.

Digital Library

[17]

H. Li, H. Zhang, Y. Wang, J. Cao, A. Shamir, and D. Cohen-or. Curve style analysis in a set of shapes. Computer Graphics Forum, 32(6):77--88, 2013.

Digital Library

[18]

Q. Li, X. Chen, Y. Song, Y. Zhang, X. Jin, and Q. Zhao. Geodesic propagation for semantic labeling. IEEE Trans. Image Process., 23(11):4812--4825, 2014.

[19]

S. Liu, S. Yan, T. Zhang, C. Xu, J. Liu, and H. Lu. Weakly supervised graph propagation towards collective image parsing. IEEE TMM, 14(2):361--373, 2012.

Digital Library

[20]

X. Liu, D. Zhao, J. Zhou, W. Gao, and H. Sun. Image interpolation via graph-based bayesian label propagation. 23(3):1084--1096, 2014.

Digital Library

[21]

Y. Matsuda. Color Design. Asakura Shoten, 1995.

[22]

P. Merrell, E. Schkufza, Z. Li, M. Agrawala, and V. Koltun. Interactive furniture layout using interior design guidelines. ACM Trans. Graph., 30(4):87:1--87:10, 2011.

Digital Library

[23]

L. Nan, K. Xie, and A. Sharf. A search-classify approach for cluttered indoor scene understanding. ACM Trans. Graph., 31(6):137:1--137:10, 2012.

Digital Library

[24]

L. Neumann, A. Nemcsics, and A. Neumann. Computational color harmony based on coloroid system. In Computational Aesthetics, pages 231--240, 2005.

Digital Library

[25]

P. O'Donovan, A. Agarwala, and A. Hertzmann. Color compatibility from large datasets. ACM Trans. Graph., 30(4):63:1--63:12, 2011.

Digital Library

[26]

E. Reinhard, M. Ashikhmin, B. Gooch, and P. Shirley. Color transfer between images. IEEE Comput. Graph. Appl., 21(5):34--41, 2001.

Digital Library

[27]

S. Roweis and L. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290:2323--2326, 2000.

[28]

S. Sanchez, O. L. Roux, H. Luga, and V. Gaildrat. Constraint-based 3d-object layout using a genetic algorithm. In 3IA, 2003.

[29]

T. Shao, W. Xu, K. Zhou, J. Wang, D. Li, and B. Guo. An interactive approach to semantic modeling of indoor scenes with an RGBD camera. ACM Trans. Graph., 31(6):136:1--11, 2012.

Digital Library

[30]

L. Wang, Y. Yu, K. Zhou, and B. Guo. Example-based hair geometry synthesis. ACM Trans. Graph., 28(3):56:1--9, 2009.

Digital Library

[31]

J. Xiao, T. Fang, P. Zhao, M. Lhuillier, and L. Quan. Image-based street-side city modeling. ACM Trans. Graph., 28(5):114:1--114:12, 2009.

Digital Library

[32]

K. Xu, H. Zheng, H. Zhang, D. Cohen-Or, L. Liu, and Y. Xiong. Photo-inspired model-driven 3d object modeling. ACM Trans. Graph., 30(4):80:1--80:10, 2011.

Digital Library

[33]

Y.-T. Yeh, L. Yang, M. Watson, N. D. Goodman, and P. Hanrahan. Synthesizing open worlds with constraints using locally annealed reversible jump MCMC. ACM Trans. Graph., 31(4):56, 2012.

Digital Library

[34]

L.-F. Yu, S. K. Yeung, C.-K. Tang, D. Terzopoulos, T. F. Chan, and S. Osher. Make it home: automatic optimization of furniture arrangement. ACM Trans. Graph., 30(4):86, 2011.

Digital Library

[35]

H. Zhang, J. Xiao, and L. Quan. Supervised label transfer for semantic segmentation of street scenes. In ECCV, pages 561--574, 2010.

Digital Library

[36]

B. Zhou, X. Chen, Q. Fu, K. Guo, and P. Tan. Garment modeling from a single image. Computer Graphics Forum, 32(7):85--91, 2013.

Cited By

Fu QHe SLi XFu H(2024)PlanNet: A Generative Model for Component-Based Plan SynthesisIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.327520030:8(4739-4751)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3275200
Fu QHe SFu HLi XDeng Z(2023)Fuzzy-based indoor scene modeling with differentiated examplesComputational Visual Media10.1007/s41095-022-0299-z9:4(717-732)Online publication date: 23-May-2023
https://doi.org/10.1007/s41095-022-0299-z
Xiong GFu QFu HZhou BLuo GDeng Z(2021)Motion Planning for Convertible Indoor Scene Layout DesignIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2020.300568027:12(4413-4424)Online publication date: 1-Dec-2021
https://doi.org/10.1109/TVCG.2020.3005680
Show More Cited By

Index Terms

Image2Scene: Transforming Style of 3D Room
1. Applied computing
  1. Document management and text processing
    1. Document capture
      1. Graphics recognition and interpretation
2. Computing methodologies
  1. Computer graphics
    1. Image manipulation
      1. Texturing

Recommendations

Automatic natural language style classification and transformation
IRSG'08: Proceedings of the 2008 BCS-IRSG conference on Corpus Profiling

Style is an integral part of natural language in written, spoken or machine generated forms. Humans have been dealing with style in language since the beginnings of language itself, but computers and machine processes have only recently begun to process ...
Jigsaw puzzles with pieces of unknown orientation
CVPR '12: Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

This paper introduces new types of square-piece jigsaw puzzles: those for which the orientation of each jigsaw piece is unknown. We propose a tree-based reassembly that greedily merges components while respecting the geometric constraints of the puzzle ...
Multifocal Projection: A Multiprojector Technique for Increasing Focal Depth

In this paper, we describe a novel multifocal projection concept that applies conventional video projectors and camera feedback. Multiple projectors with differently adjusted focal planes, but overlapping image areas are used. They can be either ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '15: Proceedings of the 23rd ACM international conference on Multimedia

October 2015

1402 pages

ISBN:9781450334594

DOI:10.1145/2733373

General Chairs:
Xiaofang Zhou
The University of Queensland, Australia
,
Alan F. Smeaton
Dublin City University, Ireland
,
Qi Tian
The University of Texas at San Antonio, USA
,
Program Chairs:
Dick C.A. Bulterman
FXPAL, USA
,
Heng Tao Shen
The University of Queensland, Australia
,
Ketan Mayer-Patel
The University of North Carolina, USA
,
Shuicheng Yan
National University of Singapore, Singapore

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 October 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National High-tech R&D Program of China
Open Funding Project of State Key Laboratory of Virtual Reality Technology and Systems
National Natural Science Foundation of China
Specialized Research Fund for the Doctoral Program of Higher Education

Conference

MM '15

Sponsor:

SIGMM

MM '15: ACM Multimedia Conference

October 26 - 30, 2015

Brisbane, Australia

Acceptance Rates

MM '15 Paper Acceptance Rate 56 of 252 submissions, 22%;

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
669
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)3

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Fu QHe SLi XFu H(2024)PlanNet: A Generative Model for Component-Based Plan SynthesisIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.327520030:8(4739-4751)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3275200
Fu QHe SFu HLi XDeng Z(2023)Fuzzy-based indoor scene modeling with differentiated examplesComputational Visual Media10.1007/s41095-022-0299-z9:4(717-732)Online publication date: 23-May-2023
https://doi.org/10.1007/s41095-022-0299-z
Xiong GFu QFu HZhou BLuo GDeng Z(2021)Motion Planning for Convertible Indoor Scene Layout DesignIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2020.300568027:12(4413-4424)Online publication date: 1-Dec-2021
https://doi.org/10.1109/TVCG.2020.3005680
Zhang SHan ZLai YZwicker MZhang H(2021)Active Arrangement of Small Objects in 3D Indoor ScenesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2019.294929527:4(2250-2264)Online publication date: 1-Apr-2021
https://doi.org/10.1109/TVCG.2019.2949295
Pan TDai YHu MCheng W(2019)Furniture style compatibility recommendation with cross-class triplet lossMultimedia Tools and Applications10.1007/s11042-018-5747-578:3(2645-2665)Online publication date: 1-Feb-2019
https://dl.acm.org/doi/10.1007/s11042-018-5747-5
Zhang SHan ZLai YZwicker MZhang H(2019)Stylistic scene enhancement GANThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-019-01691-w35:6-8(1157-1169)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1007/s00371-019-01691-w
Zhang YChen XLi JWang CXia CLi J(2018)Semantic Object Segmentation in Tagged Videos via DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2017.272704940:7(1741-1754)Online publication date: 1-Jul-2018
https://doi.org/10.1109/TPAMI.2017.2727049
Fu QChen XWang XWen SZhou BFu H(2017)Adaptive synthesis of indoor scenes via activity-associated object relation graphsACM Transactions on Graphics10.1145/3130800.313080536:6(1-13)Online publication date: 20-Nov-2017
https://dl.acm.org/doi/10.1145/3130800.3130805
Zhang SHan ZMartin RZhang H(2017)Semantic 3D indoor scene enhancement using guide wordsThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-017-1394-533:6-8(925-935)Online publication date: 1-Jun-2017
https://dl.acm.org/doi/10.1007/s00371-017-1394-5
Hua BPham QNguyen DTran MYu LYeung S(2016)SceneNN: A Scene Meshes Dataset with aNNotations2016 Fourth International Conference on 3D Vision (3DV)10.1109/3DV.2016.18(92-101)Online publication date: Oct-2016
https://doi.org/10.1109/3DV.2016.18
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents