research-article

Example-based synthesis of 3D object arrangements

Authors:

Matthew Fisher,

Daniel Ritchie,

Thomas Funkhouser,

Pat HanrahanAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 31, Issue 6

Article No.: 135, Pages 1 - 11

https://doi.org/10.1145/2366145.2366154

Published: 01 November 2012 Publication History

Abstract

We present a method for synthesizing 3D object arrangements from examples. Given a few user-provided examples, our system can synthesize a diverse set of plausible new scenes by learning from a larger scene database. We rely on three novel contributions. First, we introduce a probabilistic model for scenes based on Bayesian networks and Gaussian mixtures that can be trained from a small number of input examples. Second, we develop a clustering algorithm that groups objects occurring in a database of scenes according to their local scene neighborhoods. These contextual categories allow the synthesis process to treat a wider variety of objects as interchangeable. Third, we train our probabilistic model on a mix of user-provided examples and relevant scenes retrieved from the database. This mixed model learning process can be controlled to introduce additional variety into the synthesized scenes. We evaluate our algorithm through qualitative results and a perceptual study in which participants judged synthesized scenes to be highly plausible, as compared to hand-created scenes.

References

[1]

Akaike, H. 1973. Information theory and an extension of the maximum likelihood principle. In Second International Symposium on Information Theory, vol. 1, 267--281.

[2]

Bokeloh, M., Wand, M., and Seidel, H.-P. 2010. A connection between partial symmetry and inverse procedural modeling. In ACM SIGGRAPH 2010 papers, ACM, New York, NY, USA, SIGGRAPH '10, 104:1--104:10.

Digital Library

[3]

Chaudhuri, S., Kalogerakis, E., Guibas, L., and Koltun, V. 2011. Probabilistic reasoning for assembly-based 3D modeling. ACM Transactions on Graphics 30 (December).

Digital Library

[4]

Deng, J., Berg, A. C., and Fei-Fei, L. 2011. Hierarchical semantic indexing for large scale image retrieval. Computer Vision and Pattern Recognition, IEEE Computer Society Conference on 0, 785--792.

Digital Library

[5]

Diez, Y., and Sellarès, J. A. 2007. Efficient colored point set matching under noise. In Proceedings of the 2007 international conference on Computational science and its applications - Volume Part I, Springer-Verlag, Berlin, Heidelberg, ICCSA'07, 26--40.

Digital Library

[6]

Fisher, M., Savva, M., and Hanrahan, P. 2011. Characterizing structural relationships in scenes using graph kernels. In ACM SIGGRAPH 2011 papers, 34:1--34:12.

Digital Library

[7]

Hays, J., and Efros, A. A. 2007. Scene completion using millions of photographs. ACM Transactions on Graphics (SIGGRAPH 2007) 26, 3.

Digital Library

[8]

Kalogerakis, E., Chaudhuri, S., Koller, D., and Koltun, V. 2012. A probabilistic model for component-based shape synthesis. ACM Transactions on Graphics 31, 4.

Digital Library

[9]

Kalvin, A., and Taylor, R. 1996. Superfaces: Polygonal mesh simplification with bounded error. Computer Graphics and Applications, IEEE 16, 3, 64--77.

Digital Library

[10]

Kazhdan, M. 2007. An approximate and efficient method for optimal rotation alignment of 3d models. Pattern Analysis and Machine Intelligence, IEEE Transactions on 29, 7 (july), 1221--1229.

Digital Library

[11]

Kuhn, H. 1955. The hungarian method for the assignment problem. Naval research logistics quarterly 2, 1--2, 83--97.

[12]

Lawson, A., Linderman, M., Leonard, M., Stauffer, A., Pokines, B., and Carlin, M. 2009. Perturbation and pitch normalization as enhancements to speaker recognition. In Acoustics, Speech and Signal Processing, 2009. ICASSP 2009. IEEE International Conference on, IEEE, 4533--4536.

Digital Library

[13]

Merrell, P., Schkufza, E., Li, Z., Agrawala, M., and Koltun, V. 2011. Interactive furniture layout using interior design guidelines. In ACM SIGGRAPH 2011 papers, 87:1--87:10.

Digital Library

[14]

Murtagh, F. 1984. Complexities of hierarchic clustering algorithms: state of the art. Computational Statistics Quarterly 1, 2, 101--113.

[15]

Silverman, B. 1986. Density estimation for statistics and data analysis, vol. 26. Chapman & Hall/CRC.

[16]

Steinbach, M., Karypis, G., and Kumar, V. 2000. A comparison of document clustering techniques. In KDD workshop on text mining, vol. 400, 525--526.

[17]

Torsello, A., Albarelli, A., and Pelillo, M. 2007. Matching relational structures using the edge-association graph. In Image Analysis and Processing, 2007. ICIAP 2007. 14th International Conference on, 775--780.

Digital Library

[18]

Varga, T., and Bunke, H. 2003. Generation of synthetic training data for an hmm-based handwriting recognition system. In Document Analysis and Recognition, 2003. Proceedings. Seventh International Conference on, IEEE, 618--622.

Digital Library

[19]

Wolfson, H., and Rigoutsos, I. 1997. Geometric hashing: an overview. Computational Science Engineering, IEEE 4, 4 (oct-dec), 10--21.

Digital Library

[20]

Xu, K., Stewart, J., and Fiume, E. 2002. Constraint-based automatic placement for scene composition. In Graphics Interface 2002, 25--34.

[21]

Xu, K., Zhang, H., Cohen-Or, D., and Chen, B. 2012. Fit and diverse: Set evolution for inspiring 3d shape galleries. ACM Transactions on Graphics 31, 4.

Digital Library

[22]

Yeh, Y.-T., Yang, L., Watson, M., Goodman, N. D., and Hanrahan, P. 2012. Synthesizing open worlds with constraints using locally annealed reversible jump mcmc. ACM Transactions on Graphics 31, 4.

Digital Library

[23]

Yu, L.-F., Yeung, S.-K., Tang, C.-K., Terzopoulos, D., Chan, T. F., and Osher, S. J. 2011. Make it home: automatic optimization of furniture arrangement. In ACM SIGGRAPH 2011 papers, 86:1--86:12.

Digital Library

Cited By

Ji XPan ZGao XPan J(2024)Text-Guided Synthesis of Crowd AnimationACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657516(1-11)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657516
Sun JYang JMo KLai YGuibas LGao L(2024)Haisor: Human-aware Indoor Scene Optimization via Deep Reinforcement LearningACM Transactions on Graphics10.1145/363294743:2(1-17)Online publication date: 3-Jan-2024
https://dl.acm.org/doi/10.1145/3632947
Fu QHe SLi XFu H(2024)PlanNet: A Generative Model for Component-Based Plan SynthesisIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.327520030:8(4739-4751)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3275200
Show More Cited By

Index Terms

Example-based synthesis of 3D object arrangements
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Image and video acquisition
        3D imaging
  2. Computer graphics
    1. Animation
    2. Shape modeling
2. Theory of computation
  1. Randomness, geometry and discrete structures
    1. Computational geometry

Recommendations

Reshuffle-based interior scene synthesis
VRCAI '13: Proceedings of the 12th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry

We present a reshuffle-based approach for synthesizing interior scenes from only a small set of input scenes. Our system can analyze the input scenes to generate functionally plausible scene variations while for previous methods, the small input may not ...
Interactive example-based urban layout synthesis

We present an interactive system for synthesizing urban layouts by example. Our method simultaneously performs both a structure-based synthesis and an image-based synthesis to generate a complete urban layout with a plausible street network and with ...
Adaptive synthesis of indoor scenes via activity-associated object relation graphs

We present a system for adaptive synthesis of indoor scenes given an empty room and only a few object categories. Automatically suggesting indoor objects and proper layouts to convert an empty room to a 3D scene is challenging, since it requires ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 31, Issue 6

November 2012

794 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2366145

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 2012

Published in TOG Volume 31, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

244
Total Citations
View Citations
1,657
Total Downloads

Downloads (Last 12 months)106
Downloads (Last 6 weeks)17

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ji XPan ZGao XPan J(2024)Text-Guided Synthesis of Crowd AnimationACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657516(1-11)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657516
Sun JYang JMo KLai YGuibas LGao L(2024)Haisor: Human-aware Indoor Scene Optimization via Deep Reinforcement LearningACM Transactions on Graphics10.1145/363294743:2(1-17)Online publication date: 3-Jan-2024
https://dl.acm.org/doi/10.1145/3632947
Fu QHe SLi XFu H(2024)PlanNet: A Generative Model for Component-Based Plan SynthesisIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.327520030:8(4739-4751)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3275200
Zhang STam HLi YRen KFu HZhang S(2024)SceneDirector: Interactive Scene Synthesis by Simultaneously Editing Multiple Objects in Real-TimeIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.326811530:8(4558-4569)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3268115
Hu FMa YZhong WYe LYang XFang LZhang Q(2024)A Dataset and Benchmark for 3D Scene Plausibility AssessmentIEEE Transactions on Multimedia10.1109/TMM.2024.335345626(6529-6541)Online publication date: 2024
https://doi.org/10.1109/TMM.2024.3353456
Kutzias Dvon Mammen S(2024)Recent Advances in Procedural Generation of Buildings: From Diversity to IntegrationIEEE Transactions on Games10.1109/TG.2023.326250716:1(16-35)Online publication date: Mar-2024
https://doi.org/10.1109/TG.2023.3262507
Zhang RXiong KDu HNiyato DKang JShen XPoor H(2024)Generative AI-Enabled Vehicular Networks: Fundamentals, Framework, and Case StudyIEEE Network: The Magazine of Global Internetworking10.1109/MNET.2024.339176738:4(259-267)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/MNET.2024.3391767
Zeng WLiu X(2024)RoomGen: 3D Interior Furniture Layout Generation2024 Asia-Pacific Conference on Image Processing, Electronics and Computers (IPEC)10.1109/IPEC61310.2024.00096(529-534)Online publication date: 12-Apr-2024
https://doi.org/10.1109/IPEC61310.2024.00096
Tang JNie YMarkhasin LDai AThies JNießner M(2024)DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.01938(20507-20518)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.01938
Weber RMueller CReinhart C(2024)A hypergraph model shows the carbon reduction potential of effective space use in housingNature Communications10.1038/s41467-024-52506-z15:1Online publication date: 27-Sep-2024
https://doi.org/10.1038/s41467-024-52506-z
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents