research-article

Automatic extraction and synthesis of regular repeatable patterns

Authors:

Carlos Rodriguez-Pardo,

Jorge Lopez-Moreno,

Elena GarcesAuthors Info & Claims

Volume 83, Issue C

Pages 33 - 41

https://doi.org/10.1016/j.cag.2019.06.010

Published: 01 October 2019 Publication History

Highlights

•

Automatic method capable of synthesizing regular repeating patterns in images.

•

Latent spaces of convolutional networks are used to find minimum tileable patterns.

•

Image regularization improve consistency in the estimation of the pattern.

•

Deep perceptual metrics are effective evaluating the quality of the texture tiling.

Graphical abstract

Display Omitted

Abstract

Textures made of regular repeating patterns are ubiquitous in the real world, most notably in man-made environments. They are defined by the presence of a repeating element, which can show a significant amount of random variations, non-rigid deformations or color noise. We propose an end-to-end pipeline capable of finding the size of the minimal repeating pattern in single images, as well as obtaining the single repetition that, when tiled, produces the most similar synthesis to the complete image. We do this by combining state-of-the-art algorithms in image transformations, repeating pattern detection, image stitching and deep perceptual losses. Additionally, we show how our pipeline can find the minimal color pattern in woven fabrics, which can be useful for both surface-based render methods and computer vision tasks in the textile domain.

References

[1]

J. Moritz, S. James, T.S. Haines, T. Ritschel, T. Weyrich, Texture stationarization: turning photos into tileable textures, Computer graphics forum (Proc. Eurographics), 36, Wiley Online Library, 2017, pp. 177–188.

[2]

Y. Liu, W.-C. Lin, J. Hays, Near-regular texture analysis and manipulation, ACM transactions on graphics, 23, ACM, 2004, pp. 368–376.

[3]

P. Upchurch, J. Gardner, G. Pleiss, R. Pless, N. Snavely, K. Bala, et al., Deep feature interpolation for image content changes, Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 7064–7073.

[4]

L. Lettry, M. Perdoch, K. Vanhoey, L. Van Gool, Repeated pattern detection using cnn activations, Proceedings of the IEEE winter conference on applications of computer vision (WACV), IEEE, 2017, pp. 47–55.

[5]

A.A. Efros, W.T. Freeman, Image quilting for texture synthesis and transfer, Proceedings of the 28th annual conference on computer graphics and interactive techniques, ACM, 2001, pp. 341–346.

Digital Library

[6]

A. Kaspar, B. Neubert, D. Lischinski, M. Pauly, J. Kopf, Self tuning texture optimization, Computer graphics forum, 34, Wiley Online Library, 2015, pp. 349–359.

[7]

L. Gatys, A.S. Ecker, M. Bethge, Texture synthesis using convolutional neural networks, Advances in neural information processing systems, abs/1505.07376, 2015, pp. 262–270.

[8]

O. Sendik, D. Cohen-Or, Deep correlations for texture synthesis, ACM Trans Gr 36 (2017) 161.

[9]

Y. Zhou, Z. Zhu, X. Bai, D. Lischinski, D. Cohen-Or, H. Huang, Non-stationary texture synthesis by adversarial expansion, 2018.

[10]

Y. Liu, R.T. Collins, Y. Tsin, A computational model for periodic pattern perception based on frieze and wallpaper groups, Proceedings of the IEEE transactions on pattern analysis and machine intelligence, 26, IEEE, 2004, pp. 354–371.

[11]

P. Zhao, L. Quan, Translation symmetry detection in a fronto-parallel view, Proceedings of the IEEE conference on computer vision and pattern recognition, IEEE, 2011, pp. 1009–1016.

[12]

J. Liu, Y. Liu, Grasp recurring patterns from a single view, Proceedings of the IEEE conference on computer vision and pattern recognition, 2013, pp. 2003–2010.

[13]

J. Pritts, O. Chum, J. Matas, Detection, rectification and segmentation of coplanar repeated patterns, Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 2973–2980.

[14]

A. Torii, J. Sivic, T. Pajdla, M. Okutomi, Visual place recognition with repetitive structures, Proceedings of the IEEE conference on computer vision and pattern recognition, 2013, pp. 883–890.

[15]

D.G. Lowe, Object recognition from local scale-invariant features, Proceedings of the Seventh IEEE International Conference on Computer Vision, IEEE, Kerkyra, Greece, Greece, 1999.

[16]

R. Szeliski, M. Uyttendaele, D. Steedly, Fast poisson blending using multi-splines, Proceedings of the IEEE international conference on computational photography (ICCP), IEEE, 2011, pp. 1–8.

[17]

A. Levin, A. Zomet, S. Peleg, Y. Weiss, Seamless image stitching in the gradient domain, Proceedings of the European conference on computer vision, Springer, 2004, pp. 377–389.

[18]

E. Adel, M. Elmogy, H. Elbakry, Image stitching system based on orb feature-based technique and compensation blending, Proceedings of the international journal of advanced computer science and applications, 6, 2015.

[19]

V.G. Kim, Y. Lipman, T.A. Funkhouser, Symmetry-guided texture synthesis and manipulation, ACM Trans Gr 31 (2012) 22.

[20]

T. Weibel, C. Daul, D. Wolf, R. Rösch, Contrast-enhancing seam detection and blending using graph cuts, Proceedings of the 21st international conference on pattern recognition, IEEE, 2012, pp. 2732–2735.

[21]

S. Darabi, E. Shechtman, C. Barnes, D.B. Goldman, P. Sen, Image melding: Combining inconsistent images using patch-based synthesis., ACM Trans Gr 31 (2012).

[22]

T. Dekel, T. Michaeli, M. Irani, W.T. Freeman, Revealing and modifying non-local variations in a single image, ACM Trans Gr 34 (2015) 227.

[23]

K. Jafari-Khouzani, H. Soltanian-Zadeh, Radon transform orientation estimation for rotation invariant texture analysis, IEEE Trans Pattern Anal Mach Intell 27 (2005) 1004–1008.

[24]

D. Zheng, Y. Han, J.L. Hu, A new method for classification of woven structure for yarn-dyed fabric, Text Res J 84 (2014) 78–95.

[25]

G.C. Guarnera, P. Hall, A. Chesnais, M. Glencross, Woven Fabric Model Creation from a Single Image, ACM Trans Gr 36 (2017) 1–13.

[26]

C. Barnes, E. Shechtman, D.B. Goldman, A. Finkelstein, The generalized patchmatch correspondence algorithm, Proceedings of the European conference on computer vision, Springer, 2010, pp. 29–43.

[27]

K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.

[28]

J. Redmon, S. Divvala, R. Girshick, A. Farhadi, You only look once: Unified, real-time object detection, Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 779–788.

[29]

L.A. Gatys, A.S. Ecker, M. Bethge, Image style transfer using convolutional neural networks, Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2016.

[30]

A. Mahendran, A. Vedaldi, Visualizing deep convolutional neural networks using natural pre-images, Int J Comput Vis 120 (2016) 233–255.

Digital Library

[31]

G. Deng, L. Cahill, An adaptive gaussian filter for noise reduction and edge detection, Proceedings of the IEEE conference record nuclear science symposium and medical imaging conference, IEEE, 1993, pp. 1615–1619.

[32]

V. Kwatra, I. Essa, A. Bobick, N. Kwatra, Texture optimization for example-based synthesis, ACM Trans Gr 24 (2005) 795–802.

[33]

K. Briechle, U.D. Hanebeck, Template matching using fast normalized cross correlation, Optical pattern recognition XII, 4387, International Society for Optics and Photonics, 2001, pp. 95–103.

[34]

P. Pérez, M. Gangnet, A. Blake, Poisson image editing, ACM Trans Gr 22 (2003) 313–318.

[35]

R. Zhang, P. Isola, A.A. Efros, E. Shechtman, O. Wang, The unreasonable effectiveness of deep features as a perceptual metric, Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 586–595.

[36]

A. Sanakoyeu, D. Kotovenko, S. Lang, B. Ommer, A style-aware content loss for real-time hd style transfer, Proceedings of the European conference on computer vision (ECCV), 2018, pp. 698–714.

[37]

Y. Liu, Y.-M. Zhang, X.-Y. Zhang, C.-L. Liu, Adaptive spatial pooling for image classification, Pattern Recognit 55 (2016) 58–67.

[38]

J. Johnson, A. Alahi, L. Fei-Fei, Perceptual losses for real-time style transfer and super-resolution, Proceedings of the European conference on computer vision, 2016.

[39]

A. Krizhevsky, I. Sutskever, G.E. Hinton, Imagenet classification with deep convolutional neural networks, Proceedings of the 25th international conference on neural information processing systems - Volume 1, Curran Associates Inc., USA, 2012, pp. 1097–1105.

[40]

J. Deng, W. Dong, R. Socher, L.-J. Li, Kai Li, Li Fei-Fei, ImageNet: A large-scale hierarchical image database, Proceedings of the IEEE conference on computer vision and pattern recognition, IEEE, 2009, pp. 248–255.

[41]

A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, et al., Automatic differentiation in pytorch, 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA., 2017.

[42]

A. Canziani, A. Paszke, E. Culurciello, An analysis of deep neural network models for practical applications, 2016, arXiv:1605.07678.

[43]

C. Tomasi, R. Manduchi, Bilateral filtering for gray and color images, Proceedings of the ICCV, 98, 1998, p. 2.

[44]

P. Müller, G. Zeng, P. Wonka, L. Van Gool, Image-based procedural modeling of facades, Proceedings of the ACM transactions on graphics, 26, ACM, New York, NY, USA, 2007.

[45]

J. Xiao, T. Fang, P. Zhao, M. Lhuillier, L. Quan, Image-based street-side city modeling, Proceedings of the ACM transactions on graphics, 28, ACM, New York, NY, USA, 2009, pp. 114:1–114:12.

[46]

F.N. Iandola, M.W. Moskewicz, K. Ashraf, S. Han, W.J. Dally, K. Keutzer, Squeezenet: alexnet-level accuracy with 50x fewer parameters and <1mb model size, 2016, abs/1602.07360.

Cited By

Sartor SPeers P(2024)Content-aware Tile Generation using Exterior Boundary InpaintingACM Transactions on Graphics10.1145/368798143:6(1-12)Online publication date: 19-Dec-2024
https://dl.acm.org/doi/10.1145/3687981
Aigerman NGroueix T(2024)Generative Escher MeshesACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657452(1-11)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657452
Guerrero-Viu JHasan MRoullier AHarikumar MHu YGuerrero PGutiérrez DMasia BDeschaintre V(2024)TexSliders: Diffusion-Based Texture Editing in CLIP SpaceACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657444(1-11)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657444
Show More Cited By

Index Terms

Automatic extraction and synthesis of regular repeatable patterns
1. Computing methodologies

Index terms have been assigned to the content through auto-classification.

Recommendations

Mining Regular Patterns in Transactional Databases

The frequency of a pattern may not be a sufficient criterion for identifying meaningful patterns in a database. The temporal regularity of a pattern can be another key criterion for assessing the importance of a pattern in several applications. A ...
mpLBP: A point-based representation for surface pattern description
Highlights
- The mpLBP extend the LBP to surfaces represented by point cloud.
- The mpLBP ...
Graphical abstract

Display Omitted

Abstract
The Local Binary Pattern (LBP) is a very popular pattern descriptor for images that is widely used to classify repeated pixel arrangements in a query image. Several extensions of the LBP to surfaces exist, for both geometric and ...
Hierarchical spatial pyramid max pooling based on SIFT features and sparse coding for image classification

It is essential to build good image representations for many computer vision tasks. In this study, the authors propose a hierarchical spatial pyramid max pooling method based on scale‐invariant feature transform (SIFT) features and sparse coding, which ...

Comments

Information & Contributors

Information

Published In

cover image Computers and Graphics

Computers and Graphics Volume 83, Issue C

Oct 2019

122 pages

ISSN:0097-8493

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 01 October 2019

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Sartor SPeers P(2024)Content-aware Tile Generation using Exterior Boundary InpaintingACM Transactions on Graphics10.1145/368798143:6(1-12)Online publication date: 19-Dec-2024
https://dl.acm.org/doi/10.1145/3687981
Aigerman NGroueix T(2024)Generative Escher MeshesACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657452(1-11)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657452
Guerrero-Viu JHasan MRoullier AHarikumar MHu YGuerrero PGutiérrez DMasia BDeschaintre V(2024)TexSliders: Diffusion-Based Texture Editing in CLIP SpaceACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657444(1-11)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657444
Tang YLi ZHasan MYang JWang B(2024)Woven Fabric Capture with a Reflection-Transmission Photo PairACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657410(1-10)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657410
Cortes DBermejo BJuiz C(2024)The use of CNNs in VR/AR/MR/XR: a systematic literature reviewVirtual Reality10.1007/s10055-024-01044-628:3Online publication date: 30-Aug-2024
https://dl.acm.org/doi/10.1007/s10055-024-01044-6
Rodriguez-Pardo CGarces E(2023)SeamlessGAN: Self-Supervised Synthesis of Tileable Texture MapsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.314361529:6(2914-2925)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1109/TVCG.2022.3143615
Guo SDeschaintre VNoll DRoullier A(2022)U-Attention to Textures: Hierarchical Hourglass Vision Transformer for Universal Texture SynthesisProceedings of the 19th ACM SIGGRAPH European Conference on Visual Media Production10.1145/3565516.3565525(1-10)Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1145/3565516.3565525
Jin WWang BHasan MGuo YMarschner SYan L(2022)Woven Fabric Capture from a Single PhotoSIGGRAPH Asia 2022 Conference Papers10.1145/3550469.3555380(1-8)Online publication date: 29-Nov-2022
https://dl.acm.org/doi/10.1145/3550469.3555380

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents