Information theory in computer graphics and visualization
Article No.: 10, Pages 1 - 58
Abstract
We present a half-day course to review several information theory applications for computer graphics and visualization. Information theory tools, widely used in scientific fields such as engineering, physics, genetics and neuroscience, are also emerging as useful transversal tools in computer graphics and related fields. We introduce the basic concepts of information theory and how they map into application areas. Application areas in computer graphics include viewpoint selection, mesh saliency, scene exploration, ambient occlusion, geometry simplification, radiosity, adaptive ray-tracing and shape descriptors. Application areas in visualization are view selection for volume data, flow visualization, ambient occlusion, time-varying volume visualization, transfer function definition, time-varying volume visualization, iso-surface similarity maps and quality metrics. The applications fall broadly into two categories: the mapping of the problem to an information channel - as in viewpoint applications - and the direct use of measures such as entropy, Kullback-Leibler distance, Jensen-Shannon divergence, and f-divergences. These would be used to evaluate, for instance, the homogeneity of a set of samples being used as metrics. We will also discuss the potential applications of the information bottleneck method that allows us to progressively extract or merge information in a hierarchical structure.
References
[1]
M. S. Ali and S. D. Silvey. A general class of coefficient of divergence of one distribution from another. Journal of Royal Statistical Society (Serie B), 28(1):131--142, 1966.
[2]
J. Burbea and C. R. Rao. On the convexity of some divergence measures based on entropy functions. IEEE Transactions on Information Theory, 28(3):489--495, May 1982,
[3]
R. Coleman. Stochastic Processes. George Allen & Unwin Ltd., London, UK, 1974.
[4]
T. M. Cover and J. A. Thomas. Elements of Information Theory. Wiley Series in Telecommunications, 1991.
[5]
I. Csiszár. Eine Informationshcoretische Ungleichung und ihre Anwendungen auf den Beweis der Ergodizität von Markoffsclien Ketten16. Magyar Tudományos Akadémia Közleményei, 8:85--108, 1963.
[6]
I. Csiszár and P. C. Shields. Information theory and statistics: A tutorial. Foundations and Trends in Communications and Information Theory, 1(4), 2004.
[7]
D. M. Endres and J. E. Schindelin. A new metric for probability distributions. IEEE Transactions on Information Theory, 49(7):1858--1860, 2003.
[8]
D. P. Feldman. A brief introduction to: Information theory, excess entropy and computational mechanics. 2002.
[9]
D. P. Feldman and J. P. Crutchfield. Discovering noncritical organization: Statistical mechanical, information theoretic and computational views of patterns in one-dimensional spin systems. Working Paper 98-04-026, Santa Fe Institute, Santa Fe (NM), USA, April 1998.
[10]
R. M. Gray. Entropy and Information Theory. Springer-Verlag. New York (NY), USA, 1990.
[11]
J. Harvda and F. Charvát. Quantification method of classification processes. Concept of structural α-entropy. Kybernetika, 3:30--35, 1967.
[12]
E. D. Hellinger. Neue Begründung der Theorie der Quadratischen Formen von Unendlichen Vielen Veränderlichen17. Journal für Reine und Angewandte Mathematik, 136:210-271, 1909.
[13]
E. T. Jaynes. Information theory and statistical mechanics. Physical Review, 106(4):620--630, May 1957.
[14]
A. N. Kolmogorov. On the Shannon theory of information transmission in the case of continuous signals. IRE Transactions on Information Theory, 2:102--108, 1956.
[15]
S. Kullback and R. A. Leibler. On information and sufficiency. Annuls of Mathematical Statistics, 22:76--86, 1951.
[16]
A. Papoulis. Probability, Random Variables, and Stochastic Processes. McGraw-Hill, New York (NY), USA. 1984.
[17]
M. Pardo and I. Vajda. On asymptotic properties of information-theoretic divergences. IEEE Transactions on Information Theory, 49(7): 1860--1868, 2003.
[18]
K. Pearson. On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Philosophical Magazine, V(1):157--175. 1900.
[19]
M. S. Pinsker. Information and Stability of Random Variables and Processes. Izdatel'stvo Akademii Nauk SSSR, Moscow, Russia, 1960.
[20]
A. Rényi. On measures of entropy and information. In Proc. Fourth Berkeley Symp. Math. Stat, and Probability' 60, volume 1, pages 547--561, Berkeley (CA), USA, 1961. University of California Press.
[21]
M. Sbert, M. Feixas, J. Rigau, M. Chover, and I. Viola. Information Theory Tools for Computer Graphics. Synthesis Lectures on Computer Graphics and Animation. Morgan & Claypool Publishers, 2009, http://dx.doi.org/10.2200/S00208ED1V01Y200909CGR012.
[22]
C. E. Shannon. A mathematical theory of communication. The Bell System Technical Journal, 27:379--423, 623--656, July. October 1948.
[23]
B. D. Sharma and D. P. Mittal. New non-additive measures of entropy for a. discrete probability distribution. Journal of Mathematical Sciences (India), 10:28--40. 1975.
[24]
B. D. Sharma and I. J. Taneja. Entropy of type (α, β) and other generalized measures in information theory. Metrika. 22(1):205--215, 1975.
[25]
N. Slonim and N. Tishby. Agglomerative information bottleneck. In Proceedings of NIPS-12 (Neural Information Processing Systems), pages 617--623. MIT Press, 2000,
[26]
N. Slonim and N. Tishby. Document clustering using word clusters via the information bottleneck method. In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 208--215. ACM. Press. 2000.
[27]
I. J. Taneja. Bivariate measures of type a and their applications. Tamkang Journal of Mathematics, 19(3):63--74, 1988.
[28]
I. J. Taneja. On generalized information measures and their applications. In Advances in Electronics and Electron Physics, volume 76, pages 327--413. Academic-Press Ltd., 1989.
[29]
N. Tishby, F. C. Pereira, and W. Bialek. The information bottleneck method. In Proceedings of the 37th Annual Allerton Conference on Communication, Control and Computing, pages 368--377, 1999.
[30]
C. Tsallis. Possible generalization of Boltzmann-Gibbs statistics. Journal of Statistical Physics, 52(1/2):479--487, 1988.
[31]
C. Tsallis. Generalized entropy-based criterion for consistent testing. Physical Review E, 58:1442--1445, 1998.
[32]
C. Tsallis. Entropic nonextensivity: A possible measure of complexity. Chaos, Solitons, & Fractals, 13(3):371--391, 2002.
[33]
S. Verdú. Fifty years of Shannon theory. IEEE Transactions on Information Theory, 44(6):2057--2078: October 1998.
[34]
R. W. Yeung. Information Theory and Network Coding. Information Technology: Transmission. Processing and Storage. Springer, 2008.
[35]
{Cover06} T. M. Cover and J. A. Thomas. Elements of Information Theory. Wiley, 1991, 2006
[36]
{Yeung08} R. W. Yeung. Information Theory and Network. Springer, 2008
[37]
{DeWeese99} M. R. DeWeese and M. Meister. How to measure the information gained from one symbo., Network: Computation in Neural Systems, 10, 4, 325--340, 1999
[38]
{Butts03} D. A. Butts. How much information is associated with a particular stimulus?. Network: Computation in Neural Systems, 14, 177--187, 2003
[39]
{Burbea82} J. Burbea and C. R. Ra. On the convexity of some divergence measures based on entropy functions. IEEE Transactions on Information Theory, 28, 3, 489--495, 1982
[40]
{Slonim99} Noam Slonim and Naftali Tishby. Agglomerative Information Bottleneck. NIPS, 617--623, 1999
[41]
{Csiszár04} Imre Csiszár and Paul C. Shields. Information Theory and Statistics: A Tutorial. Communications and Information Theory, 1, 4, 2004
[42]
{Sbert09} M. Sbert, M. Feixas, J. Rigau, M. Chover, I. Viola. Information Theory Tools for Computer Graphics. Morgan and Claypool Publishers, 2009
[43]
P. Hanrahan, D. Salzman, and L. Aupperle. A rapid hierarchical radiosity algorithm. Computer Graphics (Proceedings of SIGGRAPH '91), 25(4): 197--206, July 1991.
[44]
M. F. Cohen and J. R. Wallace. Radiosity and Realistic Image Synthesis. Academic Press Professional, Boston (MA), USA, 1993
[45]
B. E. Smits, J. Arvo, and D. Salesin. An importance-driven radiosity algorithm. Computer Graphics (Proceedings of SIGGRAPH '92), 26(2):273--282, July 1992.
[46]
S. J. Gortler, P. Schröder, M. F. Cohen, and P. Hanrahan. Wavelet radiosity. In J. T. Kajiya, editor, Computer Graphics (Proceedings of SIGGRAPH '93), volume 27 of Annual Conference Series, pages 221--230. ACM, New York (NY), USA, August 1993.
[47]
P. Bekaert. Hierarchical and Stochastic Algorithms for Radiosity. PhD thesis, Katholieke Universiteit Leuven, Leuven, Belgium, December 1999.
[48]
M. Feixas. An Information-Theory Framework for the Study of the Complexity of Visibility and Radiosity in a Scene. PhD thesis, Universitat Politècnica de Catalunya, Barcelona, Spain, December 2002.
[49]
J. Rigau, M. Feixas, and M. Sbert. A generalised-mutual-information-based oracle for hierarchical radiosity. In Y. Shi, G. Albada, J. Dongarra, and P. M. Sloot, editors, Computational Science - ICCS 2007, 7th International Conference, volume 4488 (II) of LectureNotes in Computer Science, pages 105--113. Springer-Verlag, May 2007
[50]
J. Rigau, M. Feixas, and M. Sbert. Refinement criteria based on f-divergences. In P. H. Christensen and D. Cohen-Or, editors, Rendering Techniques 2003 (14th Eurographics Symposium on Rendering), pages 260--269, New York (NY), USA, June 2003. Association for Computing Machinery.
[51]
W. Purgathofer. A statistical method for adaptive stochastic sampling. Eurographics '86: Proceedings of the European Conference and Exhibition, 11(2):157--162, August 1986.
[52]
D. P. Mitchell. Generating antialiased images at low sampling densities. Computer Graphics (Proceedings of SIGGRAPH '87), 21(4):65--72, July 1987
[53]
R. Tamstorf and H. W. Jensen. Adaptive sampling and bias estimation in path tracing. In J. Dorsey and P. Slusallek, editors, RenderingTechniques '97 (Proceedings of the 8th Eurographics Workshop on Rendering), pages 285--295, New York (NY), USA, June 1997. Springer-Verlag
[54]
M. Simmons and C. H. Séquin. Tapestry: A dynamic mesh-based display representation for interactive rendering. In B. Péroche and H. Rushmeier, editors, Rendering Techniques 2000 (Proceedings of the 11th EurographicsWorkshop on Rendering), pages 329--340, New York (NY), USA, June 2000. Springer-Verlag
[55]
J. Rigau, M. Feixas, P. Bekaert, and M. Sbert. View-dependent information theory measures for pixel sampling and scene discretization in flatland. In Proceedings of Spring Conference on Computer Graphics '01, pages 173--180, Los Alamitos (CA), USA, April 2001. IEEE Computer Society.
[56]
J. Rigau, M. Feixas, and M. Sbert. New contrast measures for pixel supersampling. In J. Vince and R. Earnshaw, editors, Advances in Modeling, Animation and Rendering (Proceedings of CGI'02), pages 439--451, London, UK, July 2002. Springer-Verlag London Limited.
[57]
J. Rigau, M. Feixas, and M. Sbert. Entropy-based adaptive sampling. In Graphics Interface, pages 149--157. Canadian Information Processing Society, A. K. Peters Ltd., June 2003.
[58]
Qing Xu, Mateu Sbert, Lianping Xing, and Jianfeng Zhang. A Novel Adaptive Sampling by Tsallis Entropy. Proc. CGIV 2007 (Fourth International Conference on Computer Graphics, Imaging and Visualization, August 2007), pp. 5--10, 2007, IEEE Computer Society.
[59]
Qing Xu, Mateu Sbert, Miquel Feixas, Riccardo Scopigno, A New Refinement Criterion for Adaptive Sampling in Path Tracing, ISIE 2010, Bari, Italy
[60]
D. L. Page, A. F. Koschan, S. R. Sukumar, B. Roui-Abidi, and M. A. Abidi. Shape analysis algorithm based on information theory. In IEEE International Conference on Image Processing (Proceedings of ICIP'03), volume 1, pages 229--232, September 2003
[61]
J. Feldman and M. Singh. Information along contours and object boundaries. Psychological Review, 112(1):243--252, January 2005.
[62]
R. Osada, T. Funkhouser, B. Chazelle, and D. Dobkin. Shape distributions. ACM Transactions on Graphics, 21(4):807--832, October 2002
[63]
J. Rigau, M. Feixas, and M. Sbert. Shape complexity based on mutual information. In International Conference on Shape Modeling and Applications (SMI '05), pages 355--360, Los Alamitos (CA), USA, June 2005. IEEE Computer Society
[64]
Francisco González Garcia, Miquel Feixas and Mateu Sbert. View-based Shape Similarity using Mutual Information Spheres. Eurographics'07 Short Paper, Prague, Czech Republic, 2007
[65]
P. A. Viola. Alignment by maximization of mutual information. Ph.D. dissertation, MIT Artificial Intelligence Laboratory (TR 1548), Massachusetts (MA), USA, 1995
[66]
F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens. Multimodality image registration by maximization of mutual information. IEEE Transactions on Medical Imaging, vol. 16, no. 2, pp. 187--198, 1997
[67]
A. Bardera, J. Rigau, I. Boada, M. Feixas, and M. Sbert. Image segmentation using information bottleneck method. IEEE Transactions on Image Processing, vol. 18, no. 7, pp. 1601--1612, 2009
[68]
{Chen and Jänicke 2010} M. Chen, H. Jänicke: An Information-Theoretic Framework for Visualization, IEEE TVCG (Proceedings of IEEE Visualization), 2010
[69]
{Takahashi et al. 2005} S. Takahashi, I. Fujishiro, Y. Takeshima, T. Nishita: A Feature-Driven Approach to Locating Optimal Viewpoints for Volume Visualization, In Proceedings of IEEE Visualization, 2005
[70]
{Bordoloi and Shen 2005} U. Bordoloi and H. W. Shen: View Selection for Volume Rendering, In Proceedings of IEEE Visualization, 2005
[71]
{Ji and Shen 2006} G. Ji and H. W. Shen: Dynamic View Selection for Time-Varying Volumes, In IEEE TVCG (Proceedings of IEEE Visualization), 2006
[72]
{Viola et al. 2006} I. Viola, M. Feixas, M. Sbert, M. E. Gröller: Importance-Driven Focus of Attention, In IEEE TVCG (Proceedings of IEEE Visualization), 2006
[73]
{Bruckner and Möller 2010} S. Bruckner and T. Möller: isosurface Similarity Maps, In EG CGF (Proceedings of EuroVis), 2010
[74]
{Ruiz et al. 2008} M. Ruiz, I. Viola, I. Boada, S. Bruckner, M. Feixas, M. Sbert: Similarity-based Exploded Views, In Springer LNCS (Proceedings of Smart Graphics), 2008
[75]
{Haidacher et al. 2008} M. Haidacher, S. Bruckner, A. Kanitsar, and M. E. Gröller: Information-based Transfer Functions for Multimodal Visualization, In Proceedings of EG VCBM, 2008
[76]
{Bramon et al. 2012} R. Bramon, I. Boada, A. Bardera, J. Rodriguez, M. Feixas, J. Puig, M. Sbert: Multimodal Data Fusion based on Mutual Information, IEEE TVCG (to appear), 2012
[77]
{Haidacher et al. 2011} M. Haidacher, S. Bruckner, M. E. Gröller: Volume Analysis Using Multimodal Surface Similarity, IEEE TVCG (Proceedings of IEEE Visualization), 2011
[78]
{Wang and Shen 2006} C. Wang and H. W. Shen: LOD Map - A Visual Interface for Navigating Multiresolution Volume Visualization, In IEEE TVCG (Proceedings of IEEE Visualization), 2006
[79]
{Jänicke and Scheuermann 2009} H. Jänicke, G. Scheuermann: Visual Analysis of Flow Features Using Information Theory, IEEE CG&A, 2009
[80]
{Xu et al. 2010} L. Xu, T.-Y. Lee, H.-W. Shen: An Information-Theoretic Framework for Flow Visualization, IEEE TVCG (Proceedings of IEEE Visualization), 2011
[81]
{Ruiz et al. 2011} M. Ruiz, A. Bardera, I. Boada, I. Viola, M. Feixas, M. Sbert: Automatic Transfer Functions based on Informational Divergence, IEEE TVCG (Proceedings of IEEE Visualization), 2010
[82]
{Wang et al. 2008} C. Wang, H. Yu, K.-L. Ma: Importance-Driven Time-Varying Data Visualization, IEEE TVCG (Proceedings of IEEE Visualization), 2008
[83]
{Wang and Chen 2011} C. Wang and H.-W. Shen: Information Theory in Scientific Visualization, Entropy, 13, 2011
[84]
{Mühler et al. 2007} K. Mühler, M. Neugebauer, C. Tietjen, and B. Preim: Viewpoint Selection for Intervention Planning, EG CGF (Proceedings of EuroVis), 2007
[85]
{Chen 2008} C. Chen: An Information-Theoretic View of Visual Analytics, IEEE CG&A, 2008
[86]
{Haidacher 2011} M. Haidacher: Information-based Feature Enhancement in Scientific Visualization, PHD Thesis, Vienna University of Technology, 2011
[87]
{Sbert et al. 2009} M. Sbert, M. Feixas, J. Rigau, M. Chover, I. Viola: Information Theory Tools for Computer Graphics, Morgan and Claypool Publishers, 2009
Index Terms
- Information theory in computer graphics and visualization
Index terms have been assigned to the content through auto-classification.
Recommendations
Visualization and Computer Graphics on Isotropically Emissive Volumetric Displays
The availability of commodity volumetric displays provides ordinary users with a new means of visualizing 3D data. Many of these displays are in the class of isotropically emissive light devices, which are designed to directly illumi-nate voxels in a 3D ...
Computer graphics visualization for acoustic simulation
Special issue: Proceedings of the 1989 ACM SIGGRAPH conferenceComputer simulations can be used to generate the spatial and temporal data describing the acoustical behavior of performance halls, but typically the analytical results are difficult to assimilate and compare. By using computer graphics to display the ...
Comments
Information & Contributors
Information
Published In
Copyright © 2011 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 12 December 2011
Check for updates
Qualifiers
- Research-article
Conference
SA '11
Sponsor:
Acceptance Rates
Overall Acceptance Rate 178 of 869 submissions, 20%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 728Total Downloads
- Downloads (Last 12 months)2
- Downloads (Last 6 weeks)0
Reflects downloads up to 08 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in