Computer vision and computer graphics analysis of paintings and drawings: An introduction to the literature David G. Stork Ricoh Innovations, 2882 Sand Hill Road Suite 115, Menlo Park CA 94025 USA and Department of Statistics, Stanford University, Stanford CA 94305 USA ABSTRACT In the past few years, a number of scholars trained in computer vision, pattern recognition, image processing, computer graphics, and art history have developed rigorous computer methods for addressing an increasing number of problems in the history of art. In some cases, these computer methods are more accurate than even highly trained connoisseurs, art historians and artists. Computer graphics models of artists’ studios and subjects allow scholars to explore “what if” scenarios and determine artists’ studio praxis. Rigorous computer ray-tracing software sheds light on claims that some artists employed optical tools. Computer methods will not replace tradition art historical methods of connoisseurship but enhance and extend them. As such, for these computer methods to be useful to the art community, they must continue to be refined through application to a variety of significant art historical problems. Keywords: pattern recognition, computer image analysis, brush stroke analysis, painting analysis, image forensics, compositing, computer graphics reconstructions 1. INTRODUCTION There is a long history of the use of sophisticated imaging and, in the past several decades digital imaging, in the study of art.1 Shortly after the 19th-century discovery of x-rays such rays were used to reveal underdrawings and pentimenti. Later, infra-red photography and reflectography were exploited to similar ends; multispectra, fluoroesence and ultra-violet imaging have become a widespread, and used in revealing pigment composition and more.2–5 In such techniques, the resulting image is generally interpretted by an art scholar. In the past few years, however, we have entered a new era: one where some of the image interpretation relies in great part upon sophisticated algorithms developed from computer vision, the discipline seeking to make computers “see.”6, 7 In some circumstances, computers can analyze certain aspects of perspective, lighting, color, the subtleties of the shapes of brush strokes better than even a trained art scholar, artist, or connoisseur. Rather than replacing connoisseurship, these methods—like other scientific methods such as imaging and material studies8 — hold promise to enhance and extend it, just as microscopes extend the powers of biologists. The source of the power of these computer methods arises from the following: • The computer methods can rely on visual features that are hard to determine by eye, for instance subtle relationships among the structure of a brush stroke at different scales or colors, as in Perugino’s Holy family, or lighting in de la Tour’s Christ in the carpenter’s studio, or perspective anomalies in van Eyck’s Arnolfini portrait. • The computer methods can abstract information from lots of visual evidence, for example, in principle every brush stroke executed by van Gogh and his contemporaries—a wealth of information that few scholars even experience, much less fully exploit. Send correspondence to David G. Stork, artanalyst@gmail.com. 1 • Computer methods are objective—which need not mean they are “superior” to subjective methods, but rather promise to extend the language of to include terms that are not highly ambiguous. While today an art historian may describe a brush stroke as “bold” or “tentative” or “fluid” someday this scholar may also use technical terms and mathematical measures derived from computer image analysis, terms that other scholars will understand as well. • Rigorous computer graphics techniques can reveal new three-dimensional views based on two-dimensional artwork, and provide new views into tableaus by dewarping the images reflected in mirrors depicted within a painting. This brief paper lists some of these new computer techniques and how they have been used in the study of art. The set of topics and reference works here is by no means complete but is meant to show art scholars the power of these method and to encourage art scholars to propose new art historical problems amenable to attach through computer methods.9–14 We shall not consider many other areas of computer use in arts, for instance computer art databases and retrieval, nor the task of imaging of art—the lighting, spectral filtering, exposure, and so on. Instead, we focus on the application of computer vision, image analysis and computer graphics algorithms to process and understand digital images of scanned art, particularly paintings and drawings. We begin by describing traditional point- or pixel-based processes such as color adjustment, then consider algorithms based on a number of pixels in a digital image of a painting, and then successively more complex methods of high-level computer vision and graphics, such as dewarping, perspective analysis, lighting analysis, and three-dimensional computer modelling. 2. POINT-BASED PROCEDURES Here and below we assume we have a digital image of a painting or drawing, the format required for computer analysis. The conceptually simplest class of computer image methods in the study of art are point- or pixel-based processing, that is, methods to alter the color and brightness of each pixel based solely on the color of that pixel. Such algorithms are better described as image processing than as image analysis.15, 16 Multispectral imaging and processing has been used for pigment analysis, color rejuvenation and predicting the effects of curatorial treatment.17–22 Pixel-based image processing has been used to adjust the relative weights of different spectral bands to enhance readability,23 as in the Archimedes palimpsest,24–28 and to reveal details and structure in art that otherwise difficult to discern by the unaided eye.29 The range of light intensities in the natural world—from a darkened room to bright sunlight—spans as much as a factor of 1014 while the dynamic range in oil painting might be a factor of merely 102 , even in the works of Caravaggio, de la Tour, Joseph Wright of Derby and others who exploited chiaroscuro. As such all artists must compress the luminance range in their works.30 Graham and Field explored the nonlinear compression of the dynamic range in several classes of realist art work, a process that is based on the individual pixel values.31, 32 3. AREA-BASED PROCEDURES A very large class of image processing algorithms involve filtering a source image, where the color or grayscale value of a pixel is a function of the values of pixels in an area or region of the input image. In linear filtering the output value (color or gray level) is a linear combination of the values of the input pixels, while in non-linear filtering allows arbitrary functions of the input pixels. Typically the input image is a photograph of a painting and the output image a digital image processed to reveal some properties that are difficult to discern by unaided eye. Such filtering can remove gradual variations in the color across a painting and leave or even enhance the edges or contours as, for instance, the (nonlinear) Canny edge detector.33–35 The Chamfer transform (or distance transform) is useful for quantifying similarity or difference between two shapes, for example when testing the fidelity that artists can achieve using different copying methods.36, 37 Another class of non-linear filters are the morphological operators. Such operators are generally used on binary (black and white) images rather than color or grayscale, where the shape (rather than the color) is the matter of interest. For example, a skeletonization operator yields a single-pixel-wide curve down the center of 2 a black brush stroke, regardless of the varying width of the stroke.38 Other popular morphological operators implement erosion, dilation, opening and closing. For instance, Stork, Meador and Noble compared the shapes of different passages of brick work in a painting from the Dutch Golden Age despite variations and irregularities in the width of the painted lines. To this end they preprocessed a high-resolution photograph of the painting using a morphological filter to create an image where the mortar lines were of uniform thickness. They then computed the cross-correlation of this image to search for repeated patterns.33, 39 4. PERSPECTIVE ANALYSIS The analysis of perspective, scale and geometry has a long and important history in the study of realist art, particularly in art history of the Renaissance.40 Most of these analytic methods involve simple drawing of perspective lines, finding horizon lines, vanishing points, and so on and can be done without computers. Recently, however, a number of sophisticated computer-based methods for analysis of perspective and geometry have been developed. Criminisi and his colleagues have pioneered rigorous methods for recovering three-dimensional space from single “uncalibrated” images, such as in paintings. These methods have produced three-dimensional virtual spaces of artworks such as Masaccio’s Trinità.41–44 Smith, Stork and Zhang reconstructed the three-dimensional space of the tableau based on multiple views depicted in plane reflections within a single painting.45 This method also reveals spatial inconsistencies between direct and reflected views and thereby sheds light on the artist’s working methods. While one can use simple commercial software, such as Adobe Photoshop, to perform perspective transformation between two images or passages—for instance two arms on the chandelier in van Eyck’s Arnolfini portrait 46 —but such a technique suffers from a number of drawbacks, the most severe is that the experimenter can arbitrarily choose which portions of one image should match their partners in the other image. Criminisi derived rigorous, principled methods for finding optimal transformations that minimized the shape differences, thus eliminating this drawback.36, 42, 47 Such analysis of perspective, perspective inconsistencies, and subtleties in shape have shed light on a number of topics, including the question of whether an artist used optical aids.48–58 5. ANAMORPHIC ART Anamorphic art is distorted art that appears distorted when viewed directly but undistorted when viewed from a special position or in reflection from a curved surface. Such a mirrored cylinder or cone is called an “anamorphoscope.” The root word comes from the Greek ανα, “again,” and µoρφη, “form”—the image in anamorphic art is formed again. The earliest surviving deliberate anamorphic image appears to be a sketch of a slant anamorphic eye, drawn by Leonardo around 1485. Perhaps the most celebrated example of such slant anamorphoses is the skull along the bottom in Hans Holbein’s The ambassadors.59, 60 There was much experimentation, mathematical analysis and flourishing of anamorphic art in the seventeenth and eighteenth centuries, particularly in France. Today the transformations required by anamorphic art are easily performed by computer. The optics and perspective underlying such art appears in a number of basic texts,30, 61 but perhaps the most complete and rigorous explanation is given by Hunt, Nickel and Gigault.62, 63 6. DEWARPING OF CURVED ART Many frescos and mosaics on architectural spandrels, barrel vaults and markings on pottery are curved and warped, and art scholars use computer methods to dewarp them to better study the images.64, 65 Often such digital dewarping requires an estimate of the camera position and the curvature properties of the surface itself. Another class of dewarping centers on dewarping the virtual image appearing in depictions of curved mirrors, such as in Parmigianino’s Self portrait in a convex mirror, van Eyck’s Arnolfini portrait, and Campin’s Heinrich von Werl and St. John the Baptist. Here one models the optics of the spherical or parabolic mirror and adjusts parameters to yield an undistorted image.41, 66, 67 The mirror properties (curvature and focal length) inferred by such a technique have been used to address claims that artists used such mirrors for optical projectors.49, 50, 54, 55 On can also use computer graphics methods (cf., Sect. ??, below) to dewarp reflected images, for instance in the analysis of Hans Memling’s van Nieuwenhove Diptych.67 3 7. ANALYSIS OF LIGHTING AND ILLUMINATION Some problems in art history require knowing or estimating the position and direction of lighting in a tableau. Such knowledge can be used for determining the studio conditions when the painting was executed; significant differences in the lighting on different subjects within a tableau may indicate the different studio conditions or presence of different hands, for example. Moreover, this information may indicate whether the artist used optical aids: if the light source was local rather than distant solar illumination, for instance, then it is highly unlikely projections were used.68 If the illuminant can be assumed to be small and relatively distant from the tableau, the simple method of cast-shadow analysis is particularly effective in locating the illuminant: one merely draws a line from a point on a cast shadow, through its associated occluder. This line should pass through the position of the illuminant.69 Several such lines, from a set of occluder-shadow pairs, will intersect at the position of the illuminant. A more sophisticated method, occluding-contour analysis, derives from forensic analysis of digital photographs, and is based on the pattern of light along an object’s outer boundary or occluding contour.70 Stork and Johnson recently extended the technique to apply to the case of diffuse illumination, where the pattern of illumination along an occluding boundary is described as a weighted set of spherical harmonics. If two figures in a painting differ significantly in their sets of coefficients, then they were likely painted under different studio lighting conditions. They studied the lighting in different figures by realist portraitist Garth Herrick and showed that different subjects were executed under different illumination conditions.71 Stork and Kale modeled the physics flat surfaces, and thereby inferred the position of the illuminant from the floor in Georges de la Tour’s Christ in the carpenter’s studio and Caravaggio’s The calling of St. Matthew.72, 73 This analysis showed that the light source in these works was likely local, rather than distant solar illumination, a result that rebuts claims that these works were executed by means of optical projections of very bright tableaus. Bayesian statistical methods can be used to integrate estimates derived from different sources, for example cast shadows and occluding contours, thereby refining and improving overall estimates.70, 74 Computer shape-from-shading methods infer the properties of illumination given a known or assumed threedimensional model of objects and a shaded image of those objects, such as in a highly realistic painting. One can assume a generic three-dimensional model (for instance the face in Girl with a pearl earring) and refine both the model and the direction to illumination.75 Another method for estimating the lighting in a realist tableau is to create a full computer graphics model of the scene and adjust the positions of the virtual illuminants so that the rendered image matches the painting as closely as possible. This has been used to estimate the direction of illumination in Vermeer’s Girl with a pearl earring, and Georges de la Tour’s Christ in the carpenter’s studio,75, 76 as described in Sect. 12. 8. ANALYSIS OF BRUSH STROKES AND MARKS One of the most extensively explored areas of computer analysis of art is the analysis of marks and drawing tools. The basic approach is to use techniques of statistical pattern recognition to learn visual properties of brush strokes that correspond to a particular painter or marking tool.33, 38, 77–95 In related techniques, Hedges analyzed the changes in marks in Renaissance copperplate prints as the plates were cleaned; his method yielded an image-based “clock” for estimating the age of such prints.96, 97 Shahram, Stork and Donoho developed the De-pict algorithm, which removed successive layers of brush strokes in a digital image of a painting, such as van Gogh’s Self portrait in a grey felt hat. When such a sequence of images is displayed in “reverse” (i.e., showing the sequence of brush strokes as likely added to the painting), scholars can see and better understand the aesthetic choices made by the artist.98 A somewhat separate class of mark analyses are those for the analysis of dripped painting, particularly in the works of American Abstract Expressionist Jackson Pollock. Here the analyses are generally based on fractals, a mathematical structure that exhibits regularities at different scales or sizes.99 Taylor and his colleagues first proposed that Pollock’s drip paintings exhibited fractal structure,100 though a number of scholars have questioned the recent claim that simple fractal dimension suffices to distinguish genuine Pollocks from forgeries or other apparently random images.101–106 Recent work has returned to the application of traditional image measures for the analysis of Pollock’s works, for instance curvature, connected components, and statistical pattern classification methods.?, 107 4 9. OPTICAL ANALYSIS OF ART It is well known that some artists used optical aids during the execution of passages in some of their works, e.g., Canaletto, Thomas Eakins, photo-realists such as Richard Estes and Robert Bechtle, and many others. Computer image analysis has addressed claims that some artists used optical aids when executing some passages in some of their works, for instance that Lorenzo Lotto secretly used a concave mirror projector to execute Husband and wife,108 that a wide range of artists as early as 1430 secretly traced optical images,109 that Jan Vermeer traced images projected in a camera obscura,110 and so on. While there are a number of perspective and lighting analyses brought to bear on such claims,42, 57, 111–114 as well as textual and material analyses,115, 116 the first and only analysis of paintings done by sophisticated computer ray tracing programs was by Stork and Robinson.117, 118 This research analyzed the aberrations and other properties of the setup in purported use of optics for Lorenzo Lotto and ultimately questioned the claim he used optics. 10. ANALYSIS OF CRAQUELURE Craquelure is the pattern of fine cracks on the surface of paintings and several scholars have used computer image analysis to characterize the patterns for art imagery retrieval.119–121 There remain opportunities for additional algorithms, for instance to detect and classify changes to craquelure due to injury to paintings. 11. ANALYSIS OF COMPOSITION Many artists have characteristic compositional styles and it is natural that computer methods be applied to learning and classifying these styles. A particularly attractive oeuvre is that of the neo-plastic abstractionist Piet Mondrian, where the formal elements are simple (horizontal and vertical lines, rectangles of a small number of colors, etc.) and the two-dimensional composition of central importance. A few scholars have approached this problem for Mondrian122, 123 but there are additional painters whose works might yield information through these methods, such as the large Abstract Expressionist works of Franz Kline. 12. COMPUTER GRAPHICS Computer graphics allows scholars to understand realist artists’ working methods by exploring “what if” scenarios. Note that creating a three-dimensional model based on a two-dimensional painting is formally “ill-posed,” that is, an infinite number of three-dimensional tableaus are consistent with a given two-dimensional projection.124 As such, the creation of a tableau virtuel is part art, part science. Nevertheless, the weal the (2D) image information such as occlusion and physical constraints such as objects toughing or being supported on a given floor, and lighting consistency, strongly constrain the three-dimensional models. It is important that the assumption—for instance that bodies have normal proportions, that faces are approximately left-right symmetric, and so on—not bias or favor one conclusion over another. Stork and Furuichi built a full three-dimensional model of Georges de la Tour’s Christ in the carpenter’s studio and adjusted the location of the virtual illuminant in the tableau virtuel until the digitally rendered image matched the painting as closely as possible. In this way, these authors found the illuminant was likely at the position of the candle, rather than in place of the figures, and thereby rebutted the claim that painting was executed by means of optical projections.76 Savarese and colleagues built a very simple model of the convex mirror and a planar model of the tableau in the left panel of Hans Memling’s van Nieuwenhove Diptych to test the consistency between the tableau and the image depicted in the convex mirror. The discrepancies between the mirror and the simple tableau suggested that Memling added the mirror later, as an afterthought.67 Johnson and colleagues built a computer model of Vermeer’s Girl with a pearl earring to estimate the direction of illumination.75 Most recently, Stork and Furuichi created a model of both the tableau in Diego Velàsquez’ Las meninas as well as the viewer’s space to explore the relationship between these two spaces, for instance whether the position of the viewer corresponded to that of the king and queen visible in the plane mirror.125 5 13. WEBSITES There are a few of websites addressing computer image analysis of art. • Computer image analysis in the study of art: www.diatrope.com/stork/FAQs.html • Digital painting analysis: digitalpaintinganalysis.org • IAPR computer vision in cultural heritage applications: iapr-tc19.prip.tuwien.ac.at/ • Antonio Criminisi’s publications: research.microsoft.com/∼antcrim/papers.htm • Christopher W. Tyler’s research on perception of art by humans and machines: www.diatrope.com/projects • Computers in the history of art: www.chart.ac.uk ACKNOWLEDGMENTS My thanks to all the scholars who provided references for this compilation. REFERENCES 1. A. Pelagotti, A. D. Mastio, A. D. Rosa, and A. Piva, “Multispectral imaging of paintings,” IEEE Signal Processing magazine 25(4), pp. 27–36, 2008. 2. K. Martinez, J. Cupitt, D. Saunders, and R. Pillay, “Ten years of art imaging research,” Proceedings of the IEEE 90(1), pp. 28–41, 2002. 3. M. Barni, A. Pelagotti, and A. Piva, “Image processing for the analysis and conservation of paintings: Opportunities and challenges,” IEEE Signal Processing magazine 22(5), pp. 141–144, 2005. 4. I. E. Berezhnoy, E. O. Postma, and J. van den Herik, “Computer analysis of van Gogh’s complementary colours,” Pattern Recognition Letters 28(6), pp. 703–709, 2006. 5. H. Maitre, F. Schmitt, and C. Lahanier, “15 years of image processing and the fine arts,” Proceedings International Conference on Image Processing (ICIP) 1, pp. 557–561, 2001. 6. D. G. Stork and J. Coddington, eds., Computer image analysis in the study of art, vol. 6810, SPIE/IS&T, Bellingham, WA, 2008. 7. D. G. Stork, J. Coddington, and A. Bentkowska-Kafel, eds., Computer vision and image analysis of art, SPIE/IS&T, Bellingham, WA, 2010 (forthcoming). 8. A. Kirsh and R. S. Levenson, Seeing through paintings: Physical examination in art historical studies, Yale U. Press, New Haven, CT, 2000. 9. D. G. Stork, “Computer vision, image analysis and master art, Part I,” IEEE Multimedia 13(3), pp. 16–20, 2006. 10. D. G. Stork and M. K. Johnson, “Computer vision, image analysis and master art, Part II,” IEEE Multimedia 14(3), pp. 12–17, 2006. 11. D. G. Stork and M. Duarte, “Computer vision, image analysis and master art, Part III,” IEEE Multimedia 14(1), pp. 14–18, 2007. 12. D. G. Stork, “Imaging technology enhances the study of art,” Vision Systems Design 12(10), pp. 69–71, 2007. 13. E. van de Wetering, “Thirty years of the Rembrandt Research Project: The tension between science and connoisseurship in authenticating art,” IFAR Journal 4(2), pp. 14, 24, 2001. 14. H. Leung, S. T. S. Wong, and H. H.-S. Ip, “In the name of art,” IEEE Signal Processing magazine 25(4), pp. 49–52, 2008. 15. X. Huang, A. Mohan, and J. Tumblin, “Deep shadows in a shallow box,” in Computer image analysis in the study of art, D. G. Stork and J. Coddington, eds., 6810, pp. 681003–1–9, IS&T/SPIE, (Bellingham, WA), 2008. 6 16. I. E. Berezhnoy, E. O. Postma, and H. J. van den Herik, “Computerized visual analysis of paintings,” in Proceedings of the 16th International Conference of the Association for History and Computing, pp. 28–32, Royal Netherlands Academy of Arts and Sciences, (Amsterdam, The Netherlands), 2005. 17. R. S. Berns, “Rejuvenating Seurat’s palette using color and imaging science: A simulation,” in Seurat and the making of La Grande Jatte, R. L. Herbert, ed., pp. 214–227, Art Institute of Chicago, Chicago, IL, 2004. 18. Y. Zhao, R. S. Berns, L. A. Taplin, and J. Coddington, “An investigation of multispectral imaging for the mapping of pigments in paintings,” in Computer image analysis in the study of art, D. G. Stork and J. Coddington, eds., 6810, pp. 681007–1–9, IS&T/SPIE, (Bellingham, WA), 2008. 19. K. Martinez and M. Goodall, “Colour clustering analysis for pigment identification,” in Computer image analysis in the study of art, D. G. Stork and J. Coddington, eds., 6810, pp. 681008–1–8, IS&T/SPIE, (Bellingham, WA), 2008. 20. M. Pappas and I. Pitas, “Digital color restoration of old paintings,” IEEE Transactions on Image Processing 9(2), pp. 291–294, 2000. 21. H. Chahine, J. Cupitt, D. Saunders, and K. Martinez, “Investigation and modelling of color change in paintings during conservation treatment,” in Imaging the past: Electronic imaging and computer graphics in museums and archaeology, Occasional papers of the British Museum 114, pp. 23–33, 1996. 22. A. D. Mastio, A. Piva, M. Barni, V. Cappellini, and L. Stefanini, “Color transplant for reverse ageing of faded artworks,” in Computer image analysis in the study of art, D. G. Stork and J. Coddington, eds., 6810, pp. 681006–1–12, IS&T/SPIE, (Bellingham, WA), 2008. 23. G. Verri, D. Comelli, S. Cather, D. Saunders, and F. Piqué, “Post-capture data analysis as an aid to the interpretation of ultraviolet-induced fluorescence images,” in Computer image analysis in the study of art, D. G. Stork and J. Coddington, eds., 6810, pp. 681002–1–12, IS&T/SPIE, (Bellingham, WA), 2008. 24. R. L. Easton, Jr., K. T. Knox, and W. A. Christens-Barry, “Multispectral imaging of the Archimedes palimpsest,” in Proceedings of the 32nd Applied Imagery Pattern Recognition Workshop, AIPR03, pp. 111– 116, IEEE, (El Segundo, CA), 2003. 25. R. Netz and W. Noel, The Archimedes Codex: How a Medieval prayer book is revealing the true genius of Antiquity’s greatest scientist, Da Capo Press, Philadelphia, PA, 2007. 26. K. T. Knox, “Enhancement of overwritten text in the Archimedes Palimpset,” in Computer image analysis in the study of art, D. G. Stork and J. Coddington, eds., 6810, pp. 681004–1–11, IS&T/SPIE, (Bellingham, WA), 2008. 27. D. Walvoord, A. Bright, and R. L. Easton, Jr., “Multispectral processing of combined visible and x-ray fluoroescence imagery in the Archimedes palimpsest,” in Computer image analysis in the study of art, D. G. Stork and J. Coddington, eds., 6810, pp. 681004–1–11, IS&T/SPIE, (Bellingham, WA), 2008. 28. D. Walvoord and R. L. Easton, Jr., “Digital transcription of the Archimedes palimpsest,” IEEE Signal Processing magazine 25(4), pp. 100–104, 2008. 29. K. Minturn, “Digitally-enhanced evidence: MoMA’s reconfiguration of Namuth’s Pollock,” Visual Resources 17(1), pp. 127–145, 2001. 30. D. Falk, D. Brill, and D. Stork, Seeing the Light: Optics in nature, photography, color, vision and holography, Wiley, New York, NY, 1986. 31. D. J. Graham and D. Field, “Global nonlinear compression of natural luminances in painted art,” in Computer image analysis in the study of art, D. G. Stork and J. Coddington, eds., 6810, pp. 68100K–1– 11, IS&T/SPIE, (Bellingham, WA), 2008. 32. D. J. Graham and D. Field, “Variations in intensity statistics for representational and abstract art, and for art from the eastern and western hemispheres,” Perception 37(9), pp. 1341–1352, 2008. 33. D. G. Stork, S. Meador, and P. Noble, “Painted or printed? Correlation analysis of the brickwork in Jan van den Heyden’s V iew of Oudezijds V oorburgwal and the Oude Kerk in Amsterdam,” in Electronic Imaging: Human vision and electronic imaging XIV, B. E. Rogowitz and T. N. Pappas, eds., 7240, pp. 72401O1–10, SPIE/IS&T, Bellingham, WA, 2009. 7 34. M. Lettner, M. Diem, R. Sablatnig, P. Kammerer, and H. Miklas, “Registration of multi-spectral manuscript images as prerequisite for computer aided script description,” in Proceedings of the 12th Computer Vision Winter Workshop, M. Grabner and H. Grabner, eds., pp. 51–58, (St. Lambrecht, Austria), 2007. 35. T. Ketelsen, O. Simon, I. Reiche, S. Merchel, and D. G. Stork, “Evidence for mechanical (not optical) copying and enlarging in Jan van Eyck’s Portrait of Niccolò Albergati,” in Optical Society of American Annual Meeting, OSA, (Rochester, NY), 2004. 36. A. Criminisi and D. G. Stork, “Did the great masters use optical projections while painting? Perspective comparison of paintings and photographs of Renaissance chandeliers,” in Proceedings of the 17th International Conference on Pattern Recognition, J. Kittler, M. Petrou, and M. S. Nixon, eds., IV, pp. 645–648, 2004. 37. M. Duarte and D. G. Stork, “Image contour fidelity analysis of mechanically aided enlargements of Jan van Eyck’s P ortrait of Cardinal Niccolò Albergati,” Leonardo 42, p. (in press), 2009. 38. M. Lettner and R. Sablatnig, “Estimating the original drawing trace of painted strokes,” in Computer image analysis in the study of art, D. G. Stork and J. Coddington, eds., 6810, pp. 68100C–1–10, IS&T/SPIE, (Bellingham, WA), 2008. 39. P. Noble and D. G. Stork, “Jan van der Heyden’s V iew of Oudezijds V oorburgwal and the Oude Kerk in A msterdam, examined and restored,” p. (submitted), 2010. 40. H. Damisch, The origin of perspective, MIT Press, Cambridge, MA, 1995. 41. A. Criminisi, M. Kemp, and A. Zisserman, “Bringing pictorial space to life: Computer techniques for the analysis of paintings,” in Digital art history: A subject in transition, A. Bentkowska-Kafel, T. Cashen, and H. Gardner, eds., pp. 77–100, Intellect Books, Bristol, UK, 2005. 42. A. Criminisi, “Machine vision: The answer to the optical debate?,” Optical Society of American Annual Meeting Rochester, NY (abstract), 2004. 43. M. Kemp and A. Criminisi, “Computer vision and painter’s vision in Italian and Nederlandish art of the fifteenth century,” in Perspective, projections and design technologies of architectural representation, M. Carpo and F. Lemerle, eds., pp. 31–46, Routledge, New York, NY, 2008. 44. M. Kemp and A. Criminisi, “Paolo Uccello’s ‘Rout of San Romano’: Order from chaos,” NEW magazine 1(1), pp. 99–104, 2005. 45. B. Smith, D. G. Stork, and L. Zhang, “Three-dimensional reconstruction from multiple reflected views within a realist painting: An application to Scott Fraser’s Three way vanitas,” in Electronic imaging: 3D imaging metrology, J. A. Beraldin, G. S. Cheok, M. McCarthy, and U. Neuschaefer-Rube, eds., 7239, pp. 72390U1–10, SPIE/IS&T, Bellingham, WA, 2009. 46. D. Hockney and C. M. Falco, “Quantitative analysis of qualitative images,” in Electronic Imaging, SPIE Press, Bellingham, WA, 2005. 47. A. Criminisi, Accurate visual metrology from single and multiple uncalibrated images, ACM Distinguished Dissertation Series, Springer-Verlag, London, 2001. 48. D. G. Stork, “Were optical projections used in early Renaissance painting? A geometric vision analysis of Jan van Eyck’s Arnolfini portrait and Robert Campin’s Mérode Altarpiece,” in SPIE Electronic Imaging: Vision Geometry XII, L. J. Latecki, D. M. Mount, and A. Y. Wu, eds., pp. 23–30, SPIE, (Bellingham, WA), 2004. 49. D. G. Stork, “Optics and the old masters revisited,” Optics and Photonics News 15(3), pp. 30–37, 2004. 50. D. G. Stork, “Optics and realism in Renaissance art,” Scientific American 291(6), pp. 76–84, 2004. 51. D. G. Stork, “Did Jan van Eyck build the first ‘photocopier’ in 1432?,” in SPIE Electronic Imaging: Color Imaging IX: Processing, Hardcopy and Applications, R. Eschbach and G. G. Marcu, eds., pp. 50–56, SPIE, (Bellingham, WA), 2004. 52. D. G. Stork, “Did Hans Memling employ optical projections when painting Flower still-life?,” Leonardo 38(2), pp. 57–62, 2005. 53. D. G. Stork, “Asymmetry in ‘Lotto carpets’ and implications for Hockney’s optical projection theory,” in SPIE Electronic Imaging: Human vision and electronic imaging X, B. E. Rogowitz, T. N. Pappas, and S. J. Daly, eds., 5666, pp. 337–343, SPIE, (Bellingham, WA), 2005. 8 54. D. G. Stork, “Optique et réalisme dans l’art de la Renaissance,” Revue Pour la Science 327, pp. 74–86, 2005. 55. D. G. Stork, “Spieglein, Spieglein and der Wand,” Spektrum der Wissenschaft: Forschung und Technik in der Renaissance Spezial 4/2004(4), pp. 58–61, 2005. 56. D. G. Stork, “Mathematical foundations for quantifying shape, shading and cast shadows in realist master drawings and paintings,” in SPIE Electronic Imaging: Mathematics of data/image pattern recognition, compression and encryption with applications IX, G. X. Ritter, M. S. Schmalz, J. Barrera, and J. T. Astola, eds., 6314, pp. 63150K–1–6, SPIE, (Bellingham, WA), 2006. 57. C. W. Tyler, “‘Rosetta stone?’ Hockney, Falco and the sources of ‘opticality’ in Renaissance art,” Leonardo 37(5), pp. 397–401, 2004. 58. A. Kulkarni and D. G. Stork, “Optical or mechanical aids to drawing in the early Renaissance? A geometric analysis of the trellis in Robert Campin’s Mérode Altarpiece,” in SPIE Electronic Imaging: Machine vision applications II, K. S. Niel and D. Fofi, eds., 7251, pp. 72510R1–9, SPIE/IS&T, Bellingham, WA, 2009. 59. F. Leeman, Hidden images: Games of perception, anamorphic art, and illusion, Abrams, New York, NY, 1976. 60. J. Baltrušaitis, Anamorphic art, Abrams, New York, NY, 1977. 61. D. G. Stork, “Anamorphic art and photography: Deliberate distortions that can be easily undone,” Optics and Photonics News 3(11), pp. 8–12, 1992. 62. J. L. Hunt, B. G. Nickel, and C. Gigault, “Anamorphic images,” American Journal of Physics 68(3), pp. 232–237, 2000. 63. A. J. DeWeerd and S. E. Hill, “Comment on ‘Anamorphic images,’ by J. L. Hunt, B. G. Nickel and Christian Gigault,” American Journal of Physics 74(1), pp. 83–84, 2006. 64. H. Farid, “Reconstructing ancient Egyptian tombs,” in Virtual and augmented architecture: The international symposium on virtual and augmented reality, B. Fisher, K. Dawson-Howe, and C. O. Sullivan, eds., pp. 23–34, 2001. 65. H. Farid and S. Farid, “Unfolding Sennedjem’s tomb,” KMT: A modern journal of ancient Egypt 12(1), pp. 46–59, 2001. 66. A. Criminisi, M. Kemp, and S.-B. Kang, “Reflections of reality in Jan van Eyck and Robert Campin,” Historical methods 3(37), pp. 109–121, 2004. 67. S. Savarese, R. Spronk, D. G. Stork, and A. DelPozo, “Reflections on praxis and facture in a devotional portrait diptych: A computer analysis of the mirror in Hans Memling’s V irgin and Child and Maarten van Nieuwenhove,” in Computer image analysis in the study of art, D. G. Stork and J. Coddington, eds., 6810, pp. 68100G–1–10, SPIE/IS&T, Bellingham, WA, 2008. 68. D. G. Stork, “Color and illumination in the Hockney theory: A critical evaluation,” in Proceedings of the 11th Color Imaging Conference (CIC11), 11, pp. 11–15, IS&T, (Scottsdale, AZ), 2003. 69. D. G. Stork, “Did Georges de la Tour use optical projections while painting Christ in the carpenter’s studio?,” in SPIE Electronic Imaging: Image and video communications and processing, A. Said and J. G. Apolstolopoulos, eds., 5685, pp. 214–219, SPIE, (Bellingham, WA), 2005. 70. D. G. Stork and M. K. Johnson, “Estimating the location of illuminants in realist master paintings: Computer image analysis addresses a debate in art history of the Baroque,” in Proceedings of the 18th International Conference on Pattern Recognition, I, pp. 255–258, IEEE Press, (Hong Kong), 2006. 71. D. G. Stork and M. K. Johnson, “Lighting analysis of diffusely illuminated tabeaus in realist paintings: An application to detecting ‘compositing’ in the portraits of Garth Herrick,” in Electronic Imaging: Media forensics and security XI, E. J. Delp III, J. Dittmann, N. D. Memon, and P. W. Wong, eds., 7254, pp. 72540L1–8, SPIE/IS&T, Bellingham, WA, 2009. 72. D. G. Stork, “Locating illumination sources from lighting on planar surfaces in paintings: An application to Georges de la Tour and Caravaggio,” in Optical Society of American Annual Meeting, Optical Society of America, (Rochester, NY), 2008. 73. D. Kale and D. G. Stork, “Estimating the position of illuminants in paintings under weak model assumptions: An application to the works of two Baroque masters,” in Electronic Imaging: Human vision and electronic imaging XIV, B. E. Rogowitz and T. N. Pappas, eds., 7240, pp. 72401M1–12, SPIE/IS&T, Bellingham, WA, 2009. 9 74. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern classification, John Wiley and Sons, New York, NY, Second ed., 2001. 75. M. K. Johnson, D. G. Stork, S. Biswas, and Y. Furuichi, “Inferring illumination direction estimated from disparate sources in paintings: An investigation into Jan Vermeer’s Girl with a pearl earring,” in Computer image analysis in the study of art, D. G. Stork and J. Coddington, eds., 6810, pp. 68100I–1–12, SPIE/IS&T, Bellingham, WA, 2008. 76. D. G. Stork and Y. Furuichi, “Image analysis of paintings by computer graphics synthesis: An investigation of the illumination in Georges de la Tour’s Christ in the carpenter’s studio,” in Computer image analysis in the study of art, D. G. Stork and J. Coddington, eds., 6810, pp. 68100J–1–12, SPIE/IS&T, Bellingham, WA, 2008. 77. I. E. Berezhnoy, E. O. Postma, and H. J. van den Herik, “AUTHENTIC: Computerized brushstroke analysis,” in International Conference on Multimedia and Expo, 6, pp. 1586–1588, IEEE Press, (Amsterdam, The Netherlands), 2005. 78. K. Jones-Smith and H. Mathur, “Fractal analysis: Revisiting Pollock’s drip paintings,” Nature 444(doi: 10.1038/nature05398), pp. E9–E10, 2006. 79. J. Li and J. Z. Wang, “Studying digital imagery of ancient paintings by mixtures of stochastic models,” IEEE Transactions on Image Processing 13(3), pp. 340–353, 2004. 80. S. Lyu, D. Rockmore, and H. Farid, “A digital technique for art authentication,” Proceedings of the National Academy of Sciences 101(49), pp. 17006–17010, 2004. 81. T. Melzer, P. Kammerer, and E. Zolda, “Stroke detection of brush strokes in portrait miniatures using a semi-parametric and a model based approach,” Proceedings of the 14th International Conference on Pattern Recognition 1, pp. 474–476, 1998. 82. R. Sablatnig, P. Kammerer, and E. Zolda, “Hierarchical classification of painted portraits using face- and brush stroke models,” in Proceedings of the 14th International Conference on Pattern Recognition, A. K. Jain, S. Venkatesh, and B. C. Lovell, eds., I, pp. 172–174, IEEE Press, (Los Alamitos, CA), 1998. 83. J. Z. Wang, G. Wiederhold, O. Firschein, and S. X. Wei, “Content-based image indexing and searching using Daubechies’ wavelets,” International Journal on Digital Libraries 1(4), pp. 311–328, 1998. 84. P. Kammerer, M. Lettner, E. Zolda, and R. Sablatnig, “Identification of drawing tools by classification of textural and boundary features of strokes,” Pattern Recognition Letters 28(6), pp. 710–718, 2007. 85. I. Tastl, R. Sablatnig, and W. G. Kropatsch, “Model-based classification of painted portraits,” in Pattern Recognition 1996: Proceedings of the 20th ÖAGM Workshop, A. Pinz, ed., OCG Schriftenreihe 90, pp. 237– 250, Oldenbourg Wien, München, 1996. 86. R. Sablatnig, P. Kammerer, and E. Zolda, “Structural analysis of paintings based on brush strokes,” in Fakebusters: Scientific detection of fakery in art, W. C. McCrone and R. J. Weiss, eds., pp. 222–244, Hansen, Stoughton, Massachusetts, 1999. 87. P. Kammerer, E. Zolda, and R. Sablatnig, “Computer aided analysis of underdrawings in infrared reflectograms,” in Proceedings of the 4th International Symposium on Virtual Reality, Archaeology and Intelligent Cultural Heritage, D. Arnold, A. Chalmers, and F. Nicolucci, eds., pp. 19–27, (Brighton, United Kingdom), 2003. 88. M. Lettner, P. Kammerer, and R. Sablatnig, “Texture analysis of painted strokes,” in Digital Imaging in Media and Education, Proceedings of the 28th Workshop of the Austrian Association for Pattern Recognition (OAGM/AAPR), W. Burger and J. Scharinger, eds., 179, pp. 269–276, Schriftenreihe der OCG, (Hagenberg, Austria), 2004. 89. M. Lettner and R. Sablatnig, “Texture based drawing tool classification in infrared reflectograms,” in Proceedings of the 10th Computer Vision Winter Workshop, A. Hanbury and H. Bischof, eds., pp. 63–72, (Zell an der Pram, Austria), 2005. 90. M. Lettner and R. Sablatnig, “Texture based drawing tool classification,” in Joint Hungarian-Austrian Conference on Image Processing and Pattern Recognition, Proceedings of the 29th Workshop of the Austrian Association for Pattern Recognition (OAGM/AAPR), D. Chetverikov, L. Czuni, and M. Vinzce, eds., 189, pp. 171–178, Schriftenreihe der OCG, 2005. 10 91. M. Lettner and R. Sablatnig, “Stroke trace estimation in pencil drawings,” in Proceedings of the 8th International Symposium on Virtual Reality, Archaeology and Cultural Heritage, D. Arnold, A. Chalmers, and F. Nicolucci, eds., pp. 43–47, (Brighton, UK), 2007. 92. M. C. Vill and R. Sablatnig, “Drawing tool recognition by stroke ending analysis,” in Computer image analysis in the study of art, D. G. Stork and J. Coddington, eds., 6810, pp. 68100B–1–11, IS&T/SPIE, (Bellingham, WA), 2008. 93. H. J. van den Herik and E. O. Postma, “Discovering the visual signature of painters,” in Future directions for intelligent systems and information sciences: The future of image technologies, brain computers, WWW and bioinformatics, N. Kasabov, ed., pp. 129–147, Physica-Verlag, Heidelberg, Germany, 2000. 94. M. C. Vill and R. Sablatnig, “Stroke ending shape features for stroke classification,” in Proceedings of the Computer Vision Winter Workshop 2008, J. Perš, ed., pp. 91–98, (Moravske Toplice, Slovenia), 2008. 95. C. R. Johnson, E. Hendriks, I. J. Berezhnoy, E. Brevdo, S. M. Hughes, I. Daubechies, J. Li, E. Postma, and J. Z. Wang, “Image processing for artist identification,” IEEE Signal Processing magazine 25(4), pp. 37–48, 2008. 96. S. B. Hedges, “A method for dating early books and prints using image analysis,” Proceedings of the Royal Society A 462(2076), pp. 3555–3573, 2006. 97. S. B. Hedges, “Image analysis of Renaissance copperplate prints,” in Computer image analysis in the study of art, D. G. Stork and J. Coddington, eds., 6810, pp. 681009–1–20, IS&T/SPIE, (Bellingham, WA), 2008. 98. M. Shahram, D. G. Stork, and D. Donoho, “Recovering layers of brushstrokes through statistical analysis of color and shape: An application to van Gogh’s Self portrait with grey felt hat,” in Computer image analysis in the study of art, D. G. Stork and J. Coddington, eds., 6810, pp. 68100D–1–8, SPIE/IS&T, Bellingham, WA, 2008. 99. B. Mandelbrot, The fractal geometry of nature, W. H. Freeman, New York, NY, 1982. 100. R. P. Taylor, A. P. Micolich, and D. Jonas, “Fractal analysis of Pollock’s drip paintings,” Nature 399, p. 422, 1999. 101. J. Coddington, J. Elton, D. Rockmore, and Y. Wang, “Multifractal analysis and authentication of Jackson Pollock’s paintings,” in Computer image analysis in the study of art, D. G. Stork and J. Coddington, eds., 6810, pp. 68100F–1–12, IS&T/SPIE, (Bellingham, WA), 2008. 102. J. Alvarez-Ramirez, C. Ibarra-Valdez, E. Rodrı́guez, and L. Dagdug, “1/f-noise structures in Pollock’s drip paintings,” Physica A 387(1), pp. 281–295, 2008. 103. D. Fernandez and A. J. Wilkins, “Uncomfortable images in art and nature,” Perception 37(7), pp. 1098– 1113, 2008. 104. D. J. Graham and D. Field, “Statistical regularities of art images and natural scenes: Spectra, sparseness and nonlinearities,” Spatial vision 21(1-2), pp. 149–164, 2007. 105. J. R. Mureika, C. C. Dyer, and G. C. Cupchik, “On multifractal structure in non-representational art,” Physical Review 72(4), p. 046101, 2005. 106. C. Redies, J. Hasenstein, and J. Denzler, “Fractal-like image statistics in visual art: Similarity to natural scenes,” Spatial Vision 21(1–2), pp. 137–148, 2007. 107. M. Irfan and D. G. Stork, “Multiple visual features for the computer authentication of Jackson Pollock’s drip paintings: Beyond box-counting and fractals,” in SPIE Electronic Imaging: Machine vision applications II, K. S. Niel and D. Fofi, eds., 7251, pp. 72510Q1–11, SPIE/IS&T, Bellingham, WA, 2009. 108. D. Hockney and C. M. Falco, “Optical insights into Renaissance art,” Optics and Photonics News 11(7), pp. 52–59, 2000. 109. D. Hockney, Secret knowledge: Rediscovering the lost techniques of the old masters, Viking Studio, New York, NY, 2001. 110. P. Steadman, Vermeer’s camera: Uncovering the truth behind the masterpieces, Oxford U. Press, Oxford, UK, 2002. 111. D. G. Stork, “Did early Renaissance painters trace optical projections? Evidence pro and con,” in SPIE Electronic Imaging: Vision geometry XIII, L. J. Latecki, D. M. Mount, and A. Y. Wu, eds., 5675, pp. 25– 31, SPIE, (Bellingham, WA), 2005. 11 112. D. G. Stork, “Tracing the history of art: Review of Early Science and Medicine: Optics, instruments and painting, 1420-1720: Reflections on the Hockney-F alco theory,” Nature 438(7070), pp. 916–917, 2005. 113. D. G. Stork and M. Duarte, “Revisiting computer image analysis and art,” IEEE Multimedia 14(3), pp. 108–109, 2007. 114. C. W. Tyler and D. G. Stork, “Did Lorenzo Lotto use optical projections when painting Husband and wife?,” in Optical Society of American Annual Meeting, Optical Society of America, (Rochester, NY), 2004. 115. S. Dupré, ed., Early Science and Medicine: A Journal for the Study of Science, Technology and Medicine in the Pre-modern Period: Optics, instruments and painting 1420–1720: Reflections on the Hockney-Falco Thesis, vol. X, no. 2, Brill Academic Publishers, Leiden, The Netherlands, 2005. 116. C. Lüthy, “Reactions of historians of science and art to the Hockney thesis: Summary of the European Science Foundation’s conference of 12–15 November, 2003,” Optical Society of American Annual Meeting Rochester, NY (Abstract), 2004. 117. D. G. Stork, “Aberration analysis of the putative projector for Lorenzo Lotto’s Husband and wife,” Optical Society of American Annual Meeting San Jose, CA (Abstract), 2007. 118. M. D. Robinson and D. G. Stork, “Aberration analysis of the putative projector for Lorenzo Lotto’s Husband and wife: Image analysis through computer ray-tracing,” in Computer image analysis in the study of art, D. G. Stork and J. Coddington, eds., 6810, pp. 68100H–1–11, SPIE/IS&T, Bellingham, WA, 2008. 119. F. S. Abas and K. Martinez, “Craquelure analysis for content-based retrieval,” in Proceedings of the 14th Conference on Digital Signal Processing, pp. 111–114, (Santorini, Greece), 2002. 120. F. S. Abas, Analysis of craquelure patterns for content-based retrieval. Ph.D. thesis, University of Southampton, Southampton, UK, 2004. 121. S. Bucklow, “A stylometric analysis of craquelure,” Computers and Humanities 31(6), pp. 503–521, 1998. 122. A. Colagrossi, F. Sciarrone, and C. Seccaroni, “A method for automating the classification of works of art using neural networks,” Leonardo 36(1), pp. 69–96, 2003. 123. F. Heitz, H. Maitre, and C. de Couessin, “Application of autoregressive models to fine arts painting analysis,” Signal Processing 13(1), pp. 1–14, 1987. 124. F. Remondino, S. F. El-Hakim, A. Grün, and L. Zhang, “Turning images into 3-D models,” IEEE Signal Processing magazine 25(4), pp. 55–65, 2008. 125. D. G. Stork and Y. Furuichi, “Computer graphics synthesis for interring artist studio practice: An application to Diego Velázquez’s Las meninas,” in Electronic imaging: The engineering reality of virtual reality, I. E. McDowall and M. Dolinsky, eds., 7238, pp. 7238061–9, SPIE/IS&T, Bellingham, WA, 2009. 12