International Journal of Automation and Computing 1 (2006) 69-75

A Photo-Modeling Approach to Restituting 3D Model

Data from Single 2D Imagery for Rapid Prototyping of
Artifact Products
Zheng-Xu Zhao
Department of Instrumentation Science and Engineering, Southeast University, Nanjing 210096, PRC
School of Computing, The University of Derby, Kedleston Road, Derby DE22 1GB, UK

Kai-Ling Li
School of Mechanical & Manufacturing Engineering, Shandong University, Jinan 250061, PRC

Abstract: This article rst generalizes the basic engineering phases of modern rapid prototyping processes, and then
describes the techniques of data capture for data modeling and model making. The article also provides a brief overview of
the photogrametric techniques of restitution of 3D objects, and highlights the diculties and limitations of existing methods.
It therefore presents a novel approach to photo-modeling for acquiring 3D model data from single 2D photorealistic images.
Implementation of the approach is then described against a background of rapid prototyping processes to demonstrate the
eectiveness of photo-modeling practice.
Keywords: Rapid prototyping, photo-modeling, photogrammetry, virtual reality.

Introduction

Rapid prototyping has become a mainstream engineering process in modern industry, for its integrated,
time-compression, fast-response and customized production nature; it is a fast developing engineering practice, and its future is likely to be both prosperous and
consolidating[1,2] .
A typical rapid prototyping practice usually involves three general phases, as depicted in Fig.1: the
initial capture of product data, followed by the establishment of a data model (normally a computer aided
design le or CAD model), and nally the manufacture
of the physical product[3] .

uring and engineering process that usually requires various data acquisition techniques, expensive systems,
precision tools, and highly specialized software and
hardware; which are commonly involved in engineering practice such as CAD (computer aided design),
3D computer graphic design, geometry, measuring, and
digitizing, surface scanning, sensing and tomography.
Captured data is then further processed and modeled, (either in a CAD system or using other 3D modelers), to form a 3D data model or CAD le. Such
a CAD le will serve as the basis on which manufacture planning and preparation is carried out for the
nal shop oor model making process, as illustrated in
Fig.2. This article dedicates its contents only to the
data capture phase, and thereby presents a novel apph-

Fig.1 General rapid prototype phases

(Source: Courtesy of DELCAM )

Data capture is a complex and complicated meas

Manuscript received February 13, 2005; revised September 15,
2005.
This work is supported by Cheung Kong Scholars Program of
the Peoples Republic of China, in association with Southeast
University, Nanjing, PRC. (No.seuzx042001)
Corresponding

author. E-mail address: Z.Zhao@derby.ac.uk

Fig.2 A typical rapid prototype process

70

International Journal of Automation and Computing 1 (2006) 69-75

proach to the acquisition of 3D model data from a single 2D otorealistic image. The notion behind such a
research eort is twofold in relation to digital photorealistic images.
1) A photorealistic image is the most popular data
media, has high realism, is commonly available and is
easy to obtain.
2) Photorealistic images are widely delivered and
communicated over the Internet and other networks,
and are technically compatible with almost all CAD
and CAM (computer aided manufacturing) systems.
Therefore to utilize them in rapid prototyping is cost
eective.

Restitution of 3D object models

Restitution of 3D objects, or 3D models, is based

on photogrammetry, which is the science of extracting
measurements from photographs[46].
Photogrammetry is the technique of measuring objects, (either 2D or 3D), from photo-grammes (commonly called photographs); but it may also involve imagery stored electronically on tape or disk, taken by
video, cameras or radiation sensors such as scanners.
Results can be as follows:
1) Coordinates of required object-points that constitute a mesh of polygons, representing the surface of
an object.
2) Topographical and thematical maps, that often
dene complex and large-scale terrains, and
3) Rectied photographs, which are in fact orthogonal photographs that represent architectural structure
and engineering components.
The most important feature of photogrammetry
is the fact that objects are measured without being
touched. Therefore, photogrammetry is a subject of
remote sensing, that is closely associated with image
interpretation. Principally, depending on the lensesetting of a camera, photogrammetry can be divided
into two categories of discipline. One is far range
photogrammetry with a camera distance setting to an
indenite range, while the other is close range photogrammetry, with camera distance settings to nite
values.
The application of photogrammetry is wide spread.
Principally, it is utilized for object recognition and interpretation, (identity, type, quality, and quantity of
an object), and object measurement, (position, form,
and size of an object). Among the users of close-range
photogrammetry are architects and civil engineers, (to
supervise building, and to document the current state,
deformation, or damage, of, or to, buildings), archaeologists, surgeons (for plastic surgery), and police departments (for the documentation of trac accidents,

crime scenes, and forensic analysis). The following subsections provide a brief description of photogrammetrical techniques.

2.1

Photographing devices

A photographic image is a central perspective. This

implies that every light ray which reached the lm surface during exposure, passes through the camera lens,
(which is mathematically considered as a single point, a
so called perspective center). To take measurements of
objects from photographs, the ray bundle must be reconstructed. Therefore, the internal geometry of the
camera used, (which is dened by the focal length,
which is the position of the principal point and lens distortion), has to be precisely known. The focal length,
is called the principal distance, which is the distance
of the projection center from the principal point of
the image plane. Depending on the availability of this
knowledge, a photogrammetrist normally divides photographing devices into three categories. The rst are
metric cameras that have stable and precisely known
internal geometries, and have very low lens distortion.
The second are stereometric cameras that have an overlapping area of two photographs, called a stereo-pair,
which can be seen in 3D, thereby simulating human
stereoscopic vision. While the third are amateur cameras, in which internal geometry is not stable, and is
unknown, as is the case with any normal commercially
available camera.
More detail about photographing devices can be
found in [7].

2.2

Photogrammetric techniques

Depending on available material (such as

metric/non-metric cameras, stereo-pairs, shape of
recorded object, control information), and required
results (2D or 3D resolution and accuracy), dierent
photogrammetric techniques can be applied. In addition, depending on the number of photographs, three
main categories of modeling technique can be distinguished.
2.2.1

Modeling from multi-photographs

This restitution, which can be done in 3D, has only

become possible via analytical and digital photogrammetry. It requires more than two photographs of a 3D
object, taken from dierent angles or positions. First, a
bundle adjustment has to be calculated. Using control
and triangulation points, the geometry of photographs
are reconstructed with high precision. Then the image
coordinates of any desired object-point, measured in
at least two photographs, can be intersected. The results are the coordinates of the required points. There-

Zheng-Xu Zhao et al./ A Photo-Modeling Approach to Restituting 3D Model Data from Single 2D Imagery for

after, the whole 3D object is digitally reconstructed.

Since the required hardware and software is normally
expensive and highly specialized, this technique is only
suitable for applications of established engineering processes and well resourced projects.
2.2.2

Modeling of stereo-photogrammetry

This technique essentially requires stereo-pairs, as

its name implies. It requires two partially overlapping photographs, taken from dierent positions, but
matching the conditions of the so-called normal case
(for example, vertical aerial photographs come most
close to the normal case). The overlapping part of
each stereo-pair can be viewed in 3D, and consequently
mapped in 3D perspectives, using one of three common
methods, i.e. an analogue, analytical or digital model.
A detailed description can be found in [7].
2.2.3

Modeling from a single photograph

In theory, this technique is only useful for plane (or

2.5D) objects. Rectication can be neglected only if
the object is at and the picture taken from a vertical
position towards the object. In this case, the photograph will have a unique scale factor, which can be
determined if the length of at least one distance to the
object is known. Objects photographed at an oblique
plane show perspective deformations, which have to be
rectied. For rectication there exists a range of techniques, of which some are very simple, but they have
limitations. To obtain ideal results even with the simple techniques, the object should be a plane (such as
an engine gasket), and since only a single photograph
is used, restitution can only be normally done in 2D.
Traditionally, the following common methods are available:
1) The paper strip method This is the easiest and
cheapest method, since it needs only a ruler, a piece of
paper with a straight edge, and a pencil. Four points
must be identied in a photograph, and in an object
map. From one point, lines have to be drawn to the
others (on the image, and the object map), and to a
required object point (on the image). Then, the paper
strip is placed on the image, and intersections with the
lines marked. The strip is then placed on the map, and
adjusted so that marks again coincide with the lines.
After this, a line can be drawn on the map to the mark
of the required object point. The whole process is repeated from another point, providing the object-point
on the map as the intersection of the two object-lines.
2) Optical rectication This method is achieved
using devices called photographic enlargeners. These
should fulll the Scheimpug and vanishing-point
conditions. Again, at least four control points are required, not three on one line. The control points are

71

plotted at a certain scale. The control point plot is

rotated and displaced, until two points match corresponding object points on the projected image. Subsequently, the table has to be tilted by two rotations,
until the projected negative ts all control points. Then
an exposure is made and developed.
3) Numerical rectication Again, an object has
to be a plane, and four control points are required.
In numerical rectication, image coordinates of desired
object-points are transformed into the desired coordinate system (which is again 2D). The result is the coordinates of the projected points.
4) Dierential rectication If the object is uneven, it has to be divided into smaller parts, which are
planes. Each part can then be rectied with one of the
techniques shown above. Of course, even objects may
also be rectied piecewise, dierentially. A prerequisite for dierential rectication is the availability of a
digital object model, that is, a dense raster of points
on an object, with known distances from a reference
plane; in aerial photogrammetry this is called a DTM
(Digital Terrain Model).
5) Monoplotting This technique is similar to numerical rectication, except that coordinates are transformed into a 3D coordinate system. First, orientation
elements, that are the coordinates of the projection
center, and three angles dening the view of the photograph, are calculated by spatial resection. Then, using
the calibration data of the camera, any ray that comes
from the archaeological feature through the lense onto
the photograph, can be reconstructed and intersected
with the digital terrain model.
6) Digital rectication Digital rectication is a
new technique. It is similar to monoplotting. But here,
a scanned image is transformed pixel by pixel into a
3D real-world coordinate system. The result is an orthogonal photograph, a rectied photograph that has
a unique scale.

2.3 Photo-modeling for acquisition of 3D

model data from a single 2D image
Creating accurate and high quality 3D models and
measurements from photographs[8], has become a professional and engineering practice; research and development work along a similar line has spread throughout
many industry sectors, such as accident reconstruction,
archaeology and anthropology, architecture and preservation, lm, video and animation, forensic, and plant
and mechanical engineering, to name only a few.
However, as described in Section 2.2.3, creating 3D
models from single photographs has long been accepted
as a technically-limited method, and suitable only for
certain applications. This may stand as a truism as

72

International Journal of Automation and Computing 1 (2006) 69-75

long as complex structures and objects are to be restituted. For many artifact products, mechanical components, and even objects of free form surface, it is viable
and technically attainable to model them based on single 2D image data. In this article, such a modeling
process is called photo-modeling. It has the objectives
to (1) simplify the process, and eliminate the eort
of data capture that is essential in conventional rapid
prototyping processes, (2) to generate as-built product design models and drawings, (3) to create hightessellation surface models, (4) to measure and reversemodel mechanical parts, and (5) to export models for
use in rapid prototyping of artifact products.

stance using quads, each vertex (excluding the corner

and border vertices), is shared by four polygons. The
resulting normal at a vertex, should be computed as
the normalized sum, of all the unit length normals to
each face the vertex shares.

3 Photo-modeling approach to rapid

prototyping of artifact products
In simple terms, photo-modeling is an approach to
raising the pixels of a 2D at photo-image to a precise
height, so that they will gain an accurate depth value.
Therefore, corresponding 3D coordinates can be restituted as a vertex of the 3D surface. The color and light
eect of a pixel provided by the photo-image will then
be recalibrated, with an appropriate normal computation around that vertex, to rectify the nearby geometry
of the surface. The computational algorithms that are
exploited for establishing this approach are described
as follows.

3.1

As shown in Fig.3, v represents the normal at the

center vertex, and each vij represents a normal for each
face that shares the center vertex. For instance v12 is
the unit normal for the bottom right face. The vertex
normal v is computed as the normalized sum of all vij
vectors:

Interpretation of image pixels

There are several ways to interpret an image to obtain its depth value. If the image is grayscale, then the
process is straightforward. Pixel intensity corresponds
to a relative depth value. If the image is of RGB format, it needs to be converted to grayscale rst, and
then a depth value is related to the grayscale. The
other possible pixel type is RGBA, in which case a
depth value is established from the alpha channel, and
then rectied using RGB colors. Technical details can
be found in literature[9] .

3.2

Fig.3 Normal per vertex

Computation of normals

To apply lighting to a 3D vertex, it is necessary

to rst compute a surface normal. A normal is a vector that denes how a surface responds to light. The
amount of light reected by a surface is proportional
to the angle between the lights direction and the normal. The smaller the angle, the brighter the surface
will look.
As illustrated in Fig.3, normals are dened per vertex (normal averaging), rather than per face. This ensures a smooth surface is obtained. When computing
normals per vertex, it is necessary to take into account
the faces that share that vertex. Therefore, if for in-

v = N ormalized((v12 , v23 , v34 , v41 ))

(1)

where vij = normalized(vi vj ).

3.3

Computation of simulating lights

Lighting a surface can be a complex operation. In

this section, it will be shown how to compute the colors
of a vertex as if it was being lit by a light source. For
the sake of simplicity, only the diuse component will
be taken into account. Two types of light source will
be considered: directional and point lights.
The diuse component of vertex color is dened as:
max{L.n, 0} diuseLight diuseMaterial (2)
where L is a unit length vector that points from the
vertex to the light position, n is a unit normal for the
vertex, and L. n is inner product between the two vectors.
For a directional light, the vector L is constant, because light rays are considered to be parallel, i.e. the
vector from a vertex to a light position is not dependent
of the vertex position. In fact, a directional light is dened not with its position, but the direction L. To compute the diuse component, it needs to compute the
vector L = (Lx, Ly, Lz), (from the vertex to the light

Zheng-Xu Zhao et al./ A Photo-Modeling Approach to Restituting 3D Model Data from Single 2D Imagery for

source), and, the vertex normal n = (nx, ny, nz), the

diuse component dL = (dLR, dLG, dLB) of the light
RGB, and the diuse component dv = (dvR, dvG, dvB)
of the vertex RGB. The complete computation process
for lighting a vertex with a directional light, involves
the following steps:
1) Normalize L
l = sqrt(Lx Lx + Ly Ly + Lz Lz)

(3)

where Lx = Lx/l, Ly = Ly/l, and Lz = Lz/l.

2) Compute the inner product between L and n
ip = Lx nx + Ly ny + Lz nz.

(4)

3) Check to see if it is larger than zero, such that

(ip < 0), then ip = 0.
4) Compute the vertex lit color, vc
=
(vcR, vcG, vcB) using following equations:
vcR = ip dLR dvR

(5)

vcG = ip dLG dvG

(6)

vcB = ip dLB dvB.

(7)

In the case of directional light step 1), the normalization of L, should be done only once, because L is
the same for every vertex. For point lights, the process is similar, the only dierence being in step 1). In
this case a computation of L is required. Assuming
that the vertex position is v = (vx, vy, vz), and light
position l = (lx, ly, lz), then step 1) becomes:
1) Compute the vector from the vertex to the light
source as vl = (vlx, vly, vlz), where vlx = lx px,
vly = ly py, and vlz = lz pz.
2) Normalize vl to obtain L. First, length =
sqrt(vlx vlx + vly vly + vlz vlz), and then
Lx = lx/length, Ly = ly/length, and Lz = lz/length.
For steps 2) to 4), the computation is identical for
both light types.

3.4

73

This lter can be applied to a row as shown in

pseudo code below:
f or (x = 1; x < maxW idth; x + +)
depth(x, z) = depth(x 1, z) (1 k)
+ depth(x, z) k;

Implementation

The photo-modeling approach was implemented

through the development of a test program (standard
C code with an OpenGL library), under Windows
XP. The program was then tested with both a RGBA
photo image, shown in Fig.4(a) and a grayscale image
Fig.4(b).

(a) RGBA photo image

(b) Grayscale photo image

(c) View of 3D model

(d) Depth view of model

(e) Normals of 3D model

(f) 3D model to prototype

Surface smooth filtering

The 3D artifact surface generated so far, may look

rough; a ltering technique is required to smooth the
surface. The concepts here are borrowed from image processing lters. The simplest smoothing lter
is probably the band smoothing lter. This lter can
be applied to rows and/or columns. For any point on
a surface, its new depth will be computed taking into
account the value of one of its neighbors:
D(x, z) = D(x 1, z) (1 k) + D(x, z) k

(8)

where D stands for depth, and k, a value between 0 and

1, is a roughness constant. If k = 1, then the surface
will remain unchanged. If k = 0, then the new depth
will be set to the value of its respective neighbor.

Fig.4 Implementation and testing result

The program rst loads a 2D image, (which must be

in BMP, GIF, JPG or TGA format, as these formats
have been implemented in its present functionality),
and will then automatically generate a 3D model from
the image, as shown in Fig.4(c). This model can be
viewed from dierent perspectives as shown in Fig.4(d)
and validated with normal averaging and smooth ltering, as in Fig.4(e). It can be adjusted to a specied depth, and rectied to a nal model, as shown in
Fig.4(f). The nal model is required for rapid prototyping. The program, as shown in Fig.5, is crude, but

74

International Journal of Automation and Computing 1 (2006) 69-75

has all the essential functionality to demonstrate that

the algorithms and techniques presented in this article
are eective and useful. Further evidence is provided
by an application example in the next section.

badge, as shown in Fig.6(a). Then, a pattern of the

badge is generated, as shown in Fig.6(b). This leads to
the nal creation of a 3D data model Fig.6(c). The 3D
model is then used in a CAD system to create a CNC
milling tool path, as shown in Fig.6(d).
A nal badge is manufactured in two versions, one
is a wooden model, and the other a metal product.

Fig.5 Implementation of photo-modeling techniques

Example of applications

The photo-modeling techniques described, have

been applied to reverse engineering practice in the
rapid prototyping of a Jaguar badge, as shown in Fig.6.

Conclusions

Photo-modeling can generate 3D data models from

single 2D photo-image les. It is a useful and cost effective technique, which can be implemented to acquire
3D model data not only for rapid prototyping of artifact
products, but also for other industrial applications.
The important implication of this technique, is that
photorealistic images are nowadays the most popular
and commonly available data media. As such, they will
be made even more useful in engineering applications,
especially manufacturing, since they can be widely delivered and communicated over the Internet, and other
networks. In addition, they are technically compatible with almost all CAD and CAM systems, and can
therefore be adopted and eectively utilized by manufacturing processes like rapid prototyping.

Acknowledgments
The work presented in this article is sponsored by
Virtual Engineering Research Center at Shandong University, Jinan, PRC.

References
(a) 2D photo-image

(b) 2D pattern

(c) Generated 3D model

(d) CNC tool paths

(e) Wooden model making

(f) Metal model making

Fig.6 An application example of rapid prototyping

The process starts with a normal photograph of the

[1] C. A. Johnson. Recent developments in rapid prototyping &

manufacturing technologies. In Proceedings of Conference
of Time Compression Technology 2002 (TCT2002), Future
Focuss Session, Repaid News Publications plc. Time Compression Technologies, Cheshire, UK, 2002.
[2] W. Schoenmakers. Time compression technologies: Where
next? Keynote Address, Conference of Time Compression
Technology 2002 (TCT2002), Repaid News Publications plc.
Time Compression Technologies, Cheshire, UK, 2002.
[3] DELCAM.
Reverse Engineering, [Online].
Availabe:
http://www.delcam.com, 2003.
[4] M. Carbonnell, R. W. A. Dallas. The International Committee for Architectural Photogrammetry (CIPA)-aims, achievements, activities. Photogrammetria, vol. 40, no. 9, pp. 193,
1985.
[5] M. Carbonnell, R. W. A. Dallas. The International Committee for Architectural Photogrammetry: A review of
past achievements and future intentions. Photogrammetric
Record, vol. 62, no. 11, pp. 193, 1985.
[6] J. Badekas. 3D Detailed Reconstruction of a Demolished
Building by Using Old Photographs. The International
Archives of Photogrammetry and Remote Sensing, vol. 31,
Part B5, pp. 16, 1996.
[7] Introduction to Photogrammetry, [Online].
Availabe:
http://www.univie.ac.at/Luftbildarchiv/wgv/intro.html,
2004.
[8] Measuring and modeling the real world, [Online]. Availabe:
http://www.photomodeler.com, 2004.

Zheng-Xu Zhao et al./ A Photo-Modeling Approach to Restituting 3D Model Data from Single 2D Imagery for

75

[9] R. S. Wright Jr, M. Sweet. OpenGL SuperBible, 2nd edition, Waite Group Press, Indianapolis, USA, pp. 198206,
2000.

at the Virtual Engineering Research Center at Shandong University, PRC. It was at these two institutions where he accomplished
this paper and the related research.

Zheng-Xu Zhao is a Professor and

Chair in Applied Computing, and Director of the Computer Integrated
Manufacturing Research Center at the
University of Derby, UK. He obtained
his BSc. in Industrial Process Engineering in 1980 and MSc. in Mechanical & Manufacturing Engineering in
1984 from Shandong University, PRC,
and PhD. in Computing Science in
1992 from CNAA, UK. Professor Zhao
is currently a British Chartered Engineer, and Chartered IT Professional, and a Senior Member of the British Computer Society. He has been a Senior Technical Advisor to the United Nations Development Program since 1997, a board member of the
IEEE Technical Committee of Data Engineering since 1996, and
a member of the UK VR Special Interest Group since 1997. He
has authored and co-authored 5 books, and has published over
140 papers in 3D Computer Graphics, Virtual Reality Technology, and Computer Aided Engineering, with special research interests in 3D visualization systems, virtual environment systems,
3D programming, process control and engineering simulation.
Professor Zhao holds a Cheung Kong Scholars Programs full
Professorship in Virtual Reality Technology, as appointed in 2001
by the Chinese Ministry of Education, and is the Director of the
Virtual Reality Technology Research Center at Southeast University, PRC. He has also been a Visiting Professor since 2002

Kai-Ling Li is an Associate Professor in the Engineering Application of

Computers, and Head of the Division of Computer Application, and Director of the 3D Digitalization, Design and Manufacturing Technology
Research Center in the School of Mechanical & Manufacturing Engineering
at Shandong University, PRC. He obtained his BSc. in Manufacturing Processes and Systems in 1981, from Shandong University, PRC.
Mr. Li was a CAD/CAM Engineer and Mechanical Engineer at CAD/CAM Services Ltd. Hong Kong from 1992-1997.
His professional fields include Injection Molding and Die Cast
Molding, Design and Manufacturing, CNC Machining Process
Programming and Simulating, and CAD/CAM in Mold Design
and Machining. He has authored and co-authored 5 books in
Mechanical Manufacturing Processes, Foundation of Mechanical Manufacturing Techniques, Foundation of CAD/CAM Techniques, and Principles of Machining and Manufacturing, and has
Published over 30 papers in 3D Computer Graphics, 3D Injection Molding, Computer Automatic Design, Rapid Prototyping
Technologies, the Theory of Machining Process Size Chains, and
Computer Aided Design and Engineering; with special research
interests in 3D mold design and manufacture, and system design
and process simulation.