Basic Introduction to

Image Processing

This presentation has been put together as a common effort of Urs Ziegler, Anne Greet Bittermann, Mathias
Hoechli. Many pages are copied from Internet web pages or from presentations given by Leica, Zeiss and
other companies. Please browse the internet to learn interactively all about optics. For questions &
registration please contact www.zmb.unizh.ch .

Presentation of multidimensional data

3D-data has to be presented in in a 2D-fashion for publication
on paper. The data set might be represented i.e. as image
gallery, top+side view, projection. A virtual light source and/or
shadows on a virtual projection plane are helping to recognize
spatial relations.

Interactive models, movies and animations can be published

on web-pages or into power point-presentations.

Image processing and analysis

After registration on the microscope the digital images are loaded to image
processing software for further processing. The data includes information about
pseudo color, pixel dimensions, time scale etc.
First image data get adjusted by background subtraction, contrast enhancement,
etc. Colors might be assigned; subvolumes selected; z-mismatchs corrected by
pixel-shifts.
The softwares offer different options to look at the multidimensional data sets.
i.e. slice viewer, gallery view, section view, projections, full 3D volume
representations, surface models, time bar, color coded overlays of several
channels, transparencies, ...
The software offers analytical tools for measurement and quantification: automated
counting of features, measurements of areas and volumes, tracing of filaments,
measuring of distances, evaluation of colocalization, ...

Automated Multidimensional
Data Processing
Dimensions:

micrograph processing softwares:

xy
= 2D
xyz = 3D
xyzt = 4D
xyzt! = 5D

Imaris (Bitplane)*
Volocity (Improvision)
NIH image **
BioImageXD **
Campus-Lizenz an der Uni Zrich
**scientific freeware on the internet

Digital images:
2-dimensional distribution of image Points (Pixel)
x
y

Digital resolution
Detectors record a limited amount of image points (pixel number)
within a xy grid. Each image point has its own grey level (dynamic
range).
Increasing the amount of image points as well as the number of
grey levels leads to bigger image files and longer calculation
times.
256 grey levels are coded by 8 bit. 256 grey levels are presented
by a computer monitor.
Today, detectors are pushed to discriminate 1024, 4096 or more
grey levels. The human eye can discriminate about 60 gray levels
(6 bit).

3D Data set
x
y
z

)!z
)!z
)!z
)!z

The information within

the optical sections along
the z-axis can be used to
reconstruct a 3-dimensional
image.

4D Data set
3-D stacks recorded along the time course

x
y

)!z
)!z
)!z
)!z

t1

)!z
)!z
)!z
)!z

t2

5D Data set
Wavelenghts adding another dimension of fluorescent data. Time laps of
multi-channel 3D stacks generate a 5D data set. Wavelenght information
is displayed as pseudo-colors.

x
y
z

)!z
)!z
)!z
)!z

)!z
)!z
)!z
)!z

t1, t2, t3, ...

)!z
)!z
)!z
)!z

Voxels
A voxel (= volume element) is the 3D-equivalent of the
2D-pixel. It is the smallest unit of a sampled volume.

The given maximal lateral (x,y) resolution of 0.2 m

and the axial (z) resolution of 0.4 m of a voxel
results in an elongated shape (point spread function).

Neighbours
For the calculation and visualisation are the neighbor voxels
of great importance.

2D
-> each Pixel has
4 neighbor pixels

3D
-> each Voxel has
6 neighbor voxels

Presentation and effort:

* Simple presentations (fast, allows 2D-publishing):
gallery view, section view, projections
* Intense calculations (time consuming, for analysis):
full 3D volume representation, surface rendering,
shadowing, stereo view
* Animations (time consuming, analysis & presentation):
rotating 3D models, time sequences of 3D
volumes

Image Gallery
Galleries of
images are the
most simple
data presentation.
for xyz
xyt
xy! ...

Projecting optical sections to one plane

Optical section
through a cube
containing fibers

Projecting the structures

of all sections to the ground
level (Extended Focus)

Projection types
Average Projection:
Simple to very complex mathematical procedures. Summing up the grey
values of all voxels with identical xy-coordinates along the z-stack,
divided by the numbers of optical sections.

Maximal Intensity Projection (MIP):

Only the voxel in the z-stack, which has the highest grey value, will be
projected.

Background signal gets projected too and might cause noise/blur.

Suppress background first!!

x1

x1

y1

Z1

y1

x2

x2
Z2

y2

Maximal
Intensity
Point
Projection
-> sharp image

Z2

y2

x3
y3

Z1

x3
Z3

y3

Z3

Z4

Z4

Z5

Z5

Projektion

Averaging
may lead to
enlarged
structures
and background

Projektion

Gallery presentation of a neuron

10

Maximal
intensity
projection
of the
optical
sections
of the
neuron

Maximum
intensity
projection
with one
sided
illumination
and
shadow.
(easy3D)

stack of images
gallery of images
x
y

Section through
the stack
Image of the section

x
z

Sectioning through
a stack of images
x

- perpendicular
y

y
z

Section through
the stack along
the y-axis

Y-Z

{
{
{
{

X-Y

Computer
representation
of section levels
in XY, XZ, YZ

X-Z

Intense calculation
for 3-D representations
1. Volume rendering
Ray tracing

2. Surface rendering
Segmentation of z-stacks
Depth encoding of voxels
Shadowing

3. Animations
time course
rotations
zooms etc.

Volume rendering

Even if fog (background) limits the visibility, we get an

idea of the structure of the trees.

Volume rendering
Volume

Screen

Virtual ray

Ray Tracing
A virtual ray passing the volume accumulates the grey levels of the
voxels, normalizes the summed value and presents it on the screen.

Volume rendering with adjustments of the grey values

Adjustment of the grey

level according to the
distance between voxel
and screen.

Voxels hit by the virtual ray

Adjustment of the grey

value according to the grey
value of the voxel just
passed.

screen

Volume rendering - example

3D representation of a multifluorsescent cellmonolayer

(4 channels)

Surface rendering

Creating objects with solid surfaces.

Surface rendering: Iso-Surface modelling

1st Step: Segmentation of the z-stacks. Identification of Voxels
belonging to an object.The criteria for the identification is the grey
value of the voxel.
All voxels, whose grey value are higher (brighter) than the chosen
threshold belong to the object, the others belong to the background
and will be discriminated. This threshold value is chosen by the
scientist.
(Neighborhood rule: If a voxel belongs to the
object, but one of its 6 neighbor voxels does
not belong to the object, it will be defined as a
surface voxel.)

Surface rendering: Iso-Surface modelling

2nd Step: Depth encoding of the Voxels.
The previously identified surface voxels have all the same grey
value and would result as a non structured evenly grey image on the
screen of the monitor. Therefore, in a second step, the grey values of
the voxels are adjusted according to the distance of the surface voxels
to the screen.
z

distance (depth)

y
x

All voxels have the

same grey value

Depth dependent
adjustment of the grey
values.

Surface rendering: Iso-Surface modelling

3rd step: Shadowing
The topology can be accentuated using a one sided shadowing effect.
To do that, neighboring surface voxels are connected to form a polygon.
The grey values of the surface voxels are adjusted dependent on the angle between the viewing direction and the normal of the polygon surface.

Viewing direction
and incident light
!

Surface voxels
define polygons

The normal
to the polygon and the
viewing
direction
include the
angle " .

Representation of several surfaces

hite) + Transparency
(w
60
d
un
d)
(re
0
11
s
old
sh
re
Th

Surface modeling: setting the threshhold

Threshold 68

Threshold 138

Surface models of the same dendrites using different threshold values

Which model shows the real surface ?

Adequate Filament Imaging

Stereo-Representation
The depth feeling can be simulated by calculating two separate slightly tilted
3D-models of the same scene as if they were viewed by the left eye and the right
eye. The final stereo pair can be observed using different techniques.

The 3D impression can be achieved squinting the eyes or using special stereo
viewers (or crossing the eyes).

StereoRepresent
ation II

The 2 pictures
of the stereo
pair
are colored in
red & green and
superimposed.
The 3D impression can be
achieved using
bicolor goggels.

Looking inside
Surface view

Surface view combined

with the visualization of
internal structures

Gallery view of 20 optical sections

Section view of 20 sections

x-y

y-z

x-z
Mo l,stained with acridine orange - 20 optical sections

3D-representation: x-y, x-z, y-z

Looking inside

Transparency & slicer tool

Looking inside

... by using transparency

Animations - fly through

Volume and surface
rendering allow you to
turn and zoom the data
set. Extreme Zoom
allows you to virtually
enter the sample.

Measurements
i.e.:
- Automated data segmentation
- Particle counting
- Size regognition
- Distance measuerments
- Filament tracking
- Movement tracing
" Results are visualized in the 3D model
" Results are listed as numbers in Exel-sheets

Colocalisation
The relation of the intensity values
from 2 channels are presented in
a two dimensional histogram.
In case of colocalization, the intensity clouds of both channels are
overlapping.
Colocalization is not an absolute
fact but allways relate to voxel size
and resolution.

Animations
Animations are series of single images put together into a movie. The images might be a
volume view, a projection, a slice, a time point. The animation is done by just playing the
sequential data set, or by rotating 3D models or volume representations, by zoom-in & flythrough motions, changing of surfaces and transparencies, etc.

Today#s computer allow to calculate and

represent animated sequences reasonably
fast. Movie files can be published i.e in
power point or on the web. Also interactive
file formats are possible.

Animation in time
t1
t2
t3
t4

Changes of a 3D-volume with time might be

presented as a gallery of projection views or as a movie. Animation and stereo view
facilitate the recognition of spheric relations
in this context.

Analysis
&
Animation
Particles
recognition
and tracing
in time

Deconvolution
What is to be gained?
Increase in resolution x, y, z
Noise is reduced
The image formation process is optimized
(astigmatism, point spread function, ...)
Widefield fluorescent data can be improved a lot by deconvolution.
Confocal data show less z-distortions, less out-of-focus blur,...
-> deconvolution shows only very little effect.

Convolution - Theory

Fluorescent bead with a diameter of 0,1 m

Deconvolution procedure
measured

Measure object of known size, but smaller than the

resolution of the microscope (i.e. 100nm fluorescent
beads)
Compare the microscope image with the
ideal/theoretical representation of the object.
Determine the difference of the measured and the
real object.

real

Correct unknown objects with the determined

difference.

Deconvolution effect

3D, 4D, 5D- data reconstruction is time consuming!!!

=>Only correctly recorded images are worth to spend the time to deal
with the 3D presentation!!!
=> Keep your data small:
Reduce image resolution (512 x 512 pixel = 262 kB).
Crop images so that they containing only the most important structural details.
Work with as less channels as possible.
stay with 8 bit
=>Keep the coffee pot hot in order to wait patiently until the calculations are finished.
=>Use classical image processing tools to improve the quality of the images.

and: Don$t expect to much of a 3-D presentation.