Selected Topics in

Computer Science
Application of Image Processing #4

Gera

2020

1
Image Retrieval
Growth of Image Data
• The impact of Image in our information-driven society is
growing since Images are being captured, stored & shared at
an unprecedented scale, mainly, because of advancements in
✓Image compression standards (e.g., JPEG, GIF, etc.) that
enable efficient storage & communication.
✓Image collection devices (like scanner, digital camera &
remote sensor, …)
✓Introduction of broadband networks (e.g. ISDN, ATM, …)
that are providing much higher bandwidth for many
practical applications
2
Image Retrieval…
Sample Statistics of Multimedia Data Collection
• According to a press release by Google, the search engine offers access to:
• over 3 billion Web documents and its Image search comprises more than 500
million images, videos, and audio clips.
• AltaVista performs more than 40 million search queries each day in more
than 25 languages,
• AltaVista www.altavista.com, cover over 250 million Web pages.
• AltaVista features multimedia search over 45 million images, videos, and
audio clips.
• University of Florida Digital Collections:
• The Audio-Visual Archives houses millions of sound recordings, moving
images, and still photographs
3
Image Retrieval…
Storage & Retrieval
• This explosive growth of image data accessible to users poses a whole
new set of challenges related to data storage, search and retrieval.
• The current technology of text-based indexing & retrieval implemented for
relational databases does not provide practical solutions for this problem of
managing huge image and other multimedia repositories.
• Most of the commercially available image indexing & search systems index the
media based on keyword annotations & use standard text-based indexing &
retrieval mechanisms to store & retrieve.
• There are often many limitations with this method of keyword-based
indexing and retrieval, especially in the context of image and other
multimedia databases.
4
Image Retrieval…

Limitations of Keyword-based indexing & retrieval

Text only schemes have limitations: A lot of manual work and Inaccurate
description.
• It is often difficult to describe with human languages the content of a
multimedia object
E.g., an image having complicated texture patterns.
• A manual annotation of text phrases for a large database is prohibitively
laborious in terms of time & effort.
• Since users may have different interests in the same image, it is difficult to
describe it with a complete set of keywords.
• Even if all relevant object characteristics are annotated, difficulty may still
arise due to the use of different indexing languages or vocabularies by
different users.
5
Image Retrieval…

Content-based Image Retrieval

• To solve drawbacks of searching image based on textual annotations, there
is an increase in interest in content-based solutions.
• In content-based retrieval, manual annotation of visual media is avoided,
and indexing and retrieval are instead performed on the basis of media
content itself.
• In applications such as digital & virtual libraries, automatic data analysis
has to be done to extract semantic meanings from images and videos for
content-based image retrieval.
• There have been extensive studies on the design and advancement of
automatic content-based image indexing & retrieval systems from many
different communities of engineering, computer & information science,
and psychology.
6
Image Retrieval…

What is Image Retrieval?

• Information retrieval is the activity of obtaining information resources
relevant to an information need from a collection of information
resources.
• Good IR involves understanding information needs and interests,
developing an effective search technique, system, presentation,
distribution and delivery.
• Image Retrieval can therefore be defined as the process of searching
for relevant Images from a collection of Images that satisfy users
information need.
7
Image Retrieval…

The two Subsystems of IR system

• Indexing:
• It is the process of organizing index terms identified from
multimedia document corpus.
• Indexing is used to speed up access to desired information from
document collection as per users query
■ Searching
• A searching process begins when a user enters a query into the
system.
• Searching then scans document corpus to find relevant documents
that matches users query
8
Image Retrieval…
Indexing Subsystem

Image documents
Assign document identifier
Corpus
Image
Preprocessing
Document
IDs
Segmentation &
Feature Extraction

Features
of terms Index File 9
Image Retrieval…
Searching Subsystem
query Preprocessing Query terms

Segmentation &
Feature Extraction
ranked list
Ranking Features
of terms
Relevant Matching
multimedia
data

Index File 10
Image Retrieval…

Content Based Image Retrieval

• The query
• A text description.
• One or more exemplar images.
• A sketch, e.g., how to search for a picture that shows sunset scene?
• “a dark background with an orange disk in the middle”.
• Or the combination of them.
• Search result
• A shortlist of images.
• The instances of the event you want to find in images, videos or audios.
• Usually allows user feedback to improve the result.
• The basic task in content based image retrieval is comparing and searching
for image by image.
11
Character Recognition

• Recognition of document images is at the heart of any document

image understanding system.
• Optical character recognition (OCR) systems take scanned images
of paper documents as input, and automatically convert them
into digital format for computer-aided data processing.
• The potential of OCR for data entry application is obvious: it
offers a faster, more automated, and presumably less expensive
alternative to the manual data entry devices, thereby improving
the accuracy and speed in transcribing data into the computer
system.
• Consequently, it increases efficiency and effectiveness (by
reducing cost, time and labor) in information storage and
retrieval. 12
Character Recognition…

• OCRs have wide range of applications in government

and business organizations, as well as individual
companies and industries.
• Some of the major applications of OCR include: (i)
Library and office automation, (ii) Form and bank
check processing, (iii) Document reader systems for
the visually impaired, (iv) Postal automation, and (v)
Database and corpus development for language
modeling, text-mining and information retrieval. 13
Character Recognition…

• OCR can be done offline or online.

• OCR converts document images to text that are printed or handwritten.
• Handwriting recognition are often called optical handwriting recognition (OHR)
analogous to optical character recognition (OCR).
• OCR usually deals with the interpretation of offline data which describes printed or
typewritten objects.
• The goal of OHR is to interpret the contents of the handwritten data and generate a
description of that interpretation in the desired format.
• The OHR task is also referred to as off-line handwriting recognition or online handwriting
recognition.
• On-line handwriting recognition deals with a data stream which is coming from a
transducer while the user writes, and off-line handwriting recognition deals with a
dataset which has been obtained from a scanned handwritten document.
14
Character Recognition…

Overview of OCR Design

15
Overview of conventional approach to OCR
Character Recognition…

Overview of OCR Design…

• Given a page for recognition, first it is preprocessed.
• The aim of the preprocessing module is to prepare the image for
recognition.
• Preprocessing involves binarization, skew correction and
normalization.
• It undergoes some image enhancements such as filtering out noise
and increasing the contrast.
• Then, the image is segmented to separate the characters from each
other.
• Segmentation occurs at two levels. 16
Character Recognition…

Overview of OCR Design…

• On the first level, text, graphics and other parts are
separated.
• On the second level, text lines, words and characters in the
image are located.
• Information from connected component analysis and
projection analysis can be used to assist text segmentation.
• Segmentation is followed by feature extraction, which is
concerned with the representation of the object. 17
Character Recognition…

Overview of OCR Design…

• Feature extraction and classification are the heart of OCR.
• The character image is mapped to a higher level by extracting special
characteristics and patterns of the image in the feature extraction phase.
• Feature extraction is expected to make the image invariant to rotation,
translation, scaling, line-thickness, etc.
• It could also remove redundant information to compress the data
amount and a lot of other things.
• A number of global and local features have been employed for this,
including profiles, moments, structural features, discrete Fourier
descriptors, etc.
• The classifier is then trained with the extracted features for classification
task. 18
Character Recognition…

Overview of OCR Design…

• The classification stage identifies each input character image by
considering the detected features.
• A range of classifiers are in use for these purpose.
• Classifiers such as Template Matching, Neural Networks, Syntactical
Analysis, Hidden Markov Models, Bayesian theory, SVM, etc. have been
explored.
• For improving the recognition result, post-processing module is
incorporated.
• The post-processor is typically intended to improve accuracy by
detection and correction of OCR errors.
• Contextual information such as character n-gram, dictionary, semantic
knowledge and domain knowledge are exploited in postprocessing. 19
Medical Imaging

• Medical imaging is a mechanism used to create the visual

representation of the internal parts of the body; that is,
any organs or tissues, mainly for clinical analysis such as to
diagnose or to treat disease.
• For better diagnostic, prognostic, and follow-up processes
of many diseases, the obtained images should be very
effective.
• In medical fields nowadays, have witnessed significant
advances in medical imaging and computerized medical
image processing.
20
Medical Imaging…

• These advances have led to new two-, three-, and

multidimensional imaging modalities that have become
important clinical tools in diagnostic radiology.
• The clinical significance of radiological imaging modalities in
diagnosis and treatment of diseases is overwhelming.
• While planar X-ray imaging was the only radiological imaging
method in the early part of the last century, several modern
imaging modalities are in practice today to acquire
anatomical, physiological, metabolic, and functional
information from the human body.
21
Medical Imaging…

• However, in many critical radiological applications, the

multidimensional visualization and quantitative analysis of
physiological structures provide unprecedented clinical
information extremely valuable for diagnosis and treatment.
• Computerized processing and analysis of medical imaging
modalities provides a powerful tool to help physicians.
• Thus, computer programs and methods to process and
manipulate the raw data from medical imaging scanners must be
carefully developed to preserve and enhance the real clinical
information of interest rather than introducing additional
artifacts.
22
Medical Imaging…

• The ability to improve diagnostic information from medical images can

be further enhanced by designing computer processing algorithms
intelligently.
• Often, incorporating relevant knowledge about the physics of imaging,
instrumentation, and human physiology in computer programs provides
outstanding improvement in image quality as well as analysis to help
interpretation.
• For example, incorporating knowledge about the geometrical location of
the source, detector, and patient can reduce the geometric artifacts in
the reconstructed images.
• Further, the use of geometrical locations and characteristic signatures in
computer-aided enhancement, identification, segmentation, and
analysis of physiological structures of interest often improves the clinical
interpretation of medical images.
23
References
➢ T. S. Huang and K. Aizawa, “Image processing: Some challenging problems,” in Proceedings of
the National Academy of Sciences of the United States of America, 1993, vol. 90, no. 21, pp.
9766–9769.
➢ E. Miranda, M. Aryuni, and E. Irwansyah, “A survey of medical image classification techniques,”
Proc. 2016 Int. Conf. Inf. Manag. Technol. ICIMTech 2016, November, pp. 56–61, 2017.
➢ R. C. Gonzalez and R. E. Woods, Digital Image Processing, 3rd ed., no. September. Upper Saddle
River, New Jersey: Pearson Education, Inc, 2015.
➢ Million Meshesha, “Recognition and Retrieval from Document Image Collections,” August 2008.
➢ E. Escolar, G. Weigold, A. Fuisz, and N. J. Weissman, “New imaging techniques for diagnosing
coronary artery disease Esteban,” CMAJ, vol. 174, no. 4, 2006.
➢ P. Chinmayi, L. Agilandeeswari, and M. Prabukumar, “Survey of Image Processing Techniques in
Medical Image Analysis: Challenges and Methodologies,” in Advances in Intelligent Systems and
Computing, 2018, vol. 614, no. 10.1007/978-3-319-60618–7_45, p. 12.

Gerabirhan Paulos
24
ToCourseInfo@gmail.com