Chapter 2.

Issues and
Trends on
Multimedia
Applications
Dr Nurulfajar bin Abd Manap
FKEKK, UTeM (2015)
 Contents
• Multimedia Design Issues

• Content, Technical and Visual Design Requirement

• Multimedia System Design

• Issues in Multimedia Systems Design: Network,

Traffic, Scheduling and Synchronization

• Multimedia Applications: Types and classification

Multimedia Design Issues
There are various issues in Multimedia authoring. We
can summarise issues involved in Multimedia content
and technical design as follows:

Content Technical Visual


Design Design Design

C T V
Content Design
Content design deals with:
what to say, what vehicle to
use.

"In multimedia, there are five

ways to format and deliver
your message. You can write
it, illustrate it, wiggle it, hear it,
and interact with it."
 s
Sc

hic
rip

ap
tin

Gr
g Content
Design Anim
io a
(wig tion
A u d gling
)

Interactivity
 Scripting
Rules for good writing:

1. Understand your audience and correctly address

them.

2. Keep your writing as simple as possible. (e.g.,

write out the full message(s) first, then shorten it.)

3. Make sure technologies used complement each

other.
 • Make use of pictures to
effectively deliver your
messages.

• Create your own (draw,

(color) scanner, PhotoCD),
or keep "copy files" of art
Graphics works.

Graphics Styles

• Fonts, colors, pastels,

Illustrating earth-colors, metallic,
primary color, neon color
 ‘Wiggling’ Animation
• Types of animation

• Character animation - humanise an object

• Highlights and sparkles (to pop a word in/out of the screen,

to sparkle a logo - to draw attention)

• Moving text (put up one character at a time like a typewriter

e.g., "pulsing" - the word grows/shrinks or changes color)

• Video (more powerful than still images, but takes a lot of

disk space)
Music - set the mood of the
presentation, enhance the
emotion, illustrate points

Sound effects - to make

specific points (e.g.,
squeaky doors, explosions,
Audio
wind)

Narration - most direct

message, often effective
 Interactivity
• Menu driven programs/presentations: often a
hierarchical structure (main menu, sub-menus)

• Hypermedia: less structured, cross-links between

subsections of the same subject -> non-linear, quick
access to information,easier for introducing more
multimedia features

• Simulations / Performance-dependent Simulations:

Games - SimCity, Flight Simulators
Design
 Menus for Navigation

• A user navigates through content using a menu.

• A simple menu consists of a text list of topics.
• Created using hypertext
 Interactive Buttons
• A button is a clickable object that executes a
command when activated.

• Users can create their own buttons from bitmaps

and graphics.

• The design and labeling of the buttons should be

treated as an industrial art project.

• Created using hypermedia

 HTML Documents
• HTML stands for Hypertext Markup Language.

• It is the standard document format used for Web pages.

• HTML documents are marked using tags.

• An advanced form of HTML is DHTML.

• DHTML stands for Dynamic Hypertext Markup Language.

• DHTML uses Cascading Style Sheets (CSS).

• Some of the commonly used tags are:

1. The <B> tag for making text bold faced.

2. The <OL> tag for creating an ordered list.

3. The <IMG> tag for inserting images.

• HTML doesn't provide much flexibility to make pretty text

• Graphical images are added to make pretty text in HTML

• Layouts, such as tables, frames and horizontal rules can help create a pleasant HTML document
Creating Attractive Texts
Applications that are used to enhance texts and images
include:

• Adobe Photoshop

• TypeStyler

• COOL 3D

• HotTEXT

• TypeCaster
Attractive text
Using Hypertext Systems
• Information management and hypertext programs
present electronic text, images, and other elements in a
database fashion.

• Software robots visit Web pages and index entire Web

sites.

• Hypertext databases make use of proprietary indexing

systems.

• Server-based hypertext and database engines are

widely available.
 Searching for Words
Typical methods for word • Alternates
searching in
hypermedia systems • Association
are:
• Negation
• Categorical search
• Truncation
• Word relationship
• Intermediate words
• Adjacency
• Frequency
 Hypermedia Structures

• Links.

• Nodes.

• Anchors.

• Navigating hypermedia structures.

! Links are connections between conceptual elements.

! Links are the navigation pathways and menus.

! Nodes are accessible topics, documents, messages, and content elements.

! Nodes and links form the backbone of a knowledge access system.

! Anchor is defined as the reference from one document to another document,

image, sound, or file on the Web.

! The source node linked to the anchor is referred to as a link anchor.

! The destination node linked to the anchor is referred to as a link end.

! The simplest way to navigate hypermedia structures is via buttons.

! Location markers must be provided to make navigation user-friendly.

 Technical Design
• Technological factors may limit the capability of your multimedia
presentation

• Technical parameters that affect the design and delivery of

multimedia applications

Video Mode Resolution Colors
---------- -------------- --------
CGA 320 x 200 4
MCGA 320 x 200 256
EGA 640 x 350 16
VGA 640 x 480 256
S-VGA 1,024 x 768 $>$= 256
S-VGA 1,280 x 1,024 $>$= 256
.
.
.
16-bit color --$>$ 65536 colors
24-bit color --$>$ 16.7 million colors
 Technical Design
1. Video Mode and Computer Platform 

PC <-> Macintosh 

There are many "portable", "cross-platform" software
and "run-time modules", but many of them lose quality/
performance during the translation.

2. Memory and Disk Space Requirement 


Rapid progress in hardware alleviates the problem, but
software is too "greedy", especially the multimedia ones.

3. Delivery

Live presentation, storage, electronic delivery
 Visual Design
• Themes and Styles

• A multimedia presentation
should have a consistent
theme/style, it should not
be disjointed and cluttered
with multiple themes.

• The choice of theme/style

depends on the styles and
emotions of your audience.
 Visual Design
Some Possible Themes
• Cartoon theme
• Traditional theme: straightforward
• High tech theme: contemporary
computer art work, attractive,
easy to animate
• Technical theme: blueprints, 3D
models of the product, (start with
a drawing, then transformed into
a rendered image).
 Visual Design
Basic Layout
(a) Title (b) Action area (c) Narration
(d) Dialog (e) Interactive controls

• Make sure that the information delivery path in the

layout is smooth, not irregular/jumpy

• Use headlines/subtitles, additional shapes, buttons,

fonts, backgrounds and textures to enhance the
visual appearance.
 What is typography?
• Typography is the visual component of the written
word.

• A text is a sequence of words. A text stays the

same no matter how it’s rendered. Consider the
sentence “I like pizza.”

• We can print that text on a piece of paper, or read it

aloud, or save it in a file on my laptop. It’ll be the
same text, just rendered different ways—visually,
audibly, digitally.
 What is typography?

• But when “I like pizza” is printed, typography gets

involved.

• All visually displayed text involves typography—

whether it’s on paper, a computer screen, or a
billboard.

http://practicaltypography.com
 Fonts
• Text is a vital element of multimedia menus, navigation systems,
and content.

• Visual representation of a character is called a glyph (element of

writing or the meaning of what is written).

• Must replace characters with glyphs for display.

• Glyphs are arranged into collections called fonts.

• Fonts are stored in specified locations on a computer system, may

be embedded in documents.

• If font is not embedded, document may not display properly on

systems where that font is not installed.
 Classification of fonts
• Spacing – monospaced (fixed width)

• Serifs – serified/sans serif

• Serifs are the small strokes added to the ends of character

shapes in conventional book fonts

• Shape – upright/italic/slanted

• Slant is a vertical shear effect, italic uses different glyph

shapes with a slant

• Weight – bold/normal/light
 Classification of fonts
Factors affecting legibility of text:

• Size.

• Background and foreground color.

• Style.

• Leading.
 Understanding Fonts and
Typefaces
!A ‘typeface’ is a family of graphic characters that
usually includes many type sizes and styles.
!A ‘font’ is a collection of characters of a single size
and style belonging to a particular typeface family.
 Fonts vs Typefaces
“The distinction between font and typeface is that a font
designates a specific member of a type family such as
roman, boldface, or italic type, while typeface designates
a consistent visual appearance or style which can be a
"family" or related set of fonts.

For example, a given typeface such as Arial may include

roman, bold, and italic fonts. In the metal type era, a font
also meant a specific point size, but with digital scalable
outline fonts this distinction is no longer valid, as a single
font may be scaled to any size”

‐Wikipedia
 Understanding Fonts and
Typefaces

• Helvetica, Times and Courier are TYPEFACE

• Times 12-point italic is a font

• In computer world, font is usually used

 Font terminology
○ Points – 1pt = 1/72” = 0.3528mm
○ Exact size is not standard; 1/72” is invariably used by computer systems
○ Font’s body size is not necessarily the size of any particular character.
○ E.g. 10pt Times Roman
 Character Metrics
Character metrics consist of the
following character attributes:

• height

• ascent

• left adjust

• move

• right adjust
 Spacing
• Kerning – adjustment of space between certain
pairs of letters (e.g. AV) to make them look more
uniform.

• Kerning pairs for a font are defined by its designer,

stored with the font metrics.

• Ligatures – single composite characters used to

replace pairs of letters that don’t look right next to
each other (e.g. fi)

• Ligatures are stored as extra characters in the font.

Spacing
 Serif VS Sans Serif

• Serif is the little decoration at the end of a letter stroke.

• Serif fonts are used for body text.

• Sans serif fonts do not have a serif at the end of a letter stroke.

• These fonts are used for headlines and bold statements.

 Choice of fonts
• Text fonts – suitable for continuous text (e.g. body of a book or article)

• Must be easy to read

• Display fonts – suitable for isolated pieces of short text (e.g. headings, signs or
slogans)

• Intention is to get a short message across, so eye-catching design that would be

inappropriate for continuous text is OK

• Text fonts may be problematical.

• Low resolution of computer displays leads to loss of details (e.g. fine serifs) and
distortion of letter shapes.

• Use larger sizes than in print, prefer sans serif, use fonts such as Arial and Verdana
designed to be readable at low resolution.

• Display fonts work better and may be suitable for small pieces of continuous text.
 Using Text Elements in
Multimedia Presentation
! The text elements used in multimedia are:

1. Menus for navigation.

2. Interactive buttons.

3. Fields for reading.

4. HTML documents.

5. Symbols and icons.

! Anti aliased text must be used when a gentle and blended look for titles and headlines is
needed to jagged edges.

! Ideas and concepts can be highlighted by making the text bold or by emphasizing text.

! A pleasant look can be created by experimenting with different font faces, sizes, leadings, and
kerning.
Sample Documents 

on Typography
Multimedia System Design
• Multimedia systems design is the process of
building a system with the technologies and the
architecture able to support multimedia applications.

• Here, we study the operations that applications

might be expected to perform on each media type
and those factors involved in integration of the
media.

• Summarise the main criteria needed to be

considered in multimedia systems design.
Applications & Operations
Each multimedia application handles one or more media
type of information. There are TWO (2) levels of description
for an application:

• Functional level — what the application does and source

and display data formats. E.g. An application may deal
with the retrieval and display of movies. It retrieves movies
and displays them in a window.

• System Level — The entities that the application deals

with and how it deals with them. In the above example we
deal with video frames and voice samples and need to
process these accordingly.
 Functionality
• Digital Representation of Media - Many
standardised formats for many media

• Capture: Digitisation of Media - special Hardware/

Software

• Creation and editing - assemble media and alter it

• Storage Requirements - significant for multimedia

 Functionality
• Compression - related to above and below, ie can
save on storage but can hinder retrieval

• Structuring and retrieval methods of media - simple

to advanced Database Storage

• Display or Playback methods - effect of retrieval

must view data

• Media Synchronisation - display multimedia as it is

intended
 Operation
Text The operations involve
• Editing basic processing such as:
• Formatting • Character
Manipulation
• Sorting
• String Manipulation
• Indexing
• Searching
• Encrypting
 Operation
Audio
• Audio Editing The operations involve
• Synchronisation basic processing such as:
• Conversion/ • Sample Manipulation
Translation • Waveform
• Filtering/ Sound Manipulation
Enhancing
Operators
• Compression
• Searching
• Indexing
 Operation
The operations involve
Image basic processing such
• Pixel operations as:
• Geometric • Pixel Manipulation
Operations • Area Manipulation,
• Filtering copy image areas,
• Conversion delete area, process
• Indexing areas of interest
• Compression
• Searching
 Operation
Animation The operations involve
• Primitive/Group basic processing such as:
Editing • Pixel Manipulation
• Structural Editing
• Area Manipulation,
• Image Overlaying
copy image areas,
• Rendering
delete area, process
• Synchronisation
areas of interest
• Searching
• Frame Manipulation,
• Indexing
Copy frame(s) to
frame(s), Process
frame(s)
 Operation
Graphics
• Graphic primitive The operations involve
Editing basic processing such
• Shading as:
• Mapping • Primitive
Manipulation
• Lighting
• Structural/Group
• Viewing
Manipulation
• Rendering
• Searching
• Indexing
 `
Video
• Pixel Operations The operations involve
• Frame Operations basic processing such as:
• Editing • Pixel Manipulation
• Synchronisation • Area Manipulation,
• Conversion copy image areas,
delete area, process
• Mixing
areas of interest
• Indexing
• Frame Manipulation,
• Searching Copy frame(s) to
• Video Effects/ frame(s), Process
Filtering frame(s)
 Distributed System
• The development of large networked of distributed has had a major impact
and also been influenced by the requirements of multimedia systems.
Huge need in the requirements for performance and application.

• Many of the issues that concern distributed computing research is

complimentary to multimedia systems. For Examples:

• Network design and choice of network

• Delivery of data, Scheduling and bandwidth

• Load balancing and load sharing.

 Distributed System
• Multimedia systems are concerned with real-time delivery and
scheduling which is a challenge for any distributed systems.
Many multimedia applications would not be possible without a
distribute systems architecture, for example:

• Video conferencing

• Video on Demand

• Hypermedia based application (www etc.)

• Multimedia Databases

• Interactive TV and many more

 Multimedia Issues
• Functionality Issues

• Presentation Design

• Multimedia content issues

• Images
• Colors & Color Palettes
• Video, Flash, Animation
• Text, titles, fonts
• Audio
 Issues : functionality
! Which mode is selected (highlighted?)

! What any button or “hot” text or graphic does?

! How large the multimedia file is that they are about to download?

! What plug-in they need to view a multimedia file?

! How to start & stop a video? An Audio clip?

! How to get around in the text?

! How much text there is?

! What the hyperlinks mean?

! How to find something?

Issues : presentation design
! Define a visual theme and style

! Define consistent global & local navigation for ease of use

! Get your main message across on each page

" Get all important elements ABOVE THE FOLD
" Site design has to accomplish your developer goals for site

! Design has to appeal to potential users so they can accomplish

their goals for using your website
" Emotionally
" In a way they can understand and experience
" Accessible

! Design has to convey your message

Issues : image resolution
• Computer screen is low resolution: 72 dpi (refer
space between sub-pixel of R,G and B.)

• Image memory size varies depending on

compression and file format

• Web color on client varies depending on:

• Bit-depth settings of computer
• Color calibration & gamma default settings of
monitor
• Operating system, Web browser
 Issues : color & color
palettes
• In an interactive product when you have more than
one image on the screen,
• each image may use a different palette of colors
• all of the colors in both images may not be the
palette in use

• Good ways to limit image file size

• reduce the color palette – minimize the range of
colors: All reds/oranges or All greens/blues, but
NOT red, orange, green, blue, yellow, etc.
Issues : text, titles & fonts
• Get the message across (accomplish developer
goals)
• spelling; readability, correct grammar & spelling

• Bitmapped text looks better but text is smaller,

easier to revise, less troublesome to program for
user with disabilities

• For most flexibility use elastic layout & CSS

 Issues : video & flash
• Video can be used to

• Convey powerful emotional messages

• Entertain: live action, functional object, view object from different

points

• teach something

• video as art form

• Takes a lot of memory – needs to serve a purpose

• Divide larger video into smaller pieces that viewer can view consecutively

• Warn viewer about file sizes before download

 Issues : animation
• Elements that change, move. appear & disappear

• Be sure doesn’t flicker so much it can set off

seizures in users with certain disabilities

• Can use a series of text, pictures or photos

• Photoshop & Flash: provide beginning & ending

images & program “tweens” changes

• How can you make animation (e.g., Flash)

accessible to people who cannot see it?
 Issues : audio
• Key characteristic is sampling rate
• resolution at which sound is digitized
• the higher the sampling rate, the better the quality the larger
the file.

• Easy to modify and edit

• Sound should suit the style

• Background music
• helps integrate project
• Creates emotional mood for viewer

• How can you make audio accessible to the hearing impaired or

deaf user?
Issues : Links & navigation
• Links for navigating
• Links for performing certain actions
• Link issues:
• Avoid multiple links to the same page from a
single page
• Non-working links
• Descriptive link names help blind users
• Links within a long page w/multiple parts
• Dead ends– pages that go nowhere
• Links to page itself – confuses users
Issues : Links & navigation
• Concerns user’s relationship with site content
• How will they know where they are?
• How will users navigate?
• Semantic page structure: <h1> largest, <h6> smallest, etc.
• Tags not used for formatting but to indicate what kind of content
(ul, li) or hierarchy of importance
• Contextual clues & features (implicit support)
• e.g. color coding – blind & color-blind users?
• Landmarks – introductory pages to sections; consistent buttons;
menus, etc.
• Explicit support: maps, page labels, tabs
Issues : Hypertext System

• Hypertext is defined as the organized cross-linking

of words, images, and other Web elements.

• A system in which words are keyed or indexed to

other words is referred to as a hypertext system.

• A hypertext system enables the user to navigate

through text in a non-linear way.
 File size : digital video
tradeoff
• The higher the image quality, the lower the playback
quality
• Requires more data about each frame & more frames
per second
• Rate of transfer from disk is limited, so video has skips
and gaps
• If the frame rates are lower, the image looks better
because the frames contain more information

• Video files get very big

• Compressing in Flash Video encoder lets you stream

video without having streaming server
 File size : images
• Smaller Image Files fasten download

• Reduce Image File Size for the Web

• Reduce number of colors
• Reduce image dimensions
• Blur unimportant areas of image
• Select important areas before compressing to increase their quality
• Compress to reduce color depth

• To check download speed of a page in Web

Developers Toolbar: Tools>Speed Report
Develop workflow methods
• Buy high, sell low
• Digitize images, sound, video at highest possible
resolution & reduce it later
• Digitizing at low resolution saves memory & storage
up front, but loses too much data

• Save the best for last

• Reduce the resolution of the content last, after
you’ve done your editing and altering
• Compress from the high quality version – otherwise
the file degrades rapidly if you recompress
previously compressed documents
Develop workflow methods
• Protect the original copy -
• Archive a copy of the original media file so you can redo
the process if you need

• Keep an audit trail

• Use Adobe Bridge
• If you use non-original content, CHECK that you have
permission & save that information in your ADMIN folder
• List file & its source in your source log document
• Keep a record of changes you make to a file
• In Photoshop, make each alteration a separate layer &
save the PSD version which preserves all the
layers
• Save versions so you don’t have to go back to beginning
 Problems & solutions
• Limited bandwidth
• Solution: Compression

• Packet Jitter
• Solution: Fixed/adaptive play out delay for Audio
(example: phone over IP)

• Packet loss
• Solution: Forward Error Correction (FEC) etc
Multimedia System Issues
The major differences between a multimedia
application and a typical computer application are
(although this distinction may be getting weaker with
time as many applications now involve multimedia):

• Media types and their composition

• Relationship between the media (synchronisation)

Multimedia System Issues
There are several implications that need to concerned
with multimedia systems and their delivery of data:

• Quality of Service (QoS)

• Measuring the QoS

• How to recognise and quantify the QoS? How

may the QoS be guaranteed?

• Synchronising the data

Quality of Service (QoS)
Basically a collection of parameters that relate to a sequence
as seen at the source and as seen at the destination of a
multimedia presentation. This measure probably is the ultimate
measure of a multimedia system.

Four essential parameters are:

• Bandwidth — capacity of the transfer mechanism between
source and destination.
• Delay — the time a multimedia unit spends in transmission
from source to destination.
• Delay Jitter — Variation in delay delivery of data
• Loss Probability — the ratio of units of information that an
application can afford to lose.
 Architecture of
Multimedia System
Network Subsystem
LAYER 1

End-to-End QoS Control

LAYER 2

Media Management
LAYER 3

Application
LAYER 4
 Architecture of
Multimedia System
Network Subsystem
LAYER 1

Network Subsystem (Layer 1) : This layer takes care

of the functionalities up layer 3 in the OSI model.
Network specific functions depend on the
technology used in this layer.

Essentially this level provides a possible connection

through a network with a specified bandwidth and
error probability as supported by the underlying
technology.
 Architecture of
Multimedia System
End-to-End QoS Control
LAYER 2

End-to-End QoS Control (Layer 2) : This layer

maintains the connection between the source and
destination and can be conceptually viewed as a
single connection — even though there may be
physically many more.

Each connection is managed to ensure that a given

Quality of Service (QoS) is maintained.
 Architecture of
Multimedia System
Media Management
LAYER 3

Media Management (Layer 3) : This layer provides

generic services to applications so far as media
management is concerned.

A primary functions is synchronisation across the

media.
 Architecture of
Multimedia System
Application
LAYER 4

Application (Layer 4) : The direct interface with the user.

The application will also interface with the operating
system, if required.

For example calls to storage media or specific library

functions (subroutines).
 Challenges in 

Multimedia Network
Multimedia data (audio, video, etc.) is often called
continuous data. They have the following characteristics:

• voluminous

• real-time (synchronization, esp. between audio and

video)

• interactive (e.g., in video conferencing, interactive TV)

• sometimes bursty (e.g., video-on-demand)