Chapter 2

Multimedia:
Streams

Media and Data

The following chapter introduces terminology and gives a senseof the commonality of the elements of multimedia. The introduction of terminology begins with a followed by a description of media and the clarification of the notion multimed,i,a, important properties of multimedia systems. Subsequently,characteristics of data streamsin such systemsand the introduction of the notion LogicalData Unit (LDU) follow. One way of defining multimedia can be found in the meaning of the composed word.

o Multi-

much]

much; multiple.

c l,!.ed,i,um .: middls] A.1_il-1Sly"-"j$ [at !It".@ 9.f.b'-1-qlg"g of.mass A t3l'gi!*4,9-f*9,?l-ti"q9n1 m.e:,ns i-9*_rnglc?t"'gryg9$*9J:-g3P9r, 1991). I{eltageElectronig picti9n3,ry, or -!9l9yigi91 4.*taic11, (from ,magazine,
This description is derived from the common forms of human interaction. It is not very exact and has to be adapted to computer processing. Therefore, we discuss in the next section the notion medium in more detail with respect to computer processing.

10

CHAPTER 2. MULTIMEDIA:

MEDIA AND DATA STREAMS

2.L

Medium

In general, one describes medium as a means for distribution and presentation of information., Examples of a medium are text, graphics, speechand music. In the same way' one can also add water and atmoiphei; io lT-(acior-ding to the above medium description from the American Heritage Dictionary). Media can be classifledwith respect to different criteria [ISO93a]. We classify media according to,pe,rception, representation, presentation, storage, transmissiJn;*;rd information exchange.

2.L.1

The Perception

Medium

Perception media help humans to sensetheir environment. The central question is: How- d,ohumi"," perceiue information in a computer enuiro,nm,ent? The-anJ*u. is that the perception of information occurs mostly ihrough seeingor h,ea,iing the information, although tactile perception increasesits presencein a co-pui"i""rironment. There is a primary difference between seeing and.hearing information when using a computer. For the perception of information through seeing, iil" -"aiu "ir"it roih ur t"*l,iiog" and.uideoare used. Foithe pur."ploo.i;i;r;;tion irLiough hearing, auditory media such as music, noise and, speechare relevant. The differenceamong media can be further refined. For example, video can be further decomposedinto different video scenes, which again are composedof individual images.

2.L.2

The Representation

Medium

Representation media are characterized by internal computer representationsof information. The central question is: How is the computer information cod,ed? The answer is tiat various formats are used to represent media information ilu.-=;; puter. For example:

2.1. MEDIUM
o A text characteris coded in ASCII or EBCDIC code.

11

o Graphics are coded according to CEPT or CAPTAIN videotext standard. The graphics standard GKS can also serve as a basis for coding,o An audio stream can be represented using a simple PCM (Pulse Coding Method) with a linear quantization of 16 bits per sample. o An image can be coded as a facsimile (the gloup 3 according to the ISO StandardSpecification)or in JPEG format. o A combined audio/video sequencecan be coded in different TV standard formats (e.g.,PAL, SECAM, NTSC), and stoied in the computer using i" MFEG format.

2.1.3

The Presentation

Medium

Presentation media refer to the tools and devicesfor the input and output of information. The central question is: Through which medium is information d,eiiuered The media, e.9., paper' screen lA the computer, or introduced into the computer? and speaker are used to deliver the information by the computer (output media); keyboard, mouse, camera and microphone are the input media,

2.L.4

The Storage Medium

Storagemedia refer to a data carrier which enablesstorageof information. However, the storageof data is not limited only to the available componentso[ a .ompui"i. Therefore, paper is also a storage medium. The central question is: Where will the information be stored,? Microfilm, floppy disk, hard disk, and CD-ROM are examples of storage media.

2.L.5

The tansmission

Medium

The transmission medium characterizes different information carriers, that enable continuousdata transmission.Therefore,storagemedia are excludedfrom this kind

L2

CHAPTER 2. MULTIMEDIA:

MEDIA AND DATA STREAMS

of medium. The central question is: Ouer what will the information be transmitted? The answer ii that information is transmitted over networks. which use wire and cable transmission, such as coaxial cable and flber optics, as well as free air space

2.1.6

The Information

Exchange Medium

The information exchangemedium includes all information carriers for transmission, i.e., all storageand transmissionmedia. The central questionis: Which information carrier.will be usedfor information erchangebetween different places?The answer is,tha-t information .in flo* through intermed,iatestorage me{.ia, *ii"t9 1h,9'.-t.93gu medium is transported outside of computer networks to the destination, through direcl transmission using computer networks,or through combinedusageof storage and transmissionmedia (e.g., electronicmailing system).

2,1,7

Representation

Values and Representation

Spaces

The above classification of media can be used as a basis for characterizing the notion medium in the context of information processing. Here,,the de.scriptig-n-.of perception medium comesclosestto our notion of a medium: the media appeal to the human senses.Each medium defines ,"pr"r"itotion ualuesand representation tporrt [HD90, HS91],which involvethe five senses. Example:s-o{ yilq"! representation spacesare paper.or s_.c19n. During a computercontrolled slide show with simultaneous projection of the computer screen content, the whole movie screencounts as a representationspace. Stereo and quadraphony determinethe acousticrepresentation spaces. Representation spaces.un ulro"bl part of the above describedpresentationmedia for inftrmation output. considered Representationvalues dglermine the information representation of different media: while the lerl medium visually represents sentence a through a sequence .i;h;; ters, this sentence will be represented the speech by medium in the form of a pressure wave. Somerepresentation valuesare self-contai by their media. In other words, ned they can be properly interpreted by the recipient. Examples here are temperatuie,

2.1. MEDIUM

13

taste, and smell. Other media require a predeflned symbol set, which the users must are ugt"" opo". Text, spegchan$ Beslure_s exlmPl_e19{.lu-clie_qia. of can Repr_esentlt]on yal_geg be consideredeither as a continuum or a sequence discrete values. Pt"rioi. wave fluctuations do not appear as discrete values; instea-d th"y d"i"rmine the acoustic signals. Electromagnetic waves for human eye percepiather they are a Continuum.' eliaiicteii of'a tion ;G-nol diicrete vilues "ithe.; of text and audio sample values in electronic form are sequences discrete valuas.

2.1.8

Representation

Dimensions

Ea-chrepresentation space consistg.gf one ot more representation d'imensions. A. computer screen has two spatial dimensions; holography and stereophony .19=q:I" an additional spatial dimension. Time can occur inside each representation spaceas an additional dimension, as it has central meaning to multimedia systems. Media space: are divided into two types with respectto time in their representation are 1 . Some media, srlc! as !e-1t an$ Sr-1n-!1c.s? tlmg-rndeqln_!9nt' Information of i sequenceof indivitiuil elementior of in these media consist "i.i"ri""ly a continuum without a time component. Such media -f ilno*n as ti'iiodiscrete'is sometirnG;""frfi"4' (ot d,isirete). Note, the notion independ,ent becausea medium can also be discrete in value but continuous in time.). The text of a book is, for example, a disCietemeiiiuin. Piocessing of discrete media should happen as fast as possible,but this processingis not time critical becausethe validity (and therefore correctness)of the data does not depend on any time condition.

2 . The values of other media, such as sound and full-motion video, change over
time. Information is expressednot only in its individual value, but- also by the time of its occurrence.The semanticsdepend on the level of the ielative change of the discrete values or of the continuum. Such media ane timevalues caused by tactile or temperat"i" t"i d,epend,ent. _Also,representation sors with threshold detectors are time-dependent, and therefore also belong to the time-dependentmedia.

14

CHAPTER 2, MULTIMEDIA:

MEDIA

AND

DATA

STREAMS

Proces,qing oJ,thesemedia is time-critical becausethe validity (and therefore cortectness)of lhe data depends on a time condition. for eiimfle, i*tranJ mitted audio sample delivered too late is invalid if the following samples to the sample io qo".tiotr have already been played back. Individua_l representation values occur in audio and video as a continuous sequglcg: We underst and ui,deo a sequenceof plain images occurring perias We call these media continuous med,ia. Using this division, time-dependent representation values, which occur aperiodically, are not considered continuous media. Control commands for real-time systems are an example. Tn -ftiry4t.:rytems, we must also consider non-coi'iitr.ous sequences repof resentation nulo"r, !".1i sequ,elcgs oc_c.y-l: fgl_:fl1nqig, b_y il-utrrmiision -of information (e.g.,mousepointer position) in a cooperativeapplication within a shared window. Examples of continuous media are: uid,eocoming from natural source (e.g., video taken by a camera during a live video lransmission) or from an artificial source (e.g., video disc); aud,io, which is mostly stored as a sequenceof digitalized sound-wavesamples; and signals of different sensors)such as those that senseair pressure, temperature, humidity, pressureor rad.ioactivity. These notions of time-dependent, discrete and continuous media do not have any connection to internal representation. They relate to the impression of the viewer or Iistener. For example, a movie as a representativeof continuous media often consists of a sequence discrete values, which change in representation space according to of a time function. The inertia of the human eye only leads to the impression of continuity if a sequence at least 16 individual images per secondis provided. of

odi91111, a1_,geLt_q! igg:"_!.gggilgng:_ry" of samples with_periodic lgbyryI

2.2
2.2.L

Main Properties of a Multimedia System

If we derive a multimedia system from the meaning of the words in the American Heritage Dictionary, then a multimedia system is any system which supports more

2.2. MAIN PROPERTIESOF A MULTIMEDIA SYSTEM

15

than a single kind of media. This characterization is insufflcient becauseit only deals with a quantitatiue evaluation of the system. For example, each system processing text and graphics would be classified as a multimedia system according to this narrow definition. Such systems already existed before the multimedia notion was usedin a computer environment. Hence,the notion multimedia implies a new quality in a computer environment. We understand multimedia more in a qualitatiue rather than a quantitative way. Therefore, the kind rather than the number of supported media should determine if a system is a multimedia system. It should be pointed out that this definition is controversial.Even in the standardizationbodies,e.9.,ISO, a weakerinterpretation is often used. A multimedia system distinguishesitself from other systems through severalproperties. We elaborate on the most important properties such as combination of the computer control and integration' media, media-independence,

2.2.2

Combination

of Media

Not every arbitrary combination of media justifies the usage of the term multimed,i,a.A simple text processing program with incorporated images is often called a multimedia application becausetwo media are processedthrough one program. But one should talk about multimedia only when both continuous and discrete media are utilized. A text processingprogram with incorporated images is therefore not a multimedia application.

2.2.3

Independence

An important aspect of different media is their level of independencefrom each other. In general, there is a request for independenceof different media, but multimedia may require several levels of independence. On the one hand, a computercontrolled video recorder stores audio and video information, but there is an inherently tight connection between the two types of media. Both media are coupled together through the common storage medium of the tape. On the other hand, for

16

CHAPTER 2. MULTIMEDIA:

MEDIA

AND DATA STREAMS

the purpose of presentations,the combination of DAT recorder (DigitaI Audio Tape) signals and computer-availabletext satisfiesthe request for media-independence.

2.2.4 Computer-supportedIntegration
The media-independenceprerequisite provides the possibility of combining media in arbitrary forms. Computers are the ideal tool for this purpose. The system should be capable of computer-controlled media processing. Moreover, the system should be programmable by a system programmer or even a user. Simple input or output of different media through one system (".g., u video recorder) does not satisfy the requirement for a computer-controlled solution. Computer-controlled data of independent media can be integrated to accomplish certain functions. For such a purpose, timing, spatial and semantic synchronization relations will be included. A text processingprogram that supports text, table calculations and video clips does not satisfy the demand for integration if program supporting the connection between the data cannot be estab[shed. A high integration level is accomplishedif changing the content of a table row causescorresponding video sceneand text changes. Such flexible processingof media is not obvious - even in many of the best available multimedia products. Therefore, this aspect must be emphasizedin terms of an integrated multimedia system. Simply put, in such systems, everything can be presented with video and sound that is presented with text and graphics today [AGHg0]. For example, in conventional systems, a text messagecan be sent to other usersl but, a multimedia system with a high level of integration allows this function also for audio messages even for a combination of audio and text. or

2.2.5

Communication

Systems

Communication-capablemultimedia systems must be approached. A reason for this is that most of today's computers are interconnected; considering multimedia functions from only the local processingviewpoint would be a restriction, if not a step back. Another reason is that distributed environments enable particularly interesting multimedia applications. Here multimedia information cannot only be created, processed,presented and stored, but also distributed above the single computer,s

2.3. MULTIMEDIA boundary.

77

2.3

Multimedia

Considering the first explanation of multimedia at the beginning of this chapter, it is apparent that the notion is insufficient. We derive the following definition for multimedia from the American Heritage Dictionary definitions and the above with respectto the medium (Section2.1) and to the main properties considerations of a multimedia system (Section2.2): integratedproduction, A multimed,iasystem is characterizedby computer-controlled', 'inforrnation, mani,pulation,presentation, storage and commun'icationof independent and a discrete (timeat wh1chis encod,ed, leastthrough a continuous (tirne-depend,ent) medium. independ,ent) is Multi,med,ia very often used as an attribute of many systems, components, products, ideas, etc., without satisfying the above presentedcharacteristics.From this viewpoint our definition is deliberately restrictive. Thus, two notions of multimedia can be distinguished: o (Multimedia"o strictly speaking:

This notion was explained in Section 2.2 and will be used further. In this context, continuous media will always be included in a multimedia system. At the same time important timely marginal conditions (through the continuous media) for the processing of discrete media will be introduced' They have barely been consideredin computer use until now. o 66Multimediatt, in the broader sense: is Often the notion multimed,i,a used to describe the processing of individua'l and text, although no continuous medium is present. Many of the images processingtasks in this environment will also be necessaryin the multimedia system according to the restrictive definition. In any case, if more media are processedtogether, one can talk about multimedia according to this second notion.

18

CHAPTER 2. MULTIMEDIA:

MEDIA AND DATA STREAMS

2.4

Tbaditional Data Streams Characteristics

In Sections 2.I, 2.2,, and 2.3 we clarified the multimedia notion from the local computer-based point of view. But the presented work also includes the consideration of multimedia communication systems. Therefore, we need to specify the multimedia notion from the communication point of view. In distributed multimedia communication systems, data of discrete and continuous media are transmitted and information exchange takes place. Moreover, in each digital system, transmitted information is divided into indiuidual units (in general, these are packets) and subsequently sent away from one system component (the source) to another (the destination). The source and destination can be located either on the same computer or on different machines. A sequenceof individual packets transmitted in a time-dependent fashion is called a data stream (The term "data stream" will be used here as a synonym for "data flow".). Packets can cariy information of either continuous or discrete media. An example of a continuous media data stream is the transmission of speechin a telepironesystem. ttre retrievat of a document from a database can be seen as setting up a discrete media data stream. Transmissionof information carrying different media leads to data streamswith very different features. The attributes of asynchronous, synchronous, and. isochronous data transmission come from the fields of computer communication and r*itihing. fhey are also used, for example, in FDDI (Fiber Distributed Data Interface) networks for the description of different data transmission modes with respect to end-to-enddelay of individual packets(seeFigures 10.5 and 10.6).

2.4.1

Asynchronous

TYansmission Mode

The asynchronous transmission mode provides for communication with no timely restrictions. fg,cketq reach the receiver as fast as possible. Protocols of the worldwide Internet for electronic mail transmission are an example. In local area networks, Ethernet is i-fuither example. All information of discreie media can be transmitted u, un uryochronous data stream. Data of discrete me-

TRADITIONAL D ATA STREAMSCH ARACTERISTICS

19

dia can also include time restrictions through the timely connection to continuous transmissionmight not be ap-media synchronization.In this casean asynchronous propriui". tf an asynchronousmode is chosenfor transmission of continuous media, uaaltionut techniquesmust be applied to provide the time restrictions

2.4.2

Synchronous TYansmission Mode

The sgnchronous transmission mod,e defines a maximum end-to-end delay for each

packg!9j a data 'tl9?-.

b9 T!i9 "pp"l bound will nevgr ,Y,i-*t9d: U"lg-o'gr-packet iutr i"u.[ the iecei"ei at any arbitrary earlier.time. Thus, an important
claim of multimedia applicitions is siiish"d: guaranteed. a maximal end-to-end delay ca1 be

Additionally, an audio connection can be establishedover a local area network which supports synchronoustransmiision mode. The ui;omtieaseil iianitei of video ditb, in a retrieval mode is charicterized by a high dati iafe and-ieiiti"ely ttigh -'t"ii""t endlto-end delay. ttere ihe typical data rate is 140 Mbit/s and a maximal delay can be 1 second. In extreme casespackets arrive at the receiver 1 second too early and nu"_9 L" stored i;l;;"adi"iv. _to storageof about 17.5 MbYtes. t" oui example, a receiver would need a l:TPot.tv

2.4.3

Isochronous TYansmission Mode

The isochronous.transmission mode- defines, besides a maximum end-to-end delay for each packet of a data stream, a minimum end-to-end delay. This means that the deiay jitiei (in shoif,"ii;itter") of individual packets is bounded' yideo d3l1at ttre releiver, mentioned in the ng.-gtt?1l- t-tol?,ggo{= In this c-ase, !b--", elample, would be strongly reduced. These demands on intermediate s!3rage ab-ove along the data route between source(s) and must be consideredin all compot "oft sink(s).

20

CHAPTER 2. MULTIMEDIA:

MEDIA AND DATA STREAMS

2-5

Data stream characteristics for continuous Media

The following section describesdata stream characteristics that relate to any audio and video data transfer in a multimedia systems (multimed,ia d,ata streams).Moreover' we consider the effects of compressionon data stream characteristics during data transfer. These data stream characteristicsapply to distributed as well as local environments.

2.5.r

The Time rnterval Betweena complete Tbansmission con_ of secutive Packets

This first property relates to the time interval between a complete transmission of consecutivepackets. Based on ihe availability of paCkets,t; disiingui;n rmongtm following possibilities:

If the time interval between two consecutivepackets is constant, a datastream . ,:----is called strongly periodic. The1ef91e, an ideal case,jiiter rrii ir," ,rrd,",.;. in Figure 2.1 shows such a data stream. An exampr"'ir'FCvr.oa"a-tp"*r, i"

Figure2.7: stronrly y::r!r":t-ream, (T-time rimit between consecutiue two packets), i'e.,timeinteFffiEar'i:.;f7E,;am,eIengthbetwieb|itiioconsecuIiuepackets.
traditional telephone switching systems. t lltg duration of the time interval between two consecutive packets can be describedtlrough a periodical function with finite p"rioa, Urt ifr" ti-" irrt"rrii between consecutivepackets is not constant (otherwise it *ooia 6e a stiongly p"tiodi. data stream). Th; dita siieam is called weakly per:iodic. f urrs caseis shown in !'igure 2.2.

MEDIA FOR CHARACTERISTICS CONTII{UOUS 2.5. DATA STREAM

2I

packets Figure 2.2: Weakly peri'odicstream, i.e., time interuals betweenconsecutiue are of periodic nature.

e All olhgrp9g_!!rli!iel tl3"'qission *'-!l--'-9't9.-t.i'-!*q,tltervalareknown 9l

of Figure 2.3: Aperiod,icstream, i.e., the sequence time interaals is neither strongly nor weakly periodic. An example of an aperiodic data stream can be found in a cooperative application with shared windows. Very often, the status and actual coordinates of the user's mouse must be distributed among all participants of the multimedia conference. If this information is transmitted periodically, extremely high redundancy is present. Thus, given that an optimal system should transmit information only when necessary,after an initialization phase, data are exchangedonly when a change in position or status occurs'

2.5.2

Variation

of Consecutive

Packet Amount

A secondcharacteristic of data streams is the variation of the amount of consecutive packets. o If the amount of data stays constant during the lifetime of a data stream,

risutai. s"ih i datilt*tm l; tho*" i" ;;. ;;11rth; aJu ;tt"ffi ii,trongty digitil dataiir"'-isiio". TGuiJ.4. T-fi'-TJJt"iliJ tyt[:'t foi uiicdmpiessed

CHAPTER 2. MULTIMEDIA:

MEDIA AND DATA STREAMS ,t

lr

Dt

Figure 2.4: strongly regular stream, i.e., constant data size of all packets. Examples are the video stream taken from a camera in uncompressedform and the audio stream from an audio CD. If the amount of data varies periodically (with time), this is a weakly regular data,s,tr-eam.{g gxample of a weakly regular data gtream i; ; ;i,mpi-essa video stream which uses a compressionmethod as follows: individual images are.coded and compressed an individual,whole unit, which represents relas a atively large packet inside the data stream (bounded packet length of network transmissionis left out in this consideration.). Packetswill be periodically transmitted. e.g., every two seconds.Inbetweenthe two secondperiods, additional packets will be sent which include the information abouf the rlifference of the two consecutive compressed images. an 91?!1n! _gf-a-compression method which works similarly to the above description the MPEG compression is method (seesection 6.7). MpEc air felentiltgg among I, P and B images i" iompi"rr"d lrid"o stream. I-i-u,g", " representcompressed individual images,while p- and B-imagestake into account lmage differences. With this approach the data rate is reduced essentially. There is no constant bit rate for individual l, p, B compressed packets, but the I:B:P relation of the created data amount for every image is known (often used'oalue the I:B:P relation is 10:1:2for individual iinigbs-). Suih a of data stream can be characterized average on over a long time period as weakly regular(Figure 2.5). Data streams are irregulor if the amount of data is neither constant nor changes according to a periodic function (seeFigure 2.6). Transmission and processing

MEDIA FOR 2.5. DATASTREAMCHARACTERISTICS CONTIT\TUOUS

23

ry
D3

Dr

4
D3

periodically. Figure 2.5: WeaklAregular streanl, 'i.e.,d,atasize of the packets changes

r--;I
of Figure 2.6: Irregular d,atastrearn, i.e., data s?,ze the pacltets is nei'ther constanl periodi,cally. nor changing

method is in is more complicated this case.In the casewhen a compression the applied to the {at.a 9!_reg,m, data stream has-a t'-ariible6j! f"t31d1F" packet.(derived from aniliiyt9gd imagg)*i9 sizeof an individ.ual 49!g1q[9a from the content of the previouschangedimage. The size of the created therefore dependenton the information "t399jg09"*9 31{ffi?!9 "oit itstreamis irregular.

2.5.3

Contiguous Packets

continuity,or the connectionbetweenconsecutive A third property characterizes paciiietJtiaiismittea riiTectt!oii;ftei tnot6ei, or ii there packets.Are consecutive .t utilizatiott oi a certain-tyti". *uo betweenttre paitiefsf tfiis iun U" "u"r, suchas a network. resource,

CHAPTER 2. MULTIMEDIA:

MEDIA AND DATA STREAMS

o Figure 2,1 showsa connected information transfer. Al1 packetsare trans-

Figure 2.7: Continuousstream,i.e., the packetsare transmitted, without intermediate gapsmitted successively ll,th9ut a gap- Necessary additional information (e.g., system resource is 100% utilized. A connected data stieam allowJ'maximai data throughput u;d ;;;.ii"s optimat utiliiation of the syitu- i"r"fi". A B-channel of ISDN with trarsmission of 64 kbit/s dati is an example. 1lqi"

"tror

Coriroicodes) the dila-ii'.""Jia;i;d. Tn"this"cise, of t6"-6ndia;;"a

. T\9 through channel a with a higher !11ry35.io" 9.11.9119.t-"-d stream 1?1"

rl.,!:g-gtg_ ,I!9.-"y"1, i' t9-94-9yq ?,s, it-

capacity leads to gaps between individual packets. A d3.!a stieam yi1_trg_ap-g t.t*""" iniotmaiion-units is called un ur"orr,"cted'dai;a stream. An example ,;;

i{ 3! ",q!iqpo,rtant -g",pr,-u_TIl-e*g

Figure 2.8: Discrete stream, i.e., gaps erist among the packets. lf !!r-9!ulu!i"" of lhe gaps varies. For example, the transmission of data coded with the Jppc method, wiitr r.z Mlit/r thioughput on _a {l:lli average? will lead to gaps among individual packetson an FDDI network. In the following example, the properties described above should be made clear: an NTSC video signal is captured from a video camera and digitized in a computer, yet no compressionis done. The created data stream is strongly periodic, strongly packets ot

IIfFORMATIOI\I UNI?S
regular and connected, as shown in Figure 2.4. There ale no gaps among the packets. During the digitizing process, the DVISTV method for compression, using the ActionMedia IITMcard, is performed. The resulting data stteam (considered over a longer period of time) is now weakly periodic, weakly regular, and, through transmissionover a 16 Mbit/second Token Ring, unconnected.

2.6

Information Units

"bt u" Lou ii" u" ain"i""t'--

Continuous media consist of a time-dependent sequenceof individual information (LDU); *tric,[]J .tose Such an information unit is called a L,ogicnl ntata iii ""iii to a Protocol Data (Jnit (PDU). The meaning of the information and daiu u-outtt --

1. Considerfor example the symphony "The beaf'by JosephHaydn. It consists allegretto, minuef and fi,nale u'iuace.Each senof four sentences:uiuace assa'i, tence is an independent, self-containedpart of this composition, consisting of a sequenceof notes for different instruments. The notes are representedin a digital system as a sequenceof samples (no compressionis consideredin our per there are 44,100.samples second,which example.). With CD-DA qua-li!y-, are coded with 16 bits per sample. On a CD the samplesale grouped into units of 1/75 secondduration. One could talie as the LDU the whoie symphony, individual sen[bnces, individual notes, gi;up"d sami]leJ i/ZS seCbnd "f

durationoi just ildiyi4uSl l?-pl"':

determines r!3 narll9ul1rapplication

what is consideredto be the LDU. For example, applications using output functions of the whglg rymphony will take the whole symphony.,asthe iDU. Other applications use functions which consider the smallest meaningful units as samples the LDUs. (in our case,notes). A digital systemconsiders 2. An example of an uncompressedvideo sequenceconsisting of individual video clips, which present u rpe.ific ,."n","ir-riio*o i" fig"i" i'g. S".ft i.""" i. " of comprisedof a sequence images.An image can b" divided, tor exirnpteinto 16x16 groups of .pixels. E1ch pixel co:rgistsagain of 19min3,199*g$-=:!tgginance values. The image is therefore not the only possible LDU of a video sequence.A sceneor a pixel can also bein LDU. In ; nideileililnce,'Coaea

26

CHAPTER 2. MULTIMEDIA:

MEDIA AND DATA STREAMS

Film Clip Frame

Figure 2.9: Granularitg of a motion picture sequence. with MPEG or DVI, existent -redundancies can be used through applying an interframe compressionmethod. The smallest self-containedmeaningful units here are imaqe sequences. The notion of granularftgrcharacterizes the hierarchical division of audio or video datq qtrypms.!4tg lheir componenti. in our examples, the most geneiu,fna--ilind best-known information units are the symphony and the movie. Yet there exists also another ciassification LDU with respect to duration. ClosedLDIIs have a of pre-iefined duration. An example of such an LDU stieam is i data stream of audio samplesin the computer. If the duration is not known in advance,we encounter an open LDU. An example of such an LDU stream is a data stream sent fiom a iu,mera or microphoneto the computer.