Chapter 3

Arithmetic for Computers

3.1 Intro oduction

Arithmetic for Computers

Operations on integers

Addition and subtraction Multiplication and division Dealing with overflow Representation and operations

Floating-point real numbers

Chapter 3 Arithmetic for Computers 2

3.2 Addition and Subtractio on

Integer Addition

Example: 7 + 6

Overflow if result out of range

Adding +ve and ve operands, no overflow Adding two +ve ve operands

Overflow if result sign is 1 Overflow if result sign is 0

Chapter 3 Arithmetic for Computers 3

Adding two ve operands

Integer Subtraction

Add negation of second operand Example: 7 6 = 7 + (6) ( 6)

+7: 6: +1: 0000 0000 0000 0111 1111 1111 1111 1010 0000 0000 0000 0001

Overflow if result out of range

Subtracting two +ve or two ve operands, no overflow Subtracting +ve from ve operand

Overflow if result sign is 0 Overflow if result sign is 1

Subtracting ve from +ve operand

Chapter 3 Arithmetic for Computers 4

Dealing with Overflow

Some languages (e.g., C) ignore overflow

Use MIPS addu, addui, subu instructions

Other languages (e.g., Ada, Fortran) require raising an exception

Use MIPS add, addi, sub instructions On overflow, O o e o , invoke o ee exception cep o handler a de

Save PC in exception program counter (EPC) register J Jump t predefined to d fi d h handler dl address dd mfc0 (move from coprocessor reg) instruction can retrieve EPC value, to return after corrective action
Chapter 3 Arithmetic for Computers 5

Arithmetic for Multimedia

Graphics and media processing operates on vectors of 8-bit and 16-bit data

Use 64-bit adder, with partitioned carry chain

Operate on 88-bit 88-bit, 416-bit 416-bit, or 232-bit vectors

SIMD (single-instruction, multiple-data) On overflow, result is largest representable value

Saturating operations

c.f. 2s-complement modulo arithmetic

E g clipping in audio E.g., audio, saturation in video

Chapter 3 Arithmetic for Computers 6

3.3 Mult tiplication

Multiplication

Start with long-multiplication approach

1000 1001 1000 0000 0000 1000 1001000

multiplicand multiplier

product

Length of product is the sum of operand lengths

Chapter 3 Arithmetic for Computers 7

Multiplication Hardware

Initially y0

Chapter 3 Arithmetic for Computers 8

Optimized Multiplier

Perform steps in parallel: add/shift

One cycle per partial-product addition

Thats That s ok, ok if frequency of multiplications is low

Chapter 3 Arithmetic for Computers 9

Faster Multiplier

Uses multiple adders

Cost/performance tradeoff

Can be pipelined

S Several l multiplication lti li ti performed f di in parallel ll l

Chapter 3 Arithmetic for Computers 10

MIPS Multiplication

Two 32-bit registers for product

HI: most-significant most significant 32 bits LO: least-significant 32-bits mult rs, rt

Instructions

multu rs, rt

64-bit product in HI/LO

mfhi f rd

mflo f rd

Move from HI/LO to rd Can test HI value to see if product overflows 32 bits Least-significant 32 bits of product > rd

mul rd, rs, rt

Chapter 3 Arithmetic for Computers 11

3.4 Division

Division

Check for 0 divisor Long division approach

quotient ti t dividend

If divisor dividend bits

1 bit in quotient, subtract 0 bit in quotient, bring down next dividend bit

1001 1000 1001010 -1000 divisor 10 101 1010 -1000 10 remainder i d

n-bit operands yield n-bit quotient and remainder

Otherwise

Restoring division

Do the subtract, and if remainder goes < 0, add divisor back Divide using absolute values Adjust sign of quotient and remainder as required

Signed division

Chapter 3 Arithmetic for Computers 12

Division Hardware
Initially divisor in left half

Initially dividend

Chapter 3 Arithmetic for Computers 13

Optimized Divider

One cycle per partial-remainder subtraction Looks a lot like a multiplier!

Same hardware can be used for both

Chapter 3 Arithmetic for Computers 14

Faster Division

Cant use parallel hardware as in multiplier

Subtraction is conditional on sign of remainder

Faster dividers (e.g. SRT devision) generate multiple m ltiple q quotient otient bits per step

Still require multiple steps

Chapter 3 Arithmetic for Computers 15

MIPS Division

Use HI/LO registers for result

HI: 32-bit 32 bit remainder LO: 32-bit quotient div rs, rt / divu rs, rt No overflow or divide-by-0 checking

I t ti Instructions

Software must perform checks if required

Use mfhi fhi, mflo fl to access result

Chapter 3 Arithmetic for Computers 16

3.5 Floa ating Point t

Floating Point

Representation for non-integral numbers

Including very small and very large numbers 2.34 2 34 1056 +0.002 104 +987.02 109 1.xxxxxxx2 2yyyy
normalized not normalized

Like scientific notation

In binary

Types ypes float oat a and d doub double e in C

Chapter 3 Arithmetic for Computers 17

Floating Point Standard

Defined by IEEE Std 754-1985 Developed in response to divergence of representations

P t bilit i Portability issues f for scientific i tifi code d

Now almost universally adopted Two representations

Single g p precision ( (32-bit) ) Double precision (64-bit)

Chapter 3 Arithmetic for Computers 18

IEEE Floating-Point Format

single: 8 bits double: 11 bits single: 23 bits double: 52 bits

S Exponent

Fraction

p Bias) ) x = ( 1)S (1+ Fraction) 2((Exponent

S: sign bit (0 non-negative, 1 negative) Normalize significand: 1.0 |significand| < 2.0

Always has a leading pre-binary-point 1 bit, so no need to represent it explicitly (hidden bit) Si ifi Significand d is i Fraction F ti with ith th the 1 1. restored t d Ensures exponent is unsigned Si l Bi Single: Bias = 127 127; D Double: bl Bi Bias = 1203
Chapter 3 Arithmetic for Computers 19

Exponent: excess representation: actual exponent + Bias

Single-Precision Range

Exponents 00000000 and 11111111 reserved Smallest value

Exponent: 00000001 actual exponent = 1 127 = 126 Fraction: 00000 significand = 1.0 1.0 2126 1.2 1038 exponent: 11111110 actual exponent = 254 127 = +127 Fraction: 11111 significand 2.0 2 0 2+127 3.4 2.0 3 4 10+38
Chapter 3 Arithmetic for Computers 20

Largest value

Double-Precision Range

Exponents 000000 and 111111 reserved Smallest value

Exponent: 00000000001 actual exponent = 1 1023 = 1022 Fraction: 00000 significand = 1.0 1.0 21022 2.2 10308 Exponent: 11111111110 actual exponent = 2046 1023 = +1023 Fraction: 11111 significand 2.0 2 0 2+1023 1.8 2.0 1 8 10+308
Chapter 3 Arithmetic for Computers 21

Largest value

Floating-Point Precision

Relative precision

all fraction bits are significant Single: approx 223

Equivalent to 23 log102 23 0.3 0 3 6 decimal digits of precision Equivalent to 52 log102 52 0.3 16 decimal digits of precision

Double: approx 252

Chapter 3 Arithmetic for Computers 22

Floating-Point Example

Represent 0.75

1 0.75 0 75 = (1) ( 1)1 1.1 1 12 21 S=1 Fraction = 100000 1000 002 Exponent = 1 + Bias

Single: 1 Single 1 + 127 = 126 = 011111102 Double: 1 + 1023 = 1022 = 011111111102

Single: 101111110100000 1011111101000 00 Double: 101111111110100000

Chapter 3 Arithmetic for Computers 23

Floating-Point Example

What number is represented by the singleprecision p ec s o float oat 1100000010100000

S=1 Fraction = 01000002 Fxponent po e = 10000001 000000 2 = 129 9 =( (1) 1) 1.25 22 = 5.0

x = (1)1 (1 + 012) 2(129 127)

Chapter 3 Arithmetic for Computers 24

Denormal Numbers

Exponent = 000...0 hidden bit is 0

x = ( 1)S (0 + Fraction) 2 Bias

Smaller than normal numbers

allow for gradual underflow, with diminishing precision

Denormal with fraction = 000...0

x = ( 1)S (0 + 0) 2Bias = 0.0
Two representations p of 0.0!
Chapter 3 Arithmetic for Computers 25

Infinities and NaNs

Exponent = 111...1, Fraction = 000...0

Infinity Can be used in subsequent calculations, avoiding need for overflow check Not-a-Number N t N b (N (NaN) N) Indicates illegal or undefined result

Exponent = 111...1, Fraction 000...0

e.g., 0.0 0 0 / 0.0 00

Can be used in subsequent calculations

Chapter 3 Arithmetic for Computers 26

Floating-Point Addition

Consider a 4-digit decimal example

9.999 101 + 1.610 101 Shift number with smaller exponent 9.999 101 + 0.016 101 9 999 101 + 0.016 9.999 0 016 101 = 10 10.015 015 101 1.0015 102 1.002 102
Chapter 3 Arithmetic for Computers 27

1. Align decimal points

2. Add significands

3. Normalize result & check for over/underflow

4. Round and renormalize if necessary

Floating-Point Addition

Now consider a 4-digit binary example

1.0002 21 + 1.1102 22 (0.5 + 0.4375) Shift number with smaller exponent 1 + 0.111 21 1 1.0002 21 2
1 + 0.111 1 1 0002 21 1.000 0 1112 21 = 0 0.001 0012 21

1. Align binary points

2. Add significands

3. Normalize result & check for over/underflow

1.0002 24, with no over/underflow 1.0002 24 (no change) = 0.0625

Chapter 3 Arithmetic for Computers 28

4. Round and renormalize if necessary

FP Adder Hardware

Much more complex than integer adder Doing it in one clock cycle would take too long

Much M hl longer th than i integer t operations ti Slower clock would penalize all instructions Can be pipelined

FP adder usually takes several cycles

Chapter 3 Arithmetic for Computers 29

FP Adder Hardware

Step 1

Step 2

Step 3 Step 4

Chapter 3 Arithmetic for Computers 30

Floating-Point Multiplication

Consider a 4-digit decimal example

1.110 1010 9.200 105 For biased exponents, subtract bias from sum New exponent = 10 + 5 = 5 1.110 9.200 = 10.212 10.212 105 1.0212 106 1 021 106 1.021 +1.021 106

1 Add exponents 1. t

2. Multiply significands

3 Normalize result & check for over/underflow 3.

4. Round and renormalize if necessary

5. Determine sign of result from signs of operands

Chapter 3 Arithmetic for Computers 31

Floating-Point Multiplication

Now consider a 4-digit binary example

1.0002 21 1.1102 22 (0.5 0.4375) Unbiased: 1 + 2 = 3 Biased: (1 + 127) + (2 + 127) = 3 + 254 127 = 3 + 127 1.0002 1.1102 = 1.1102 1.1102 23 1.1102 23 (no change) with no over/underflow
3 (no 1 1102 23 1.110 ( change) h )

1 Add exponents 1. t

2. Multiply significands

3 Normalize result & check for over/underflow 3.

4. Round and renormalize if necessary

5. Determine sign: +ve ve ve

1.1102 23 = 0.21875
Chapter 3 Arithmetic for Computers 32

FP Arithmetic Hardware

FP multiplier is of similar complexity to FP adde adder

But uses a multiplier for significands instead of an adder Addition, subtraction, multiplication, division, reciprocal, square-root FP integer conversion Can be pipelined
Chapter 3 Arithmetic for Computers 33

FP arithmetic hardware usually does

Operations usually takes several cycles

FP Instructions in MIPS

FP hardware is coprocessor 1

Adjunct processor that extends the ISA 32 single-precision: $f0, $f1, $f31 Paired for double-precision: double precision: $f0/$f1, $f2/$f3,

Separate FP registers

Release 2 of MIPs ISA supports 32 64-bit FP regs

FP instructions operate only on FP registers

Programs generally dont don t do integer ops on FP data data, or vice versa More registers with minimal code-size impact lwc1, ldc1, swc1, sdc1

FP load l d and d store t instructions i t ti

e.g., g ldc1 $f8, 32($sp) p

Chapter 3 Arithmetic for Computers 34

FP Instructions in MIPS

Single-precision arithmetic

add.s, sub.s, mul.s, div.s

e.g., add.s dd $f0, $f0 $f1, $f1 $f6

Double-precision arithmetic

add.d, sub.d, mul.d, div.d

e.g., mul.d $f4, $f4, $f6

Single- and double-precision comparison

c.xx.s c s, c. c xx.d d (xx is eq, lt, le, ) ) Sets or clears FP condition-code bit

e.g. c.lt.s $f3, $f4

B Branch h on FP condition diti code d t true or f false l

bc1t, bc1f

e.g., bc1t TargetLabel

Chapter 3 Arithmetic for Computers 35

FP Example: F to C

C code:
float f2c (float fahr) { return ((5.0/9.0)*(fahr - 32.0)); } fahr in $f12, result in $f0, literals in global memory space

Compiled MIPS code:

f2 f2c: lwc1 l 1 lwc2 div.s l lwc1 sub.s mul.s jr $f16, $f16 $f18, $f16, $f18, $f $f18, $f0, $ra const5($gp) ($ ) const9($gp) $f16, $f18 const32($gp) ($ ) $f12, $f18 $f16, $f18
Chapter 3 Arithmetic for Computers 36

FP Example: Array Multiplication

X=X+YZ

All 32 32 matrices, 64-bit double-precision elements

C code:
void mm (double x[][], double y[][] y[][], double z[][]) { int i, j, k; for (i = 0; i! = 32; i = i + 1) for (j = 0; j! = 32; j = j + 1) for (k = 0; k! = 32; k = k + 1) x[i][j] = x[i][j] + y[i][k] * z[k][j]; } Addresses of x, y, z in $a0, $a1, $a2, and i, j, k in $s0, $s1, $s2
Chapter 3 Arithmetic for Computers 37

FP Example: Array Multiplication

MIPS code:
$t1, 32 $s0, 0 $s1, 0 $s2, 0 $t2, $ 2 $s0, $ 0 5 $t2, $t2, $s1 $t2, $t2, 3 $t2 $t2, $a0, $a0 $t2 $f4, 0($t2) $t0, $s2, 5 $t0, $t0, $s1 $t0, $t0, 3 $t0, $a2, $t0 $f16, 0($t0) # # # # # # # # # # # # # # $t1 = 32 (row size/loop end) i = 0; initialize 1st for loop j = 0; restart 2nd for loop k = 0; restart 3rd for loop $t2 $ 2 = i * 32 (size ( i of f row of f x) ) $t2 = i * size(row) + j $t2 = byte offset of [i][j] $t2 = byte address of x[i][j] $f4 = 8 bytes of x[i][j] $t0 = k * 32 (size of row of z) $t0 = k * size(row) + j $t0 = byte offset of [k][j] $t0 = byte address of z[k][j] $f16 = 8 bytes of z[k][j]

li li L1: li L2: li sll ll addu sll addu l.d L3: sll addu sll addu l.d

Chapter 3 Arithmetic for Computers 38

FP Example: Array Multiplication

sll $t0, $s0, 5 addu $t0 $t0, $t0, $t0 $s2 sll $t0, $t0, 3 addu $t0, $a1, $t0 l.d $f18, , 0($t0) ( ) mul.d $f16, $f18, $f16 add.d $f4, $f4, $f16 addiu $s2, $s2, 1 bne $s2, $t1, L3 s.d $f4, 0($t2) addiu $s1, $s1, 1 bne $s1 $s1, $t1, $t1 L2 addiu $s0, $s0, 1 bne $s0, $t1, L1 # # # # # # # # # # # # # # $t0 = i*32 (size of row of y) $t0 = i i*size(row) size(row) + k $t0 = byte offset of [i][k] $t0 = byte address of y[i][k] $f18 = 8 bytes y of y[i][k] y[ ][ ] $f16 = y[i][k] * z[k][j] f4=x[i][j] + y[i][k]*z[k][j] $k k + 1 if (k != 32) go to L3 x[i][j] = $f4 $j = j + 1 if (j != ! 32) go to L2 $i = i + 1 if (i != 32) go to L1

Chapter 3 Arithmetic for Computers 39

Accurate Arithmetic

IEEE Std 754 specifies additional rounding control

Extra bits of precision (guard, round, sticky) Choice of rounding modes Allows programmer to fine fine-tune tune numerical behavior of a computation Most programming languages and FP libraries just use defaults

Not all FP units implement p all options p

Trade-off Trade off between hardware complexity complexity, performance, and market requirements

Chapter 3 Arithmetic for Computers 40

Interpretation of Data
The BIG Picture

Bits have no inherent meaning

Interpretation p depends p on the instructions applied Finite range and precision Need to account for this in programs

Computer p representations p of numbers

Chapter 3 Arithmetic for Computers 41

3.6 Para allelism an nd Compu uter Arithm metic: Asso ociativity

Associativity

Parallel programs may interleave operations in unexpected orders

Assumptions of associativity may fail

(x+y)+z x -1.50E+38 y 1.50E+38 1 50E+38 0 0.00E+00 00E+00 z 1.0 1.0 1.50E+38 1.00E+00 0.00E+00 x+(y+z) -1.50E+38

Need to validate parallel programs under varying degrees of parallelism

Chapter 3 Arithmetic for Computers 42

3.7 Rea al Stuff: Flo oating Point in the x x86

x86 FP Architecture

Originally based on 8087 FP coprocessor

8 80 80-bit bit extended-precision extended precision registers Used as a push-down stack Registers indexed from TOS: ST(0), ST(1), Converted on load/store of memory operand Integer operands can also be converted on load/store Result: poor FP performance

FP values are 32-bit or 64 in memory

V Very diffi difficult lt t to generate t and d optimize ti i code d

Chapter 3 Arithmetic for Computers 43

x86 FP Instructions
Data transfer
FILD mem/ST(i) FISTP mem/ST(i) FLDPI FLD1 FLDZ

Arithmetic
FIADDP FISUBRP FIMULP FIDIVRP FSQRT FABS FRNDINT mem/ST(i) mem/ST(i) mem/ST(i) mem/ST(i)

Compare
FICOMP FIUCOMP FSTSW AX/mem

Transcendental
FPATAN F2XMI FCOS FPTAN FPREM FPSIN FYL2X

Optional variations

I: integer g operand p P: pop operand from stack R: reverse operand order But not all combinations allowed
Chapter 3 Arithmetic for Computers 44

Streaming SIMD Extension 2 (SSE2)

Adds 4 128-bit registers

Extended to 8 registers in AMD64/EM64T 2 64-bit 64 bi d double bl precision i i 4 32-bit double precision Instructions operate on them simultaneously

Can be used for multiple FP operands

Single-Instruction Multiple-Data

Chapter 3 Arithmetic for Computers 45

3.8 Falla acies and Pitfalls

Right Shift and Division

Left shift by i places multiplies an integer by 2i Right shift divides by 2i?

Only for unsigned integers Arithmetic right shift: replicate the sign bit e.g., 5 / 4

For signed integers

111110112 >> 2 = 111111102 = 2 Rounds toward

c.f. 111110112 >>> 2 = 001111102 = +62

Chapter 3 Arithmetic for Computers 46

Who Cares About FP Accuracy?

Important for scientific code

But for everyday consumer use?

My bank balance is out by 0.0002! /

The Intel Pentium FDIV bug

The market expects accuracy S C See Colwell, l ll The Th Pentium P ti Chronicles Ch i l

Chapter 3 Arithmetic for Computers 47

3.9 Con ncluding R Remarks

Concluding Remarks

ISAs support arithmetic

Signed and unsigned integers Floating-point approximation to reals Operations can overflow and underflow Core instructions: 54 most frequently used

B Bounded d d range and d precision i i

MIPS ISA

100% of SPECINT, 97% of SPECFP

Other instructions: less frequent

Chapter 3 Arithmetic for Computers 48