Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
77 views

Theory of Computing: MAS 714 Hartmut Klauck

This document provides an overview of the Theory of Computing course. It discusses the following key points: - The course will cover algorithms, data structures, complexity theory, computability, formal languages, and other topics in the first and second halves. - Algorithms are analyzed based on their time and space complexity, usually using asymptotic analysis. Common models of computation include Turing machines and random access machines (RAM). - Examples of algorithms discussed include addition, multiplication, and Gaussian elimination. Data structures like arrays are also introduced. Complexity classes like P, NP, and more will be covered later in the course.

Uploaded by

PunithRossi
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
77 views

Theory of Computing: MAS 714 Hartmut Klauck

This document provides an overview of the Theory of Computing course. It discusses the following key points: - The course will cover algorithms, data structures, complexity theory, computability, formal languages, and other topics in the first and second halves. - Algorithms are analyzed based on their time and space complexity, usually using asymptotic analysis. Common models of computation include Turing machines and random access machines (RAM). - Examples of algorithms discussed include addition, multiplication, and Gaussian elimination. Data structures like arrays are also introduced. Complexity classes like P, NP, and more will be covered later in the course.

Uploaded by

PunithRossi
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 27

Theory of Computing

Lecture 1
MAS 714
Hartmut Klauck
Organization:
Lectures:
Tue 9:30-11:30 TR+20
Fr 9:30-10:30
Tutorial:
Fr:10:30-11:30
Exceptions: This week no tutorial, next
Tuesday no lecture
Organization
Final: 60%, Midterm: 20%, Homework: 20%

Homework 1 out on Aug. 15, due on Aug. 15

http://www.ntu.edu.sg/home/hklauck/MAS714.htm
Books:
Cormen Leiserson Rivest Stein: Introduction to
Algorithms
Sipser: Introduction to the Theory of
Computation
Arora Barak: Computational Complexity- A
Modern Approach
Computation
Computation: Mapping inputs to outputs in a
prescribed way, by small, easy steps

Example: Division
Div(a,b)=c such that bc=a
How to find c?
School method
Outline: Theory of Computing

ALGORITHMS
CRYPTOGRAPHY

DATA STRUCTURES COMPLEXITY

MACHINE MODELS
UNCOMPUTABLE
FORMAL LANGUAGES
UNIVERSALITY

PROOF SYSTEMS
This course
First half: algorithms and data-structures
Second half: Some complexity, computability,
formal languages
NP-completeness
Computable and uncomputable problems
Time-hierarchy
Automata and Circuits
Algorithms
An algorithm is a procedure for performing a
computation
Algorithms consist of elementary
steps/instructions
Elementary steps depend on the model of
computation
Example: C++ commands
Algorithms: Example
Gaussian Elimination
Input: Matrix M
Output: Triangular Matrix that is row-equivalent to
M
Elementary operations: row operations
swap, scale, add
Algorithms
Algorithms are named after Al-Khwrizm
(Ab Abdallh Muammad ibn Ms al-Khwrizm)
c. 780-850 ce
Persian mathematician and astronomer
(Algebra is also named after his work)
His works brought the positional system of
numbers to the attention of Europeans
Algorithms: Example
Addition via the school method:
Write numbers under each other
Add number position by position moving a carry
forward
Elementary operations:
Add two numbers between 0 and 9
(memorized)
Read and Write
Can deal with arbitrarily long numbers!
Datastructure
The addition algorithm uses (implicitly) an
array as datastructure
An array is a fixed length vector of cells that can
each store a number/digit
Note that when we add x and y then x+y is at most
1 digit longer than max{x,y}
So the result can be stored in an array of length
n+1
Multiplication
The school multiplication algorithm is an
example of a reduction
First we learn how to add n numbers with n
digits each
To multiply x and y we generate n numbers
xi y 2i and add them up
Reduction from Multiplication to Addition
Complexity
We usually analyze algorithms to grade the
performance
The most important (but not the only)
parameters are time and space
Time refers to the number of elementary steps
Space refers to the storage needed during the
computation
Example: Addition
Assume we add two numbers x,y with n
decimal digits
Clearly the number of elementary steps
(adding digits etc) grows linearly with n
Space is also n

Typical: asymptotic analysis


Example: Multiplication
We generate n numbers with at most 2n digits, add
them
Number of steps is O(n2)
Space is O(n2)

Much faster algorithms exist


(but not easy to do with pen and paper)
Question: Is multiplication harder than addition?
Answer: we dont know...
Our Model of Computation
We could use Turing machines...
Will consider algorithms in a richer model, a
RAM
RAM:
random access machine

Basically we will just use pseudocode/informal


language
RAM
Random Access Machine
Storage is made of registers that can hold a number
(an unlimited amount of registers is available)
The machine is controlled by a finite program
Instructions are from a finite set that includes
Fetching data from a register into a special register
Arithmetic operations on registers
Writing into a register
Indirect addressing
RAM
Random Access Machines and Turing
Machines can simulate each other
There is a universal RAM
RAMs are very similar to actual computers
machine language
Computation Costs
The time cost of a RAM step involving a register
is the logarithm of the number stored
logarithmic cost measure
adding numbers with n bits takes time n etc.
The time cost of a RAM program is the sum of
the time costs of its steps
Space is the sum (over all registers ever used) of
the logarithms of the maximum numbers stored
in the register
Other Machine models
Turing Machines (we will define them later)
Circuits
Many more!

A machine model is universal, if it can simulate


any computation of a Turing machine
RAMs are universal
Vice versa, Turing machines can simulate RAMs
Types of Analysis
Usually we will use asymptotic analysis
Reason: next years computer will be faster, so constant
factors dont matter (usually)
Understand the inherent complexity of a problem (can
you multiply in linear time?)
Usually gives the right answer in practice
Worst case
The running time of an algorithm is the maximum time
over all inputs
On the safe side for all inputs
Types of Analysis
Average case:
Average under which distribution?
Often the uniform distribution, but may be unrealistic
Amortized analysis
Sometimes after some costly preparations we can
solve many problem instances cheaply
Count the average cost of an instance (preparation
costs are spread between instances)
Often used in datastructure analysis
Asymptotic Analysis
Different models lead to slightly different
running times
E.g. depending on the instruction set
Also computers become faster through faster
processors
Same sequence of operations performed faster
Therefore we generally are not interested in
constant factors in the running time
Unless they are obscene
O, ,
Let f,g be two monotone increasing functions
that send N to R+
f=O(g) if 9 n0,c 8 n>n0: f(n) c g(n)
Example:
f(n)=n, g(n)=1000n+100 ) g(n)=O(f(n))
Set c=1001 and n0=100
Example:
f(n)=n log n, g(n)=n2
O, ,
Let f,g be two monotone increasing functions
that send N to R+
f = (g) iff g=O(f)
Definition by Knuth
f = (g) iff [ f=O(g) and g=O(f) ]
o, !: asymptotically smaller/larger
E.g., n=o(n2)
But 2n2 + 100 n=(n2)
Some functions

2n+10
n2/500
n log(n)/2

You might also like