Complexity Theory 1 Complexity Theory 2

Formalising Algorithms
Complexity Theory
Lecture 2 To prove a lower bound on the complexity of a problem, rather
than a specific algorithm, we need to prove a statement about all
algorithms for solving it.
In order to prove facts about all algorithms, we need a
mathematically precise definition of algorithm.

Anuj Dawar We will use the Turing machine.

The simplicity of the Turing machine means it’s not useful
University of Cambridge Computer Laboratory for actually expressing algorithms, but very well suited for
Easter Term 2011 proofs about all algorithms.

http://www.cl.cam.ac.uk/teaching/1011/Complexity/

Anuj Dawar May 4, 2011 Anuj Dawar May 4, 2011

Complexity Theory 3 Complexity Theory 4

Turing Machines Configurations

For our purposes, a Turing Machine consists of: A complete description of the configuration of a machine can be
given if we know what state it is in, what are the contents of its
• Q — a finite set of states;
tape, and what is the position of its head. This can be summed up
• Σ — a finite set of symbols, including ⊔. in a simple triple:
• s ∈ Q — an initial state; Definition
• δ : (Q × Σ) → (Q ∪ {a, r}) × Σ × {L, R, S} A configuration is a triple (q, w, u), where q ∈ Q and w, u ∈ Σ⋆

A transition function that specifies, for each state and symbol a

next state (or accept acc or reject rej), a symbol to overwrite The intuition is that (q, w, u) represents a machine in state q with
the current symbol, and a direction for the tape head to move the string wu on its tape, and the head pointing at the last symbol
(L – left, R – right, or S - stationary) in w.

The configuration of a machine completely determines the future

behaviour of the machine.

Anuj Dawar May 4, 2011 Anuj Dawar May 4, 2011

Complexity Theory 5 Complexity Theory 6

Computations Computations

Given a machine M = (Q, Σ, s, δ) we say that a configuration The relation →⋆M is the reflexive and transitive closure of →M .
(q, w, u) yields in one step (q ′ , w′ , u′ ), written A sequence of configurations c1 , . . . , cn , where for each i,
ci →M ci+1 , is called a computation of M .
(q, w, u) →M (q ′ , w′ , u′ )
The language L(M ) ⊆ Σ⋆ accepted by the machine M is the set of
if strings
• w = va ;
{x | (s, ⊲, x) →⋆M (acc, w, u) for some w and u}
′
• δ(q, a) = (q , b, D); and
• either D = L and w′ = v u′ = bu
or D = S and w′ = vb and u′ = u
A machine M is said to halt on input x if for some w and u, either
or D = R and w′ = vbc and u′ = x, where u = cx. If u is
(s, ⊲, x) →⋆M (acc, w, u) or (s, ⊲, x) →⋆M (rej, w, u)
empty, then w′ = vb⊔ and u′ is empty.

Anuj Dawar May 4, 2011 Anuj Dawar May 4, 2011

Complexity Theory 7 Complexity Theory 8

Decidability Example

A language L ⊆ Σ⋆ is recursively enumerable if it is L(M ) for some Consider the machine with δ given by:
M.
⊲ 0 1 ⊔
A language L is decidable if it is L(M ) for some machine M which
halts on every input. s s, ⊲, R rej, 0, S rej, 1, S q, ⊔, R
q rej, ⊲, R q, 1, R q, 1, R q ′ , 0, R
A language L is semi-decidable if it is recursively enumerable.
q′ rej, ⊲, R rej, 0, S q ′ , 1, L acc, ⊔, S

A function f : Σ⋆ → Σ⋆ is computable, if there is a machine M ,

such that for all x, (s, ⊲, x) →⋆M (acc, f (x), ε)
This machine, when started in configuration (s, ⊲, ⊔1n 0) eventually
halts in configuration (acc, ⊲⊔, 1n+1 , 0).

Anuj Dawar May 4, 2011 Anuj Dawar May 4, 2011

Complexity Theory 9 Complexity Theory 10

Multi-Tape Machines Running Time

The formalisation of Turing machines extends in a natural way to With any Turing machine M , we associate a function rIN → IN
multi-tape machines. For instance a machine with k tapes is called the running time if M .
specified by: r(n) is defined to be the largest value R such that there is a string
• Q, Σ, s; and x of length n so that the computation of M starting with
configuration (s, ⊲, x) is of length R (i.e. has R successive
• δ : (Q × Σk ) → Q ∪ {a, r} × (Σ × {L, R, S})k
configurations in it) and ends with an accepting configuration.

Similarly, a configuration is of the form:

In short, r(n) is the length of the longest accepting computation of
M on an input of length n.
(q, w1 , u1 , . . . , wk , uk )

Anuj Dawar May 4, 2011 Anuj Dawar May 4, 2011

Complexity Theory 11 Complexity Theory 12

Complexity Nondeterminism

For any function f : IN → IN, we say that a language L is in If, in the definition of a Turing machine, we relax the condition on
TIME(f (n)) if there is a machine M = (Q, Σ, s, δ), such that: δ being a function and instead allow an arbitrary relation, we
obtain a nondeterministic Turing machine.
• L = L(M ); and
• The running time of M is O(f (n)). δ ⊆ (Q × Σ) × (Q ∪ {a, r} × Σ × {R, L, S}).

Similarly, we define SPACE(f (n)) to be the languages accepted by a

The yields relation →M is also no longer functional.
machine which uses O(f (n)) tape cells on inputs of length n.
In defining space complexity, we assume a machine M , which has a We still define the language accepted by M by:
read-only input tape, and a separate work tape. We only count
cells on the work tape towards the complexity. {x | (s, ⊲, x) →⋆M (acc, w, u) for some w and u}

though, for some x, there may be computations leading to

accepting as well as rejecting states.

Anuj Dawar May 4, 2011 Anuj Dawar May 4, 2011

Complexity Theory 13 Complexity Theory 14

Computation Trees Decidability and Complexity

For every decidable language L, there is a computable function f

With a nondeterministic machine, each configuration gives rise to a such that
tree of successive configurations.
(s, ⊲, x) L ∈ TIME(f (n))
(q0 , u0 , w0 )(q1 , u1 , w1 )(q2 , u2 , w2 )

(q00 , u00 , w00 ) (rej, u2 , w2 )

(q10 , u10 , w10 )

.
(q11 , u11 , w11 )
. If L is a semi-decidable (but not decidable) language accepted by
. .
. .
M , then there is no computable function f such that every
(acc, . . .)
accepting computation of M , on input of length n is of length at
most f (n).

Anuj Dawar May 4, 2011 Anuj Dawar May 4, 2011