cp1 K2opt PDF
cp1 K2opt PDF
cp1 K2opt PDF
Foreword
vi
Preface
vii
Authors Proles
viii
Convention
ix
Abbreviations
List of Tables
xi
List of Figures
xii
1 Introduction
1.1 Competitive Programming . . . . . . . . . . . .
1.2 Tips to be Competitive . . . . . . . . . . . . .
1.2.1 Tip 1: Quickly Identify Problem Types
1.2.2 Tip 2: Do Algorithm Analysis . . . . . .
1.2.3 Tip 3: Master Programming Languages
1.2.4 Tip 4: Master the Art of Testing Code .
1.2.5 Tip 5: Practice and More Practice . . .
1.3 Getting Started: Ad Hoc Problems . . . . . . .
1.4 Chapter Notes . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
. 1
. 2
. 4
. 5
. 7
. 9
. 10
. 11
. 13
. . .
. . .
. . .
Hash
. . .
. . .
. . .
. . .
. . .
. . . .
. . . .
. . . .
Table)
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
14
14
15
15
16
18
18
19
22
25
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
26
26
27
29
32
32
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
Binary Search
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
3.3
3.4
3.5
Greedy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.1 Classical Example . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.2 Non Classical Example . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.3 Remarks About Greedy Algorithm in Programming Contests . . .
Dynamic Programming . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4.1 DP Illustration . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4.2 Several Classical DP Examples . . . . . . . . . . . . . . . . . . . .
3.4.3 Non Classical Examples . . . . . . . . . . . . . . . . . . . . . . . .
3.4.4 Remarks About Dynamic Programming in Programming Contests
Chapter Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4 Graph
4.1 Overview and Motivation . . . . . . . . . . . . . .
4.2 Depth First Search . . . . . . . . . . . . . . . . . .
4.3 Breadth First Search . . . . . . . . . . . . . . . . .
4.4 Kruskals . . . . . . . . . . . . . . . . . . . . . . .
4.5 Dijkstras . . . . . . . . . . . . . . . . . . . . . . .
4.6 Bellman Fords . . . . . . . . . . . . . . . . . . . .
4.7 Floyd Warshalls . . . . . . . . . . . . . . . . . . .
4.8 Edmonds Karps (excluded in IOI syllabus) . . . .
4.9 Special Graphs . . . . . . . . . . . . . . . . . . . .
4.9.1 Tree . . . . . . . . . . . . . . . . . . . . . .
4.9.2 Directed Acyclic Graph . . . . . . . . . . .
4.9.3 Bipartite Graph (excluded in IOI syllabus)
4.10 Chapter Notes . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
35
35
35
37
40
40
45
49
54
57
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
58
58
58
67
70
74
75
77
81
85
86
87
89
92
5 Mathematics
5.1 Overview and Motivation . . . . . . . . . . . . . . . . . . . . . . . .
5.2 Ad Hoc Mathematics Problems . . . . . . . . . . . . . . . . . . . . .
5.3 Number Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.3.1 Prime Numbers . . . . . . . . . . . . . . . . . . . . . . . . . .
5.3.2 Greatest Common Divisor (GCD) & Least Common Multiple
5.3.3 Eulers Totient (Phi) Function . . . . . . . . . . . . . . . . .
5.3.4 Extended Euclid: Solving Linear Diophantine Equation . . .
5.3.5 Modulo Arithmetic . . . . . . . . . . . . . . . . . . . . . . . .
5.3.6 Fibonacci Numbers . . . . . . . . . . . . . . . . . . . . . . . .
5.3.7 Factorial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.4 Java BigInteger Class . . . . . . . . . . . . . . . . . . . . . . . . . .
5.4.1 Basic Features . . . . . . . . . . . . . . . . . . . . . . . . . .
5.4.2 Bonus Features . . . . . . . . . . . . . . . . . . . . . . . . . .
5.5 Miscellaneous Mathematics Problems . . . . . . . . . . . . . . . . . .
5.5.1 Combinatorics . . . . . . . . . . . . . . . . . . . . . . . . . .
5.5.2 Cycle-Finding . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.5.3 Existing (or Fictional) Sequences and Number Systems . . .
5.5.4 Probability Theory (excluded in IOI syllabus) . . . . . . . . .
5.5.5 Linear Algebra (excluded in IOI syllabus) . . . . . . . . . . .
5.6 Chapter Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . .
. . . . .
. . . . .
. . . . .
(LCM)
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
93
93
94
94
94
98
98
99
100
101
101
102
102
103
105
105
106
107
108
108
108
6 String Processing
6.1 Overview and Motivation . . . . . . . . . . . .
6.2 Ad Hoc String Processing Problems . . . . . .
6.3 String Processing with Dynamic Programming
6.3.1 String Alignment (Edit Distance) . . . .
6.3.2 Longest Common Subsequence . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
110
110
110
112
112
113
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
6.4
6.5
6.3.3 Palindrome . . . . . . . . .
Sux Tree and Sux Array . . . .
6.4.1 Sux Tree: Basic Ideas . .
6.4.2 Applications of Sux Tree
6.4.3 Sux Array: Basic Ideas .
Chapter Notes . . . . . . . . . . .
7 (Computational) Geometry
7.1 Overview and Motivation . .
7.2 Geometry Basics . . . . . . .
7.3 Grahams Scan . . . . . . . .
7.4 Intersection Problems . . . .
7.5 Divide and Conquer Revisited
7.6 Chapter Notes . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
113
114
114
115
116
119
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
120
120
121
128
130
131
132
A Problem Credits
133
134
Bibliography
135
Acknowledgements
My father Lin Tjie Fong and mother Tan Hoey Lan for raising us and encouraging us to do
well in our study and work.
School of Computing, National University of Singapore, for employing me and allowing me
to teach CS3233 - Competitive Programming module from which this book is born.
NUS/ex-NUS professors/lecturers who have shaped my competitive programming and coach-
ing skills: Prof Andrew Lim Leong Chye, Dr Tan Sun Teck, Aaron Tan Tuck Choy, Dr Sung
Wing Kin, Ken, Dr Alan Cheng Holun.
Fellow Teaching Assistants of CS3233 and ACM ICPC Trainers @ NUS: Su Zhan, Ngo Minh
Duc, Melvin Zhang Zhiyong, Bramandia Ramadhana.
My CS3233 students in Sem2 AY2008/2009 who inspired me to come up with the lecture
notes and CS3233 students in Sem2 AY2009/2010 who help me verify the content of this
book plus the Live Archive contribution.
My friend Ilham Winata Kurnia for proof reading the manuscript.
Copyright
This book is written mostly during National University of Singapore (NUS) oce hours as part
of the lecture notes for a module titled CS3233 - Competitive Programming. Hundreds of hours
Therefore, no part of this book may be reproduced or transmitted in any form or by any means,
Foreword
Long time ago (exactly the Tuesday November 11th 2003 at 3:55:57 UTC), I received an e-mail
with the following sentence: I should say in a simple word that with the UVa Site, you have given
birth to a new CIVILIZATION and with the books you write (he meant Programming Challenges:
The Programming Contest Training Manual [23], coauthored with Steven Skiena), you inspire the
soldiers to carry on marching. May you live long to serve the humanity by producing super-human
programmers.
Although its clear that was an exaggeration, to tell the truth I started thinking a bit about and
I had a dream: to create a community around the project I had started as a part of my teaching
job at UVa, with persons from everywhere around the world to work together after that ideal. Just
by searching in Internet I immediately found a lot of people who was already creating a web-ring
of sites with excellent tools to cover the many lacks of the UVa site.
The more impressive to me was the Methods to Solve from Steven Halim, a very young
student from Indonesia and I started to believe that the dream would become real a day, because
the contents of the site were the result of a hard work of a genius of algorithms and informatics.
Moreover his declared objectives matched the main part of my dream: to serve the humanity. And
the best of the best, he has a brother with similar interest and capabilities, Felix Halim.
Its a pity it takes so many time to start a real collaboration, but the life is as it is. Fortunately,
all of us have continued working in a parallel way and the book that you have in your hands is the
best proof.
I cant imagine a better complement for the UVa Online Judge site, as it uses lots of examples
from there carefully selected and categorized both by problem type and solving techniques, an
incredible useful help for the users of the site. By mastering and practicing most programming
exercises in this book, reader can easily go to 500 problems solved in UVa online judge, which will
place them in top 400-500 within 100000 UVa OJ users.
Then its clear that the book Competitive Programming: Increasing the Lower Bound of
Programming Contests is suitable for programmers who wants to improve their ranks in upcoming
ICPC regionals and IOIs. The two authors have gone through these contests (ICPC and IOI)
themselves as contestants and now as coaches. But its also an essential colleague for the newcomers,
because as Steven and Felix say in the introduction the book is not meant to be read once, but
several times.
Moreover it contains practical C++ source codes to implement the given algorithms. Because
understand the problems is a thing, knowing the algorithms is another, and implementing them
well in short and ecient code is tricky. After you read this extraordinary book three times you
will realize that you are a much better programmer and, more important, a more happy person.
Miguel A. Revilla
UVa Online Judge site creator
ACM-ICPC International Steering Committee Member and Problem Archivist
University of Valladolid
http://uva.onlinejudge.org
http://acmicpc-live-archive.uva.es
Preface
This is a book that every competitive programmer must read and master, at least during the
middle phase of their programming career: when they want to leap forward from just knowing
some programming language commands and some algorithms to become a top programmer.
Typical readers of this book will be: 1). Thousands University students competing in annual
ACM International Collegiate Programming Contest (ICPC) [27] regional contests, 2). Hundreds
(IOI) [12], 3). Their coaches who are looking for a comprehensive training materials [9], and 4).
for readers who have knowledge in basic programming methodology, familiar with at least one
programming language (C/C++/Java), and have passed basic data structures and algorithms (or
There are many exercises and programming problems scattered throughout the body text of this
book which can be skipped at rst if solution is not known at that point of time, but can be
revisited in latter time after the reader has accumulated new knowledge to solve it. Solving these
exercises help strengthening the concepts taught in this book as they usually contain interesting
twists or variants of the topic being discussed, so make sure to attempt them.
Use uva.onlinejudge.org/index.php?option=com_onlinejudge&Itemid=8&category=118,
We know that one probably cannot win an ACM ICPC regional or get a gold medal in IOI just
by mastering the current version of this book. While we have included a lot of material in this
book, we are well aware that much more than what this book can oer, are required to achieve
that feat. Some pointers are listed throughout this book for those who are hungry for more.
We believe this book is and will be relevant to many University and high school students as
ICPC and IOI will be around for many years ahead. New students will require the basic knowledge
presented in this book before hunting for more challenges after mastering this book. But before
you assume anything, please check this books table of contents to see what we mean by basic.
We will be happy if in year 2010 and beyond, the level of competitions in ICPC and IOI increase
because many of the contestants have mastered the content of this book. We hope to see many
ICPC and IOI coaches around the world, especially in South East Asia, adopt this book knowing
that without mastering the topics in and beyond this book, their students have no chance of doing
well in future ICPCs and IOIs. If such increase in required lowerbound knowledge happens, this
book has fullled its objective of advancing the level of human knowledge in this era.
To obtain PowerPoint slides/other instructional materials (only for coaches), send a personal request
email to stevenhalim@gmail.com.
Authors Proles
(SoC, NUS). He teaches several programming courses in NUS, ranging from basic programming
methodology, intermediate data structures and algorithms, and up to the Competitive Program-
ming module that uses this book. He is the coach of both NUS ACM ICPC teams and Singapore
IOI team. He participated in several ACM ICPC Regional as student (Singapore 2001, Aizu 2003,
Shanghai 2004). So far, he and other trainers @ NUS have successfully groomed one ACM ICPC
World Finalist team (2009-2010) as well as two silver and two bronze IOI medallists (2009).
Felix Halim is currently a PhD student in the same University: SoC, NUS. In terms of programming
contests, Felix has much colorful reputation than his older brother. He was IOI 2002 contestant.
His teams (at that time, Bina Nusantara University) took part in ACM ICPC Manila Regional
2003-2004-2005 and obtained rank 10th, 6th, and 10th respectively. Then, in his nal year, his
team nally won ACM ICPC Kaohsiung Regional 2006 and thus became ACM ICPC World Finalist
@ Tokyo 2007 (Honorable Mention). Today, felix halim actively joins TopCoder Single Round
Convention
There are a lot of C++ codes shown in this book. Many of them uses typedefs, shortcuts, or
macros that are commonly used by competitive programmers to speed up the coding time. In this
#define _CRT_SECURE_NO_DEPRECATE // suppress some compilation warning messages (for VC++ users)
// To simplify repetitions/loops, Note: define your loop style and stick with it!
#define REP(i, a, b) \
for (int i = int(a); i <= int(b); i++) // a to b, and variable i is local!
#define TRvi(c, it) \
for (vi::iterator it = (c).begin(); it != (c).end(); it++)
#define TRvii(c, it) \
for (vii::iterator it = (c).begin(); it != (c).end(); it++)
#define TRmsi(c, it) \
for (msi::iterator it = (c).begin(); it != (c).end(); it++)
Abbreviations
CC : Coin Change
CCW : Counter ClockWise
CS : Computer Science
ED : Edit Distance
Contest
IOI : International Olympiad in Informatics
OJ : Online Judge
PE : Presentation Error
RB : Red-Black (BST)
RMQ : Range Minimum Query
RSQ : Range Sum Query
RTE : Run Time Error
List of Tables
1.1
1.2
1.3
1.4
. .
. .
. .
n
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
4
4
5
6
3.1
DP Decision Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.1
4.2
4.3
5.1
6.1
Some String Processing Problems in Recent ACM ICPC Asia Regional . . . . . . . . 110
7.1
Some (Computational) Geometry Problems in Recent ACM ICPC Asia Regional . . 120
List of Figures
1.1
1.2
1.3
1.4
1.5
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
10
10
11
11
13
2.1
2.2
2.3
2.4
2.5
.
.
.
.
17
18
20
20
2.6
2.7
2.8
.
.
.
.
21
21
22
23
3.1
3.2
3.3
3.4
3.5
3.6
3.7
3.8
3.9
3.10
3.11
3.12
3.13
3.14
4.1
4.2
4.3
4.4
4.5
4.6
4.7
4.8
4.9
4.10
4.11
4.12
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
28
30
36
36
36
37
38
44
46
47
51
53
54
55
. . . .
. . . .
. . . .
. . . .
. . . .
(SCC)
. . . .
[17]) .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
59
62
63
64
64
65
68
70
71
71
72
72
4.13
4.14
4.15
4.16
4.17
4.18
4.19
4.20
4.21
4.22
4.23
4.24
4.25
4.26
4.27
4.28
4.29
4.30
73
73
75
76
76
78
79
79
81
82
84
84
85
88
90
90
91
91
6.1
6.2
6.3
6.4
6.5
113
114
116
116
116
7.1
7.2
7.3
7.4
7.5
7.6
Circles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Triangles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Quadrilaterals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Great-Circle and Great-Circle Distance (Arc A-B) (Figures from [46])
Convex Hull CH(P ) of Set of Points P . . . . . . . . . . . . . . . . . .
Athletics Track (from UVa 11646) . . . . . . . . . . . . . . . . . . . .
122
122
124
125
128
131
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Chapter 1
Introduction
I want to compete in ACM ICPC World Final!
A dedicated student
In this chapter, we introduce readers to the world of competitive programming. Hopefully you enjoy the ride
and continue reading and learning until the very last page of this book, enthusiastically.
1.1
Competitive Programming
Competitive Programming in summary, is this: Given well-known Computer Science (CS) problems, solve them as quickly as possible!.
Lets digest the terms one by one. The term well-known CS problems implies that in competitive programming, we are dealing with solved CS problems and not research problems (where the
solutions are still unknown). Denitely, some people (at least the problem setter) have solved these
problems before. Solve them implies that we must push our CS knowledge to a certain required
level so that we can produce working codes that can solve these problems too in terms of getting
the same output as the problem setter using the problem setters secret input data. As quickly as
possible is the competitive element which is a very natural human behavior.
Please note that being well-versed in competitive programming is not the end goal, it is just
the means. The true end goal is to produce all-rounded computer scientists/programmers who are
much more ready to produce better software or to face harder CS research problems in the future.
The founders of ACM International Collegiate Programming Contest (ICPC) [27] have this vision
and we, the authors, agree with it. With this book, we play our little roles in preparing current
and future generations to be more competitive in dealing with well-known CS problems frequently
posed in recent ICPCs and International Olympiad in Informatics (IOI).
Illustration on solving UVa Online Judge [17] Problem Number 10911 (Forming Quiz Teams).
Abridged problem description: Let (x,y) be the coordinate of a students house on a 2-D plane.
There are 2N students and we want to pair them into N groups. Let di be the distance
N
between the houses of 2 students in group i. Form N groups such that i=1 di is minimized.
Constraints: N 8; 0 x, y 1000. Think rst, try not to ip this page immediately!
Now, ask yourself, which one is you? Note that if you are unclear with the materials or terminologies shown in this chapter, you can re-read it after going through this book once.
Non-competitive programmer A (a.k.a the blurry one):
Step 1: Read the problem... confused @-@, never see this kind of problem before.
Step 2: Try to code something... starting from reading non-trivial input and output.
Step 3: Realize that all his attempts fail:
Greedy solution: pair students based on shortest distances gives Wrong Answer (WA).
Complete search using backtracking gives Time Limit Exceeded (TLE).
After 5 hours of labor (typical contest time), no Accepted (AC) solution is produced.
Non-competitive programmer B (Give up):
Step 1: Read the problem...
Then realize that he has seen this kind of problem before.
But also remember that he has not learned how to solve this kind of problem...
He is not aware of a simple solution for this problem: Dynamic Programming (DP)...
Step 2: Skip the problem and read another problem.
(Still) non-competitive programmer C (Slow):
Step 1: Read the problem and realize that it is a matching on general graph problem.
In general, this problem must be solved using Edmonds Blossom Shrinking [34].
But since the input size is small, this problem is solve-able using Dynamic Programming!
Step 2: Code I/O routine, write recursive top-down DP, test the solution, debug >.<...
Step 3: Only after 3 hours, his solution is judged as AC (passed all secret test data).
Competitive programmer D:
Same as programmer C, but do all those steps above in less than 30 minutes.
Very Competitive programmer E:
Of course, a very competitive programmer (e.g. the red target coders in TopCoder [26])
may solve this classical problem in less than 15 minutes...
1.2
Tips to be Competitive
If you strive to be like competitive programmer D or E in the illustration above. That is, you want
to do well to qualify and get a medal in IOI [12]; to qualify in ACM ICPC [27] national, regional,
and up to world nal; or in other programming contests, then this book is denitely for you!
In subsequent chapters, you will learn basic to medium data structures and algorithms frequently
appearing in recent programming contests, compiled from many sources [19, 6, 20, 2, 4, 14, 21, 16,
23, 1, 13, 5, 22, 15, 47, 24] (see Figure 1.5). But you will not just learn the algorithm, but also how
to implement them eciently and apply them to appropriate contest problem.
On top of that, you will also learn many tiny bits of programming tips from our experience that
can be helpful in contest situation. We will start by giving you few general tips below:
Tip 0: Type Code Faster!
No kidding! Although this tip may not mean much as ICPC nor IOI are about typing speed
competition, but we have seen recent ICPCs where rank i and rank i + 1 are just separated by few
minutes. When you can solve the same number of problems as your competitor, it is now down to
coding skill and ... typing speed.
Try this typing test at http://www.typingtest.com and follow the instructions there on how to
improve your typing skill. Stevens is 85-95 wpm and Felixs is 55-65 wpm. You also need to
familiarize your ngers with the position of frequently used programming language characters, e.g.
braces {} or () or <>, semicolon ;, single quote for char and double quotes for string, etc.
As a little practice, try typing this C++ code (a UVa 10911 solution above) as fast as possible.
/* Forming Quiz Teams. This DP solution will be explained in Section 3.4 */
#include <iostream>
#include <algorithm>
#include <string.h>
#include <math.h>
using namespace std;
int N;
double dist[20][20], memo[1 << 16]; // 1 << 16 is 2^16, recall that max N = 8
double matching(int bit_mask) {
if (memo[bit_mask] > -0.5) // see that we initialize the array with -1 in the main function
return memo[bit_mask];
if (bit_mask == (1 << 2 * N) - 1) // all are matched
return memo[bit_mask] = 0;
double matching_value = 32767 * 32767; // initialize with large value
for (int p1 = 0; p1 < 2 * N; p1++)
if (!(bit_mask & (1 << p1))) { // if this bit is off
for (int p2 = p1 + 1; p2 < 2 * N; p2++)
if (!(bit_mask & (1 << p2))) // if this different bit is also off
matching_value = min(matching_value,
dist[p1][p2] + matching(bit_mask | (1 << p1) | (1 << p2)));
break; // this break is necessary. do you understand why?
}
// hint: it helps reducing time complexity from O((2N)^2 * 2^(2N)) to O((2N) * 2^(2N))
return memo[bit_mask] = matching_value;
int main() {
char line[1000], name[1000];
int i, j, caseNo = 1, x[20], y[20];
// freopen("10911.txt", "r", stdin); // one way to simplify testing
while (sscanf(gets(line), "%d", &N), N) {
for (i = 0; i < 2 * N; i++)
sscanf(gets(line), "%s %d %d", &name, &x[i], &y[i]);
1.2.1
In ICPCs, the contestants will be given a set of problems ( 7-11 problems) of varying types. From
our observation of recent ICPC Asia Regional problem sets, we can categorize the problems types
and their rate of appearance as in Table 1.1. For IOI, please refer to IOI syllabus 2009 [8] and [28].
No
1.
Category
Ad Hoc
Sub-Category
Straightforward
Simulation
Iterative
Backtracking
2.
Complete Search
3.
4.
5.
Dynamic Programming
6.
7.
8.
9.
10.
Graph
Mathematics
String Processing
Computational Geometry
Some Harder Problems
Classic
Original
Classic
Original
In This Book
Section 1.3
Section 1.3
Section 3.1
Section 3.1
Section 3.2
Section 3.3.1
Section 3.3.2
Section 3.4.2
Section 3.4.3
Chapter 4
Chapter 5
Chapter 6
Chapter 7
Total in Set
Appearance Frequency
1-2
0-1
0-1
0-1
0-1
0
1
0
1-2 (can go up to 3)
1-2
1-2
1
1
0-1
7-16 (usually 11)
The classication in Table 1.1 is adapted from [18] and by no means complete. Some problems,
e.g. sorting, are not classied here as they are trivial and only used as sub-routine in a bigger
problem. We do not include recursion as it is embedded in other categories. We also omit data
more than one type, e.g. Floyd Warshalls is either a solution for graph problem: All-Pairs Shortest
Paths (APSP, Section 4.7) or a Dynamic Programming (DP) algorithm (Section 3.4).
In the future, these classications may grow or change. One signicant example is DP. This
technique was not known before 1940s, not frequently used in ICPCs or IOIs before mid 1990s, but
it is a must today. There are 3 DP problems (out of 11) in recent ICPC World Finals 2010.
As an exercise, read the UVa [17] problems shown in Table 1.2 and determine their problem
types. The rst one has been lled for you. Filling this table is easy after mastering this book.
UVa
10360
10341
11292
11450
11635
11506
10243
10717
11512
10065
Title
Rat Attack
Solve It
Dragon of Loowater
Wedding Shopping
Hotel Booking
Angry Programmer
Fire! Fire!! Fire!!!
Mint
GATTACA
Useless Tile Packers
Problem Type
Complete Search or Dynamic Programming
Hint
Section
Section
Section
Section
Section
Section
Section
Section
Section
Section
3.1 or 3.4
3.2
3.3
3.4
4.3 + 4.5
4.8
4.9.1
5.3.2
6.4
7.3
The goal is not just to map problems into categories as in Table 1.1. After you are familiar with
most of the topics in this book, you can classify the problems into just four types as in Table 1.3.
No
A.
B.
C.
D.
Category
I have solved this type before
I have solved this type before
I have seen this type before
I have not seen this type before
To be competitive, you must frequently classify the problems that you read in the problem set into
1.2.2
Once you have designed an algorithm to solve a particular problem in a programming contest,
you must now ask this question: Given the maximum input bound (usually given in a good prob-
lem description), can the currently developed algorithm, with its time/space complexity, pass the
Sometimes, there are more than one way to attack a problem. However, some of them may be
incorrect and some of them are not fast enough... The rule of thumb is: Brainstorm many possible
algorithms - then pick the simplest that works (fast enough to pass the time and memory limit,
For example, the maximum size of input n is 100K, or 105 (1K = 1, 000), and your algorithm
is of order O(n2 ). Your common sense told you that (100K)2 is an extremely big number, it is
1010 . So, you will try to devise a faster (and correct) algorithm to solve the problem, say of order
O(n log2 n). Now 105 log2 105 is just 1.7 106 ... Since computer nowadays are quite fast and can
process up to order 1M , or 106 (1M = 1, 000, 000) operations in seconds, your common sense told
you that this one likely able to pass the time limit.
Now how about this scenario. You can only devise an algorithm of order O(n4 ). Seems pretty
bad right? But if n 10... then you are done. Just directly implement your O(n4 ) algorithm since
104 is just 10K and your algorithm will only use relatively small computation time.
So, by analyzing the complexity of your algorithm with the given input bound and stated
time/memory limit, you can do a better judging whether you should try coding your algorithm
(which will take your time, especially in the time-constrained ICPCs and IOIs), or attempt to
improve your algorithm rst or switch to other problems in the problem set.
In this book, we will not discuss the concept of algorithm analysis. We assume that you have
this basic skill. Please check this reference book: Introduction to Algorithms [4] and make sure
the O(|Q| + occ) complexity for nding an exact string matching of query string Q with help
of Sux Tree (see Section 6.4).
Many novice programmers usually skip this phase and tempted to directly code the rst algorithm
that they can think of (usually the nave version), after that they ended up realizing that the chosen
data structure is not ecient or their algorithm is not fast enough (or wrong). Our advice: refrain
from coding until you are sure that your algorithm is both correct and fast enough.
To help you in judging how fast is enough, we produce Table 1.4. Variants of such Table 1.4
can also be found in many algorithms book. However, we put another one here from programming
contest perspective. Usually, the input size constraints are given in the problem description. Us-
ing some logical assumptions that typical year 2010 CPU can do 1M operations in 1s and time
limit of 3s (typical time limit used in most UVa online judge [17] problems), we can predict the
worst algorithm that can still pass the time limit. Usually, the simplest algorithm has poor time
complexity, but if it can already pass the time limit, just use it!
From Table 1.4, we see the importance of knowing good algorithms with lower order of growth
as they allow us to solve problems with bigger input size. Beware that a faster algorithm is usually
non trivial and harder to code. In Section 3.1.2 later, we will see a few tips that may allow us to
enlarge the possible input size n for the same class of algorithm.
n
10
20
50
100
1K
100K
1M
Worst AC Algorithm
O(n!), O(n6 )
O(2n ), O(n5 )
O(n4 )
O(n3 )
O(n2 )
O(n log2 n)
O(n), O(log 2 n), O(1)
Comment
e.g. Enumerating a Permutation
e.g. DP + Bitmask Technique
e.g. DP with 3 dimensions + O(n) loop, choosing n Ck=4
e.g. Floyd Warshalls
e.g. Bubble/Selection/Insertion Sort
e.g. Merge Sort, building Segment Tree
Usually, contest problem has n 1M (e.g. to read input)
Table 1.4: Rule of Thumb for the Worst AC Algorithm for various input size n (single test case
only), assuming that year 2010 CPU can compute 1M items in 1s and Time Limit of 3s.
1. There are n webpages (1 n 10M ). Each webpage i has dierent page rank ri . You want
to pick top 10 pages with highest page ranks. Which method is more feasible?
(a) Load all n webpages page rank to memory, sort (Section 2.2.1), and pick top 10.
(b) Use priority queue data structure (heap) (Section 2.2.2).
2. Given a list L of up to 10K integers, you want to frequently ask the value of sum(i, j), i.e.
the sum of L[i] + L[i+1] + ... + L[j]. Which data structure should you use?
(a) Simple Array (Section 2.2.1).
(b) Balanced Binary Search Tree (Section 2.2.2).
(c) Hash Table (Section 2.2.2).
(d) Segment Tree (Section 2.3.3).
(e) Sux Tree (Section 6.4).
(f) Simple Array that is pre-processed with Dynamic Programming (Section 2.2.1 & 3.4).
3. You have to compute the shortest path between two vertices on a weighted Directed Acyclic
Graph (DAG) with |V |, |E| 100K. Which algorithm(s) can be used?
(a) Dynamic Programming + Topological Sort (Section 3.4, 4.2, & 4.9.2).
(b) Breadth First Search (Section 4.3).
(c) Dijkstras (Section 4.5).
(d) Bellman Fords (Section 4.6).
(e) Floyd Warshalls (Section 4.7).
4. Which algorithm is faster (based on its time complexity) for producing a list of the rst 10K
prime numbers? (Section 5.3.1)
(a) Sieve of Eratosthenes (Section 5.3.1).
(b) For each number i [1 10K], test if i is a prime with prime testing function.
1.2.3
There are several programming languages allowed in ICPC, including C/C++ and Java. Which
one should we master? Our experience gives us the following answer: although we prefer C++
with built-in Standard Template Library (STL), we still need to master Java, albeit slower, since
this language has a powerful BigInteger, String Processing, and GregorianCalendar API. Simple
illustration is shown below (part of the solution for UVa problem 623: 500!):
Compute 25! (factorial of 25). The answer is very large: 15,511,210,043,330,985,984,000,000.
This is way beyond the largest built-in data structure (unsigned long long: 264 1) in C/C++.
Using C/C++, you will hard time coding this simple problem as there is no native support for Big
Integer data structure in C/C++ yet. Meanwhile, the Java code is simply this:
import java.util.*;
import java.math.*;
class Main { // standard class name in UVa OJ
public static void main(String[] args) {
BigInteger fac = new BigInteger.valueOf(1); // :)
for (int i = 2; i <= 25; i++)
fac = fac.multiply(BigInteger.valueOf(i)); // wow :)
System.out.println(fac);
} }
Another illustration to reassure you that mastering a programming language is good: Read this
input: There are N lines, each line always start with character 0 followed by ., then unknown
number of digits x, nally each line always terminated with three dots .... See an example below.
2
0.1227...
0.517611738...
Not many C/C++ programmers are aware of the trick above. Although scanf/printf are C-style
I/O routines, they can still be used in C++ code. Many C++ programmers force themselves to
use cin/cout all the time which, in our opinion, are not as exible as scanf/printf and slower.
In ICPCs, coding should not be your bottleneck at all. That is, once you gure out the worst
AC algorithm that will pass the given time limit, you are supposed to be able to translate it into
bug-free code and you can do it fast! Try to do some exercises below. If you need more than
10 lines of code to solve them, you will need to relearn your programming language(s) in depth!
Mastery of programming language routines will help you a lot in programming contests.
1. Given a string that represents a base X number, e.g. FF (base 16, Hexadecimal), convert it
to base Y, e.g. 255 (base 10, Decimal), 2 X, Y 36. (More details in Section 5.4.2).
2. Given a list of integers L of size up to 1M items, determine whether a value v exists in L?
(More details in Section 2.2.1).
3. Given a date, determine what is the day (Monday, Tuesday, ..., Sunday) of that date?
4. Given a long string, replace all the occurrences of a character followed by two consecutive
digits in with ***, e.g. S = a70 and z72 will be replaced, but aa24 and a872 will not will
be transformed to S = *** and *** will be replaced, but aa24 and a872 will not.
1.2.4
You thought you have nailed a particular problem. You have identied its type, designed the
algorithm for it, calculated the algorithms time/space complexity - it will be within the time and
memory limit given, and coded the algorithm. But, your solutoin is still not Accepted (AC).
Depending on the programming contests type, you may or may not get credit by solving the
problem partially. In ICPC, you will only get credit if your teams code solve all the judges secret
test cases, thats it, you get AC. Other responses like Presentation Error (PE), Wrong Answer
(WA), Time Limit Exceeded (TLE), Memory Limit Exceeded (MLE), Run Time Error (RTE), etc
do not increase your teams points. In IOI (2009 rule), there exists a partial credit system, in which
you will get scored based on the number of correct/total number of test cases for the latest code
that you have submitted for that problem, but the judging will only be done after the contest is
over, so you must be very sure that your code is doing OK.
In either case, you will need to be able to design good, educated, tricky test cases. The sample
input-output given in problem description is by default too trivial and therefore not a good way
for measuring your codes correctness.
Rather than wasting submissions (and get time or point penalties) by getting non AC responses,
you may want to design some tricky test cases rst, test it in your own machine, and ensure your
code is able to solve it correctly (otherwise, there is no point submitting your solution right?).
Some coaches ask their students to compete with each other by designing test cases. If student
As test cases can break other students code, then A will get bonus point. You may want to try
this in your team training too :). This concept is also used in TopCoder [26] challenge phase.
Here are some guidelines for designing good test cases, based on our experience:
1. Must include sample input as you have the answer given... Use fc in Windows or di in
UNIX to help checking your codes output against the sample output.
2. Must include boundary cases. Increase the size of input incrementally up to the maximum
possible. Sometimes your program works for small input size, but behave wrongly when input
size increases. Check for overow, out of bounds, etc.
3. For multiple input test cases, use two identical test cases consecutively. Both must output the
same result. This is to check whether you have forgotten to initialize some variables, which
will be easily identied if the 1st instance produce correct output but the 2nd one does not.
4. Create tricky test cases by identifying cases that are hidden in the problem description.
5. Do not assume the input will always be nicely formatted if the problem description does not
say so (especially for badly written programming problem). Try inserting white spaces (space,
tabs) in your input, and check whether your code is able to read in the values correctly.
6. Finally, generate large random test cases. See if your code terminates on time and still give
reasonably ok output (correctness is hard to verify here this test is only to verify that your
code runs within time limit).
However, after all these careful steps, you may still get non-AC responses. In ICPC, you and your
team can actually use the judges response to determine your next action. With more experience
in such contests, you will be able to make better judgment. See the next exercises:
1. You receive a WA response for a very easy problem. What should you do?
(a) Abandon this problem and do another.
(b) Improve the performance of the algorithm.
(c) Create tricky test cases and nd the bug.
(d) (In team contest): Ask another coder in your team to re-do this problem.
2. You receive a TLE response for an your O(N 3 ) solution. However, maximum N is just 100.
What should you do?
(a) Abandon this problem and do another.
(b) Improve the performance of the algorithm.
(c) Create tricky test cases and nd the bug.
3. Follow up question (see question 2 above): What if maximum N is 100.000?
1.2.5
Competitive programmers, like real athletes, must train themselves regularly and keep themselves
programming-t. Thus in our last tip, we give a list of websites that can help you improve your
University of Valladolid (from Spain) Online Judge [17] contains past years ACM contest prob-
lems (usually local or regional) plus problems from another sources, including their own contest
problems. You can solve these problems and submit your solutions to this Online Judge. The
correctness of your program will be reported as soon as possible. Try solving the problems men-
tioned in this book and see your name on the top-500 authors rank list someday :-). At the point
of writing (9 August 2010), Steven is ranked 121 (for solving 857 problems) while Felix is ranked
70 (for solving 1089 problems) from 100386 UVa users and 2718 problems.
Figure 1.1: University of Valladolid (UVa) Online Judge, a.k.a Spanish OJ [17]
UVa sister online judge is the ACM ICPC Live Archive that contains recent ACM ICPC Regionals
and World Finals problem sets since year 2000. Train here if you want to do well in future ICPCs.
USA Computing Olympiad has a very useful training website [18] for you to learn about programming contest. This one is more geared towards IOI participants. Go straight to their website,
register your account, and train yourself.
1.3
We will end this chapter by introducing you to the rst problem type in ICPC: the Ad Hoc
problems. According to USACO training gateway [18], Ad Hoc problems are problems that cannot
be classied anywhere else, where each problem description and the corresponding solution are
unique. Ad Hoc problems can be further classied into two: straightforward where the solution
just requires translation of problem requirement to code; or simulation problem where there are
some set of rules that must be simulated to obtain the answer.
Ad Hoc problems almost usually appear in a programming contest. Using a benchmark of total
10 problems, there may be 1-2 Ad Hoc problems. If the Ad Hoc problem is easy, it will usually
be the rst problem being attacked by teams in a programming contest. But there exists Ad Hoc
problems that are complicated to code and some teams will strategically defer solving them until
the last hour. Assuming a 60 teams contest, your team is probably in lower half (rank 30-60) if
your team can only do this type of problem during an ICPC regional contest.
Get your coding skills up and running by solving these Ad Hoc problems before continuing to
the next chapter. We have selected one Ad Hoc problem from every volume in UVa online judge
[17] (there are 28 volumes as of 9 August 2010) plus several ones from ACM ICPC Live Archive
[11]. Note that some simple Ad Hoc problems below are tricky.
1.4
Chapter Notes
Figure 1.5: Some Reference Books that Inspired the Authors to Write This Book
This and subsequent chapters are supported by many text books (see Figure 1.5) and Internet
resources. Tip 1 is an adaptation from introduction text in USACO training gateway [18]. More
details about Tip 2 can be found in many CS books, e.g. Chapter 1-5, 17 of [4]. Reference for
(Tip 4), a little detour to software engineering books may be worth trying. There are many other
SPOJ http://www.spoj.pl,
POJ http://acm.pku.edu.cn/JudgeOnline,
TOJ http://acm.tju.edu.cn/toj,
ZOJ http://acm.zju.edu.cn/onlinejudge/,
Chapter 2
2.1
Data Structures
Data structure is a way to store and organize data in order to support ecient insertions, queries,
searches, updates, and deletions. Although a data structure in itself does not solve the given
programming problem the algorithm operating on it does, using the most ecient data structure
for the given problem may be a dierence between passing or exceeding the problems time limit.
There are many ways to organize the same data and sometimes one way is better than the other
on dierent context, as we will see in the discussion below. Familiarity with the data structures
discussed in this chapter is a must in order to understand the algorithms in subsequent chapters.
As stated in the preface of this book, we assume that you are familiar with the basic data
structures listed in Section 2.2, and thus we will not review them again in this book. We simply
highlight the fact that they all have built-in libraries in C++ STL and Java API (Note that in this
version of the book, we write most example codes from C++ perspective). If you feel that you
are not sure with any of the terms or data structures mentioned in Section 2.2, pause reading this
book, quickly explore and learn that term in the reference books, e.g. [3]1 , and resume when you
get the basic ideas of those data structures.
Note that for competitive programming, you just have to be able to use (i.e. know the strengths,
weaknesses, and time/space complexities) a certain data structure to solve the appropriate contest
problem. Its theoretical background is good to know, but can be skipped.
This chapter is divided into two parts. Section 2.2 contains basic data structures with their
basic operations that currently have built-in libraries. Section 2.3 contains more data structures for
which currently we have to build our own libraries. Because of this, Section 2.3 has more detailed
discussions than Section 2.2.
1
Materials in Section 2.2 are usually taught in level-1 data structures and algorithms course in CS curriculum.
High school students who are planning to join competitions like IOI are encouraged to do self-study on these material.
2.2
2.2.1
A data structure is classied as linear if its elements form a sequence. Mastery of all these basic
linear data structures below is a must to do well in todays programming contests.
Static Array in C/C++ and in Java
This is clearly the most commonly used data structure in programming contests whenever
there is a collection of sequential data to be stored and later accessed using their indices.
As the maximum input size is normally mentioned in a programming problem, then usually
the declared array size is this value + small extra buer. Typical dimensions of the array
are: 1-D, 2-D, 3-D, and rarely goes beyond 4-D. Typical operations for array are: accessing
certain indices, sorting the array, linearly scanning, or binary searching the array.
Resizeable Array a.k.a. Vector: C++ STL <vector> (Java ArrayList)
All else the same as static array but has auto-resize feature. Using vector over array is better
if array size is unknown beforehand, i.e. before running the program. Usually, we initialize
the size with some guess value for better performance. Typical operations are: push back(),
at(), [] operator, erase(), and typically use iterator to scan the content of the vector.
2.2.2
For some computational problems, there are better ways to organize data other than ordering it
sequentially. With ecient implementation of non-linear data structures shown below, you can
search items much faster, which can speed up the algorithms that use them.
For example, if you want to store a dynamic collection of pairs (e.g. name index pairs), then
using C++ STL <map> below can give you O(log n) performance for insertion/search/deletion
with just few lines of codes whereas storing the same information inside one static array of struct
may requires O(n) insertion/search/deletion and you have to code it by yourself.
Balanced Binary Search Tree (BST): C++ STL <map>/<set> (Java TreeMap/TreeSet)
BST is a way to organize data as a tree-structure. In each subtree rooted at x, this BST
property holds: items on the left subtree of x are smaller than x and items on the right
subtree of x are greater (or equal) than x. Organizing the data like this (see Figure 2.1, left)
allows O(log n) insertion, search, and deletion as only O(log n) worst case root-to-leaf scan is
needed to perform those actions (details in [4]) but this only works if the BST is balanced.
Implementing a bug-free balanced BST like AVL2 Tree or Red-Black (RB) Tree is tedious
and hard to do under time constrained contest environment. Fortunately, C++ STL has
<map> and <set> which are usually the implementation of RB Tree, thus all operations
are in O(log n). Mastery of these two STL templates can save a lot of precious coding time
during contests! The dierence is simple: <map> stores (key data) pair whereas <set>
only stores the key.
Heap: C++ STL <queue>: priority queue (Java PriorityQueue)
Heap is another way to organize data as a tree-structure. Heap is also a binary tree like BST
but it must be complete. Instead of enforcing BST property, Heap enforces Heap property:
In each subtree rooted at x, items on the left and the right subtrees of x are smaller than x
(see Figure 2.1, right). This property guarantees that the top of the heap is the maximum
element. There is usually no notion of search in Heap, but only insertion and deletion, which
can be easily done by traversing a O(log n) leaf-to-root or root-to-leaf path [4].
2
However, for some tricky contest problems where you have to augment your data structure (see Chapter 14 of
[4]), AVL Tree knowledge is needed as C++ STL <map> or <set> cannot be augmented easily.
is done by taking the max element k times (k is the number of the top most items to be
sorted). As each delete-max is in O(log n), partial sort has O(k log n) time complexity.
Hash Table: no native C++ STL support (Java HashMap/HashSet/HashTable)
Hash Table is another form of non-linear data structures, but we do not recommend using
it in contests unless necessary. Reasons: designing a good performing hash function is quite
tricky and there is no native C++ STL support for it. Moreover C++ STL <map> or <set>
are usually good enough as the typical input size of programming contest problems is 1M ,
making O(1) for Hash Table and O(log 1M ) for balanced BST actually do not dier by much.
However, a form of Hash Table is actually used in contests, namely Direct Addressing Table
(DAT), where the key itself determines the index, bypassing the need of hash function. For
example, UVa 11340 - Newspaper, where all possible ASCII characters [0-255] are assigned
to certain monetary values, e.g. a 3, W 10, . . . , I 13.
Programming exercises to practice using basic data structures and algorithms (with libraries):
Static array, C++ STL <vector>, <bitset>, Direct Addressing Table
1. UVa 482 - Permutation Arrays (simple array manipulation)
2. UVa 594 - One Two Three Little Endian (manipulate bit string easily with <bitset>)
3. UVa 11340 - Newspaper (Direct Addressing Table)
C++ STL <algorithm>
1. UVa 146 - ID Codes (use next permutation)
2. UVa 10194 - Football a.k.a. Soccer (multi-elds sorting, use sort)
3. UVa 10258 - Contest Scoreboard (multi-elds sorting, use sort)
Sorting-related problems
1. UVa 299 - Train Swapping (inversion index3 problem solvable with bubble sort)
2. UVa 612 - DNA Sorting (inversion index + stable sort)
3. UVa 10810 - Ultra Quicksort (inversion index - requires O(n log n) merge sort)
3
Inversion index problem: count how many swaps are needed to make the list sorted.
2.3
As of 9 August 2010, important data structures shown in this section do not have built-in sup-
port yet in C++ STL or Java API. Thus, to be competitive, contestants must have a bug-free
implementations during contests. In this section, we only discuss the ideas of these data structures.
2.3.1
Graph
Graph is a pervasive data structure which appears in many CS problems. Graph is simply a
collection of vertices and edges (that store connectivity information between those vertices). In
Chapter 3 & 4, we will explore many important graph problems and algorithms. In this subsection,
we only briey discuss four basic ways (there are others) to store graph information. Assuming
that we have a graph G with V vertices and E edges, here are the ways to store them:
2.3.2
Union-Find Disjoint Sets is a data structure to model a collection of disjoint sets which has abilities
to eciently4 1). nd which set an item belongs to (or to test whether two items belong to the
same set) and 2). union two disjoint sets into one bigger set. These two operations are useful for
algorithms like Kruskals (Section 4.4) or problems that involve partitioning, like keeping track of
connected components of an undirected graph (see Section 4.2).
These seemingly simple operations are not eciently supported by C++ STL <set> which only
deals with a single set. Having a vector of sets and looping through each one to nd which set
4
M operations of this data structure runs in O(M (n)), but (n) is just less than 5 for most practical values
of n. For more details, see [33].
an item belongs to is expensive! C++ STL <algorithm>s set union is also not ecient enough
although it combines two sets in linear time, as we still have to deal with the shuing of the content
inside the vector of sets! Thus, we need our own library to support this data structure. One such
example is shown in this section.
The key ideas of this data structure are like this: Keep a representative (parent) item of each
set. This information is stored in vector<int> pset, where pset[i] tells the representative item
of the set that contains item i. Example: suppose we have 5 items: {A, B, C, D, E} as 5 disjoint
sets of 1 item each. Each item initially has itself as the representative, as shown in Figure 2.3.
In Figure 2.4, we see what is happening when we call unionSet(i, j): every union is simply done
by changing the representative item of one item to point to the others representative item.
5
There is another heuristic called union-by-rank [4] that can further improve the performance of this data
structure. But we omit this enhancing heuristic from this book to simplify this discussion.
Figure 2.5: Calling findSet(i) to Determine the Representative Item (and Compressing the Path)
In Figure 2.5, we see what is happening when we call findSet(i). This function recursively
calls itself whenever pset[i] is not yet itself (i). Then, once it nds the main representative item
(e.g. x) for that set, it will compress the path by saying pset[i] = x. Thus subsequent calls of
findSet(i) will be O(1). This simple heuristic strategy is aptly named as Path Compression.
In Figure 2.6, we illustrate another operation for this data structure, called isSameSet(i, j)
that simply calls findSet(i) and findSet(j) to check if both refer to the same representative
item. If yes, i and j belong to the same set, otherwise, they do not.
Figure 2.6: Calling isSameSet(i, j) to Determine if Both Items Belong to the Same Set
Exercise 1: There are two more queries commonly performed on the Union-Find Disjoint Sets data
structure: int numberOfSets() that returns the number of disjoint sets currently in the structure
and int sizeOfSet(int i) that returns the size of set that currently contains item i. Update
the codes shown in this section to support these two queries eciently!
Exercise 2: In [4], there is a union by rank heuristic to speed-up this data structure. Do you think
this heuristic will help speed up the data structure signicantly? If yes, in which case(s)? Is there
any programming tricks to achieve similar eect without using this heuristic?
2.3.3
Segment Tree
In this subsection, we discuss another data structure which can eciently answer dynamic range
queries. As a starting point, we discuss a problem of nding the index of the minimum element
in an array given a range: [i..j]. This is more commonly known as the Range Minimum Query
(RMQ). For example, given an array A of size 7 below, RMQ(1, 3) = 2, as the index 2 contains
the minimum element among A[1], A[2], and A[3]. To check your understanding of RMQ, verify
There are several ways to solve this RMQ. One of the trivial algorithm is to simply iterate the
array from index i to j and report the index with the minimum value. But this is O(n) per query.
In this section, we solve the RMQ with Segment Tree: a binary tree similar to heap, but usually
not a complete binary tree. For the array A above, the segment tree is shown in Figure 2.7. The
root of this tree contains the full segment, from [0, N - 1]. And for each segment [l, r], we split them
into [l, (l + r) / 2] and [(l + r) / 2 + 1, r] until l = r. See the O(n log n) built_segment_tree
routine below. With segment tree ready, answering an RMQ can now be done in O(log n).
For example, we want to answer RMQ(1, 3). The execution in Figure 2.7 (red solid lines) is
as follows: From root [0, 6], we know that the answer for RMQ(1, 3) is on the left of vertex [0, 6]
as [0, 6] is still larger than the RMQ(1, 3), thus the stored min(imum) value of [0, 6] = 5 is not
appropriate as it is the min value over a larger segment [0, 6] than the RMQ(1, 3).
We move to the left segment [0, 3]. At vertex [0, 3], we have to search two sides as [0, 3] is still
larger than the RMQ(1, 3) and intersect both the left segment [0, 1] and the right segment [2, 3].
The right segment is [2, 3], which is inside the required RMQ(1, 3), so from the stored min
value inside this node, we know that RMQ(2,3) = 2. We do not need to traverse further down.
The left segment is [0, 1], which is not yet inside the RMQ(1, 3), so another split is necessary.
From [0, 1], we move right to segment [1, 1], which is now inside the RMQ(1, 3). Then, we return
A[RMQ(1, 1)] > A[RMQ(2, 3)] since A[1] = 7 and A[2] = 3, we know that RMQ(1, 3) = 2.
Lets take a look at another example: RMQ(4, 6). The execution in Figure 2.7 (blue dashed line)
is as follows: We again start from the root [0, 6]. Since it is bigger than the query, we move right
to segment [4, 6]. Since this segment is exactly the RMQ(4, 6), we simply return the index of
minimum element that is stored in this node, which is 5. Thus RMQ(4, 6) = 5. We do not have to
traverse the unnecessary parts of the tree! In the worst case, we have two root-to-leaf paths which
is just O(log n). For example in RMQ(3, 4) = 4, we have one root-to-leaf path from [0, 6] to [3, 3]
and another root-to-leaf path from [0, 6] to [4, 4].
If the array A is static, then using Segment Tree to solve RMQ is an overkill as there exists
a Dynamic Programming (DP) solution that requires O(n log n) one-time pre-processing and O(1)
per RMQ. This DP solution will be discussed later in Section 3.4.3.
The Segment Tree becomes useful if array A is frequently updated. For example, if A[5] is now
changed from 1 to 100, then what we need to do is to update the leaf to root nodes which can be
done in O(log n). The DP solution requires another O(n log n) pre-processing to do the same.
Figure 2.8: Updating Array A to {8, 7, 3, 9, 5, 100, 10}. Only leaf-to-root nodes are aected.
Our library implementation for Segment Tree is shown below. The code shown here supports static
Range Minimum/Maximum/Sum queries (the dynamic update part is left as exercise). There are
of course other ways to implement segment tree, e.g. a more ecient version that only expands
the segments when needed.
#include <iostream>
#include <math.h>
#include <vector>
using namespace std;
// Segment Tree Library
// The segment tree is stored like a heap array
#define RANGE_SUM 0
#define RANGE_MIN 1
#define RANGE_MAX 2
vi segment_tree;
void init_segment_tree(int N) { // if original array size is N,
// the required segment_tree array length is 2*2^(floor(log2(N)) + 1);
int length = (int)(2 * pow(2.0, floor((log((double)N) / log(2.0)) + 1)));
segment_tree.resize(length, 0); // resize this vector and fill with 0
}
void build_segment_tree(int code, int A[], int node, int b, int e) {
if (b == e) { // as b == e, either one is fine
if (code == RANGE_SUM) segment_tree[node] = A[b]; // store value of this cell
else
segment_tree[node] = b; // if RANGE_MIN/MAXIMUM, store index
}
else { // recursively compute the values in the left and right subtrees
int leftIdx = 2 * node, rightIdx = 2 * node + 1;
build_segment_tree(code, A, leftIdx , b
, (b + e) / 2);
build_segment_tree(code, A, rightIdx, (b + e) / 2 + 1, e
);
int lContent = segment_tree[leftIdx], rContent = segment_tree[rightIdx];
if (code == RANGE_SUM) // make this segment contains sum of left and right subtree
segment_tree[node] = lContent + rContent;
else { // (code == RANGE_MIN/MAXIMUM)
int lValue = A[lContent], rValue = A[rContent];
if (code == RANGE_MIN) segment_tree[node] = (lValue <= rValue) ? lContent : rContent;
else
segment_tree[node] = (lValue >= rValue) ? lContent : rContent;
} } }
int query(int code, int A[], int node, int b, int e, int i, int j) {
if (i > e || j < b) return -1; // if the current interval does not intersect query interval
if (b >= i && e <= j) return segment_tree[node]; // if the current interval is inside query interval
// compute the minimum position in the left and right part of the interval
int p1 = query(code, A, 2 * node
, b
, (b + e) / 2, i, j);
int p2 = query(code, A, 2 * node + 1, (b + e) / 2 + 1, e
, i, j);
// return the position where the overall minimum is
if (p1 == -1) return p2; // can happen if we try to access segment outside query
if (p2 == -1) return p1; // same as above
if (code == RANGE_SUM)
return p1 + p2;
else if (code == RANGE_MIN) return (A[p1] <= A[p2]) ? p1 : p2;
else
return (A[p1] >= A[p2]) ? p1 : p2;
}
int main() {
int A[] = {8,7,3,9,5,1,10};
init_segment_tree(7); build_segment_tree(RANGE_MIN, A, 1, 0, 6);
printf("%d\n", query(RANGE_MIN, A, 1, 0, 6, 1, 3)); // answer is index 2
return 0;
}
Exercise 1: Draw a segment tree of this array A = {10, 2, 47, 3, 7, 9, 1, 98, 21, 37} and answer
RMQ(1, 7) and RMQ(3, 8)!
Exercise 2: Using the same tree as in exercise 1 above, answer this Range Sum Query(i, j) (RSQ),
i.e. a sum from A[i] + A[i + 1] + ... + A[j]. What is RSQ(1, 7) and RSQ(3, 8)? Is this a good
approach to solve this problem? (See Section 3.4).
Exercise 3: The Segment Tree code shown above lacks update operation. Add the O(log n) update
function to update the value of a certain segment in the Segment Tree!
Programming exercises that use data structures with our own libraries:
Graph (simple one, many more in Section 4)
1. UVa 291 - The House of Santa Claus (simple backtracking)
2. UVa 10928 - My Dear Neighbours (count out degrees)
2.4
Chapter Notes
Basic data structures mentioned in Section 2.2 can be found in almost every data structure and
algorithm textbooks and references to their libraries are available online in:
http://www.cppreference.com and http://java.sun.com/javase/6/docs/api.
Extra references for data structures mentioned in Section 2.3 are as follow: For Graph data
structure, a good textbook is [21]. For Union-Find Disjoint Sets, see Chapter 21 of [4]. For Segment
Tree and other geometric-related data structures, see [6].
With more experience, and especially by looking at TopCoders codes, you will master more
tricks in using these data structures. This point is hard to teach. Thus, please spend some time to
explore the sample codes listed in this book.
The discussion about string-specic data structures (Sux Tree and Sux Array) is deferred
until Section 6.4. Yet, there are still many other data structures that we cannot cover in this book.
If you want to win a programming contest, mastering more than what we present in this book will
increase your chance to do better in contests, e.g. AVL Tree, Red Black Tree, Splay Tree
which are useful for certain contest problems where you need to implement and augment a balanced
BST; Fenwick (Binary Indexed) Tree which can be used to implement cumulative frequency
tables; Quad Tree for partitioning 2-D space; etc.
Notice that many of the ecient data structures shown in this book have the spirit of Divide
and Conquer (discussed in Section 3.2).
Chapter 3
This chapter highlights four problem solving paradigms commonly used to attack problems in programming
contests, namely Complete Search, Divide & Conquer, Greedy, and Dynamic Programming. Mastery of
all these problem solving paradigms will help contestants to attack each problem with the appropriate tool,
rather than hammering every problem with brute-force solution... which is clearly not competitive. Our advice before you start reading: Do not just remember the solutions for the problems presented in this chapter,
but remember the way, the spirit of solving those problems!
3.1
Complete Search
Complete Search, also known as brute force or recursive backtracking, is a method for solving a
problem by searching (up to) the entire search space in bid to obtain the required solution.
In programming contests, a contestant should develop a Complete Search solution when there
is clearly no clever algorithm available (e.g. the problem of enumerating all permutations of
{1, 2, 3, .., N }, which clearly requires O(N !) operations) or when such clever algorithms exist, but
overkill, as the input size happens to be small (e.g. the problem of answering Range Minimum
Query as in Section 2.3.3 but on a static array with N 100 solvable with O(N ) loop).
In ICPC, Complete Search should be the rst considered solution as it is usually easy to come
up with the solution and code/debug it. A bug-free Complete Search solution should never receive
Wrong Answer (WA) response in programming contests as it explores the entire search space.
However, many programming problems do have better-than-Complete-Search solutions. Thus a
Complete Search solution may receive a Time Limit Exceeded (TLE) verdict. With proper analysis,
you can determine which is the likely outcome (TLE versus AC) before attempting to code anything
(Table 1.4 in Section 1.2.2 is a good gauge). If Complete Search can likely pass the time limit,
then go ahead. This will then give you more time to work on the harder problems, where Complete
Search is too slow. In IOI, we usually need better problem solving techniques.
Sometimes, running Complete Search on small instances of a hard problem can give us some
patterns from the output that can be exploited to design faster algorithm. The Complete Search
solution can also act as a verier for faster but non-trivial algorithms (but only on small instances).
In this section, we give two examples of this simple paradigm and provide a few tips to give
Complete Search solution a better chance to pass the required Time Limit.
3.1.1
Examples
We show two examples of Complete Search: one that is implemented iteratively and one that is
implemented recursively (backtracking). We also mention a few optimization tricks to make some
row) {
<= queen - 1; prev++) // check previously placed queens
(abs(x[prev] - row) == abs(prev - queen)))
infeasible solution if share same row or same diagonal
int main() {
scanf("%d", &TC);
while (TC--) {
scanf("%d %d", &a, &b);
memset(x, 0, sizeof x); lineCounter = 0;
printf("SOLN
COLUMN\n");
printf(" #
1 2 3 4 5 6 7 8\n\n");
NQueens(1); // generate all possible 8! candidate solutions
if (TC) printf("\n");
}
return 0;
}
3.1.2
Tips
The biggest gamble in writing a Complete Search solution is whether it will be able to pass the
Time Limit. If it is 1 minute and your program currently runs in 1 minute 5 seconds, you may
want to tweak the critical code1 of your program rst rather than painfully redo the problem with
a faster algorithm which may not be trivial to design.
Here are some tips that you may want to consider when designing your solution, especially a
Complete Search solution, to give it a higher chance for passing the Time Limit.
Tip 1: Generating versus Filtering
Programs that generate lots of candidate solutions and then choose the ones that are correct
(or remove the incorrect ones) are called lters recall the nave 8-queens solver with 88 time
complexity. Those that hone in exactly to the correct answer without any false starts are called
generators recall the improved 8-queens solver with 8! complexity plus diagonal checks.
Generally, lters are easier to code but run slower. Do the math to see if a lter is good enough
or if you need to create a generator.
Tip 2: Prune Infeasible Search Space Early
In generating solutions (see tip 1 above), we may encounter a partial solution that will never lead
to a full solution. We can prune the search there and explore other parts. For example, see the
diagonal check in 8-queens solution above. Suppose we have placed a queen at row[1] = 2, then
placing another queen at row[2] = 1 or row[2] = 3 will cause a diagonal conict and placing
another queen at row[2] = 2 will cause a row conict. Continuing from any of these branches will
never lead to a valid solution. Thus we can prune these branches right at this juncture, concentrate
on only valid positions of row[2] = {4, 5, 6, 7, 8}, thus saving overall runtime.
Tip 3: Utilize Symmetries
Some problems have symmetries and we should try to exploit symmetries to reduce execution time!
In the 8-queens problem, there are 92 solutions but there are only 12 unique (or fundamental)
solutions as there are rotations and reections symmetries in this problem [35]. You can utilize this
fact by only generating the 12 unique solutions and, if needed, generate the whole 92 by rotating
and reecting these 12 unique solutions.
Tip 4: Pre-Computation a.k.a. Pre-Calculation
Sometimes it is helpful to generate tables or other data structures that enable the fastest possible
lookup of a result - prior to the execution of the program itself. This is called Pre-Computation,
in which one trades memory/space for time.
Again using the 8-queens problem above. If we know that there are only 92 solutions, then
we can create a 2-dimensional array int solution[92][8] and then ll it with all 92 valid permutations of 8 queens row positions! Thats it, we create a generator program (which takes some
runtime) to ll this 2-D array solution, but afterwards, we generate a new program and submit
the code that just prints out the correct permutations with 1 queen at (a, b) (very fast).
1
It is said that every program is doing most of its task in only about 10% of the code the critical code.
Surprisingly, some contest problems seem far easier when they are solved backwards than when
they are solved using a frontal attack. Be on the lookout for processing data in reverse order or
building an attack that looks at the data in some order other than the obvious.
This tip is best shown using an example: UVa 10360 - Rat Attack. Abridged problem descrip-
tion: Imagine a 2-D array (up to 1024 x 1024) containing rats. There are n 20000 rats at some
cells, determine which cell (x, y) should be gas-bombed so that the number of rats killed in square
First option is to attack this problem frontally: Try bombing each of the 10242 cells and see which
one is the most eective. For each bombed cell (x, y), we need to do O(d2 ) scans to count the
number of rats killed within the square-bombing radius. For the worst case when the array has
size 10242 and d = 50, this takes 10242 502 = 2621M operations. Clearly TLE!
Second option is to attack this problem backwards: Create an array int killed[1024][1024].
For each n rat population at coordinate (x, y), add the value of array killed[i][j] with the
number of rats in (x, y) that will be killed if a bomb is placed in (i, j) and (i, j) is within the
operations. Then, to determine the most optimal bombing position, we nd the coordinate of the
highest entry in array killed, which can be done in O(n2 ) operations. This backwards approach
only requires 20000 502 + 200002 = 51M operations for the worst test case (n = 20000, d = 50)
There are many tricks that you can use to optimize your code. Understanding computer hardware,
especially I/O, memory, and cache behavior, can help you design a better program. Some examples:
1. Use the faster C-style scanf/printf rather than cin/cout.
2. Use the expected O(n log n) but cache-friendly quicksort (built-in in C++ STL sort as part
of introsort) rather than the true O(n log n) but not (cache) memory friendly mergesort.
3. Access a 2-D array in a row major fashion (row by row) rather than column by column.
4. Bitwise manipulation on integer is faster than using an array of bits (see Section 3.4.3), etc.
5. Use STL <bitset> rather than vector<bool> for Sieve of Eratosthenes (see Section 5.3.1).
6. Declare a bulky data structure just once by setting it to have global scope, so you do not
have to pass the structure as function arguments.
7. Allocate memory just once, according to the largest possible input in the problem description,
rather than re-allocating it for every test case in a multiple-input problem.
Browse the Internet or reference books to nd more information on how to speed up your code.
Tip 7: Use Better Data Structure & Algorithm :)
No kidding. Using better data structures and algorithms always outperforms any optimization tips
mentioned in Tips 1-6 above. If all else fails, abandon Complete Search approach.
Programming Exercises solvable using Complete Search:
Iterative
1. UVa 154 - Recyling (try all combinations)
2. UVa 441 - Lotto (6 nested loops!)
3. UVa 639 - Dont Get Rooked (generate 216 possible combinations, prune invalid ones)
4. UVa 725 - Division (elaborated in this section)
5. UVa 10360 - Rat Attack (this problem is also solvable using 10242 DP range sum)
6. UVa 10662 - The Wedding (3 nested loops!)
7. UVa 11242 - Tour de France (iterative complete search + sorting)
8. UVa 11804 - Argentina (5 nested loops!)
Recursive Backtracking
1. UVa 193 - Graph Coloring (Maximum Independent Set)
2. UVa 222 - Budget Travel (input not large)
3. UVa 524 - Prime Ring Problem (also see Section 5.3.1)
4. UVa 624 - CD (input size is small, use backtracking; also solve-able with DP)
5. UVa 628 - Passwords (backtracking)
6. UVa 729 - The Hamming Distance Problem (backtracking)
7. UVa 750 - 8 Queens Chess Problem (solution already shown in this section)
8. UVa 10285 - Longest Run on a Snowboard (backtracking, also solve-able with DP)
9. UVa 10496 - Collecting Beepers (small TSP instance)
10. LA 4793 - Robots on Ice (World Finals Harbin10, recommended problem for practice)
Problem I - Robots on Ice in the recent ACM ICPC World Final 2010 can be viewed as a tough
test on pruning strategy. The problem is simple: Given an M x N board with 3 check-in points
{A, B, C}, nd a Hamiltonian path of length (M x N) from coordinate (0, 0) to coordinate
(0, 1). This Hamiltonian path must hit check point {A, B, C} at one-fourth, one-half, and
three-fourths of the way through its tour, respectively. Constraints: 2 M, N 8.
A nave recursive backtracking algorithm will get TLE. To speed up, we must prune the search
space if: 1). it does not hit the appropriate target check point at 1/4, 1/2, or 3/4 distance; 2).
it hits target check point earlier than the target time; 3). it will not be able to reach the next
check point on time from the current position; 4). it will not be able to reach nal point (0, 1)
as the current path blocks the way. These 4 pruning strategies are sucient to solve LA 4793.
3.2
Divide and Conquer (abbreviated as D&C) is a problem solving paradigm where we try to make a
problem simpler by dividing it into smaller parts and conquering them. The steps:
1. Divide the original problem into sub-problems usually by half or nearly half,
2. Find (sub-)solutions for each of these sub-problems which are now easier,
3. If needed, combine the sub-solutions to produce a complete solution for the main problem.
We have seen this D&C paradigm in previous chapters in this book: various sorting algorithms like
Quick Sort, Merge Sort, Heap Sort, and Binary Search in Section 2.2.1 utilize this paradigm. The
way data is organized in Binary Search Tree, Heap, and Segment Tree in Section 2.2.2 & 2.3.3, also
3.2.1
In this section, we discuss the spirit of D&C paradigm around a well-known Binary Search algo-
rithm. We still classify Binary Search as Divide and Conquer paradigm although some references
(e.g. [14]) suggest that it should be classied as Decrease (by-half) and Conquer as it does not
actually combine the result. We highlight this algorithm because many contestants know it, but
not many are aware that it can be used in other ways than its ordinary usage.
Recall: The ordinary usage of Binary Search is for searching an item in a static sorted array. We
check the middle portion of the sorted array if it is what we are looking for. If it is or there is
no more item to search, we stop. Otherwise, we decide whether the answer is on the left or right
portion of the sorted array. As the size of search space is halved (binary) after each query, the
complexity of this algorithm is O(log n). In Section 2.2.1, we have seen that this algorithm has
library routines, e.g. C++ STL <algorithm>: lower bound, Java Collections.binarySearch.
This is not the only way to use and apply binary search. The pre-requisite to run binary search
algorithm a static sorted array (or vector) can also be found in other uncommon data structure,
Binary Search on Uncommon Data Structure (Thailand ICPC National Contest 2009)
Problem in short: given a weighted (family) tree of N vertices up to N 80K with a special trait:
vertex values are increasing from root to leaves. Find the ancestor vertex closest to root from a
starting vertex v that has weight at least P . There are up to Q 20K such queries.
Nave solution is to do this linear O(N ) scan per query: Start from a given vertex v, then move
up the family tree until we hit the rst ancestor with value < P . In overall, as there are Q queries,
this approach runs in O(QN ) and will get TLE as N 80K and Q 20K.
A better solution is to store all the 20K queries rst. Then traverse the family tree just once
from root using O(N ) Depth First Search (DFS) algorithm (Section 4.2). Search for some non-
existent value so that DFS explores the entire tree, building a partial root-to-leaf sorted array as it
goes this is because the vertices in the root-to-leaf path have increasing weights. Then, for each
vertex asked in query, perform a O(log N ) binary search, i.e. lower bound, along the current
path from root to that vertex to get ancestor closest to root with weight at least P . Finally, do
an O(Q) post-processing to output the results. The overall time complexity of this approach is
O(Q log N ), which is now manageable.
Bisection Method
What we have seen so far are the usage of binary search in nding items in a static sorted array.
However, the binary search principle can also be used to nd the root of a function that may be
dicult to compute mathematically.
Sample problem: You want to buy a car using loan and want to pay in monthly installments of
d dollars for m months. Suppose the value of the car is originally v dollars and the bank charges
i% interest rate for every unpaid money at the end of each month. What is the amount of money
d that you must pay per month (rounded to 2 digits after decimal point)?
Suppose d = 576, m = 2, v = 1000, and i = 10%. After one month, your loan becomes 1000
(1.1) - 576 = 524. After two months, your loan becomes 524 (1.1) - 576 0.
But if we are only given m = 2, v = 1000, and i = 10%, how to determine that d = 576? In
another words, nd the root d such that loan payment function f (d, 2, 1000, 10) 0. The easy
way is to run the bisection method2 . We pick a reasonable range as the starting point. In this
case, we want to nd d within range [a . . . b]. a = 1 as we have to pay something (at least d = 1
dollar). b = (1 + i) v as the earliest we can complete the payment is m = 1, if we pay exactly
(1 + i%) v = (1 10) 1000 = 1100 dollars after one month. Then, we apply bisection method
to obtain d as follows:
If we pay d = (a + b)/2 = (1 + 1100)/2 = 550.5 dollars per month, then we undershoot by
53.95 dollars after two months, so we know that we must increase the monthly payment.
If we pay d = (550.5 + 1100)/2 = 825.25 dollars per month, then we overshoot by 523.025
dollars after two months, so we know that we must decrease the payment.
If we pay d = (550.5 + 825.25)/2 = 687.875 dollars per month, then we overshoot by 234.5375
dollars after two months, so we know that we must decrease the payment.
... few logarithmic iterations after, to be precise, after O(log2 ((b a)/)) iterations where
is the amount of error that we can tolerate.
Finally, if we pay d = 576.190476... dollars per month, then we manage to nish the payment
(the error is now less than ) after two months, so we know that d = 576 is the answer.
For bisection method to work3 , we must ensure that the function values of the two extreme points
in the initial Real range [a . . . b], i.e. f (a) and f (b) have opposite signs (true in the problem above).
Bisection method in this example only takes log2 1099/ tries. Using a small = 1e-9, this is just
40 tries. Even if we use an even smaller = 1e-15, we still just need 60 tries4 . Bisection
method is more ecient compared to linearly trying each possible value of d = [1..1100]/.
2
We use the term binary search principle as a divide and conquer technique that involve halving the range
of possible answer. We use the term binary search algorithm (nding index of certain item in sorted array) and
bisection method (nding root of a function) as instances of this principle.
3
Note that the requirement of bisection method (which uses binary search principle) is slightly dierent from the
more well-known binary search algorithm which needs a sorted array.
4
Thus some competitive programmers choose to do loop 100 times which guarantees termination instead of
testing whether the error is now less than as some oating point errors may lead to endless loop.
3.3
Greedy
An algorithm is said to be greedy if it makes locally optimal choice at each step with the hope of
nding the optimal solution. For some cases, greedy works - the solution code becomes short and
runs eciently. But for many others, it does not. As discussed in [4], a problem must exhibit two
3.3.1
Classical Example
Suppose we have a large number of coins with dierent denominations, i.e. 25, 10, 5, and 1 cents.
We want to make change with the least number of coins used. The following greedy algorithm
works for this set of denominations of this problem: Keep using the largest denomination of coin
which is not greater than the remaining amount to be made. Example: if the denominations are
{25, 10, 5, 1} cents and we want to make a change of 42 cents, we can do: 42-25 = 17 17-10 =
The coin changing example above has the two ingredients for a successful greedy algorithm:
1. It has optimal sub-structures.
We have seen that in the original problem to make 42 cents, we have to use 25+10+5+1+1.
This is an optimal 5 coins solution to the original problem!
Now, the optimal solutions to its sub-problems are contained in this 5 coins solution, i.e.
a. To make 17 cents, we have to use 10+5+1+1 (4 coins),
b. To make 7 cents, we have to use 5+1+1 (3 coins), etc
2. It has a greedy property.
Given every amount V , we greedily subtract it with the largest denomination of coin which
is not greater than this amount V . It can be proven (not shown here for brevity) that using
other strategy than this will not lead to optimal solution.
However, this greedy algorithm does not work for all sets of coin denominations, e.g. {1, 3, 4}
cents. To make 6 cents with that set, a greedy algorithm would choose 3 coins {4, 1, 1} instead of
the optimal solution using 2 coins {3, 3}. This problem is revisited later in Section 3.4.2.
There are many other classical examples of greedy algorithms in algorithm textbooks, for ex-
ample: Kruskals for Minimum Spanning Tree (MST) problem Section 4.4, Dijkstras for Single-
Source Shortest Paths (SSSP) problem Section 4.5, Greedy Activity Selection Problem [4], Hu-
3.3.2
Todays contest problems usually do not ask for solution of trivial and classical greedy problems.
Instead, we are presented with novel ones that requires creativity, like the one shown below.
a list of mass of the S specimens, determine in which chamber we should store each specimen in
A=(
j=1 Mj )/C,
IMBALANCE =
C
i=1 |Xi
A|, i.e. sum of dierences between the mass in each chamber w.r.t A.
This problem can be solved using a greedy algorithm. But rst, we have to make several observa-
tions. If there exists an empty chamber, at least one chamber with 2 specimens must be moved to
this empty chamber! Otherwise the empty chambers contribute too much to IMBALANCE! See
Figure 3.4.
Next observation: If S > C, then S C specimens must be paired with one other specimen already
Now, the key insight that can simplify the problem is this: If S < 2C, add dummy 2C S specimens
Then, sort these specimens based on their mass such that M1 M2 . . . M2C1 M2C . In
this example, M = {5, 1, 2, 7, 0, 0} {0, 0, 1, 2, 5, 7}.
By adding dummy specimens and then sorting them, a greedy strategy appears. We can now:
Pair the specimens with masses M1 &M2C and put them in chamber 1, then
Pair the specimens with masses M2 &M2C1 and put them in chamber 2, and so on . . .
This greedy algorithm known as Load Balancing works! See Figure 3.6.
3.3.3
Using Greedy solutions in programming contests is usually risky. A greedy solution normally will
not encounter TLE response, as it is lightweight, but tends to get WA response. Proving that
a certain problem has optimal sub-structure and greedy property in contest time may be time
consuming, so a competitive programmer usually do this:
He will look at the input size. If it is small enough for the time complexity of either Complete
Search or Dynamic Programming (see Section 3.4), he will use one of these approaches as both will
ensure correct answer. He will only use Greedy solution if he knows for sure that the input size
given in the problem is too large for his best Complete Search or DP solution.
Having said that, it is quite true that many problem setters nowadays set the input size of such
can-use-greedy-algorithm-or-not-problems to be in some reasonable range so contestants cannot
use the input size to quickly determine the required algorithm!
Programming Exercises solvable using Greedy (hints omitted):
1. UVa 410 - Station Balance (elaborated in this section)
2. UVa 10020 - Minimal Coverage
3. UVa 10340 - All in All
4. UVa 10440 - Ferry Loading II
5. UVa 10670 - Work Reduction
6. UVa 10763 - Foreign Exchange
7. UVa 11054 - Wine Trading in Gergovia
8. UVa 11292 - Dragon of Loowater
9. UVa 11369 - Shopaholic
In this section, we want to highlight another problem solving trick called: Decomposition!
While there are only few basic algorithms used in contest problems (most of them are covered
in this book), harder problems may require a combination of two (or more) algorithms for their
solution. For such problems, try to decompose parts of the problems so that you can solve
the dierent parts independently. We illustrate this decomposition technique using a recent
top-level programming problems that combines three problem solving paradigms that we have
just learned: Complete Search, Divide & Conquer, and Greedy!
ACM ICPC World Final 2009 - Problem A - A Careful Approach
You are given a scenario of airplane landings. There are 2 n 8 airplanes in the scenario.
Each airplane has a time window during which it can safely land. This time window is specied
by two integers ai , bi , which give the beginning and end of a closed interval [ai , bi ] during which
the i-th plane can land safely. The numbers ai and bi are specied in minutes and satisfy
0 ai bi 1440. In this problem, plane landing time is negligible. Then, your task is to:
1. Compute an order for landing all airplanes that respects these time windows.
HINT: order = permutation = Complete Search?
2. Furthermore, the airplane landings should be stretched out as much as possible so that the
minimum achievable time gap between successive landings is as large as possible. For example,
if three airplanes land at 10:00am, 10:05am, and 10:15am, then the smallest gap is ve minutes,
which occurs between the rst two airplanes. Not all gaps have to be the same, but the smallest
gap should be as large as possible!
HINT: Is this similar to greedy activity selection problem [4]?
3. Print the answer split into minutes and seconds, rounded to the closest second.
See Figure 3.7 for illustration: line = the time window of a plane; star = its landing schedule.
int main() {
while (scanf("%d", &n), n) { // 2 <= n <= 8
for (i = 0; i < n; i++) {
scanf("%lf %lf", &a[i], &b[i]); // [ai, bi] is the interval where plane i can land safely
a[i] *= 60; b[i] *= 60; // originally in minutes, convert to seconds
order[i] = i;
}
3.4
Dynamic Programming
Dynamic Programming (from now on abbreviated as DP) is perhaps the most challenging problem
solving paradigm among the four paradigms discussed in this chapter. Therefore, make sure that
you have mastered the material mentioned in the previous chapters before continuing. Plus, get
3.4.1
Abridged problem statement: Given dierent models for each garment (e.g. 3 shirts, 2 belts, 4
shoes, ...), buy one model of each garment. As the budget is limited, we cannot spend more money
than the budget, but we want to spend the maximum possible. But it is also possible that we
cannot buy one model of each garment due to that small amount of budget.
The input consist of two integers 1 M 200 and 1 C 20, where M is the budget and C is
the number of garments that you have to buy. Then, there are information of the C garments. For
models for that garment id, followed by K integers indicating the price of each model [1 . . . K]
to buy one element of each garment without exceeding the initial amount of money. If there is no
M = 20, C = 3
3 models of garment id 0 6 4 8 // see that the prices are not sorted in input
2 models of garment id 1 5 10
4 models of garment id 2 1 5 3 5
Then the answer is 19, which may come from buying the underlined items (8+10+1).
Note that this solution is not unique, as we also have (6+10+3) and (4+10+5).
3 models of garment id 0 6 4 8
2 models of garment id 1 5 10
4 models of garment id 2 1 5 3 5
Then the answer is no solution as buying all the cheapest models (4+5+1) = 10 is still > M .
First, lets see if Complete Search (backtracking) can solve this problem: Start with money_left = M
and garment_id = 0. Try all possible models in that garment_id = 0 (max 20 models). If model
i is chosen, then subtract money_left with model is price, and then recursively do the same
process to garment_id = 1 (also can go up to 20 models), etc. Stop if the model for the last
garment_id = C - 1 has been chosen. If money_left < 0 before we reach the last garment_id,
prune this partial solution. Among all valid combinations, pick one that makes money_left as
close to 0 as possible yet still 0. This maximizes the money spent, which is (M - money_left).
shop(money_left, garment_id) = INVALID (in practice, we can return a large negative value)
2. if a model from the last garment_id has been bought (i.e. a candidate solution),
shop(money_left, garment_id) =
max(shop(money_left - price[garment_id][model], garment_id + 1))
We want to maximize this value (Recall that invalid ones have large negative value)
This solution works correctly, but very slow! Lets analyze its worst case time complexity. In the
largest test case, garment id 0 have up to 20 choices; garment id 1 also have up to 20 choices; ...;
and the last garment id 19 also have up to 20 choices. Therefore, Complete Search like this runs
in 20 20 ... 20 of total 20 times in the worst case, i.e. 2020 = a very very large number. If
we only know Complete Search, there is no way we can solve this problem.
Since we want to maximize the budget spent, why dont we take the most expensive model in each
garment_id which still ts our budget? For example in test case A above, we choose the most
expensive model 3 of garment_id = 0 with cost 8 (money_left = 20-8 = 12), then choose the
most expensive model 2 of garment_id = 1 with cost 10 (money_left = 12-10 = 2), and then for
garment_id = 2, we can only choose model 1 with cost 1 as money_left does not allow us to buy
other models with cost 3 or 5. This greedy strategy works for test cases A+B above and produce
the same optimal solution (8+10+1) = 19 and no solution, respectively. It also runs very fast,
But greedy does not work for many other cases. This test case below is a counter-example:
M = 12, C = 3
3 models of garment id 0 6 4 8
2 models of garment id 1 5 10
4 models of garment id 2 1 5 3 5
Greedy strategy selects model 3 of garment_id = 0 with cost 8 (money_left = 12-8 = 4), thus
we do not have enough money to buy any model in garment_id = 1 and wrongly reports no
solution. The optimal solution is actually (4+5+3 = 12), which use all our budget.
To solve this problem, we have to use DP. Lets see the key ingredients to make DP works:
1. This problem has optimal sub-structures.
This is shown in Complete Search recurrence 3 above: solution for the sub-problem is part of
the solution of the original problem. Although optimal sub-structure are the same ingredient
to make a Greedy Algorithm work, this problem lacks the greedy property ingredient.
2. This problem has overlapping sub-problems.
This is the key point of DP! The search space is actually not as big as 2020 analyzed in
Complete Search discussion above as many sub-problems are actually overlapping!
Lets verify if this problem has overlapping sub-problems. Suppose that there are 2 models in certain
garment_id with the same price p. Then, Complete Search will move to the same sub-problem
shop(money_left - p, garment_id + 1) after picking either model! Similarly, this situation
also occur if some combination of money_left and chosen models price causes money_left1 - p1
= money_left2 - p2 . This same sub-problem will be computed more than once! Inecient!
So, how many distinct sub-problems (a.k.a. states) are there in this problem? The answer
is, only 201 20 = 4,020. As there only there are only 201 possible money_left (from 0 to 200,
inclusive) and 20 possible garment_id (from 0 to 19, inclusive). Each of the sub-problem just need
to be computed only once. If we can ensure this, we can solve this problem much faster.
Implementation of this DP solution is surprisingly simple. If we already have the recursive
backtracking (the recurrence relation shown previously), we can implement top-down DP by
Analyzing DP solution is easy. If it has M states, then it requires at least O(M ) memory space. If
lling a cell in this state requires O(k) steps, then the overall time complexity is O(kM ). UVa 11450
- Wedding Shopping problem above has M = 201 20 = 4020 and k = 20 (as we have to iterate
through at most 20 models for each garment_id). Thus the time complexity is 4020 20 = 80400,
which is very manageable. We show our code below as an illustration, especially for those who
}
} // return 0;
There is another style of writing DP solutions, called the bottom-up DP. This is actually the true
form of DP a.k.a. the tabular method. The steps to build bottom-up DP are like this:
1. Identify the Complete Search recurrence as with top-down DP above.
2. Initialize some parts of the DP table with known initial values.
3. Determine how to ll the rest of the DP table based on the Complete Search recurrence,
usually involving one or more nested loops to do so.
For UVa 11450 above, we can write the bottom-up DP as follow. For clarity of this discussion,
rst, only the cells reachable by buying any of the models of garment_id = 0 are true. See Figure
3.8, leftmost, where only rows 20-6 = 14, 20-4 = 16, and 20-8 = 12 in column 0, are true.
Then, we loop through from second garment until the last garment. We set can_reach[a][b]
to be true if it is possible to reach this state from any states in previous column, i.e. from state
can_reach[a + price of any model of previous garment_id][b - 1]. See Figure 3.8, middle,
where for example, can_reach[11][1] can be reached from can_reach[11 + 5][0] by buying a
model with cost 5 in garment_id = 1; can_reach[2][1] can be reached from can_reach[2 + 10][0]
index 0 that is set to be true. In Figure 3.8, rightmost, the cell can_reach[1][2] is the answer.
This means that we can somehow reach this state (money_left = 1) by buying combination of
various garment models. The nal answer is actually M - money_left, or in this case, 20-1 = 19.
The answer is no solution if there is no cell in the last column that is set to be true.
#include <iostream>
#include <string.h>
using namespace std;
int main() {
int i, j, l, TC, M, C, K, price[25][25]; // price[garment_id (<= 20)][model (<= 20)]
bool can_reach[210][25]; // can_reach table[money_left (<= 200)][garment_id (<= 20)]
// question: is 2nd dimension (model) needed? M = (201*20) -> (201) only?
scanf("%d", &TC);
// can we compute the solution by just maintaining 2 most recent columns?
while (TC--) {
// hint: DP-on-the-fly (a.k.a space saving trick)
scanf("%d %d", &M, &C);
for (i = 0; i < C; i++) {
scanf("%d", &K);
price[i][0] = K; // to simplify coding, we store K in price[i][0]
for (j = 1; j <= K; j++)
scanf("%d", &price[i][j]);
}
memset(can_reach, false, sizeof can_reach); // clear everything
for (i = 1; i <= price[0][0]; i++) // initial values
can_reach[M - price[0][i]][0] = true; // if only using first garment_id
// Challenge: this loop is written in column major, rewrite it in row major!
// See Tips 6.3 in Section 3.1.2
for (j = 1; j < C; j++) // for each remaining garment
for (i = 0; i < M; i++) if (can_reach[i][j - 1]) // if can reach this state
for (l = 1; l <= price[j][0]; l++) if (i - price[j][l] >= 0) // flag the rest
can_reach[i - price[j][l]][j] = true; // as long as it is feasible
for (i = 0; i <= M && !can_reach[i][C - 1]; i++); // the answer is in the last column
Bottom-Up
Pro:
1. Faster if many sub-problems are revisited
as there is no overhead from recursive calls
2. Can save memory space with DP on-they technique (see comment in code above)
Cons:
1. For some programmers who are inclined
with recursion, this may be not intuitive
2. If there are M states, bottom-up DP visits
and lls the value of all these M states
3.4.2
There are many other well-known problems with ecient DP solutions. We consider these problems
as classic and therefore must be known by everyone who wish to do well in ICPC or IOI!
Longest Increasing Subsequence (LIS)
Problem: Given a sequence {X[0], X[1], . . . , X[N-1]}, determine its Longest Increasing Subsequence
(LIS)5 as the name implies. Take note that subsequence is not necessarily contiguous.
Example:
N = 8, sequence = {-7, 10, 9, 2, 3, 8, 8, 1}
The LIS is {-7, 2, 3, 8} of length 4.
Solution: Let LIS(i) be the LIS ending in index i, then we have these recurrences:
1. LIS(0) = 1 // base case
2. LIS(i) = ans, computed with a loop below:
int ans = 1;
for (int j = 0; j < i; j++) // O(n)
if (X[i] > X[j]) // if we can extend LIS(j) with i
ans = max(ans, 1 + LIS(j))
5
There are other variants of this problem: Longest Decreasing Subsequence, Longest Non Increasing/Decreasing
Subsequence, and the O(n log k) solution by utilizing the fact that the LIS is sorted and binary-searchable. See
http://en.wikipedia.org/wiki/Longest increasing subsequence for more details. Note that increasing subsequences can be modeled as a Directed Acyclic Graph (DAG). Thus nding LIS is equivalent to nding longest path
in DAG.
The answer is the highest value of LIS(k) for all k in range [0 . . . N-1].
LIS(3) is also 2, we can only form {-7, 2}. {-7, 10} + {2} or {-7, 9} + {2} are both non increasing.
LIS(4) is now 3, as we can extend {-7, 2} + {3} and this is the longest among other forms.
And so on until LIS(7). The answers are in LIS(5) or LIS(6), both with length 4.
There are clearly many overlapping sub-problems in LIS problem, but there are only N states, i.e.
the LIS ending at index i, for all i [0 . . . N-1]. As we need to compute each state with an O(n)
loop, then this DP algorithm runs in O(n2 ). The LIS solution(s) can be reconstructed by following
the arrows via some backtracking routine (scrutinize the arrows in Figure 3.9 for LIS(5) or LIS(6)).
Coin Change (CC) - The General Version
Problem6 : Given a target amount V cents and a list of denominations of N coins, i.e. We have
coinValue[i] (in cents) for coin type i [0 . . . N-1], what is the minimum number of coins that we
must use to obtain amount V ? Assume that we have unlimited supply of coins of any type.
Example 1:
V = 10, N = 2, coinValue = {1, 5}
We can use:
A. Ten 1 cent coins = 10 1 = 10; Total coins used = 10
B. One 5 cents coin + Five 1 cent coins = 1 5 + 5 1 = 10; Total coins used = 6
C. Two 5 cents coins = 2 5 = 10; Total coins used = 2 Optimal
Recall that we can use greedy solution if the coin denominations are suitable as in Example 1
above (See Section 3.3). But for general cases, we have to use DP as in Example 2 below:
Example 2:
V = 7, N = 4, coinValue = {1, 3, 4, 5}
Greedy solution will answer 3, using 5+1+1 = 7, but optimal solution is 2, using 4+3 only!
Solution: Use these Complete Search recurrences:
1. coin(0) = 0 // 0 coin to produce 0 cent
2. coin(< 0) = INVALID (in practice, we just return a large positive value)
3. coin(value) = 1 + min(coin(value - coinValue[i])) i [0..N-1]
The answer is in coin(V).
6
There are other variants of this problem, e.g. counting how many ways to do coin change and a variant where
supply of coins are limited.
We can see that there are a lot of overlapping sub-problems in this Coin Change problem, but there
are only O(V ) possible states! As we need to try O(N ) coins per state, the overall time complexity
of this DP solution is O(V N ).
Maximum Sum
Abridged problem statement: Given n x n (1 n 100) array of integers, each ranging from [-127,
127], nd a minimum sub-rectangle that have the maximum value.
Example: The 4 x 4 array (n = 4) below has 15 as the maximum sub-rectangle value.
0
-2
-7
-6
==>
-4
-4
==>
| -4
1 |
-1
-2
==>
| -1
8 |
2 | = 9 + 2 - 4 + 1 - 1 + 8 = 15
Navely attacking this problem as shown below does not work as it is an 1006 algorithm.
maxSubRect = -127*100*100; // lowest possible value for this problem
for (int i = 0; i < n; i++) for (int j = 0; j < n; j++) // start coord
for (int k = i; k < n; k++) for (int l = j; l < n; l++) { // end coord
subRect = 0; // sum items in this sub-rectangle
for (int a = i; a <= k; a++) for (int b = j; b <= l; b++)
subRect += arr[a][b];
maxSubRect = max(maxSubRect, subRect); // keep largest so far
}
Solution: There are several (well-known) DP solutions for static range problem. DP can work in
this problem as computing a large sub-rectangle will denitely involves computing smaller sub-
One possible DP solution is to turn this n x n array into an n x n sum array where arr[i][j]
no longer contains its own value, but the sum of all items within sub-rectangle (0, 0) to (i, j). This
can easily be done on-the-y when reading the input and still O(n2 ).
scanf("%d", &n);
for (int i = 0; i < n; i++) for (int j =
scanf("%d", &arr[i][j]);
if (i > 0) arr[i][j] += arr[i - 1][j];
if (j > 0) arr[i][j] += arr[i][j - 1];
if (i > 0 && j > 0) arr[i][j] -= arr[i
} // inclusion-exclusion principle
0; j < n; j++) {
// if possible, add values from top
// if possible, add values from left
- 1][j - 1]; // to avoid double count
This code turns input array (shown in the left) into sum array (shown in the right):
0
-2
-7
==>
-2
-9
-9
-6
==>
-4
-4
-4
==>
6 -11
-8
-1
-2
==>
13
-4
-3
Now, with this sum array, we can answer the sum of any sub-rectangle (i, j) to (k, l) in O(1)! Suppose
we want to know the sum of (1, 2) to (3, 3). We can split the sum array into 4 sections and compute
arr[3][3] - arr[0][3] - arr[3][1] + arr[0][1] = -3 - 13 - (-9) + (-2) = -9.
0 [-2]| -9 [-9]
----------------9
9 | -4
6 |-11
-8
4 [13]| -4 [-3]
With this O(1) DP formulation, this problem can now be solved in 1004 .
maxSubRect = -127*100*100; // lowest possible value for this problem
for (int i = 0; i < n; i++) for (int j = 0; j < n; j++) // start coord
for (int k = i; k < n; k++) for (int l = j; l < n; l++) { // end coord
subRect = arr[k][l]; // this is sum of all items from (0, 0) to (k, l): O(1)
if (i > 0) subRect -= arr[i - 1][l]; // O(1)
if (j > 0) subRect -= arr[k][j - 1]; // O(1)
if (i > 0 && j > 0) subRect += arr[i - 1][j - 1]; // inclusion-exclusion: O(1)
maxSubRect = max(maxSubRect, subRect);
}
Lesson: Not every range problems require Segment Tree as in Section 2.3.3! Problems where the
input data is static like this usually is solvable with DP technique.
Exercise 1: The solution above runs in 1004 . There exist 1003 solution. Can you gure out how to
Exercise 2: What if the given static array is 1-D? Can you form a similar O(1) DP solution to
+ arr[j]?
Exercise 3: Use the solution from Exercise 2 to nd maximum sum in 1-D array in O(n2 ). Can
Exercise 4: Now, what if the query is range minimum query(i, j) on 1-D static array. The solution
in Section 2.3.3 uses Segment Tree. Can you just utilize DP to solve the same problem assuming
Remarks
There are other classical DP problems that we choose not to cover in this book such as Matrix
Chain Multiplication [4], Optimal Binary Search Tree [14], 0-1 Knapsack [5, 14]. However, Floyd
Warshalls will be discussed in Section 4.7; String Edit Distance, Longest Common Subsequence
3.4.3
Although DP is one problem type that is most frequently appears in recent programming contests,
the classical DP problems on their pure forms above usually never appear again in recent ICPCs or
IOIs. We have to study them to understand more about DP. But once we do, we have to move on
to solve many other non classical DP problems and build up our DP technique skills. As you can
see below, there are many interesting DP techniques from simple to intermediate. Try solving these
problems at the corresponding online judges to further improve your DP skills. We have grouped
and ordered the following DP problems from our perceived level of diculty (easier to harder).
There are DP trick lessons in each example. Enjoy :).
Abridged problem statement: Given a stick of length 1 l 1000, make 1 n 50 cuts to that
sticks (coordinates within range (0 . . . l) are given). The cost of cut is determined by the length of
the stick to be cut. Find a cutting sequence so that the overall cost is minimized!
Example: l = 100, n = 3, and cut coordinates: coord = {25, 50, 75} (already sorted)
Solution: Use this Complete Search recurrences + top-down DP (memoization): cut(left, right)
1. If (left + 1 = right) where left and right are indices in array coord,
then cut(left, right) = 0 as we are left with 1 segment, there is no need to cut anymore.
2. Otherwise, try all possible cutting points and pick the minimum cost using code below:
Lesson: The Complete Search recurrences above only has 50 50 possible left/right indices congu-
ration (states) and runs in just 50 50 50 = 125K operations! It is easy to convert this Complete
For each state, we record the minimum weight of previous matchings that must be done in order
to reach this state. As we want a perfect matching, then for a currently o bit i, we must nd
the best other o bit j from [i+1 . . . M -1] using one O(M ) loop. This check is again done using
bit operation, i.e. if (!(bit_mask & (1 << i))) and similarly for j. This algorithm runs in
O(M 2M ). In problem UVa 10911, M = 2N and 2 N 8, so this DP + bitmasks approach is
feasible. For more details, please study the code shown in Section 1.2.
Lesson: DP + bitmasks can solve small instances M 20 of matching on general graph.
Exercise: The code in Section 1.2 says that this break is necessary. do you understand why?,
with hint: it helps reducing time complexity from O((2N )2 22N ) to O((2N ) 22N ). Answer it!
Abridged problem statement: Given number of cities 3 n 50, available time 1 t 1000, and
two n n matrices (one gives a travel time and another gives tolls between two cities), choose a
route from the rst city 0 in such a way that one has to pay as little money for tolls as possible to
arrive at the last city n 1 within a certain time t. Output two information: the total tolls that
one is to minimize tolls along the route. Requirement two is to ensure that we arrive in last city
within allocated time, which may mean that we have to pay higher tolls in some part of the path.
Requirement two is a hard constraint. We must satisfy it, otherwise we do not have a solution.
It should be quite clear after trying several test cases that greedy Single-Source Shortest Paths
(SSSP) algorithm like Dijkstras (Section 4.5) on its pure form will not work as picking the
shortest travel time to ensure we use less than available time t may not lead to the smallest possible
tolls. On the other hand, picking the cheapest tolls along the route may not ensure that we arrive
within available time t. Both requirements cannot be made independent!
Solution: We can use the following Complete Search recurrences + top-down DP (memoization)
go(curCity, time left) that returns pairs of information (actual toll paid, actual time used):
1. go(any curCity, < 0) = make pair(INF, INF); // Cannot go further if we run out of time.
2. go(n 1, 0) = make pair(0, 0); // Arrive at last city, no need to pay/travel anymore.
3. In general case, go(curCity, time left) is best described using the C++ code below:
pair<int, int> go(int curCity, int time_left) { // top-down DP, returns a pair
if (time_left < 0) // invalid
return make_pair(INF, INF); // a trick: return large value so that this state is not chosen
if (curCity == n - 1) // at last city
return make_pair(0, 0); // no need to pay toll, and time needed is 0
if (memo[curCity][time_left].first != -1) // visited before
return memo[curCity][time_left]; // simply return the answer
pair<int, int> answer = make_pair(INF, INF);
for (int neighbor = 0; neighbor < n; neighbor++) // try to go to another city
if (curCity != neighbor) { // force us to always move
pair<int, int> nextCity = go(neighbor,
time_left - travelTime[curCity][neighbor]); // go to the other city
// among neighboring cities, pick the one that has minimum cost
if (nextCity.first + toll[curCity][neighbor] < answer.first) {
answer.first = nextCity.first + toll[curCity][neighbor];
answer.second = nextCity.second + travelTime[curCity][neighbor];
}
}
return memo[curCity][time_left] = answer;
} // store the answer to memo table as well as returning the answer to the caller
Lesson: Some graph shortest (longest) path problems look solvable with classical (usually greedy)
graph algorithms in Chapter 4, but in reality should be solved using DP techniques7 .
4. ACORN (ACM ICPC Singapore 2007 LA 4106)
Abridged problem statement: Given t oak trees, the height h of all trees, the height f that Jayjay
loses when it ies from one tree to another, 1 t, h 2000, 1 f 500, and the positions of acorns
on each of oak trees: acorn[tree][height], determine the maximum number of acorns that Jayjay
Figure 3.11: ACM ICPC Singapore 2007 - Jayjay the Flying Squirrel Collecting Acorns
7
Nave DP Solution: Use a memo table total[tree][height] that stores the best possible acorns
collected when Jayjay is on a certain tree at certain height. Then Jayjay recursively tries to either
go down (-1) unit on the same oak tree or ies (-f ) unit(s) to t1 other oak trees from this position.
This approach requires up to 2000 2000 = 4M states and has time complexity 4M 2000 = 8B
operations. This approach is clearly TLE!
Better DP Solution: We can actually ignore the information: on which tree Jayjay is currently
at as just memoizing the best among them is sucient. Set a table: dp[height] that stores the
best possible acorns collected when Jayjay is at this height. The bottom-up DP code that requires
only 2000 = 2K states and time complexity of 2000 2000 = 4M is as follow:
for (int tree = 0; tree < t; tree++) // initialization
dp[h] = max(dp[h], acorn[tree][h]);
for (int height = h - 1; height >= 0; height--)
for (int tree = 0; tree < t; tree++) {
acorn[tree][height] +=
max(acorn[tree][height + 1], // from this tree, +1 above
((height + f <= h) ? dp[height + f] : 0)); // best from tree at height + f
dp[height] = max(dp[height], acorn[tree][height]); // update this too
}
printf("%d\n", dp[0]); // solution will be here
Lesson: When nave DP states are too large causing the overall DP time complexity not-doable,
think of dierent ways other than the obvious to represent the possible states. Remember that no
programming contest problem is unsolvable, the problem setter must have known a trick.
5. Free Parentheses (ACM ICPC Jakarta 2008 LA 4143)
Abridged problem statement: You are given a simple arithmetic expression which consists of only
addition and subtraction operators, i.e. 1 - 2 + 3 - 4 - 5. You are free to put any parentheses
to the expression anywhere and as many as you want as long as the expression is still valid. How
many dierent numbers can you make? The answer for simple expression above is 6:
1 - 2 + 3 - 4 - 5
-7
1 - (2 + 3 - 4 - 5)
1 - (2 + 3) - 4 - 5
= -13
1 - 2 + 3 - (4 - 5)
1 - (2 + 3 - 4) - 5
1 - (2 + 3) - (4 - 5) =
-3
-5
The expression consists of only 2 N 30 non-negative numbers less than 100, separated by
addition or subtraction operators. There is no operator before the rst and after the last number.
1. We only need to put an open bracket after a - (negative) sign as it will reverse the meaning of
2. You can only put X close brackets if you already use X open brackets we need to store this
information to process the subproblems correctly;
3. The max value is 100 + 100 + ... 100 (30 times) = 3000 and the min value is 100 - 100 - ... 100 (29 times) = -2800 this information needs to be stored, as we will see below.
The DP states that are easier to identify:
1. idx the current position being processed, we need to know where we are now.
2. open number of open brackets, we need this information in order to produce valid expression.
But these two states are not unique yet. For example, this partial expression: 1-1+1-1... has
state idx=3 (indices: 0,1,2,3 have been processed), open=0 (cannot put close bracket anymore),
which sums to 0. Then, 1-(1+1-1)... also has the same state idx=3, open=0 and sums to 0. But
1-(1+1)-1... has the same state idx=3, open=0, but sums to -2. This DP state is not yet unique.
So we need additional value to distinguish them, i.e. the value val, to make these states unique.
We can represent all possible states of this problem with bool state[idx][open][val], a
3-D array. As val ranges from -2800 to 3000 (5801 distinct values), the number of states is
30 30 5801 5M with only O(1) processing per state fast enough. The code is shown below:
Lesson: DP formulation for this problem is not trivial. Try to nd a state representation that can
uniquely identify sub-problems. Make observations and consider attacking the problem from there.
Abridged problem statement: Given a weighted graph G, nd Max Weighted Independent Set
(MWIS) on it. A subset of vertices of graph G is said to be Independent Set (IS)8 if there is no
edge in G between any two vertices in the IS. Our task is to select an IS with maximum weight of
G. If graph G is a tree, this problem has ecient DP solution (see Figure 3.12).
C(v, false) =
ch
Answer will be in the root: max(C(root, false), C(root, true)). DP solution can be implemented
either in a top-down or bottom-up fashion. As there are only 2 V states, the DP solution runs
in O(kV ) where k is the average number of children of a vertex in the tree!
Lesson: Optimization problems on tree may be solved with DP techniques. The solution usually
involves passing information to parent and getting information from the children of a rooted tree.
3.4.4
In the past (1990s), if a contestant is good with DP, he can become a king of programming contests
as usually DP problems are the deciding problem. But now, mastering DP is a basic requirement!
You cannot do well in contests without this knowledge. However, we have to keep reminding the
readers of this book not to claim that they know DP by memorizing the solutions of classical
DP problems! Try to go to the basics of problem solving using DP: Determine states (the table)
8
Maximum Sum
1. UVa 108 - Maximum Sum (maximum 2-D sum, elaborated in this section)
2. UVa 836 - Largest Submatrix (maximum 2-D sum)
3. UVa 10074 - Take the Land (maximum 2-D sum)
4. UVa 10667 - Largest Block (maximum 2-D sum)
5. UVa 10827 - Maximum Sum on a Torus (maximum 2-D sum)
6. UVa 507 - Jill Rides Again (maximum 1-D sum/maximum consecutive subsequence)
7. UVa 10684 - The Jackpot (maximum 1-D sum/maximum consecutive subsequence)
DP + Bitmasks
1. UVa 10364 - Square (bitmask technique can be used)
2. UVa 10651 - Pebble Solitaire
3. UVa 10908 - Largest Square
4. UVa 10911 - Forming Quiz Teams (elaborated in this section)
5. LA 3136 - Fun Game (Beijing04)
6. PKU 2441 - Arrange the Bulls
DP on Graph Problem
1. UVa 590 - Always on the Run
2. UVa 910 - TV Game (straightforward)
3. UVa 10681 - Teobaldos Trip
4. UVa 10702 - Traveling Salesman
5. LA 4201 - Switch Bulbs (Dhaka08)
6. SPOJ 101 - Fishmonger
3.5
Chapter Notes
The main source of the Complete Search material in this chapter is the USACO training gateway
[18].
We adopt the name Complete Search rather than Brute-Force as we believe that some Complete Search solution can be clever and fast enough, although it is complete, e.g. Branch &
Bound and A* Search (both are not discussed yet in this book). We believe the term clever
Brute-Force is a bit self-contradicting.
Divide and Conquer paradigm is usually used in form of its popular algorithms: binary search
(bisection method), merge/quick/heap sort, and data structures: binary search tree, heap, segment
tree, etc. We will see more Divide and Conquer later in Computational Geometry (Section 7.5).
Basic Greedy and Dynamic Programming (DP) techniques techniques are always included in
popular algorithm textbooks, e.g. Introduction to Algorithms [4], Algorithm Design [13], Algorithm
[5], Programming Challenges [23], The Art of Algorithms and Programming Contests [16], Art of
Programming Contest [1]. However, to keep pace with the growing diculties and creativity of
these techniques, especially the DP techniques, we include more references from Internet: TopCoder
algorithm tutorial [25] and recent programming contests, e.g. ACM ICPC Regional Kaohsiung 2006,
Singapore 2007, Jakarta 2008, Central European Olympiad in Informatics 2009, etc.
However, for some real-life problems, especially those that are classied as NP-Complete [4],
many of the approaches discussed so far will not work. For example, 0-1 Knapsack Problem which
has O(nW ) DP complexity is too slow if W is big. For this, people use heuristics or local search:
Beam Search, Tabu Search [10], Genetic Algorithm, Ants Colony Optimization, etc.
Chapter 4
Graph
We Are All Connected
Heroes TV Series
Many real-life problems can be classied as graph problems. Some have ecient solutions. Some do not have
them yet. In this chapter, we learn various graph problems with known ecient solutions, ranging from basic
traversal, minimum spanning trees, shortest paths, and network ow algorithms.
4.1
In this chapter, we discuss graph problems that are commonly appear in programming contests,
the algorithms to solve them, and the practical implementations of these algorithms. The issue on
how to store graph information has been discussed earlier in Section 2.3.1.
We assume that the readers are familiar with the following terminologies: Vertices/Nodes,
Simple Graph, Sparse/Dense, Path, Cycle, Isolated versus Reachable Vertices, (Strongly) Connected Component, Sub-Graph, Tree/Forest, Complete Graph, Directed Acyclic Graph, Bipartite Graph, Euler/Hamiltonian Path/Cycle. If you encounter any unknown terms. Please go to
Wikipedia [46] and search for that particular term.
Table 4.1 summarizes our research so far on graph problems in recent ACM ICPC Asia regional
contests and its ICPC Live Archive [11] IDs. Although there are so many graph problems (many
are discussed in this chapter), they only appear either once or twice in a problem set. The question
is: Which ones that we have to learn? If you want to do well in contests, you have no choice but
to study all these materials.
4.2
LA
2817
2818
3133
3138
3171
3290
3294
3678
4099
4109
4110
4138
4271
4272
4407
4408
4524
4637
4645
Problem Name
The Suspects
Geodetic Set Problem
Finding Nemo
Color a Tree
Oreon
Invite Your Friends
Ultimate Bamboo Eater
Bug Sensor Problem
Sub-dictionary
USHER
RACING
Anti Brute Force Lock
Necklace
Polynomial-time Red...
Gun Fight
Unlock the Lock
Interstar Transport
Repeated Substitution ...
Infected Land
Source
Kaohsiung03
Kaohsiung03
Beijing04
Beijing04
Manila06
Dhaka05
Dhaka05
Kaohsiung06
Iran07
Singapore07
Singapore07
Jakarta08
Hefei08
Hefei08
KLumpur08
KLumpur08
Hsinchu09
Japan09
Japan09
Table 4.1: Some Graph Problems in Recent ACM ICPC Asia Regional
DFS_WHITE = -1), DFS starts from a vertex u, mark u as visited (set dfs_num[u] to DFS_BLACK = 1),
and then for each unvisited neighbor v of u (i.e. edge u v exist in the graph), recursively visit
v. The snippet of DFS code is shown below:
typedef pair<int, int> ii; // we will frequently use these two data type shortcuts
typedef vector<ii> vii;
#define TRvii(c, it) \ // all sample codes involving TRvii use this macro
for (vii::iterator it = (c).begin(); it != (c).end(); it++)
void dfs(int u) { // DFS for normal usage
printf(" %d", u); dfs_num[u] = DFS_BLACK; // this vertex is visited, mark it
TRvii (AdjList[u], v) // try all neighbors v of vertex u
if (dfs_num[v->first] == DFS_WHITE) // avoid cycle
dfs(v->first); // v is a (neighbor, weight) pair
}
The time complexity of this DFS implementation depends on the graph data structure used. In
a graph with V vertices and E edges, dfs runs in O(V + E) and O(V 2 ) if the graph is stored as
Adjacency List and Adjacency Matrix, respectively.
Figure 4.1: Sample graph for the early part of this section
On sample graph in Figure 4.1, dfs(0) calling DFS from a start vertex u = 0 will trigger
this sequence of visitation: 0 1 2 3 4. This sequence is depth-rst, i.e. DFS goes to
the deepest possible vertex from the start vertex before attempting another branches. Note that
this sequence of visitation depends very much on the way we order neighbors of a vertex, i.e. the
void backtracking(state) {
if (hit end state or invalid state) // invalid state includes states that cause cycling
return; // we need terminating/pruning condition
for each neighbor of this state // regardless it has been visited or not
backtracking(neighbor);
}
Other Applications
DFS is not only useful for traversing a graph. It can be used to solve many other graph problems.
The fact that one single call of dfs(u) will only visit vertices that are actually connected to u can
be utilized to nd (and to count) the connected components of an undirected graph (see further
below for a similar problem on directed graph). We can simply use the following code to restart
DFS from one of the remaining unvisited vertices to nd the next connected component (until all
are visited):
#define REP(i, a, b) \ // all sample codes involving REP use this macro
for (int i = int(a); i <= int(b); i++)
//
//
//
//
Exercise: We can also use Union-Find Disjoint Sets to solve this graph problem. How?
Exercise: Flood Fill is more commonly performed on 2-D grid (implicit graph). Try to solve UVa
352, 469, 572, etc.
Graph Edges Property Check via DFS Spanning Tree
Running DFS on a connected component of a graph will form a DFS spanning tree (or spanning
forest if the graph has more than one component and DFS is run on each component). With one
more vertex state: DFS_GRAY = 2 (visited but not yet completed) on top of DFS_WHITE (unvisited)
and DFS_BLACK (visited and completed), we can use this DFS spanning tree (or forest) to classify
graph edges into four types:
1. Tree edges: those traversed by DFS, i.e. from vertex with DFS_GRAY to vertex with DFS_WHITE.
2. Back edges: part of cycle, i.e. from vertex with DFS_GRAY to vertex with DFS_GRAY too.
Note that usually we do not count bi-directional edges as having cycle
(We need to remember dfs_parent to distinguish this, see the code below).
3. Forward/Cross edges from vertex with DFS_GRAY to vertex with DFS_BLACK.
These two type of edges are not typically used in programming contest problem.
Figure 4.2 shows an animation (from top left to bottom right) of calling dfs(0), then dfs(5), and
nally dfs(6) on the sample graph in Figure 4.1. We can see that 1 2 3 1 is a (true) cycle
and we classify edge (3 1) as a back edge, whereas 0 1 0 is not a cycle but edge (1 0) is
just a bi-directional edge. The code for this DFS variant is shown below.
Motivating problem: Given a road map (undirected graph) with cost associated to all intersections
(vertices) and roads (edges), sabotage either a single intersection or a single road that has minimum
cost such that the road network breaks down. This is a problem of nding the least cost Articulation
Point (intersection) or the least cost Bridge (road) in an undirected graph (road map).
graph without any articulation points is called Biconnected. Similarly, a Bridge is dened as
an edge in a graph G whose removal disconnects G. These two problems are usually dened for
undirected graphs, although they are still well dened for directed graphs.
See Figure 4.3 for clarity. In these two sample graphs, we run articulationPointAndBridge(0).
Suppose in the graph in Figure 4.3 left side, the sequence of visitation is 0 (at iteration 0) 1 (1)
2 (2) (backtrack to 1) 4 (3) 3 (4) (backtrack to 4) 5 (5). See that these iteration counters
are shown correctly in dfs_num. Since there is no back edge in this graph, all dfs_low = dfs_num.
Figure 4.3: Introducing two more DFS attributes: dfs number and dfs low
In the graph in Figure 4.3 right side, the sequence of visitation is 0 (at iteration 0) 1 (1) 2
(2) (backtrack to 1) 3 (3) (backtrack to 1) 4 (4) 5 (5). There is an important back edge
that forms a cycle, i.e. edge 5-1 that is part of cycle 1-4-5-1. This causes vertices 1, 4, and 5 to be
able to reach vertex 1 (with dfs_num 1). Thus dfs_low of {1, 4, 5} are all 1.
When we are in a vertex u, v is a neighbor of u, and dfs_low(v) dfs_num(u), then u is
an articulation vertex. This is because the fact that dfs_low(v) is not smaller than dfs_num(u)
implies that there is no back edge connected to vertex v that can reach vertex w with a lower
dfs_num(w) (which further implies that w is the parent of u in the DFS spanning tree). Thus, to
reach that parent of u from v, one must pass through vertex u. This implies that removing the
vertex u will disconnect the graph.
Special case: The root of the DFS spanning tree (the vertex chosen as the start of DFS call) is
an articulation point only if it has more than one children (a trivial case that is not detected by
this algorithm).
Figure 4.4: Finding articulation points with dfs num and dfs low
See Figure 4.4 for more details. On the graph in Figure 4.4 left side, vertices 1 and 4 are
articulation points, because for example in edge 1-2, we see that dfs_low(2) dfs_num(1) and
in edge 4-5, we also see that dfs_low(5) dfs_num(4). On the graph in Figure 4.4 right side,
vertex 1 is the articulation point, because for example in edge 1-5, dfs_low(5) dfs_num(1).
The process to nd bridges is similar. When dfs_low(v) > dfs_num(u), then edge u-v is a
bridge. In Figure 4.5, almost all edges are bridges for the left and right graph. Only edges 1-4, 4-5,
and 5-1 are not bridges on the right graph. This is because for example edge 4-5, dfs_low(5)
dfs_num(4), i.e. even if this edge 4-5 is removed, we know for sure that vertex 5 can still reach
Figure 4.5: Finding bridges, also with dfs num and dfs low
void articulationPointAndBridge(int u) {
dfs_low[u] = dfs_num[u] = dfsNumberCounter++; // dfs_low[u] <= dfs_num[u]
TRvii (AdjList[u], v)
if (dfs_num[v->first] == DFS_WHITE) { // a tree edge
dfs_parent[v->first] = u; // parent of this children is me
if (u == dfsRoot) // special case
rootChildren++; // count children of root
articulationPointAndBridge(v->first);
if (dfs_low[v->first] >= dfs_num[u]) // for articulation point
articulation_vertex[u] = true; // store this information first
if (dfs_low[v->first] > dfs_num[u]) // for bridge
printf(" Edge (%d, %d) is a bridge\n", u, v->first);
dfs_low[u] = min(dfs_low[u], dfs_low[v->first]); // update dfs_low[u]
}
else if (v->first != dfs_parent[u]) // a back edge and not direct cycle
dfs_low[u] = min(dfs_low[u], dfs_num[v->first]); // update dfs_low[u]
}
Figure 4.6: An example of directed graph and its Strongly Connected Components (SCC)
There are at least two known algorithms to nd SCCs: Kosarajus explained in [4] and Tarjans
algorithm [45]. In this book, we adopt Tarjans version, as it extends naturally from our previous
discussion of nding Articulation Points and Bridges also due to Tarjan.
The basic idea of the algorithm is that SCCs form the subtrees of the DFS spanning tree
and the roots of the subtrees are also the roots of the SCCs. To determine whether a vertex u
is the root of an SCC, Tarjans SCC algorithm uses dfs_num and dfs_low, i.e. by checking if
dfs_low(u) = dfs_num(u). The visited vertices are pushed into a stack according to its dfs_num.
When DFS returns from a subtree, the vertices are popped from the stack. If the vertex is the root
of an SCC, then that vertex and all of the vertices popped before it forms that SCC. The code is
shown below:
Exercise: This implementation can be improved by about O(log V ) factor by using other data
structure other than set<int> in_stack. How?
Topological Sort (on a Directed Acyclic Graph)
Topological sort or topological ordering of a Directed Acyclic Graph (DAG) is a linear ordering of
the vertices in the DAG so that vertex u comes before vertex v if edge (u v) exists in the DAG.
Every DAG has one or more topological sorts. There are several ways to implement a Topological
Sort algorithm. The simplest is to slightly modify the simplest DFS implementation in this section.
void topoVisit(int u) {
dfs_num[u] = DFS_BLACK;
TRvii (AdjList[u], v)
if (dfs_num[v->first] == DFS_WHITE)
topoVisit(v->first);
topologicalSort.push_back(u); // this is the only change
}
// inside int main()
topologicalSort.clear(); // this global vector stores topological sort in reverse order
memset(dfs_num, DFS_WHITE, sizeof dfs_num);
REP (i, 0, V - 1)
if (dfs_num[i] == DFS_WHITE)
topoVisit(i);
reverse(topologicalSort.begin(), topologicalSort.end());
REP (i, 0, topologicalSort.size() - 1)
printf("%d\n", topologicalSort[i]);
In topoVisit(u), we append u to the list vertices explored only after visiting all subtrees below u.
As vector only support ecient insertion from back, we work around this issue by simply reversing
the print order in the output phase. This simple algorithm for nding (a valid) topological sort is
4.3
take out the front most vertex u from the queue and enqueue each unvisited neighbors of u. With
the help of the queue, BFS will visit vertex s and all vertices in the connected component that
contains s layer by layer. This is why the name is breadth-rst. BFS algorithm also runs in O(V +E)
on a graph represented using an Adjacency List.
Implementing BFS is easy if we utilize C++ STL libraries. We use queue to order the sequence
of visitation and map to record if a vertex has been visited or not which at the same time also
record the distance (layer number) of each vertex from source vertex. This feature is important as
it can be used to solve special case of Single-Source Shortest Paths problem (discussed below).
queue<int> q; map<int, int> dist;
q.push(s); dist[s] = 0; // start from source
while (!q.empty()) {
int u = q.front(); q.pop(); // queue: layer by layer!
printf("Visit %d, Layer %d\n", u, dist[u]);
TRvii (AdjList[u], v) // for each neighbours of u
if (!dist.count(v->first)) { // dist.find(v) != dist.end() also works
dist[v->first] = dist[u] + 1; // if v not visited before + reachable from u
q.push(v->first); // enqueue v for next steps
}
}
Exercise: This implementation uses map<STATE-TYPE, int> dist to store distance information.
This may be useful if STATE-TYPE is not integer, e.g. a pair<int, int> of (row, col) coordinate.
However, this trick adds a log V factor to the O(V + E) BFS complexity. Please, rewrite this
If we run BFS from the vertex labeled with 35 (i.e. the source vertex s = 35) on the connected
undirected graph shown in Figure 4.7, we will visit the vertices in the following order:
Layer 0: >35< (source)
Layer 1: 15, 55, 40
Layer 2: 10, 20, 50, 60
Layer 3: >30<, 25, 47, 65
Layer 4: 45
// Three layers from 35 to 30 implies that the shortest path between them
// on this unweighted graph is 3 distance units.
Other Applications
Single-Source Shortest Paths (SSSP) on Unweighted Graph
The fact that BFS visits vertices of a graph layer by layer from a source vertex turns BFS as a good
solver for Single-Source Shortest Paths (SSSP) problem on unweighted graph. This is because in
unweighted graph, the distance between two neighboring vertices connected with an edge is simply
one unit. Thus the layer count of a vertex that we have seen previously is precisely the shortest
path length from the source to that vertex. For example in Figure 4.7, the shortest path from the
vertex labeled with 35 to the vertex labeled 30, is 3, as 30 is in the third layer in BFS sequence
of visitation. Reconstructing the shortest path: 35 15 10 30 is easy if we store the BFS
spanning tree, i.e. vertex 30 remembers 10 as its parent, vertex 10 remembers 15, vertex 15
remembers 35 (the source).
Which graph traversal algorithm to choose? Table 4.2 can be helpful.
Pro
Cons
Code
O(V + E) DFS
Uses less memory
Cannot solve SSSP on unweighted graphs
Slightly easier to code
O(V + E) BFS
Can solve SSSP on unweighted graphs
Uses more memory
Slightly longer to code
4.4
Kruskals
Figure 4.8: Example of a Minimum Spanning Tree (MST) Problem (from UVa 908 [17])
This MST problem can be solved with several well-known algorithms, i.e. Prims and Kruskals,
both are greedy algorithms and explained in [4, 21, 14, 23, 16, 1, 13, 5]. For programming contests,
we opt Kruskals as its implementation is very easy with help of 2 data structures.
Joseph Bernard Kruskal Jr.s algorithm rst sort E edges based on non decreasing weight in
O(E log E). This can be easily done using priority_queue (or alternatively, use vector & sort).
Then, it greedily tries to add O(E) edges with minimum costs to the solution as long as such
addition does not form a cycle. This cycle check can be done easily using Union-Find Disjoint Sets.
The code is short and in overall runs in O(E log E).
#typedef pair<int, int> ii; // we use ii as a shortcut of integer pair data type
priority_queue< pair<int, ii> > EdgeList; // sort by edge weight O(E log E)
// PQ default: sort descending. To sort ascending, we can use <(negative) weight(i, j), <i, j> >
Figure 4.9: Kruskals Algorithm for MST Problem (from UVa 908 [17])
Exercise: The implementation shown here only stop when EdgeList is empty. For some cases, we
can stop Kruskals algorithm earlier. When and how to modify the code to handle this?
Figure 4.9 shows the execution of Kruskals algorithm on the graph shown in Figure 4.8, leftmost1 .
Other Applications
Variants of basic MST problems are interesting. In this section, we will explore some of them.
Maximum Spanning Tree
we see a comparison between MST and Maximum ST. Solution: sort edges in non increasing order.
Note that the solution for this problem is denitely a tree, i.e. no cycles in the solution!
A chord edge is dened as an edge in graph G that is not selected in the MST of G.
Figure 4.13: Second Best Spanning Tree (from UVa 10600 [17])
A simple solution is to rst sort the edges in O(E log E), then nd the MST using Kruskals in
O(E). Then, for each edge in the MST, make its weight to be INF (innite) to delete it in
practice, this is just a very big number. Then try to nd the second best ST in O(V E) as there are
E = V 1 edges in the MST that we have to try. Figure 4.14 shows this algorithm on the given
graph. Remember that both MST and second best ST are spanning tree, i.e. they are connected!
In overall, this algorithm runs in O(E log E + E + V E).
Figure 4.14: Finding the Second Best Spanning Tree from the MST
Variants
1. UVa 10147 - Highways (Partial Minimum Spanning Tree)
2. UVa 10369 - Arctic Networks (Minimum Spanning Forest)
3. UVa 10397 - Connect the Campus (Partial Minimum Spanning Tree)
4. UVa 10600 - ACM Contest and Blackout (Second Best Spanning Tree)
5. UVa 10842 - Trac Flow (nd min weighted edge in Maximum Spanning Tree)
6. LA 3678 - The Bug Sensor Problem (Minimum Spanning Forest)
7. LA 4110 - RACING (Maximum Spanning Tree)
8. POJ 1679 - The Unique MST (Second Best Spanning Tree)
4.5
Dijkstras
Motivating problem: Given a weighted graph G and a starting source vertex s, what are the shortest
paths from s to the other vertices of G?
This problem is called the Single-Source3 Shortest Paths (SSSP) problem on a weighted graph.
It is a classical problem in graph theory and ecient algorithms exist. If the graph is unweighted,
we can use the BFS algorithm as shown earlier in Section 4.3. For a general weighted graph, BFS
does not work correctly and we should use algorithms like the O((E +V ) log V ) Dijkstras algorithm
(discussed here) or the O(V E) Bellman Fords algorithm (discussed in Section 4.6).
Edsger Wybe Dijkstras algorithm is a greedy algorithm: Initially, set the distance to all vertices
to be INF (a large number) but set the dist[source] = 0 (base case). Then, repeat the following
process from the source vertex: From the current vertex u with the smallest dist[u], relax all
neighbors of u. relax(u, v) sets dist[v] = min(dist[v], dist[u] + weight(u, v)). Vertex
u is now done and will not be visited again. Then we greedily replace u with the unvisited vertex
x with currently smallest dist[x]. See proof of correctness of this greedy strategy in [4].
There can be many ways to implement this algorithm, especially in using the priority_queue.
The following code snippet may be one of the easiest implementation.
vector<int> dist(V, INF); dist[s] = 0; // INF = 2.10^9 not MAX_INT to avoid overflow
priority_queue<ii, vector<ii>, greater<ii> > pq; pq.push(ii(0, s)); // sort by distance
while (!pq.empty()) { // main loop
ii top = pq.top(); pq.pop(); // greedy: pick shortest unvisited vertex
int d = top.first, u = top.second;
if (d == dist[u]) // This check is important! We want to process vertex u only once but we can
// actually enqueue u several times in priority_queue... Fortunately, other occurrences of u
// in priority_queue will have greater distances and can be ignored (the overhead is small) :)
TRvii (AdjList[u], it) { // all outgoing edges from u
int v = it->first, weight_u_v = it->second;
if (dist[u] + weight_u_v < dist[v]) { // if can relax
dist[v] = dist[u] + weight_u_v; // relax
pq.push(ii(dist[v], v)); // enqueue this neighbor
}
} }
// regardless whether it is already in pq or not
3
This generic SSSP problem can also be used to solve: 1). Single-Pair Shortest Path problem where both source
+ destination vertices are given and 2). Single-Destination Shortest Paths problem where we can simply reverse the
role of source and destination vertices.
Figure 4.15: Dijkstra Animation on a Weighted Graph (from UVa 341 [17])
Figure 4.15 shows an example of running Dijkstras on a simple weighted graph |V | = 5 and |E| = 7:
1. At the beginning, only dist[source] = dist[2] = 0, pq is {(0,2)}.
2. From vertex 2, we relax vertices {1, 3, 5}. Now dist[1] = 2, dist[3] = 7, and dist[5] = 6.
Vertex 2 is done. The content of our priority_queue pq is {(2,1), (6,5), (7,3)}.
3. Among unprocessed vertices {1, 5, 3} in pq, vertex 1 has the least dist[1] = 2 and is in front
of pq. We dequeue (2,1) and relax all its neighbors: {3, 4} such that dist[3] = min(dist[3],
dist[1]+weight(1,3)) = min(7, 2+3) = 5 and dist[4] = 8. Vertex 1 is done. Now pq
contains {(5,3), (6,5), (7,3), (8,4)}. See that we have 2 vertex 3. But it does not matter, as
our Dijkstras implementation will only pick one with minimal distance later.
4. We dequeue (5,3) and try to do relax(3,4), but 5+5 = 10, whereas dist[4] = 8 (from
path 2-1-4). So dist[4] is unchanged. Vertex 3 is done and pq contains {(6,5), (7,3), (8,4)}.
5. We dequeue (6,5) and relax(5, 4), making dist[4] = 7 (the shorter path from 2 to 4 is
now 2-5-4 instead of 2-1-4). Vertex 5 is done and pq contains {(7,3), (7,4), (8,4)}.
6. Now, (7,3) can be ignored as we know that d > dist[3] (i.e. 7 > 5). Then (7,4) is processed
4.6
as before. And nally (8,4) is ignored again as d > dist[4] (i.e. 8 > 7). Dijkstras stops
here as the priority queue is empty.
Bellman Fords
If the input graph has negative edge weight, Dijkstra can fail. Figure 4.16 is a simple example where
Dijkstras fails. Dijkstras greedily sets dist[3] = 3 rst and uses that value to relax dist[4] =
To solve SSSP problem in the presence of negative edge weight, the more generic (but slower)
Bellman Fords algorithm must be used. This algorithm was invented by Richard Ernest Bellman
(the pioneer of DP techniques) and Lester Randolph Ford, Jr (the same person who invented Ford
Fulkersons method in Section 4.8). This algorithm is simple: Relax all E edges V 1 times!
The complexity of Bellman Fords algorithm is O(V 3 ) if the graph is stored as Adjacency Matrix
or O(V E) if the graph is stored as Adjacency List. This is (much) slower compared to Dijkstras.
Thus, Bellman Fords is typically only used to solve SSSP problem when the input graph is not
too big and not guaranteed to have all non-negative edge weights!
Bellman Fords algorithm has one more interesting usage. After relaxing all E edges V -1 times,
the SSSP problem should have been solved, i.e. there is no way we can relax any more vertex. This
fact can be used to check the presence of negative cycle, although such a problem is ill-dened.
In Figure 4.17, left, we see a simple graph with negative cycle. After 1 pass, dist[1] = 973 and
dist[2] = 1015 (middle). After V 1 = 2 passes, dist[1] = 988 and dist[1] = 946 (right).
But since there is a negative cycle, we can still do this one more time, i.e. relaxing dist[1] =
Figure 4.17: Bellman Fords can detect the presence of negative cycle (from UVa 558 [17])
bool negative_cycle_exist = false;
REP (u, 0, V - 1) // one more pass to check
TRvii (AdjList[u], v)
if (dist[v->first] > dist[u] + v->second) // should be false, but if possible
negative_cycle_exist = true;
// then negative cycle exists!
printf("Negative Cycle Exist? %s\n", negative_cycle_exist ? "Yes" : "No");
In Table 4.3, we present an SSSP algorithm decision table with programming contest in mind.
This is to help readers in deciding which algorithm to choose depending on various graph criteria.
Graph
Criteria
Unweighted
Weighted
Negative weight
Negative cycle
BFS
O(V + E)
Best: V, E 1M
WA except on Tree & DAG
WA
Cannot detect
Dijkstras
O((V + E) log V )
Ok: V, E 50K
Best: V, E 50K
WA
Cannot detect
Bellman Fords
O(V E)
Bad: V E 1M
Ok: V E 1M
Best: V E 1M
Can detect
4.7
Floyd Warshalls
s2, nd a vertex v in G that represents the best meeting point, i.e. dist[s1][v] + dist[s2][v]
is the minimum over all possible v. What is the best solution?
This problem requires the shortest path information from two sources s1 and s2 to all vertices in
G. This can be easily done with two calls of Dijkstras algorithm. One from s1 to produce shortest
distance array dist1 from s1, and one from s2 to produce dist2. Then iterates through all possible
vertices in graph to nd v such that dist1[v] + dist2[v] is minimized. Can we do better?
If the given graph is known to have V 100, then there is an even simpler algorithm in
terms of implementation to solve this problem as quickly as possible!
Load the small graph into an Adjacency Matrix and then run the following short code with
3 nested loops. When it terminates, AdjMatrix[i][j] will contain the shortest path distance
between two pair of vertices i and j in G. The original problem now become easy.
REP (k, 0, V - 1) // recall that #define REP(i, a, b) for (int i = int(a); i <= int(b); i++)
REP (i, 0, V - 1)
REP (j, 0, V - 1)
AdjMatrix[i][j] = min(AdjMatrix[i][j], AdjMatrix[i][k] + AdjMatrix[k][j]);
This algorithm is called Floyd Warshalls algorithm, invented by Robert W Floyd and Stephen
Warshall. Floyd Warshalls is a DP algorithm that clearly runs in O(V 3 ) due to its 3 nested
loops4 , but since |V | 100 for the given problem, this is do-able. In general, Floyd Warshalls
solves another classical graph problem: the All-Pairs Shortest Paths (APSP) problem. It is an
alternative algorithm (for small graphs) compared to calling SSSP algorithms multiple times:
1. V calls of O((V + E) logV ) Dijkstras = O(V 3 log V ) if E = O(V 2 ).
2. V calls of O(V E) Bellman Fords = O(V 4 ) if E = O(V 2 ).
In a programming contest setting, Floyd Warshalls main attractiveness is basically its implementation speed 4 short lines only. If the given graph is small, do not hesitate using this algorithm
even if you only need a solution for the SSSP problem.
Floyd Warshalls algorithm must use Adjacency Matrix so that the weight of edge(i, j) can be accessed in O(1).
In Figure 4.18, we want to nd sp(3,4). The shortest possible path is 3-0-2-4 with cost 3. But how
to reach this solution? We know that with direct edges only, sp(3,4) = 5, as in Figure 4.18.A.
The solution for sp(3,4) will eventually be reached from sp(3,2)+sp(2,4). But with only direct
When we allow k = 0, i.e. vertex 0 can now be used as an intermediate vertex, then sp(3,4) is
sp(3,2) which we will need later also drop from 3 to sp(3,0)+sp(0,2) = 1+1 = 2. Floyd
Warshalls will process sp(i,j) for all pairs considering only vertex 0 as the intermediate vertex.
When we allow k = 1, i.e. vertex 0 and 1 can now be used as the intermediate vertices, then it
When we allow k = 2, i.e. vertices 0, 1, and 2 now can be used as the intermediate vertices, then
1
= weight(i, j). This is the base case when we do not use any intermediate vertices.
Di,j
k = min(D k1 , D k1 + D k1 ) = min(not using vertex k, using k), for k 0, see Figure 4.19.
Di,j
i,j
i,k
k,j
This DP formulation requires us to ll the entries layer by layer. To ll out an entry in the table k,
2 , (row 3, column 4, in table
we make use of entries in the table k-1. For example, to calculate D3,4
The nave implementation is to use 3-dimensional matrix D[k][i][j] of size O(V 3 ). However,
we can utilize a space-saving trick by dropping dimension k and computing D[i][j] on-the-y.
Thus, the Floyd Warshalls algorithm just need O(V 2 ) space although it still runs in O(V 3 ).
Other Applications
The basic purpose of Floyd Warshalls algorithm is to solve the APSP problem. However, it can
Exercise: In this section, we have shown you how to solve Minimax (and Maximin) with Floyd
Warshalls algorithm. However, this problem can also be modeled as an MST problem and solved
using Kruskals algorithm. Find out the way!
Programming Exercises for Floyd Warshalls algorithm:
Floyd Warshalls Standard Application (for APSP or SSSP on small graph)
1. UVa 186 - Trip Routing (graph is small)
2. UVa 341 - Non-Stop Travel (graph is small)
3. UVa 423 - MPI Maelstrom (graph is small)
4. UVa 821 - Page Hopping (one of the easiest ICPC World Finals problem)
5. UVa 10075 - Airlines (with special great-circle distances, see Section 7.2)
6. UVa 10171 - Meeting Prof. Miguel (solution is easy with APSP information)
7. UVa 11015 - 05-32 Rendezvous (graph is small)
8. UVa 10246 - Asterix and Obelix
9. UVa 10724 - Road Construction (adding one edge will only change few things)
10. UVa 10793 - The Orc Attack (Floyd Warshalls simplies this problem)
11. UVa 10803 - Thunder Mountain (graph is small)
12. UVa 11463 - Commandos (solution is easy with APSP information)
Variants
1. UVa 334 - Identifying Concurrent Events (transitive closure is only the sub-problem)
2. UVa 534 - Frogger (Minimax)
3. UVa 544 - Heavy Cargo (Maximin)
4. UVa 869 - Airline Comparison (run Warshalls twice, then compare the AdjMatrices)
5. UVa 925 - No more prerequisites, please!
6. UVa 10048 - Audiophobia (Minimax)
7. UVa 10099 - Tourist Guide (Maximin)
4.8
Edmonds Karps
we want to know the maximum volume of water that can be passed by this pipe network)? This
problem is called the Max Flow problem. An illustration of this problem is shown in Figure 4.21.
Figure 4.21: Illustration of Max Flow (From UVa 820 [17] - ICPC World Finals 2000 Problem E)
One solution is the Ford Fulkersons method invented by the same Lester Randolph Ford. Jr who
created Bellman Fords algorithm and Delbert Ray Fulkerson. The pseudo code is like this:
max_flow = 0
while (there exists an augmenting path p from s to t) { // iterative algorithm
augment flow f along p, i.e.
f = min edge weight in the path p
max_flow += f // we can send flow f from s to t
forward edges -= f // reduce capacity of these edges
backward edges += f // increase capacity of reverse edges
}
output max_flow
There are several ways to nd an augmenting path in the pseudo code above, each with dierent
time complexity. In this section, we highlight two ways: via DFS or via BFS.
Ford Fulkersons method implemented using DFS can run in O(f E) where f is the max ow
value. This is because we can have a graph like in Figure 4.22 where every path augmentation only
decreases the edge capacity along the path by 1. This is going to be repeated f times. In Figure
4.22, it is 200 times. The fact that number of edges in ow graph is E V 1 to ensure 1 s-t
ow dictates that a DFS run is O(E). The overall time complexity is O(f E).
A better implementation of Ford Fulkersons method is to use BFS for nding the shortest path
Edmonds and Richard Karp, thus named as Edmonds Karps algorithm. It runs in O(V E 2 ) as it
can be proven that after O(V E) iterations, all augmenting paths will already exhausted. Interested
readers can browse books like [4] to study more about this algorithm. As BFS also runs in O(E),
the overall time complexity is O(V E 2 ). Edmonds Karps only needs two s-t paths in Figure 4.22:
s-a-t (send 100 unit ow) and s-b-t (send another 100).
}
printf("Max flow = %d\n", max_flow);
The code snippet above shows how to implement Edmonds Karps algorithm in a way that it still
achieves its O(V E 2 ) time complexity. The code uses both Adjacency List (for fast enumeration of
neighbors) and Adjacency Matrix (for fast access to edge weight) of the same ow graph.
In general, this O(V E 2 ) Edmonds Karps implementation is sucient to answer most network
ow problems in programming contests. However, for harder problems, we may need O(V 2 E)
Dinics or O(V 3 ) Push-Relabel (relabel-to-front) Max Flow algorithms [4].
Exercise: The implementation of Edmonds Karps algorithm shown here uses AdjMatrix to store
residual capacity of each edge. A better way is to store ow of each edge, and then derive the
Other Applications
There are several other interesting applications of Max Flow problem. We discuss six examples
here while some others are deferred until Section 4.9. Note that some tricks shown in this section
Min Cut
Lets dene an s-t cut C = (S, T ) as a partition of V G such that source s S and sink t T .
Lets also dene a cut-set of C to be the set {(u, v) E | u S, v T } such that if all edges in the
cut-set of C are removed, the Max Flow from s to t is 0 (i.e. s and t are disconnected). The cost of
an s-t cut C is dened by the sum of the capacities of the edges in the cut-set of C. The Min Cut
problem is to minimize the amount of capacity of an s-t cut. This problem is more general than
nding bridges in a graph (See Section 4.2), i.e. in this case we can cut more than just one edge,
Cut. In Figure 4.21.D, we can see that edges that are saturated, i.e. the ow on that edge equals
to that edges capacity, belong to the Min Cut!, i.e. edges 1-4 (capacity 30, ow 30), 3-4 (5/5) and
3-2 (25/25). The cost of cut is 30+5+25 = 60. This is the minimum over all possible s-t cuts. All
vertices that are still reachable from source s belong to set S. The rest belong to set T . Here,
Sometimes, we can have more than one source and/or more than one sink. However, this variant
is no harder than the original Max Flow problem with a single source and a single sink. Create a
super source ss and a super sink st. Connect ss with all s with innite capacity and also connect
tt with all t with innite capacity, then run Edmonds Karps algorithm as per normal.
We can also have a Max Flow variant where capacities are not just dened along the edges but
also on the vertices. To solve this variant, we can use the vertex splitting technique. A graph with
a vertex weight can be converted into a more familiar graph without a vertex weight by splitting
the vertex v to vout and vin , reassigning incoming edges to vout /outgoing edges to vin , and putting
the original vertex vs weight as the weight of edge (vout , vin ). For details, see Figure 4.23. Then
The problem of nding the maximum number of independent paths from source s to sink t can be
reduced to the Max Flow problem. Two paths are said to be independent if they do not share any
vertex apart from s and t (vertex-disjoint). Solution: construct a ow network N = (V, E) from G
with vertex capacities, where N is the carbon copy of G except that the capacity of each v V is
1 (i.e. each vertex can only be used once) and the capacity of each e E is also 1 (i.e. each edge
can only be used once too). Then run Edmonds Karps algorithm as per normal.
Finding the maximum number of edge-disjoint paths from s to t is similar to nding max inde-
pendent paths. The only dierence is that this time we do not have any vertex capacity (i.e. two
edge-disjoint paths can still share the same vertex). See Figure 4.24 for a comparison between
Figure 4.24: Comparison Between Max Independent Paths versus Max Edge-Disjoint Paths
4.9
Special Graphs
Some basic graph problems have simpler / faster polynomial algorithm if the given graph is special.
So far we have identied the following special graphs that commonly appear in programming
contests: Tree, Directed Acyclic Graph (DAG), and Bipartite Graph. Problem setters may
force contestants to use specialized algorithms for these special graphs simply by giving a large
input size to judge a correct algorithm for general graph as Time Limit Exceeded (TLE). In this
section, we will discuss some popular graph problems on these special graphs (see Figure 4.25).
Figure 4.25: Special Graphs (Left to Right): Tree, Directed Acyclic Graph, Bipartite Graph
4.9.1
Tree
Tree is a special graph with the following characteristics: has E = V -1 (any O(V + E) algorithm is
O(V )), it has no cycle, it is connected, and there exists one unique path from any pair of vertices.
In Section 4.2, we have seen O(V + E) Tarjans DFS algorithm for nding articulation points and
bridges of a graph. However, if the given graph is a tree, the problem becomes simpler: all edges on
a tree are bridges and all internal vertices (degree > 1) are articulation points. This is still O(V )
as we have to scan the tree to count the number of internal vertices, but the code is simpler.
In Sections 4.5 and 4.6, we have seen two general purpose algorithms (O((E + V ) log V ) Dijkstras
and O(V E) Bellman-Fords) for solving the SSSP problem on a weighted graph. However, if the
given graph is a tree, the SSSP problem becomes simpler: any O(V ) graph traversal algorithm, i.e.
BFS or DFS, can be used to solve this problem. There is only one unique path between any two
vertices in a tree, so we simply traverse the tree to nd the path connecting the two vertices and
the shortest path between these two vertices is basically the sum of edge weights of this unique
path.
In Section 4.7, we have seen a general purpose algorithm (O(V 3 ) Floyd Warshalls) for solving
the APSP problem on weighted graph. However, if the given graph is a tree, the APSP problem
becomes simpler: repeat the SSSP process V times from each vertex, thus it is O(V 2 ).
But this can still be improved to O(V +QL): Q is the number of query and L is the complexity
of the Lowest Common Ancestor (LCA) implementation (see [40] for more details). We run O(V )
DFS/BFS once from any vertex v to nd dist[v][other vertices] in tree. Then we can answer any
shortest path query (i, j) on this tree by reporting dist[v][i] + dist[v][j] - 2 dist[v][LCA(i, j)].
Diameter of Tree
The diameter of a graph is dened as the greatest distance between any pair of vertices. To nd
the diameter of a graph, we rst nd the shortest path between each pair of vertices (the APSP
problem). The greatest length of any of these paths is the diameter of the graph. For general
graph, we may need O(V 3 ) Floyd Warshalls algorithm discussed in Section 4.7. However, if the
given graph is a tree, the problem becomes simpler: do DFS/BFS from any node s to nd furthest
vertex x, then do DFS/BFS one more time from vertex x to get the true furthest vertex y from x.
The length of the unique path along x to y is the diameter of that tree. This solution only requires
In Section 3.4, we have shown a DP on Tree example that solves Max Weighted Independent Set
(MWIS) on Tree in O(V ). In Section 4.9.3 below, we will revisit this problem on a Bipartite Graph,
which can be reduced to Max Flow problem and runs in O(V E 2 ) with Edmonds Karps. However,
this problem is NP-complete on general graph.
Programming Exercises related to Tree (also see Section 3.4.3 for DP on Tree Topic):
1. UVa 112 - Tree Summing (backtracking)
2. UVa 115 - Climbing Trees (tree traversal, LCA)
3. UVa 122 - Trees on the level (tree traversal)
4. UVa 536 - Tree Recovery (tree traversal, reconstructing tree from pre + inorder)
5. UVa 615 - Is It A Tree? (graph property check)
6. UVa 699 - The Falling Leaves (preorder traversal)
7. UVa 712 - S-Trees (tree traversal)
8. UVa 10308 - Roads in the North (diameter of tree, discussed in this section)
9. UVa 10459 - The Tree Root (diameter + center of tree)
10. UVa 10701 - Pre, in and post (reconstructing tree from pre + inorder)
11. UVa 10938 - Flea Circus (use LCA)
12. UVa 11695 - Flight Planning (diameter + center of tree)
4.9.2
A Directed Acyclic Graph, abbreviated as DAG, is a special graph with the following characteristics:
it is directed and has no cycle.
Single-Source Shortest Paths on DAG
The Single-Source Shortest Paths (SSSP) problem becomes much simpler if the given graph is a
DAG as DAG has at least one topological order! We can use an O(V +E) topological sort algorithm
in Section 4.2 to nd one such topological order, then relax edges according to this order. The
topological order will ensure that if we have a vertex b that has an incoming edge from a vertex a,
then vertex b is relaxed after vertex a. This way, the distance information propagation is correct
with just one O(V + E) linear pass!
Single-Source Longest Paths on DAG
Single-Source Longest Paths problem, i.e. nding the longest path from a starting vertex s is
NP-complete on a general graph [39]. However the problem is again easy if the graph has no cycle,
which is true in DAG. The solution for the Longest Paths in DAG (a.k.a. Critical Paths) is just a
minor tweak from the SSSP solution in DAG shown above, i.e. simply negate all edge weights.
Min Path Cover on DAG
Motivating problem: Imagine that the vertices in Figure 4.26.A are passengers, and we draw an
edge between two vertices u v if a single taxi can serve passenger u then passenger v on time. The
question is: What is the minimum number of taxis that must be deployed to serve all passengers?
The answer for the motivating problem above is two taxis. In Figure 4.26.D, we see one possible
solution. One taxi (red dotted line) serves passenger 1 (colored with red), passenger 2 (blue), and
then passenger 4 (yellow). Another taxi (green dashed line) serves passenger 3 (green) and passenger
5 (orange). All passengers are served with just two taxis.
Vin , E ) from
4.9.3
Bipartite Graph
Bipartite Graph, is a special graph with the following characteristics: the set of vertices V can be
partitioned into two disjoint sets V1 and V2 and all edges in (u, v) E has the property that u V1
and v V2 . The most common application is the (bipartite) matching problem, shown below.
Max Cardinality Bipartite Matching
Motivating problem (from TopCoder [26] Open 2009 Qualifying 1): Group a list of numbers into
pairs such that the sum of each pair is prime. For example, given the numbers {1, 4, 7, 10, 11, 12},
we can have: {1 + 4 = 5}, {1 + 10 = 11}, {1 + 12 = 13}, {4 + 7 = 11}, {7 + 10 = 17}, etc.
Actual task: Given a list of numbers N , return a list of all the elements in N that could be
paired with N [0] successfully as part of a complete pairing (i.e. each element a in N is paired to a
unique other element b in N such that a + b is prime), sorted in ascending order. The answer for
the example above would be {4, 10} omitting 12. This is because even though (1+12) is prime,
there would be no way to pair the remaining 4 numbers whereas if we pair (1+4), we have (7+10),
(11+12) and if we pair (1+10), we have (4+7), (11+12).
Constraints: list N contains an even number of elements (within [2 . . . 50], inclusive). Each
element of N will be between 1 and 1000, inclusive. Each element of N will be distinct.
Although this problem involves nding prime numbers, this is not a pure math problem as the
elements of N are not more than 1K there are not too many primes below 1000. The issue is
that we cannot do Complete Search pairings as there are
50 C2
48 C2
for the second pair, ..., until 2 C2 for the last pair. DP + bitmask technique is not an option either
because 50! is too big.
The key to solve this problem is to realize that this pairing or matching is done on bipartite
graph! To get a prime number, we need to sum 1 odd + 1 even, because 1 odd + 1 odd = even
number which is not prime, and 1 even + 1 even = also even number, which is not prime. Thus we
can split odd/even numbers to set1/set2 and give edges from set1 to set2 if set1[i] + set2[j]
is prime.
After we build this bipartite graph, the solution is trivial: If size of set1 and set2 are dierent,
complete pairing is not possible. Otherwise, if the size of both sets is n/2, try to match set1[0]
with set2[k] for k = [0 . . . n/2 1] and do Max Cardinality Bipartite Matching (MCBM) for the
rest. This problem can be solved with Max Flow algorithm like O(V E 2 ) Edmonds Karps algorithm.
If we obtain n/2 1 more matchings, add set2[k] to the answer. For this test case, the answer is
{4, 10}.
Bipartite Matching can be reduced to the Max Flow problem by assigning a dummy source
vertex connected to all vertices in set1 and a dummy sink vertex connected to all vertices in set2.
By setting capacities of all edges in this graph to be 1, we force each vertex in set1 to be matched to
only one vertex in set2. The Max Flow will be equal to the maximum number of possible matchings
on the original graph (see Figure 4.27).
Motivating Problem: Suppose that there are two users: User A and B. Each user has transactions,
e.g. A has {A1 , A2 , . . . , An } and each transaction has a weight, e.g. W (A1 ), W (A2 ), etc. These
transactions use shared resources, e.g. transaction A1 uses {r1 , r2 }. Access to a resource is exclusive,
e.g. if A1 is selected, then any of user Bs transaction(s) that use either r1 or r2 cannot be selected.
It is guaranteed that two requests from user A will never use the same resource, but two requests
from dierent users may be competing for the same resource. Our task is to maximize the sum of
transactions from user B that share some or all resources cannot be selected. This is a strong hint
for Independent Set. And since we want to maximize sum of weight of selected transactions, this
is Max Weighted Independent Set (MWIS). And since there are only two users (two sets)
and the problem statement guarantees that there is no resource conict between the transactions
from within one user, we are sure that the input graph is a Bipartite Graph. Thus, this problem
Lets see Figure 4.28 for illustration. We have two users. We list down all transactions of A on the
left and all transactions of B on the right. We draw an edge between two transactions if they share
similar resource. For example, transaction A1 uses resource 1 and transaction B1 also uses resource
1. We draw an edge between A1 (with weight 45) and B1 (with weight 54) because they share the
same resource. In fact, there are two more edges between A2 B2 and A3 B3. Transaction B4
has no edge because the resources that it used: {4, 5} are not shared with any other transactions.
In this instance, {B1 (54), A2 (51), A3 (62), B4 (2)} is the MWIS, with total weight = 54+51+62+2
= 169.
To nd the solution for non-trivial cases, we have to reduce this problem to a Max Flow problem.
We assign the original vertex cost (the weight of taking that vertex) as capacity from source to
that vertex for user A and capacity from that vertex to sink for user B. Then, we give innite
capacity in between any edge in between sets A and B. See Figure 4.29.
Figure 4.29: Reducing MWIS on Bipartite Graph to Max Flow Problem (from LA 3487 [11])
Then, we run O(V E 2 ) Edmonds Karps algorithm on this Flow graph. After the Max Flow algorithm terminates, the solution is {s-component
vertices in
User B} where s-component (t-component) are the vertices still reachable to source vertex (sink vertex) after running Max Flow. In Figure 4.30, the solution is: {A1 (20), A2 (18), A4 (54)}+{B3 (47)} =
139. This value can also be obtained via: MWIS = Total Weight - Max Flow = 259 120 = 139.
4.10
Chapter Notes
Take note that recent ICPCs and IOIs usually do not just ask contestants to solve problems involving
the pure form of these graph algorithms. New problems usually require contestants to combine two
or more algorithms or to combine an algorithm with some advanced data structures, e.g. using
BFS and Dijkstras together in the same problem to compute shortest path on both weighted and
unweighted version of the same graph, to combine longest path in DAG with Segment Tree data
structure, etc.
This chapter, albeit already quite long, still omits many known graph algorithms and graph
problems that are sometimes tested in ICPCs or IOIs, namely: Kosarajus algorithm for nd-
ing Strongly Connected Component, Prims and Boruvkas algorithms for Minimum Spanning
Tree, k-th shortest paths, Eulers Path/Tour, Fleurys algorithm, Chinese Postman Problem,
Hamiltonian Path/Tour, Bitonic Traveling Salesman Problem, Arborescence, Tarjans Oine
Lowest Common Ancestor, Dinics or Push Relabel algorithms for Max Flow, Circulation
Problem, Kuhn Munkress (Hungarian) matching algorithm, Edmondss Blossom Shrinking, etc.
If you want to increase your winning chance in ACM ICPC, please spend some time to study
them beyond this book. These harder ones rarely appears in regional contests and if they are, they
usually become the decider problem. Harder graph problems like these are more likely to appear
in ACM ICPC World Finals level.
However, for IOI contestants, most graph materials in IOI syllabus are already covered in this
chapter.
Chapter 5
Mathematics
We all use math every day; to predict weather, to tell time, to handle money.
Math is more than formulas or equations; its logic, its rationality,
its using your mind to solve the biggest mysteries we know.
TV show NUMB3RS
Recent ICPCs (especially in Asia) usually contain one or two mathematics problems. This chapter aims to
prepare contestants in dealing with them.
5.1
As with the topic of graph in previous chapter, there exist mathematics problems in recent ICPC
problem sets at least one, and can be two per problem set (see Table 5.1)!
LA
2194
2195
2953
2955
3172
3399
3904
3997
4104
4203
4209
4270
4340
4406
4715
4721
Problem Name
The Luncheon
Counting Zeroes
Sum of Factorials
Vivians Problem
Period of an Innite ...
Sum of Consecutive ...
Tile Code
Numerical surprises
MODEX
Puzzles of Triangles
Stopping Dooms Day
Discrete Square Roots
Find Terrorists
Irreducible Fractions
Rating Hazard
Nowhere Money
Source
Dhaka06
Dhaka06
Guangzhou03
Guangzhou03
Manila06
Japan05
Seoul07
Danang07
Singapore07
Dhaka08
Dhaka08
Hefei08
Amrita08
KLumpur08
Phuket09
Phuket09
Prime Numbers
Combinatorics
Modular Exponentiation (Java BigInteger)
Formula Simplication + BigInt
Farey Sequence
Fibonacci, Zeckendorf Theorem
Table 5.1: Some Mathematics Problems in Recent ACM ICPC Asia Regional
The appearance of mathematics problems in programming contests is not surprising since Computer
Science is deeply rooted in Mathematics. The term computer itself comes from the word compute
as computer is built primarily to help human compute numbers.
We are aware that dierent countries have dierent emphasis in mathematics training in preUniversity education. Thus, for some newbie ICPC contestants, the term Euler Phi is a familiar
term, but for others, the term does not ring any bell. Perhaps because he has not learnt it before, or
perhaps the term is dierent in his native language. In this chapter, we want to make a more level-
playing eld for the readers by listing common mathematic terminologies, denitions, problems,
and algorithms that frequently appear in programming contests.
5.2
We start this chapter by mentioning Ad Hoc mathematics problems. They are basically contest
problems involving mathematics that requires no more than basic programming skills.
Programming Exercises related to Ad Hoc Mathematics:
1. UVa 344 - Roman Numerals (conversion from roman numerals to decimal and vice versa)
2. UVa 377 - Cowculations (base 4 operations)
3. UVa 10346 - Peters Smoke (simple math)
4. UVa 10940 - Throwing Cards Away II (nd pattern using brute force, then use the pattern)
5. UVa 11130 - Billiard bounces (use billiard table reection technique)
6. UVa 11231 - Black and White Painting (use the O(1) formula once you spot the pattern)
7. UVa 11313 - Gourmet Games (similar to UVa 10346)
8. UVa 11428 - Cubes (simple math with complete search)
9. UVa 11547 - Automatic Answer (one liner O(1) solution exists)
10. UVa 11723 - Numbering Road (simple math)
11. UVa 11805 - Bafana Bafana (very simple O(1) formula exists)
5.3
Number Theory
Mastering as many topics as possible in the eld of number theory is important as some mathematics
problems becomes easy (or easier) if you know the theory behind the problems. Otherwise, either
a plain brute force attack leads to a TLE response or you simply cannot work with the given input
as it is too large without some pre-processing.
5.3.1
Prime Numbers
Prime number is an important topic in number theory and the source for many programming
problems1 . In this section, we will discuss algorithms involving prime numbers.
Optimized Prime Testing Function
The rst algorithm presented in this section is for testing whether a given natural number N is
prime, i.e. bool isPrime(N). The most nave version is to test by denition, i.e. test if N is
divisible by divisor [2 . . . N -1]. This of course works, but runs in O(N ) in terms of number of
divisions. This is not the best way and there are several possible improvements.
q were smaller than p, then q or a prime factor of q would have divided N earlier. This is O( N )
which is already much faster than previous version, but can still be improved to be twice faster.
test odd numbers up to N . This is because there is only one even prime number, i.e. number 2,
divisible by prime divisors N . This is because if a prime number X cannot divide N , then
there is no point testing whether multiples of X divide N or not. This is faster than O( N ) which
is about O(|#primes N |). For example, there are 500 odd numbers in [1 . . . (106 )], but there
are only 168 primes in the same range. The number of primes M denoted by (M ) is bounded
by O(M/(ln(M ) 1)), so the complexity of this prime testing function is about O( N / ln( N )).
The code is shown in the next discussion below.
Sieve of Eratosthenes: Generating List of Prime Numbers
If we want to generate a list of prime numbers between range [0 . . . N ], there is a better algorithm
than testing each number in the range whether it is a prime or not. The algorithm is called Sieve
of Eratosthenes invented by Eratosthenes of Alexandria. It works as follows.
First, it sets all numbers in the range to be probably prime but set numbers 0 and 1 to be
not prime. Then, it takes 2 as prime and crosses out all multiples3 of 2 starting from 2 2 = 4,
6, 8, 10, ... until it the multiple is greater than N . Then it takes the next non-crossed number 3
as a prime and crosses out all multiples of 3 starting from 3 3 = 9, 12, 15, 18, .... Then it takes
5 and crosses out all multiples of 5 starting from 5 5 = 25, 30, 35, 40, .... After that, whatever
left uncrossed within the range [0 . . . N ] are primes. This algorithm does approximately (N (1/2
+ 1/3 + 1/5 + 1/7 + ... + 1/last prime in range N )) operations. Using sum of reciprocals of
primes up to n, we end up with the time complexity of roughly O(N log log N ) [44].
Since generating a list of small primes 10K using the sieve is fast (our library code below can
go up to 107 under contest setting), we opt sieve for smaller primes and reserve optimized prime
testing function for larger primes see previous discussion. The combined code is as follows:
In real life, large primes are used in cryptography because it is hard to factor a number xy into x y when both
are relatively prime.
2
This is a bit recursive testing whether a number is a prime by using another (smaller) prime numbers. But the
reason should be obvious after reading the next section.
3
Common sub-optimal implementation is to start from 2 i instead of i i, but the dierence is not that much.
// in int main()
sieve(10000000); // can go up to 10^7
printf("%d\n", isPrime(5915587277)); // 10 digit prime
In number theory, we know that a prime number N only have 1 and itself as factors but a composite
numbers N , i.e. the non-primes, can be written uniquely it as a multiplication of its prime factors.
Thats it, prime numbers are multiplicative building blocks of integers. For example, N = 240 =
2 2 2 2 3 5 = 24 3 5 (the latter form is called prime-power factorization).
A nave algorithm generates a list of primes (e.g. with sieve) and check how many of them can
actually divide the integer N without changing N . This can be improved!
A better algorithm utilizes a kind of Divide and Conquer spirit. An integer N can be expressed
as: N = P F N , where P F is a prime factor and N is another number which is N/P F i.e.
we can reduce the size of N by taking out its factor P F . We can keep doing this until eventually
N = 1. Special case if N is actually a prime number. The code template below takes in an integer
vi primeFactors(int N) {
vi factors;
// vi "primes" (generated by sieve) is optional
int PF_idx = 0, PF = primes[PF_idx]; // using PF = 2, 3, 4, ..., is also ok.
while (N != 1 && (PF * PF <= N)) { // stop at sqrt(N), but N can get smaller
while (N % PF == 0) { N /= PF; factors.push_back(PF); } // remove this PF
PF = primes[++PF_idx]; // only consider primes!
}
if (N != 1) factors.push_back(N); // special case if N is actually a prime
return factors;
}
// in int main()
sieve(100); // prepare list of primes [0 .. 100]
vi result = primeFactors(10000); // with that, we can factor up to 100^2 = 10000
vi::iterator last = unique(result.begin(), result.end()); // to remove duplicates
for (vi::iterator i = result.begin(); i != last; i++) // output: 2 and 5
printf("%d\n", *i);
In the worst case when N is prime, this prime factoring algorithm with trial division requires
5.3.2
The Greatest Common Divisor (GCD) of two integers (a, b) denoted by gcd(a, b), is dened as the
largest positive integer d such that d | a and d | b where x | y implies that x divides y. Example
of GCD: gcd(4, 8) = 4, gcd(10, 5) = 5, gcd(20, 12) = 4. One practical usage of GCD is to simplify
fraction, e.g.
4
8
4/gcd(4,8)
8/gcd(4,8)
4/4
8/4
= 12 .
To nd the GCD between two integers is an easy task with an eective Euclid algorithm [20, 4]
which can be implemented as a one liner code (see below). Thus nding the GCD is usually not
the actual issue in a Math-related contest problem, but just part of the bigger solution.
The GCD is closely related to Least (or Lowest) Common Multiple (LCM). The LCM of two
integers (a, b) denoted by lcm(a, b), is dened as the smallest positive integer l such that a | l and
b | l. Example of LCM: lcm(4, 8) = 8, lcm(10, 5) = 10, lcm(20, 12) = 60. It has been shown [20]
that: lcm(a, b) = a b/gcd(a, b). This can also be implemented as a one liner code (see below).
int gcd(int a, int b) { return (b == 0 ? a : gcd(b, a % b)); }
int lcm(int a, int b) { return (a * (b / gcd(a, b))); } // divide before multiply!
The GCD of more than 2 numbers, e.g. gcd(a, b, c) is equal to gcd(a, gcd(b, c)), etc, and similarly
for LCM. Both GCD and LCM algorithms run in O(log10 n), where n = max(a, b).
Programming Exercises related to GCD and/or LCM:
1. UVa 332 - Rational Numbers from Repeating Fractions
2. UVa 412 - Pi
3. UVa 530 - Binomial Showdown
4. UVa 10193 - All You Need Is Love
5. UVa 10407 - Simple Division
6. UVa 10680 - LCM
7. UVa 10717 - Mint (requires Complete Search on top of LCM/GCD)
8. UVa 10791 - Minimum Sum LCM (prime factorization + tricky cases)
9. UVa 10892 - LCM Cardinality
10. UVa 11388 - GCD LCM (must understand the relationship between GCD and LCM)
11. UVa 11417 - GCD
5.3.3
Now that we have discussed prime number and GCD, we can dene relatively prime. Two
integers a and b are said to be relatively prime if gcd(a, b) = 1, e.g. 25 and 42.
There is an interesting problem of nding positive integers below N that are relatively prime
to N (see programming exercises below). A nave algorithm starts with counter = 0, iterates
through i [1 . . . N ], and increases the counter if gcd(i, N ) = 1, but this is slow for a large N .
A better algorithm is the Eulers Totient (Phi) function [36]. The Eulers Phi (N ) is a function
to compute the solution for the problem posed above. (N ) = N
PF
1
P F ), where we iterate
36 (1 12 ) (1 13 ) = 12.
(1
In fact, there are only 12 integers less than equal to 36 that are relatively prime to 36. They
are 1, 5, 7, 11, 13, 17, 19, 23, 25, 29, 31, and 35. As we need to factor N , the complexity of
this algorithm is similar with the complexity of factoring an integer with trial division mentioned
earlier. The code is below.
int EulerPhi(int N) {
vi factors = primeFactors(N);
vi::iterator new_end = unique(factors.begin(), factors.end()); // get unique
int result = N;
for (vi::iterator i = factors.begin(); i != new_end; i++)
result = result - result / *i;
return result;
}
5.3.4
Motivating problem: Suppose a housewife buys apples and oranges at a total cost of 8.39 SGD. If
an apple is 25 cents and an orange is 18 cents, how many of each type of fruit does she buys?
This problem can be modeled as a linear equation with two variables: 25x + 18y = 839. Since
we know that both x and y must be integers, this linear equation is called the Linear Diophantine
Equation. We can solve Linear Diophantine Equation with two variables even if we only have one
equation! This is dierent from System of Linear Equations discussed later in Section 5.5.5. The
solution for the Linear Diophantine Equation is as follow [20].
Let a and b be integers with d = gcd(a, b). The equation ax + by = c has no integral solutions
if d | c is not true. But if d | c, then there are innitely many integral solutions. The rst solution
(x0 , y0 ) can be found using the Extended Euclid algorithm shown below (also see [4], Chapter 31),
and the rest can be derived from x = x0 + (b/d)n, y = y0 (a/d)n, where n is an integer.
// store x, y, and d as global variables
void extendedEuclid(int a, int b) {
if (b == 0) { x = 1; y = 0; d = a; return; }
extendedEuclid(b, a % b);
int x1 = y;
int y1 = x - (a / b) * y;
x = x1;
y = y1;
}
Using extendedEuclid, the Linear Diophantine Equation with two variables can be easily solved.
For our motivating problem above: 25x + 18y = 839, we have:
a = 25, b = 18, extendedEuclid(25, 18) = ((5, 7), 1), or
25 (5) + 18 7 = 1.
Multiplying the left and right hand side of the equation above by 839/gcd(25, 18) = 839, we have:
25 (4195) + 18 5873 = 839. Thus,
x = 4195 + (18/1)n, y = 5873 (25/1)n.
Since we need to have non-negative x and y, we have:
4195 + 18n 0 and 5873 25n 0, or
4195/18 n 5873/25, or
233.05 n 234.92.
The only possible integer for n is 234.
Thus x = 4195 + 18 234 = 17 and y = 5873 25 234 = 23,
i.e. 17 apples (of 25 cents each) and 23 oranges (of 18 cents each) of a total of 8.39 SGD.
Programming Exercises related to Extended Euclid algorithm:
1. UVa 718 - Skycraper Floors
2. UVa 10090 - Marbles (use solution for Linear Diophantine Equation)
3. UVa 10104 - Euclid Problem (pure problem involving Extended Euclid)
5.3.5
Modulo Arithmetic
Some mathematics computations in programming problems can end up having very large positive
(or very small negative) results that reside the range of largest integer data type (currently the
64-bit long long in C++). Sometimes, we are only interested with the result modulo a number.
For example in UVa 10176 - Ocean Deep! Make it shallow!!, we are asked to convert a long
binary number (up to 100 digits) to decimal. A quick calculation shows that the largest possible
number is 2100 1 which is beyond the reach of 64-bit integers. However, the problem only ask if
the result is divisible by 131071. So what we need to do is to convert binary to decimal digit by
digit, and then quickly perform modulo operation to the intermediate result by 131071. If the nal
result is 0, then the actual number in binary (which we never compute), is divisible by 131071.
Programming Exercises related to Modulo Arithmetic:
1. UVa 374 - Big Mod
2. UVa 602 - What Day Is It?
3. UVa 10174 - Couple-Bachelor-Spinster Numbers
4. UVa 10176 - Ocean Deep! Make it shallow!! (as discussed above)
5. UVa 10212 - The Last Non-zero Digit
6. UVa 10489 - Boxes of Chocolates
7. LA 4104 - MODEX
5.3.6
Fibonacci Numbers
Leonardo Fibonaccis numbers are dened as f ib(0) = 0, f ib(1) = 1, and f ib(n) = f ib(n 1) +
f ib(n 2) for n 2. This generates the following familiar patterns: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55,
89, . . . which can be derived with an O(n) DP technique. This pattern sometimes appears in some
contest problems which do not mention the term Fibonacci at all, like in some problems shown as
exercises below (e.g. UVa 900, etc). Note that Fibonacci sequence grows very fast and sometime
problems involving Fibonacci have to be solved using Java BigInteger library (see Section 5.4 for a
quick solution involving large integers).
Fibonacci numbers have many interesting properties. One of them is the Zeckendorf s theorem: every positive integer can be written in a unique way as a sum of one or more distinct
Fibonacci numbers such that the sum does not include any two consecutive Fibonacci numbers.
Programming Exercises related to Fibonacci:
1. UVa 495 - Fibonacci Freeze (use Java BigInteger class)
2. UVa 763 - Fibinary Numbers (Zeckendorf representation)
3. UVa 900 - Brick Wall Patterns (Combinatorics, the pattern is similar to Fibonacci)
4. UVa 948 - Fibonaccimal Base (Zeckendorf representation)
5. UVa 10183 - How many Fibs?
6. UVa 10229 - Modular Fibonacci
7. UVa 10334 - Ray Through Glasses (use Java BigInteger class)
8. UVa 10450 - World Cup Noise (Combinatorics, the pattern is similar to Fibonacci)
9. UVa 10497 - Sweet Child Make Trouble (Combinatorics, the pattern is Fibonacci variant)
10. UVa 10579 - Fibonacci Numbers
11. UVa 10862 - Connect the Cable Wires (the pattern ends up very similar to Fibonacci)
12. UVa 11000 - Bee
13. UVa 11161 - Help My Brother (II) (Fibonacci + median)
14. UVa 11780 - Miles 2 Km (the background of this problem is about Fibonacci Numbers)
5.3.7
Factorial
6. UVa 10061 - How many zeros & how many digits? (there exists a formula for this)
7. UVa 10139 - Factovisors (there exists a formula for this)
8. UVa 10858 - Recover Factorial
9. UVa 10220 - I Love Big Numbers! (use Java BigInteger class)
10. UVa 10323 - Factorial! You Must Be Kidding
11. UVa 10780 - Again Prime? No time.
12. UVa 11347 - Multifactorials
13. UVa 11415 - Count the Factorials
5.4
5.4.1
Basic Features
=
=
1,000,000,000,000,000,000,000
17
------------------------------ *
7,000,000,000,000,000,000,000
10,000,000,000,000,000,000,00
------------------------------ +
num1 * num2 = 17,000,000,000,000,000,000,000
Addition and subtraction are two simpler operations in BigInteger. Multiplication takes a bit more
programming job. Ecient division and raising number to a certain power are more complicated.
Anyway, coding these library routines in C/C++ under stressful contest environment can be a
buggy aair, even if we can bring notes containing such C/C++ library in ICPC. Fortunately, Java
has a BigInteger class that we can use for this purpose (as of 9 August 2010, C++ STL currently
does not have such library thus it is a good idea to use Java for BigInteger problems).
The Java BigInteger (BI) class supports the following basic integer operations: addition
example, the following short Java code is the solution for UVa 10925 - Krakovia which simply
requires BigInteger addition (to sum N large bills) and division (to divide the large sum to F
friends).
import java.io.*;
import java.util.*; // Scanner class is inside this package
import java.math.*; // BigInteger class is inside this package
5.4.2
Bonus Features
The Java BigInteger class has a few more bonus features that can be useful in programming
contests. It happens to have a built-in GCD routine gcd(BI), a modular arithmetic function
modPow(BI exponent, BI m), and Base Number converter toString(int radix).
GCD Revisited
When we need to compute the GCD of two big integers, we do not have to worry. See an example
below for UVa 10814 - Simplifying Fractions that ask us to simplify a given (large) fraction to its
simplest form by dividing both numerator and denominator with the gcd between them.
import java.io.*;
import java.util.*;
import java.math.*;
One of the problems presented in the previous section is LA 4104 - MODEX. We are asked to nd
what is the value of xy (mod n). It turns out that this problem can be solved with:
import java.io.*;
import java.util.*;
import java.math.*;
class Main { /* LA 4104 - MODEX */
public static void main(String[] args) {
Scanner sc new Scanner(System.in);
int nTC = sc.nextInt();
while (nTC-- > 0) {
BigInteger x = BigInteger.valueOf(scan.nextInt());
BigInteger y = BigInteger.valueOf(scan.nextInt());
BigInteger n = BigInteger.valueOf(scan.nextInt());
System.out.println(x.modPow(y, n)); // look ma, its in the library ;)
} } }
Programming Exercises related to Big Integer that are not mentioned elsewhere in this chapter.
1. UVa 343 - What Base Is This? (number base conversion)
2. UVa 355 - The Bases Are Loaded (number base conversion)
3. UVa 389 - Basically Speaking (number base conversion)
4. UVa 424 - Integer Inquiry (bignum addition)
5. UVa 446 - Kibbles n Bits n Bits n Bits (number base conversion)
6. UVa 636 - Squares (number base conversion++)
7. UVa 10083 - Division (bignum + number theory)
8. UVa 10551 - Basic Remains (bignum mod and base conversion)
4
5.5
In this section, we list down a few more mathematics problems that have been used a few times
in some programming contests. Mastering the solutions for these problems can be an advantage
if you are given similar problem or problem that uses these knowledge as part of solution for the
problem. The terms mentioned here also act as keywords for further study.
5.5.1
Combinatorics
Combinatorics [31] is a branch of discrete mathematics concerning the study of nite or countable discrete structures. Programming problems involving combinatorics usually titled How Many
[Object], Count [Object], etc, although some problem setters choose to hide this fact from their
problem title. The code is usually short, but nding the recurrence formula takes some mathematics brilliance and patience. In ICPC, if such problem exists in the given problem set, ask one team
member to derive the formula whereas the other two concentrates on other problems. Quickly code
the usually short formula once it is obtained.
For example, try solving UVa 10401 - Triangle Counting. This problem has a short description:
given n rods of length 1, 2, . . . , n, pick any 3 of them & build a triangle. How many distinct
triangles can you make? (3 n 1M ) . Note that, two triangles will be considered dierent if
they have at least one pair of arms with dierent lengths. If you are lucky, you may spend only a
few minutes to spot the pattern. Otherwise, this problem may end up unsolved by the time contest
is over. Hint: answers for few small n = 3, 4, 5, 6, 7, 8, 9, and 10 are 0, 1, 3, 7, 13, 22, 34, and 50,
respectively.
Programming Exercises related to Combinatorics (also see Fibonacci Numbers above):
1. UVa 326 - Extrapolation using a Dierence Table
2. UVa 369 - Combinations (be careful with overow issue)
3. UVa 991 - Safe Salutations (Catalan Numbers)
4. UVa 10007 - Count the Trees (Catalan Numbers, see [30] for details)
5. UVa 10219 - Find the Ways! (use Java BigInteger class)
6. UVa 10303 - How Many Trees (Catalan Numbers)
7. UVa 10375 - Choose and Divide
8. UVa 10784 - Diagonal
9. UVa 10790 - How Many Points of Intersection?
5.5.2
Cycle-Finding
For example in UVa 350 - Pseudo-Random Numbers, we are given a pseudo-random number
generator f (x) = (Z x + I)%M with x0 = L and we want to nd out the sequence length before
any number is repeated.
A nave algorithm that works in general for this problem uses a data structure (e.g. C++ STL
<set>, hash table or direct addressing table) to store information that a number xi has been
visited in the sequence and then for each xj found later (j > i), we test if xj is stored in the data
structure or not. If it is, it implies that xj = xi , = i, = j i. This algorithm runs in O( + )
but also requires at least O( + ) space to store past values.
There is a better algorithm called Floyds Cycle-Finding algorithm [32] that runs in the same
O( + ) time complexity but only uses O(1) memory space much smaller than the nave version.
The working C/C++ implementation of this algorithm is shown below:
pair<int, int> floyd_cycle_finding(int (*f)(int), int x0) {
// The main phase of the algorithm, finding a repetition x_i = x_2i, hare speed is 2x tortoises
int tortoise = f(x0), hare = f(f(x0)); // f(x0) is the element/node next to x0
while (tortoise != hare) { tortoise = f(tortoise); hare = f(f(hare)); }
// Find the position of mu, the hare and tortoise move at the same speeds
int mu = 0; hare = tortoise; tortoise = x0;
while (tortoise != hare) { tortoise = f(tortoise); hare = f(hare); mu += 1; }
// Find the length of the shortest cycle starting from x_mu, hare moves, tortoise stays
int lambda = 1; hare = f(tortoise);
while (tortoise != hare) { hare = f(hare); lambda += 1; }
return make_pair(mu, lambda);
}
5.5.3
Some Ad Hoc mathematic problems involve a denition of existing (or ctional) number Sequence
or Number System and our task is to produce either the sequence/number within some range
or the n-th one, verify if the given sequence/number is valid according to denition, etc. Usually,
following the description carefully is sucient to solve the problem.
Programming Exercises related to Sequences and Number Systems:
Sequences
1. UVa 100 - The 3n + 1 problem
2. UVa 413 - Up and Down Sequences
3. UVa 694 - The Collatz Sequence (similar to UVa 100)
4. UVa 10408 - Farey Sequences
5. UVa 10930 - A-Sequence
6. UVa 11063 - B2 Sequences
Number Systems
1. UVa 136 - Ugly Numbers
2. UVa 138 - Street Numbers
3. UVa 443 - Humble Numbers
4. UVa 640 - Self Numbers (DP)
5. UVa 962 - Taxicab Numbers (Pre-calculate the answer)
6. UVa 974 - Kaprekar Numbers
7. UVa 10001 - Bangla Numbers
8. UVa 10006 - Carmichael Numbers
9. UVa 10042 - Smith Numbers
10. UVa 10044 - Erdos Numbers (solvable with BFS)
11. UVa 10591 - Happy Number (solvable with the Floyds Cycle-Finding algorithm)
12. UVa 11461 - Square Numbers
13. UVa 11472 - Beautiful Numbers
5.5.4
Probability Theory
5.5.5
Linear Algebra
A linear equation is dened as an equation where the order of the unknowns (variables) is linear
2
3.
Notice the
dierence to the linear diophantine equation as the solution for system of linear equations
can be non-integers!
There are several ways to nd the solution for a system of linear equations. One of them is
Gaussian Elimination. See [5, 37] for details.
Programming Exercises related to Linear Algebra:
1. UVa 10089 - Repackaging
2. UVa 10109 - Solving Systems of Linear Equations
3. UVa 10309 - Turn the Lights O
5.6
Chapter Notes
We admit that there are still a lot of other mathematics problems and algorithms beyond this
chapter. This last section provides pointers for a few more topics.
For an even faster prime testing function than the one presented here, one can use the
non deterministic Miller-Rabins algorithm [41] which can be made deterministic for contest
environment with a known maximum input size N .
In this chapter, we have seen a quite eective trial division method for nding prime factors
of an integer. For a faster integer factorization, one can use the Pollards rho algorithm [4].
However, if the integer to be factored is a large prime number, then this is still a slow business.
function, Riemanns hypothesis, Goldbachs conjecture, twin prime conjecture, etc. However, when
such things appear in programming contests, usually their denitions are given!
We can compute f ib(n) in O(log n) using matrix multiplication, but this is usually not needed
Chinese Remainder Theorem (e.g. UVa 756 - Biorhythms), Divisibility properties (e.g. UVa
995), Pascals Triangle, Combinatorial Games (e.g. the Sprague-Grundys theorem for
games like UVa 10165 - Stone Game (Nim game), Chess, Tic-Tac-Toe, etc), problems involving
chapter for that, we reserve the discussions related to geometry problems in Chapter 7.
In terms of doing well in ICPC, it is a good idea to have at least one strong mathematician in
your ICPC team. This is because there usually exists one or two mathematics problems in the set
where the solution is short but getting the solution/formula requires a strong thinking cap.
We suggest that interested readers should browse more about number theory see books like
Chapter 6
String Processing
Human Genome has approximately 3.3 Giga base-pairs
Human Genome Project
In this chapter, we present one more topic that is tested in ICPC, namely: string processing. Processing
(long) string is quite common in the research eld of bioinformatics and some of such problems are presented
as contest problems in ICPC.
6.1
Although not as frequent as graph and mathematics problems in the previous two chapters, string
processing problems are also found in recent ICPCs (see Table 6.1). Some string-related problems
have huge inputs. Thus the solution must use ecient data structures and algorithms for string.
LA
2460
2972
3170
3669
3791
3901
3999
4200
4657
Problem Name
Searching Sequence ...
A DP Problem
AGTC
String Cutting
Team Arrangement
Editor
The longest constant gene
Find the Format String
Top-10
Source
Singapore01
Tehran03
Manila06
Hanoi06
Tehran06
Seoul07
Danang07
Dhaka08
Jakarta09
Table 6.1: Some String Processing Problems in Recent ACM ICPC Asia Regional
6.2
We start this chapter by mentioning Ad Hoc string processing problems. They are contest problems
involving string that require no more than basic programming skills. We only need to read the
requirements in the problem description carefully and code it. Sometimes, pure character array
(a.k.a. string) manipulation is sucient. Sometimes, we need string libraries like C <string.h>,
C++ <string> class, or Java String class. For example, we can use strstr in C to nd certain
substring in a longer string (also known as string matching or string searching), strtok in C to
tokenize longer string into tokens based on some delimiters. Here are some examples:
Programming Exercises related to Ad Hoc String Processing:
1. UVa 148 - Anagram Checker (+ backtracking)
2. UVa 159 - Word Crosses
3. UVa 263 - Number Chains
4. UVa 353 - Pesky Palindromes
5. UVa 401 - Palindromes
6. UVa 409 - Excuses, Excuses! (string matching)
7. UVa 422 - Word Search Wonder (string searching in a grid)
8. UVa 537 - Articial Intelligence?
9. UVa 644 - Immediate Decodability
10. UVa 865 - Substitution Cypher (simple character substitution mapping)
11. UVa 902 - Password Search
12. UVa 10010 - Wheres Waldorf? (string searching in a grid)
13. UVa 10115 - Automatic Editing
14. UVa 10197 - Learning Portuguese
15. UVa 10293 - Word Length and Frequency
16. UVa 10391 - Compound Words (Use ecient Data Structure!)
17. UVa 10508 - Word Morphing
18. UVa 10815 - Andys First Dictionary
19. UVa 10878 - Decode the Tape
20. UVa 10896 - Known Plaintext Attack
21. UVa 11056 - Formula 1 (involving case insensitive string comparison)
22. UVa 11062 - Andys Second Dictionary
23. UVa 11221 - Magic Square Palindrome (solvable without DP)
24. UVa 11233 - Deli Deli
25. UVa 11278 - One-Handed Typist
26. UVa 11362 - Phone List
27. UVa 11385 - Da Vinci Code (string manipulation + Fibonacci numbers)
28. UVa 11048 - Automatic Correction of Misspellings
29. UVa 11713 - Abstract Names (modied string comparison function)
30. UVa 11716 - Digital Fortress (simple cipher)
31. UVa 11734 - Big Number of Teams will Solve This (modied string comparison function)
However, recent contest problems in ACM ICPC usually do not ask solutions for basic string
processing except for the giveaway problem that all teams should be able to solve. Some string
processing problems are solve-able with Dynamic Programming (DP) technique. We discuss them
in Section 6.3. Some other string processing problems have to deal with long strings, thus an
ecient data structure for string like Sux Tree or Sux Array must be used. We discuss these
data structures and several specialized algorithms using these data structures in Section 6.4.
6.3
In this section, we discuss some string processing problems that solve-able with DP techniques
discussed in Section 3.4.
6.3.1
The String Alignment (or Edit Distance) problem is dened as follows: Given two strings A and B,
align1 A with B with the maximum alignment score (or minimum number of edit operations):
After aligning A with B, there are few possibilities between character A[i] and B[i] index i:
1. Character A[i] and B[i] match (assume we give +2 score),
2. Character A[i] and B[i] mismatch and we replace A[i] with B[i] (-1 score),
3. We insert a space in A[i] (also -1 score), or
4. We delete a letter from A[i] (also -1 score).
For example:
A = information
-> ___information_
B = bioinformatics -> bioinformatic_s
---222222222--- -> String Alignment Score = 9 x 2 - 6 = 12
A brute force solution that tries all possible alignments will typically end up with a TLE verdict
for long strings A and/or B. The solution for this problem is a well-known DP solution (NeedlemanWunschs algorithm [24]). Consider two strings A[1 ... n] and B[1 ... m]. We dene V (i, j)
to be the score of the optimal alignment between A[1 ... i] and B[1 ... j] and score(A, B) is
the score if character A is aligned with character B.
Base case:
V (0, 0) = 0 // no score for matching two empty strings
Recurrences: For i > 0 and j > 0:
V (0, j) = V (0, j 1) + score( , B[j]) // insert space j times to make alignment
V (i, 0) = V (i 1, 0) + score(A[i], ) // delete i times to make alignment
V (i, j) = max(option1, option2, option3), where
option1 = V (i 1, j 1) + score(A[i], B[j]) // score of match or mismatch
option2 = V (i 1, j) + score(A[i], ) // delete
option3 = V (i, j 1) + score( , B[j]) // insert
In short, this DP algorithm concentrates on the three possibilities for the last pair of characters,
xxx...xx
|
yyy...y_
delete
xxx...x_
|
yyy...yy
insert
With a simple cost function where a match gets a +2 point and mismatch, insert, delete all get
a -1 point, the detail of string alignment score of A = ACAATCC and B = AGCATGC is shown
1
Align is a process of inserting spaces to strings A or B such that they have the same number of characters.
Figure 6.1: String Alignment Example for A = ACAATCC and B = AGCATGC (score = 7)
in Figure 6.1. The alignment score is 7 (bottom right). Follow the dashed (red) arrows from the
bottom right cell to reconstruct the solution Diagonal arrow means a match or a mismatch (e.g.
the last C). Vertical arrow means a deletion (e.g. ..CAT.. to ..C_A..). Horizontal arrow means
A = A_CAAT[C]C
B = AGC_AT[G]C
As we need to ll in all entries in the table of n m matrix and each entry can be computed in
O(1), the time complexity is O(nm). The space complexity is O(nm) the size of the DP table.
6.3.2
The Longest Common Subsequence (LCS) problem is dened as follows: Given two strings A and
B, what is the longest common subsequence between them. For example, A = ACAATCC and
for mismatch as negative innity, cost for insertion and deletion as 0, and the cost for match as 1.
6.3.3
Palindrome
A palindrome is a string that can be read the same way in either direction. Some variants of
palindrome nding problems are solve-able with DP technique, as shown in this example: given a
string of up to 1000 characters, determine the length of the longest palindrome that you can make
The DP solution: let len(l, r) be the length of the longest palindrome from string A[l ... r].
Base cases:
Recurrences:
If (A[l] = A[r]), then len(l, r) = 2 + len(l + 1, r 1). // both corner characters are similar
else len(l, r) = max(len(l, r 1), len(l + 1, r)). // increase left side or decrease right side
Programming Exercises related to String Processing with DP:
1. UVa 164 - String Computer (String Alignment/Edit Distance)
2. UVa 531 - Compromise (Longest Common Subsequence + printing solution)
3. UVa 10066 - The Twin Towers (Longest Common Subsequence - but not on string)
4. UVa 10100 - Longest Match (Longest Common Subsequence)
5. UVa 10192 - Vacation (Longest Common Subsequence)
6. UVa 10405 - Longest Common Subsequence (as the problem name implies)
7. UVa 10739 - String to Palindrome
8. UVa 11151 - Longest Palindrome
9. LA 2460 - Searching Sequence Database in Molecular Biology (String Alignment)
6.4
Sux Tree and Sux Array are two ecient and related data structures for strings. We do not
put this topic in Chapter 2 as these two data structures are special for strings.
6.4.1
Figure 6.2: Sux Trie (Left) and Sux Tree (Right) of S = acacag$ (Figure from [24])
Consider a string2 S = acacag$, a Sux3 Trie of S is a tree that contains all possible suxes
of S (see Figure 6.2, left). Two suxes that share common prex will share the same rst few
vertices, e.g. cag$ and cacag$ share the rst two vertices ca before they split. The leaves
contain the indices of the suxes. Sux Tree of S is Sux Trie where we merge vertices with only
one child (see Figure 6.2, right). Notice the edge-label and path-label in the gure.
Exercise: Draw the Sux Tree of S = competitive$!
2
Notice that S is usually appended with a special char $ which is lexicographically smaller than all the alphabets
used in S.
3
A sux of a string is a special case substring that goes up to the last character of the string.
6.4.2
Assuming that a Sux Tree4 for a string S is already built, we can use it for these applications:
Exact String Matching in O(|Q| + occ)
With Sux Tree, we can nd all (exact) occurrences of a query string Q in S in O(|Q| + occ) where
|Q| is the length of the query string Q itself and occ is the total number of occurrences of Q in S
no matter how long the string S is. When the Sux Tree is already built, this approach is faster
than many exact string matching algorithms (e.g. KMP).
With Sux Tree, our task is to search for the vertex x in the Sux Tree which represents the
query string Q. This can be done by just one root to leaf traversal that follows the edge labels.
Vertex with path-label = Q is the desired vertex x. Then, leaves in the subtree rooted at x are the
occurrences of Q in S. We can then read the starting indices of such substrings that are stored in
the leaves of the sub tree.
For example, in the Sux Tree of S = acacag$ shown in Figure 6.2, right and Q = aca,
we can simply traverse from root, go along the edge label a, then the edge label ca to nd vertex
x with the path-label aca (follow the dashed red arrow in Figure 6.2, right). The leaves of this
vertex x point to index 1 (substring: acacag$) and index 3 (substring: acag$).
Exercise: Now try to nd a query string Q = ca and Q = cat!
Finding Longest Repeated Substring in O(n)
With Sux Tree, we can also nd the longest repeated substring in S easily. The deepest internal
vertex X in the Sux Tree of S is the answer. Vertex X can be found with an O(n) tree traversal.
The fact that X is an internal vertex implies that it represent more than one suxes (leaves) of
string S and these suxes shared a common prex (repeated substring). The fact that X is the
deepest internal vertex (from root) implies that its path-label is the longest repeated substring.
For example, in the Sux Tree of S = acacag$ shown in Figure 6.2, right, the longest
repeated substring is aca as it is the path-label of the deepest internal vertex.
Exercise: Find the longest repeated substring in S = cgacattacatta$!
Finding Longest Common Substring in O(n)
The problem of nding the Longest Common Substring (not Subsequence)5 of two or more
strings can be solved in linear time with Sux Tree. Consider two strings S1 and S2, we can
build a generalized Sux Tree for S1 and S2 with two dierent ending markers, e.g. S1 with
character # and S2 with character $. Then, we mark each internal vertices with have leaves
that represent suxes of both S1 and S2 this means the suxes share a common prex. We then
report the deepest marked vertex as the answer.
For example, with S1 = acgat# and S2 = cgt$, The Longest Common Substring is cg
of length 2. In Figure 6.3, we see the root and vertices with path-labels cg, g, and t all have
two dierent leaf markers. The deepest marked vertex is cg. The two suxes cgat# and cgt$
share a common prex cg.
4
5
As Sux Tree is more compact than Sux Trie, we will concentrate on Sux Tree.
In abcdef, bce (skip character d) is subsequence and bcd (contiguous) is substring and also subsequence.
Figure 6.3: Generalized Sux Tree of S1 = acgat# and S2 = cgt$ (Figure from [24])
6.4.3
Building ecient Sux Tree under contest environment is a bit complex and risky. Fortunately,
another data structure that we are going to describe, called Sux Array invented by Udi Manber
and Gene Myers, has similar functionalities as Sux Tree but simpler to implement, especially
in programming contest setting. Thus we will skip the discussion on building a Sux Tree and
consider the same S = acacag$ with n = 7. The Sux Array of S is an integer permutation of
n as shown in Figure 6.4. Note that we regard the terminating symbol $ as the lexicographically
smallest character.
Sux Tree and Sux Array are very related. As we can see in Figure 6.5, the leaves of a Sux
Tree (from left to right) is in Sux Array order. In short, a vertex in Sux Tree corresponds to
Figure 6.5: Sux Tree versus Sux Array of S = acacag$ (Figure from [24])
A Sux Array is good enough for many practical string operations in contest problems. In this
section, we present two simple ways to build a Sux Array given a string S[0 ... n-1].
#include <iostream>
#include <stdlib.h>
#include <string.h>
using namespace std;
char S[1001]; // this naive Suffix Array cannot go beyond 1000 characters
int SA[1001], n;
// compare suffixes
int SA_cmp(const void *a, const void *b) { return strcmp(S + *(int*)a, S + *(int*)b); }
int main() { // first approach: O(n^2 log n), only for
n = strlen(gets(S));
for (int i = 0; i < n; i++) SA[i] = i; // sort
qsort(SA, n, sizeof(int), SA_cmp);
// O(n log n)
for (int i = 0; i < n; i++) printf("%d %s\n", SA[i],
} // return 0;
n <= 1000
* comparison
* O(n) = O(n^2 log n)
S+SA[i]);
When applied to string S = acacag$, the simple code that simply sort all suxes with sort
library will produce the correct Sux Array = {6, 0, 2, 4, 1, 3, 5} (note that index starts from 0).
However, this is barely useful except for contest problems with n 1000.
A better way to build the Sux Array is to sort suxes O(n log n) in increasing length. We
start from suxes with length 1, length 2, length 4, length 8, ..., up to n. As the length grows
exponentially, we only need O(log n) steps. Thus the overall complexity is O(n log2 n). With this
complexity, working with strings of length n 100K the typical programming contest range is
not a problem. The library code is shown below. For explanation, see [29, 24].
#include <algorithm>
#include <iostream>
#include <stdlib.h>
#include <string.h>
using namespace std;
#define MAXN 200010
int RA[MAXN], SA[MAXN], LCP[MAXN], *FC, *SC, step;
char S[MAXN], Q[MAXN];
bool cmp(int a, int b) {
if (step==-1 || FC[a]!=FC[b]) return FC[a] < FC[b];
return FC[a+(1<<step)] < FC[b+(1<<step)];
}
void suffix_array(char *S, int n) { // O(n log^2(n))
for (int i=0; i<n; i++) RA[i] = S[SA[i] = i];
for (FC=RA, SC=LCP, step=-1; (1<<step)<n; step++) {
sort(SA, SA+n, cmp);
int cnt = 0;
for (int i=0; i<n; i++) {
if (i>0 && cmp(SA[i-1],SA[i])) cnt++;
SC[SA[i]] = cnt;
}
if (cnt==n-1) break; // all distinct, no need to continue
swap(FC,SC);
}
for (int i=0; i<n; i++) RA[SA[i]] = i;
}
Now with a Sux Array already built, we can search for a query string Q of length m in string S of
length n in O(m log n). This is O(log n) times slower than the Sux Tree version but in practice is
quite acceptable. The O(m log n) complexity comes from the fact that we can do O(log n) binary
search on a sorted suxes and do up to O(m) comparisons per sux.
The fact that the occurrences of Q in the Sux Array of S are consecutive can be used to
deal with the longest repeated substring and the longest common substring problems in similar
O(m log n) + O(occ). For example in S = acacag$, the repeated substring ca occurs in
SA[4] and SA[5]. For the common substring between S1 = acgat# and S2 = cgt$, we can
concatenate the two string into S = acgat#cgt$, build the Sux Array of the concatenated
string, and then modify the comparison function to check for the marker character # and $.
Our code for nding the query string Q in a Sux Array is shown below:
int main() {
int n = strlen(gets(S));
suffix_array(S, n + 1); // NULL is included!
for (int i = 1; i <= n; i++) // SA[0] is the NULL
printf("%d %s\n", SA[i], S + SA[i]);
gets(Q);
pair<int, int> pos = range(n, Q);
if (pos.first != -1 && pos.second != -1) {
printf("%s is found SA [%d .. %d] of %s\n", Q, pos.first, pos.second, S);
printf("They are:\n");
for (int i = pos.first; i <= pos.second; i++)
printf(" %s\n", S + SA[i]);
}
else
printf("%s is not found in %s\n", Q, S);
return 0;
6.5
Chapter Notes
The materials regarding String Alignment (Edit), Longest Common Subsequence, Sux Tree, and
Sux Array are courtesy of A/P Sung Wing Kin, Ken [24] from School of Computing, National
University of Singapore.
The string alignment problem discussed in Section 6.3 is called the global alignment problem.
The best solution so far is O(nm/ log2 n) [24] but it is complicated to code. However if the given
contest problem is limited to d insertions or deletions only, we can speed up Needleman-Wunschs
algorithm to O(dn) by just concentrating on the main diagonal of DP matrix.
For the specialized case of local alignment problem, a better algorithm similar to NeedlemanWunschs algorithm exists (Smith-Watermans algorithm [24]).
The Longest Common Subsequence problem can be solved in O(n log n) when all characters are
distinct [24].
There are other string matching algorithms that are not discussed in this chapter: Rabin-
Chapter 7
(Computational) Geometry
Let no man ignorant of geometry enter here.
Platos Academy in Athens
(Computational) Geometry is yet another topic that frequently appears in programming contests. Many
contestants afraid to tackle them due to oating point precision errors1 or the many tricky special cases
commonly found in geometry problems. Some others skip these problems as they forgot some important formulas and unable to derive the required formulas from basic concepts. Study this chapter for some ideas on
tackling (computational) geometry problems in ICPC.
7.1
Almost all ICPC problem sets have at least one geometry problem. If you are lucky, it ask you for
some geometry solution that you have learned in pre-University which you still remember or the
solution can be derived by drawing the geometrical objects. However, many geometry problems
are more complex and nding the correct algorithm is harder. Table 7.1 shows some geometry
problems in recent ACM ICPC Asia Regional.
LA
2797
3169
3616
4107
4410
4413
4532
4639
4642
4717
Problem Name
Monster Trap
Boundary Points
How I Wonder ...
TUSK
Shooting the Monster
Triangle Hazard
Magic Rope
Separate Points
Malfatti Circles
In-circles Again
Source
Aizu03
Manila06
Yokohama06
Singapore07
KLumpur08
KLumpur08
Hsinchu09
Tokyo09
Tokyo09
Phuket09
Table 7.1: Some (Computational) Geometry Problems in Recent ACM ICPC Asia Regional
1
To avoid this error, usually we do oating-point comparison test in this way: f abs(a b) < EP S where EP S
usually is a small number like 1e-9.
We divide this chapter into two parts. The rst part is geometry basics in Section 7.2. We
review many (not all) English geometric terminologies and formulas that are commonly used in
programming contests. The second part deals with computational geometry in Section 7.4 7.5, where we use data structures and algorithms which can be stated in terms of geometry.
7.2
Geometry Basics
As ACM ICPC contestants come from various nationalities, languages, and backgrounds, sometimes
these English geometric terminologies below may look alien. Here, we try to provide a list
which cannot be exhaustive to be used as a quick reference when contestants are given geometry
problems.
Lines
1. A Line can be described with mathematical equation: y = mx + c or ax + bx + c = 0.
The y = mx + c equation involves gradient / slope m.
Note: Be careful with vertical lines with innite slope. Usually, we treat the vertical
lines separately in the solution code (example of the special cases in geometry problems).
2. A Line Segment is a line with two end points with nite length.
360.0
2
Human usually works with degrees, but many mathematical functions in programming languages works with
radians. Check your programming language manual to verify this. To help with conversion, just remember that one
radian equals to 180 degrees.
5. The length of a Chord of a circle with a radius r and an angle (in degrees) can be
obtained with the Law of Cosines: 2r 2 (1 cos()) see the explanation of this law
in the discussion about Triangles below.
6. The Area A of a circle with a radius r is A = r 2
7. The area of a Sector of a circle with an area A and an angle (in degrees) is
360.0
8. The area of a Segment of a circle can be found by subtracting the area of the corresponding Sector with the area of an Isosceles Triangle with sides: r, r, and Chord-length.
a
sin()
b
sin()
c
sin() .
Rectangles
1. A Rectangle is a polygon with four edges, four vertices, and four right angles.
2. The Area A of a rectangle with width w and height h is A = w h.
3. The Perimeter p of a rectangle with width w and height h is p = 2 (w + h).
4. A Square is a special case of rectangle where w = h.
Trapeziums
1. A Trapezium is a polygon with four edges, four vertices, and one pair of parallel edges.
If the two non-parallel sides of the trapezium have the same length, we have an Isosceles
Trapezium.
2. The Area A of a trapezium with base w1, another edge parallel with the base w2 and
height h is A = 0.5 (w1 + w2) h.
Quadrilaterals
1. A Quadrilateral or Quadrangle is a polygon with with four edges (and four vertices).
Rectangles, Squares, and Trapeziums that are mentioned above are Quadrilaterals. Figure 7.3 shows a few more examples: Parallelogram, Kite, Rhombus.
Spheres
1. A Sphere is a perfectly round geometrical object in 3-D space.
2. The Great-Circle Distance between any two points A and B on sphere [38] is the
shortest distance along a path on the surface of the sphere. This path is equal to the
length of the Arc of the Great-Circle of that sphere that pass through the two points
A and B.
The Great-Circle of a sphere is dened as the circle that runs along the surface of the
sphere so that it cuts the sphere into two equal hemispheres. To nd the solution to the
Great-Circle Distance, we nd the central angle AOB (see Figure 7.4) of the Great-Circle
and use it to determine the length of the arc.
Although quite rare, some contest problems use this distance metric. Usually, the two
points on the surface of a sphere are given as (latitude, longitude) pair. The following
library code will help us obtain the shortest great-circle distance given two points on the
sphere and the radius of the sphere. For the derivation of this library code, see [38].
Figure 7.4: Great-Circle and Great-Circle Distance (Arc A-B) (Figures from [46])
double greatCircleDistance(double pLat, double pLong,
double qLat, double qLong, double radius) {
pLat *= PI / 180; pLong *= PI / 180;
qLat *= PI / 180; qLong *= PI / 180;
return radius * acos(cos(pLat)*cos(pLong)*cos(qLat)*cos(qLong) +
cos(pLat)*sin(pLong)*cos(qLat)*sin(qLong) +
sin(pLat)*sin(qLat));
}
Polygons
1. A Polygon is a plane gure that is bounded by a closed path or circuit composed of
a nite sequence of straight line segments. These segments are called edges or sides.
The point where two edges meet are the polygons vertex or corner. The interior of the
polygon is sometimes called its body.
2. A polygon is said to be Convex if any line segment drawn inside the polygon does not
intersect any edge of the polygon. Otherwise, the polygon is called Concave.
3. The area A of an n-sided polygon (either convex or concave) with n pairs of vertex
coordinates given in some order (clockwise or counter-clockwise) is:
|
|
1
|
= - * |
2
|
|
|
x1
x2
x3
.
.
.
xn
y1
y2
y3
.
.
.
yn
|
|
|
| = 1/2 * (x1y2 + x2y3 + y3y4 + ... + xny1
|
-x2y1 - x3y2 - x3y4 - ... - x1yn)
|
|
This can be written into the library code below. Notice that our default setting is allinteger coordinates and use all-integer operations whenever possible. You need to change
a part of this code if the given points are not integers:
struct point { int x, y; } // a point has two members
int determinant(vector<point> P) { // default: integer computation
int result = 0, x1, y1, x2, y2;
for (int i = 0; i < P.size(); i++) {
x1 = P[i].x; x2 = P[(i + 1) % P.size()].x;
y1 = P[i].y; y2 = P[(i + 1) % P.size()].y;
result += (x1 * y2 - x2 * y1);
}
return result;
}
// area is half of the determinant and the result may be a non-integer
double area(vector<point> P) { return fabs(determinant(P)) / 2.0); }
4. The perimeter p of an n-sided polygon with n pairs of coordinates given in some order
(clockwise or counter-clockwise) can be computed with Pythagorean theorem:
double perimeter(vector<point> P) { // point has x & y
double result = 0.0, x1, y1, x2, y2, dx, dy;
for (int i = 0; i < P.size(); i++) {
x1 = P[i].x; x2 = P[(i + 1) % P.size()].x;
y1 = P[i].y; y2 = P[(i + 1) % P.size()].y;
dx = x2 - x1;
dy = y2 - y1;
result += sqrt(dx * dx + dy * dy);
}
return result;
}
5. Testing if a polygon is convex (or concave) is easy with a quite robust3 geometric predicate test called CCW (Counter Clockwise) Test (a.k.a. Left-Turn Test).
CCW test is a simple yet important predicate test in computational geometry. This test
takes in 3 points p, q, r in a plane and determine if the sequence p q r is a left
turn4 . For example, CCW (p, q, r) where p = (0, 0), q = (1, 0), r = (0, 1) is true. This
test can be implemented with the following library code:
int turn(point
int result =
if (result <
if (result >
return 0; //
}
p, point q, point r) {
(r.x - q.x) * (p.y - q.y) 0) return -1; // P->Q->R is
0) return 1; // P->Q->R is
P->Q->R is a straight line,
With the library code above, we can now check if a polygon is convex by verifying if all
three consecutive points in the polygon make left-turns if visited in counter clockwise
order. If we can nd at least one triple where this is false, the polygon is concave.
There are of course exists many other geometric shapes, objects, and formulas that have not
been covered yet, like 3-D objects, etc. What we have covered so far are the ones which
appear more frequently in programming contests.
Other Programming Exercises related to Basic Geometry that are not listed above:
1. UVa 10088 - Trees on My Island (Georg A. Picks Theorem: A = i +
b
2
1, see [43])
Geometric programs are preferred to be robust, namely, no numerical errors. To help achieving that quality,
computations are often done by predicate tests (e.g. the CCW test) rather than by oating point calculations
which is prone to precision errors. Moreover, arithmetic operations used are limited to additions, subtractions and
multiplications on integers only (exact arithmetics).
4
Or in other words: p q r is counter-clockwise, r is on the left of line pq, triangle pqr has a positive area and
its determinant is greater than zero.
7.3
Grahams Scan
The Convex Hull of a set of points P is the smallest convex polygon CH(P ) for which each
point in P is either on the boundary of CH(P ) or in its interior (see Figure 7.5.D). Every vertex
in CH(P ) is a vertex in original P . Thus, the algorithm of nding convex hull must decide which
vertices in P to be chosen as part of the convex hull.
7.4
Intersection Problems
In geometry problems, we can virtually take any pair of objects and see if they intersect, and if
they are: what is the point/area/volume of intersection? This is the source of many computational
geometry problems in contests.
Line segment intersection is one the most frequent intersection problems. We can test whether
two line segments intersect or not by using several ccw tests (code is shown below).
struct line { point p1, p2; }
int intersect(line line1, line line2) {
return
((ccw(line1.p1, line1.p2, line2.p1) * ccw(line1.p1, line1.p2, line2.p2)) <= 0)
&&
((ccw(line2.p1, line2.p2, line1.p1) * ccw(line2.p1, line2.p2, line1.p2)) <= 0);
}
However, intersections can occur between dierent types of objects other than two line segments.
The other objects are: Cube, Box, Circle, Polygon, Triangle, Rectangle, etc. Some examples are
shown below.
Programming Exercises solvable related to intersection problem:
Line Segment Intersection
1. UVa 191 - Intersection
2. UVa 378 - Intersecting Lines
3. UVa 866 - Intersecting line segments
4. UVa 920 - Sunny Mountain
5. UVa 972 - Horizon Line
6. UVa 10902 - Pick-up sticks
7. UVa 11343 - Isolated Segments
Other Objects
1. UVa 453 - Intersecting Circles (circle and circle)
2. UVa 460 - Overlapping Rectangles (rectangle and rectangle)
3. UVa 737- Gleaming the Cubes (cube and cube)
4. UVa 904 - Overlapping Air Trac Control (3D-box and 3D-box)
5. UVa 10301 - Rings and Glue (circle and circle)
6. UVa 10321 - Polygon Intersection (convex polygon and convex polygon)
7.5
Several computational geometry problems turn out to be solvable with Divide and Conquer paradigm
that has been elaborated earlier in Section 3.2. One of the example is shown below.
7.6
Chapter Notes
Some materials from this chapter are derived from the materials courtesy of Dr Cheng Holun,
Alan from School of Computing, National University of Singapore.
There are many other Convex Hull algorithms such as Jarviss March, Gift Wrapping, and
Upper/Lower Hull with more or less similar time complexity as Grahams Scan. The Grahams
Scan algorithm presented in this chapter is usually enough for most contest problems.
There is a computational geometry technique that has not been discussed yet: plane sweep.
Interested reader should consult the following books [19, 6, 4].
If you are preparing for ICPC, it is a good idea to dedicate one person in your team to study this
topic in depth. This person should master basic geometry formulas and advanced computational
geometry techniques. He must train himself to be familiar with many degenerate (special) cases in
certain geometry problems, able to deal with precision errors, etc.
Appendix A
Problem Credits
The problems discussed in this book are mainly taken from UVa online judge [17] and ACM ICPC
Live Archive [11]. We have tried our best to contact the original authors and get their permissions.
So far, we have contacted the following problem setters and obtained their permissions: Sohel
Haz, Shahriar Manzoor, Manzurur Rahman Khan, Rujia Liu, Gordon Cormack, Jim Knisely,
yet does not allow his/her problem to be used in this book, we will replace that particular problem
Appendix B
Bibliography
[1] Ahmed Shamsul Aren. Art of Programming Contest (from Stevens Website). Gyankosh
Prokashoni (Available Online), 2006.
[2] Jon Bentley. Programming Pearls. Addison Wesley, 2nd edition, 2000.
[3] Frank Carrano. Data Abstraction & Problem Solving with C++. Pearson, 5th edition, 2007.
[4] Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Cli Stein. Introduction to
Algorithm. MIT Press, 2nd edition, 2001.
[5] Sanjoy Dasgupta, Christos Papadimitriou, and Umesh Vazirani. Algorithms. McGraw Hill,
2008.
[6] Mark de Berg, Marc van Kreveld, Mark Overmars, and Otfried Cheong Schwarzkopf. Computational Geometry: Algorithms and Applications. Springer, 2nd edition, 2000.
[7] Project Euler. Project Euler.
http://projecteuler.net/.
[8] Michal Forisek. IOI Syllabus.
http://people.ksp.sk/ misof/ioi-syllabus/ioi-syllabus-2009.pdf.
[9] Steven Halim and Felix Halim. Competitive Programming in National University of Singapore.
Ediciones Sello Editorial S.L. (Presented at Collaborative Learning Initiative Symposium CLIS
@ ACM ICPC World Final 2010, Harbin, China, 2010.
[10] Steven Halim, Roland Hock Chuan Yap, and Felix Halim. Engineering Stochastic Local Search
for the Low Autocorrelation Binary Sequence Problem. In Principles and Practice of Constraint Programming, pages 640645, 2008.
[13] Jon Kleinberg and Eva Tardos. Algorithm Design. Addison Wesley, 2006.
[14] Anany Levitin. Introduction to The Design & Analysis of Algorithms. Addison Wesley, 1st
edition, 2002.
[15] Rujia Liu. Algorithm Contests for Beginners (In Chinese). Tsinghua University Press, 2009.
[16] Rujia Liu and Liang Huang. The Art of Algorithms and Programming Contests (In Chinese).
Tsinghua University Press, 2003.
[19] Joseph ORourke. Computational Geometry in C. Cambridge University Press, 2nd edition,
1998.
[20] Kenneth H. Rosen. Elementary Number Theory and its applications. Addison Wesley Longman,
4th edition, 2000.
[21] Robert Sedgewick. Algorithms in C++, Part 1-5. Addison Wesley, 3rd edition, 2002.
[23] Steven S. Skiena and Miguel A. Revilla. Programming Challenges. Springer, 2003.
[24] Wing-Kin Sung. Algorithms in Bioinformatics: A Practical Introduction. CRC Press (Taylor
& Francis Group), 1st edition, 2010.
[28] Tom Verhoe. 20 Years of IOI Competition Tasks. Olympiads in Informatics, 3:149166, 2009.
[29] Adrian Vladu and Cosmin Negruseri. Sux arrays - a programming contest approach. 2008.
[47] Yonghui Wu and Jiang De Wang. Practical Algorithm Analysis and Program Design (In
Chinese). Posts and Telecom Press, 2009.
Index
ACM, 1
Dynamic Programming, 40
Array / Vector, 15
Articulation Points, 62
Backtracking, 27
Bellman Fords, 76
Euclid of Alexandria, 98
Bellman, Richard, 76
Eulers Totient, 98
Binary Search, 32
Euler, Leonhard, 98
Bisection Method, 33
bitset, 96
Bridges, 62
Brute Force, 26
Factorial, 101
Fibonacci Numbers, 101
Fibonacci, Leonardo, 101
Flood Fill, 61
Floyd Warshalls, 78
Floyd, Robert W, 78
Ford Fulkersons, 81
Combinatorics, 105
Competitive Programming, 1
Complete Search, 26
Connected Components, 60
Cycle-Finding, 106
Data Structures, 14
Decomposition, 38
Dijkstras, 74
Geometry, 120
Grahams Scan, 128
Graham, Ronald, 128
Graph, 58
Data Structure, 18
Greatest Common Divisor, 98
Greedy Algorithm, 35
Hash Table, 17
Heap, 16
Heron of Alexandria, 123
Herons Formula, 123
Diophantus of Alexandria, 99
ICPC, 1
IOI, 1
Prime Numbers, 94
Primality Testing, 95
Prime Factors, 96
Sieve of Eratosthenes, 95
Karp, Richard, 82
Kruskals, 70
Queue, 15
Segment Tree, 22
Libraries, 14
Sequences, 107
Linked List, 15
Negative Weight, 75
Live Archive, 10
Unweighted, 68
Weighted, 74
Special Graphs, 85
Bipartite Graph, 89
Mathematics, 93
Max Flow, 81
Max Edge-Disjoint Paths, 84
Max Flow with Vertex Capacities, 84
Max Independent Paths, 84
Min Cost (Max) Flow, 85
Min Cut, 83
Multi-source Multi-sink Max Flow, 83
Number Theory, 94
Palindrome, 113
Applications
Exact String Matching, 115
TopCoder, 10
Topological Sort, 66
USACO, 10
UVa, 10