STUART M.

SHIEBER

PROGRAMMING
WELL:
ABSTRACTION AND
DESIGN IN
C O M P U TAT I O N
2

1
 3

©2022 Stuart M. Shieber. All rights reserved for the time being, though
the intention is for this document to eventually be licensed under a CC
license. In the meantime, please do not cite, quote, or redistribute.

CI Build: 76-1e68c0c (Wed Nov 16 19:27:22 UTC 2022)

Commit 1e68c0c from Wed Nov 16 14:13:28 2022 -0500 by Stuart Shieber.
Preface

This book began as the notes for Computer Science 51, a second
semester course in programming at Harvard College, which follows
the legendary CS50 course that introduces some half of all Harvard
undergraduate students to programming. The nature of the course –
introducing a wide range of programming abstractions and paradigms
with an eye toward developing a large design space of programs, using
functional programming as its base and OCaml as the delivery vehicle
– is shared with similar courses at a number of colleges. The instruc-
tors in those courses have for many years informally shared ideas,
examples, problems, and notes in an open and free-flowing manner.
When I took over the course from Greg Morrisett (now Dean of Com-
puting and Information Sciences at Cornell University), I became the
beneficiary of all of this collaboration, including source materials from
these courses – handouts, notes, lecture material, labs, and problem
sets – which have been influential in the structure and design of these
notes, and portions of which have thereby inevitably become inter-
mixed with my own contributions in a way that would be impossible
to disentangle. I owe a debt of gratitude to all of the faculty who have
been engaged in this informal sharing, especially,

• Dan Grossman, University of Washington

• Michael Hicks, University of Maryland

• Greg Morrisett, Cornell University

• Benjamin Pierce, University of Pennsylvania

• David Walker, Princeton University

• Stephanie Weirich, University of Pennsylvania

• Steve Zdancewic, University of Pennsylvania

All of these faculty have kindly agreed to allow their contributions to

be used here and distributed openly.
Contents

1 Introduction 15
1.1 An extended example: greatest common divisor . . . . . 17
1.2 Programming as design . . . . . . . . . . . . . . . . . . . . 20
1.3 The OCaml programming language . . . . . . . . . . . . . 22
1.4 Tools and skills for design . . . . . . . . . . . . . . . . . . . 24

2 A Cook’s tour of OCaml 25

3 Expressions and the linguistics of programming languages 27

3.1 Specifying syntactic structure with rules . . . . . . . . . . 27
3.2 Disambiguating ambiguous expressions . . . . . . . . . . 30
3.3 Abstract and concrete syntax . . . . . . . . . . . . . . . . . 32
3.4 Expressing your intentions . . . . . . . . . . . . . . . . . . 33
3.4.1 Commenting . . . . . . . . . . . . . . . . . . . . . 34

4 Values and types 37

4.1 OCaml expressions have values . . . . . . . . . . . . . . . 37
4.1.1 Integer values and expressions . . . . . . . . . . . 37
4.1.2 Floating point values and expressions . . . . . . . 38
4.1.3 Character and string values . . . . . . . . . . . . . 39
4.1.4 Truth values and expressions . . . . . . . . . . . . 39
4.2 OCaml expressions have types . . . . . . . . . . . . . . . . 40
4.2.1 Type expressions and typings . . . . . . . . . . . . 42
4.3 The unit type . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.4 Functions are themselves values . . . . . . . . . . . . . . . 44

5 Naming and scope 47

5.1 Variables are names for values . . . . . . . . . . . . . . . . 47
5.2 The type of a let-bound variable can be inferred . . . . . 47
5.3 let expressions are expressions . . . . . . . . . . . . . . . 48
5.4 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
5.5 Global naming and top-level let . . . . . . . . . . . . . . 51

6 Functions 55
8

6.1 Multiple arguments and currying . . . . . . . . . . . . . . 56

6.2 Defining anonymous functions . . . . . . . . . . . . . . . 57
6.3 Named functions . . . . . . . . . . . . . . . . . . . . . . . . 58
6.3.1 Compact function definitions . . . . . . . . . . . 59
6.3.2 Providing typings for function arguments and
outputs . . . . . . . . . . . . . . . . . . . . . . . . . 60
6.4 Defining recursive functions . . . . . . . . . . . . . . . . . 62
6.5 Unit testing . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

7 Structured data and composite types 71

7.1 Tuples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
7.2 Pattern matching for decomposing data structures . . . . 73
7.2.1 Advanced pattern matching . . . . . . . . . . . . . 76
7.3 Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
7.3.1 Some useful list functions . . . . . . . . . . . . . . 79
7.4 Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
7.4.1 Field selection . . . . . . . . . . . . . . . . . . . . . 86
7.5 Comparative summary . . . . . . . . . . . . . . . . . . . . 86
7.6 Problem set: The prisoners’ dilemma . . . . . . . . . . . . 87
7.6.1 Background . . . . . . . . . . . . . . . . . . . . . . 87
7.6.2 Some practice functions . . . . . . . . . . . . . . . 89
7.6.3 Unit testing . . . . . . . . . . . . . . . . . . . . . . 89
7.6.4 The prisoner’s dilemma . . . . . . . . . . . . . . . 89
7.6.5 Tournament . . . . . . . . . . . . . . . . . . . . . . 90
7.6.6 Testing . . . . . . . . . . . . . . . . . . . . . . . . . 90

8 Higher-order functions and functional programming 91

8.1 The map abstraction . . . . . . . . . . . . . . . . . . . . . 92
8.2 Partial application . . . . . . . . . . . . . . . . . . . . . . . 94
8.3 The fold abstraction . . . . . . . . . . . . . . . . . . . . . . 96
8.4 The filter abstraction . . . . . . . . . . . . . . . . . . . . . 99
8.5 Problem set 2: Higher-order functional programming . . 100
8.5.1 Higher order functional programming . . . . . . 101

9 Polymorphism and generic programming 103

9.1 Type inference and type variables . . . . . . . . . . . . . . 104
9.2 Polymorphic map . . . . . . . . . . . . . . . . . . . . . . . 105
9.3 Regaining explicit types . . . . . . . . . . . . . . . . . . . . 106
9.4 The List library . . . . . . . . . . . . . . . . . . . . . . . . . 107
9.5 Problem section: Function composition . . . . . . . . . . 109
9.6 Problem section: Credit card numbers and the Luhn
check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
9.7 Weak type variables . . . . . . . . . . . . . . . . . . . . . . 111
 9

10 Handling anomalous conditions 115

10.1 A non-solution: Error values . . . . . . . . . . . . . . . . . 116
10.2 Option types . . . . . . . . . . . . . . . . . . . . . . . . . . 117
10.2.1 Option poisoning . . . . . . . . . . . . . . . . . . . 119
10.3 Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
10.3.1 Handling exceptions . . . . . . . . . . . . . . . . . 122
10.3.2 Zipping lists . . . . . . . . . . . . . . . . . . . . . . 124
10.3.3 Declaring new exceptions . . . . . . . . . . . . . . 128
10.4 Options or exceptions? . . . . . . . . . . . . . . . . . . . . 129
10.5 Unit testing with exceptions . . . . . . . . . . . . . . . . . 130
10.6 Problem set 3: Bignums and RSA encryption . . . . . . . 132
10.6.1 Big numbers . . . . . . . . . . . . . . . . . . . . . . 134
10.6.2 Challenge problem: Multiply faster . . . . . . . . 137
10.6.3 More background: How the RSA cryptosystem
works . . . . . . . . . . . . . . . . . . . . . . . . . . 137

11 Algebraic data types 141

11.1 Built-in composite types as algebraic types . . . . . . . . 143
11.2 Example: Boolean document search . . . . . . . . . . . . 144
11.3 Example: Dictionaries . . . . . . . . . . . . . . . . . . . . . 150
11.4 Example: Arithmetic expressions . . . . . . . . . . . . . . 153
11.5 Example: Binary trees . . . . . . . . . . . . . . . . . . . . . 155
11.6 Supplementary material . . . . . . . . . . . . . . . . . . . 157
11.6.1 Lab 6: Variants, algebraic types, and pattern
matching . . . . . . . . . . . . . . . . . . . . . . . . 157
11.7 Problem set 4: Symbolic differentiation . . . . . . . . . . 157
11.7.1 A language for symbolic mathematics . . . . . . . 159

12 Abstract data types and modular programming 167

12.1 Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
12.2 A queue module . . . . . . . . . . . . . . . . . . . . . . . . 171
12.3 Signatures hide extra components . . . . . . . . . . . . . 175
12.4 Modules with polymorphic components . . . . . . . . . . 177
12.5 Abstract data types and programming for change . . . . . 178
12.5.1 A string set module . . . . . . . . . . . . . . . . . . 181
12.5.2 A generic set signature . . . . . . . . . . . . . . . . 185
12.5.3 A generic set implementation . . . . . . . . . . . . 188
12.6 A dictionary module . . . . . . . . . . . . . . . . . . . . . . 193
12.7 Alternative methods for defining signatures and modules 197
12.7.1 Set and dictionary modules . . . . . . . . . . . . . 198
12.8 Library Modules . . . . . . . . . . . . . . . . . . . . . . . . 200
12.9 Problem section: Image manipulation . . . . . . . . . . . 201
12.10 Problem section: An abstract data type for intervals . . . 202
12.11 Problem section: Mobiles . . . . . . . . . . . . . . . . . . . 203
10

12.12 Problem set 5: Ordered collections . . . . . . . . . . . . . 206

12.12.1 Ordered collections . . . . . . . . . . . . . . . . . 207
12.12.2 Implementing ordered collections with binary
search trees . . . . . . . . . . . . . . . . . . . . . . 209
12.12.3 Priority queues . . . . . . . . . . . . . . . . . . . . 210
12.12.4 Challenge problem: Sort functor . . . . . . . . . . 213
12.12.5 Challenge problem: Benchmarking . . . . . . . . 213

13 Semantics: The substitution model 215

13.1 Semantics of arithmetic expressions . . . . . . . . . . . . 217
13.2 Semantics of local naming . . . . . . . . . . . . . . . . . . 221
13.3 Defining substitution . . . . . . . . . . . . . . . . . . . . . 224
13.3.1 Handling variable scope . . . . . . . . . . . . . . . 224
13.3.2 Free and bound occurrences of variables . . . . . 225
13.4 Implementing a substitution semantics . . . . . . . . . . 227
13.4.1 Implementing substitution . . . . . . . . . . . . . 228
13.4.2 Implementing evaluation . . . . . . . . . . . . . . 229
13.5 Problem section: Semantics of booleans and conditionals 232
13.6 Semantics of function application . . . . . . . . . . . . . . 232
13.6.1 More on capturing free variables . . . . . . . . . . 234
13.7 Substitution semantics of recursion . . . . . . . . . . . . . 236

14 Efficiency, complexity, and recurrences 241

14.1 The need for an abstract notion of efficiency . . . . . . . 242
14.2 Two sorting functions . . . . . . . . . . . . . . . . . . . . . 243
14.3 Empirical efficiency . . . . . . . . . . . . . . . . . . . . . . 245
14.4 Big-O notation . . . . . . . . . . . . . . . . . . . . . . . . . 247
14.4.1 Informal function notation . . . . . . . . . . . . . 249
14.4.2 Useful properties of O . . . . . . . . . . . . . . . . 250
14.4.3 Big-O as the metric of relative growth . . . . . . . 251
14.5 Recurrence equations . . . . . . . . . . . . . . . . . . . . . 252
14.5.1 Solving recurrences by unfolding . . . . . . . . . . 254
14.5.2 Complexity of reversing a list . . . . . . . . . . . . 255
14.5.3 Complexity of reversing a list with accumulator . 257
14.5.4 Complexity of inserting in a sorted list . . . . . . 258
14.5.5 Complexity of insertion sort . . . . . . . . . . . . 259
14.5.6 Complexity of merging lists . . . . . . . . . . . . . 260
14.5.7 Complexity of splitting lists . . . . . . . . . . . . . 260
14.5.8 Complexity of divide and conquer algorithms . . 261
14.5.9 Complexity of mergesort . . . . . . . . . . . . . . 262
14.5.10 Basic Recurrence patterns . . . . . . . . . . . . . . 263
14.6 Problem section: Complexity of the Luhn check . . . . . 263
14.7 Problem set 6: The search for intelligent solutions . . . . 264
14.7.1 Search problems . . . . . . . . . . . . . . . . . . . 264
 11

14.7.2 Structure of the provided code . . . . . . . . . . . 266

14.7.3 Testing, metering, and writeup . . . . . . . . . . . 269

15 Mutable state and imperative programming 271

15.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
15.1.1 Reference operator types . . . . . . . . . . . . . . 274
15.1.2 Boxes and arrows . . . . . . . . . . . . . . . . . . . 275
15.1.3 References and pointers . . . . . . . . . . . . . . . 276
15.2 Other primitive mutable data types . . . . . . . . . . . . . 278
15.2.1 Mutable record fields . . . . . . . . . . . . . . . . . 278
15.2.2 Arrays . . . . . . . . . . . . . . . . . . . . . . . . . 279
15.3 References and mutation . . . . . . . . . . . . . . . . . . . 279
15.4 Mutable lists . . . . . . . . . . . . . . . . . . . . . . . . . . 282
15.5 Imperative queues . . . . . . . . . . . . . . . . . . . . . . . 284
15.5.1 Method 1: List references . . . . . . . . . . . . . . 286
15.5.2 Method 2: Two stacks . . . . . . . . . . . . . . . . 286
15.5.3 Method 3: Mutable lists . . . . . . . . . . . . . . . 288
15.6 Hash tables . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
15.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . 293

16 Loops and procedural programming 295

16.1 Loops require impurity . . . . . . . . . . . . . . . . . . . . 296
16.2 Recursion versus iteration . . . . . . . . . . . . . . . . . . 297
16.2.1 Saving stack space . . . . . . . . . . . . . . . . . . 297
16.2.2 Tail recursion . . . . . . . . . . . . . . . . . . . . . 298
16.3 Saving data structure space . . . . . . . . . . . . . . . . . . 299
16.3.1 Problem section: Metering allocations . . . . . . 300
16.3.2 Reusing space through mutable data structures . 301
16.4 In-place sorting . . . . . . . . . . . . . . . . . . . . . . . . 302

17 Infinite data structures and lazy programming 309

17.1 Delaying computation . . . . . . . . . . . . . . . . . . . . 309
17.2 Streams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
17.2.1 Operations on streams . . . . . . . . . . . . . . . . 312
17.3 Lazy recomputation and thunks . . . . . . . . . . . . . . . 315
17.3.1 The Lazy Module . . . . . . . . . . . . . . . . . . . 317
17.4 Application: Approximating π . . . . . . . . . . . . . . . . 318
17.5 Problem section: Circuits and boolean streams . . . . . . 320
17.6 A unit testing framework . . . . . . . . . . . . . . . . . . . 321
17.7 A brief history of laziness . . . . . . . . . . . . . . . . . . . 325
17.8 Problem set 7: Refs, streams, and music . . . . . . . . . . 326
17.8.1 Mutable lists and cycles . . . . . . . . . . . . . . . 327
17.8.2 Lazy evaluation . . . . . . . . . . . . . . . . . . . . 327
17.8.3 The song that never ends . . . . . . . . . . . . . . 329
12

18 Extension and object-oriented programming 333

18.1 Drawing graphical elements . . . . . . . . . . . . . . . . . 334
18.2 Objects introduced . . . . . . . . . . . . . . . . . . . . . . 338
18.3 Object-oriented terminology and syntax . . . . . . . . . . 341
18.4 Inheritance . . . . . . . . . . . . . . . . . . . . . . . . . . . 343
18.4.1 Overriding . . . . . . . . . . . . . . . . . . . . . . . 345
18.5 Subtyping . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346
18.6 Problem section: Object-oriented counters . . . . . . . . 350
18.7 Problem set 8: Force-directed graph drawing . . . . . . . 350
18.7.1 Background . . . . . . . . . . . . . . . . . . . . . . 351
18.7.2 Building a force-directed graph layout system . . 353
18.7.3 Completing the graph drawing system . . . . . . 354
18.8 Problem set 9: Simulating an infectious process . . . . . 356
18.8.1 The simulation . . . . . . . . . . . . . . . . . . . . 356
18.8.2 The simulator . . . . . . . . . . . . . . . . . . . . . 357
18.8.3 Implementing the simulation . . . . . . . . . . . . 358
18.8.4 Exploration . . . . . . . . . . . . . . . . . . . . . . 360

19 Semantics: The environment model 363

19.1 Review of substitution semantics . . . . . . . . . . . . . . 363
19.2 Environment semantics . . . . . . . . . . . . . . . . . . . . 364
19.2.1 Dynamic environment semantics . . . . . . . . . 365
19.2.2 Lexical environment semantics . . . . . . . . . . 372
19.3 Conditionals and booleans . . . . . . . . . . . . . . . . . . 373
19.4 Recursion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374
19.5 Implementing environment semantics . . . . . . . . . . . 376
19.6 Semantics of mutable storage . . . . . . . . . . . . . . . . 377
19.6.1 Lexical environment semantics of recursion . . . 381

20 Concurrency 383
20.1 Sequential, concurrent, and parallel computation . . . . 384
20.2 Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . 385
20.3 Threads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
20.4 Interthread communication . . . . . . . . . . . . . . . . . 389
20.5 Futures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392
20.6 Futures are not enough . . . . . . . . . . . . . . . . . . . . 394
20.7 Locks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398
20.7.1 Abstracting lock usage . . . . . . . . . . . . . . . . 399
20.8 Deadlock . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400

21 Final project: Implementing MiniML 401

21.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
21.1.1 Grading and collaboration . . . . . . . . . . . . . 402
 13

21.1.2 A digression: How is this project different from a

problem set? . . . . . . . . . . . . . . . . . . . . . . 402
21.2 Implementing a substitution semantics for MiniML . . . 403
21.3 Implementing an environment semantics for MiniML . . 408
21.4 Extending the language . . . . . . . . . . . . . . . . . . . . 411
21.4.1 Extension ideas . . . . . . . . . . . . . . . . . . . . 411
21.4.2 A lexically scoped environment semantics . . . . 412
21.4.3 The MiniML parser . . . . . . . . . . . . . . . . . . 415
21.5 Submitting the project . . . . . . . . . . . . . . . . . . . . 415
21.6 Alternative final projects . . . . . . . . . . . . . . . . . . . 416

A Mathematical background and notations 417

A.1 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
A.1.1 Defining functions with equations . . . . . . . . . 417
A.1.2 Notating function application . . . . . . . . . . . 418
A.1.3 Alternative mathematical notations for func-
tions and their application . . . . . . . . . . . . . 418
A.1.4 The lambda notation for functions . . . . . . . . . 421
A.2 Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422
A.3 Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422
A.4 Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423
A.5 Equality and identity . . . . . . . . . . . . . . . . . . . . . 424

B A style guide 425

B.1 Formatting . . . . . . . . . . . . . . . . . . . . . . . . . . . 426
B.1.1 No tab characters . . . . . . . . . . . . . . . . . . . 426
B.1.2 80 column limit . . . . . . . . . . . . . . . . . . . . 426
B.1.3 No needless blank lines . . . . . . . . . . . . . . . 426
B.1.4 Use parentheses sparely . . . . . . . . . . . . . . . 426
B.1.5 Delimiting code used for side effects . . . . . . . 427
B.1.6 Spacing for operators and delimiters . . . . . . . 428
B.1.7 Indentation . . . . . . . . . . . . . . . . . . . . . . 429
B.2 Documentation . . . . . . . . . . . . . . . . . . . . . . . . 430
B.2.1 Comments before code . . . . . . . . . . . . . . . 430
B.2.2 Comment length should match abstraction level 431
B.2.3 Multi-line commenting . . . . . . . . . . . . . . . 431
B.3 Naming and declarations . . . . . . . . . . . . . . . . . . . 431
B.3.1 Naming conventions . . . . . . . . . . . . . . . . . 431
B.3.2 Use meaningful names . . . . . . . . . . . . . . . 432
B.3.3 Constants and magic numbers . . . . . . . . . . . 433
B.3.4 Function declarations and type annotations . . . 433
B.3.5 Avoid global mutable variables . . . . . . . . . . . 434
B.3.6 When to rename variables . . . . . . . . . . . . . . 434
B.3.7 Order of declarations in a module . . . . . . . . . 434
14

B.4 Pattern matching . . . . . . . . . . . . . . . . . . . . . . . 435

B.4.1 No incomplete pattern matches . . . . . . . . . . 435
B.4.2 Pattern match in the function arguments when
possible . . . . . . . . . . . . . . . . . . . . . . . . 435
B.4.3 Pattern match with as few match expressions as
necessary . . . . . . . . . . . . . . . . . . . . . . . 436
B.4.4 Misusing match expressions . . . . . . . . . . . . 436
B.4.5 Avoid using too many projection functions . . . . 437
B.5 Verbosity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437
B.5.1 Reuse code where possible . . . . . . . . . . . . . 437
B.5.2 Do not abuse if expressions . . . . . . . . . . . . 438
B.5.3 Don’t rewrap functions . . . . . . . . . . . . . . . 438
B.5.4 Avoid computing values twice . . . . . . . . . . . 439
B.6 Other common infelicities . . . . . . . . . . . . . . . . . . 439

C Solutions to selected exercises 441

Bibliography 499

Index 503

Image Credits 509

1
Introduction

We forget how incredible the computer is. The modern computer

executes billions of operations each second, every one of which must
work perfectly – accurately performing the right operation at each
and every cycle. How is this even possible? How can we, mere humans
with our cognitive limitations, manage to build devices that work at
this pace with this level of fidelity? Each of the billions of instructions
executed per second on a modern computer is another detail to be
managed. How can we gain control over this mass of detail?
In Jorge Luis Borges’s 1944 short story Funes the Memorious, the
protaganist, Ireneo Funes, experiences what it is like to perceive the
world at this level of streaming detail. After being thrown from a wild
horse and severely crippled, he develops a prodigious memory. He
recalls, perfectly and instantaneously, every moment of his life.

He knew by heart the forms of the southern clouds at dawn on the 30th
of April, 1882, and could compare them in his memory with the mottled
streaks on a book in Spanish binding he had only seen once.. . . Two or
three times he had reconstructed a whole day; he never hesitated, but
each reconstruction had required a whole day. (Borges, 1962)

Yet, each of his memories was individual, disconnected, divorced of

any higher structural patterns. Borges relates,

With no effort, he had learned English, French, Portuguese and Latin. I

suspect, however, that he was not very capable of thought. To think is to
forget differences, generalize, make abstractions. In the teeming world
of Funes, there were only details, almost immediate in their presence.

Without abstraction, there are only details. And it is through abstrac-

tion – forgetting differences, generalizing – that we can get control of
the sheer daunting complexity of controlling a computer.
What is abstraction? A B S T R A C T I O N is the process of viewing a set of
apparently dissimilar things as instantiating an underlying identity. Fu-
nes sees a field of flowers, hundreds of blooms. To him, they are each
16 PROGRAMMING WELL

individuals, but to the botanist, these apparently dissimilar individuals

are all instances of a type, the genus Tulipa, the tulips. By capturing
innumerable individual plants into a hierarchy of abstract families,
genera, and species, the bewildering complexity of plant life on the
planet becomes more manageable.
Programming computers is a battle against the sheer daunting
complexity of the task. The chief weapon in the battle is abstraction.
The first objective of this book is to introduce you to a broad variety
of abstraction mechanisms and their uses, providing you with an
appropriate armamentarium. The second objective is to open your
eyes to the beauty that computer programming can manifest when
those tools are elegantly applied.
You are already familiar with some of the primary abstraction mech-
anisms used in programming computers. (I assume throughout this
book that you’ve had some experience programming computers us- Figure 1.1: A model of a part of Charles
Babbage’s analytical engine, intended
ing an imperative programming language, of the sort, for instance, for the calculation of tables of mathe-
acquired in Harvard’s CS50 or CS50x course.) matical functions such as the trigono-
metric functions like sine and cosine,
Let’s take as an example the problem of generating a table of loga-
the Bernoulli numbers, or logarithms, as
rithms. The choice is not random. The building of tables of mathemat- in Figure 1.2 below.
ical functions like the logarithm was the motivating task for the earliest
computer designs, those of Charles Babbage in the 1820s and 1830s
(Figure 1.1). In the margin (Figure 1.2) is the beginning of such a table.
A program to print out this kind of table might look like this:

printf "1 0.0000\ n";

printf "2 1.0000\ n";
printf "3 1.5850\ n";
printf "4 2.0000\ n" ;; x l og 2 x

1 0.0000
and when the program is executed, it prints the table: 2 1.0000
3 1.5850
# printf "1 0.0000\ n"; 4 2.0000
# printf "2 1.0000\ n"; ⋯
# printf "3 1.5850\ n";
# printf "4 2.0000\ n" ;; Figure 1.2: A small table of logarithms
1 0.0000
2 1.0000
3 1.5850
4 2.0000
- : unit = ()

(For the moment, the details of the language in which this computa-
tion is written are immaterial. We’ll get to all that in a bit. The idea is
just to get the gist of the argument. In the meantime, you can just let
the code waft over you like a warm summer breeze.)
Now of course this code is hopelessly written. Why? Because it
treats each line of the table as an individual specimen, missing the
abstract view. The first step in viewing the lines abstractly is to note
 INTRODUCTION 17

that they are actually instances of an underlying uniformity: Each

string is of the form of an integer (call it x) and the log (with base 2) of
x. They are instances of the underlying pattern

printf "%2d %2.4f\ n" x (log2 x);

for each of several values of the variable x. (Again, the details of the
language being used are postponed, but you hopefully get the idea.)
This mechanism, the S TAT E VA R I A B L E , is thus a mechanism for ab-
straction – for making apparently dissimilar computations manifest an
underlying identity. To take full advantage of this type of variable, we’ll
need to specify the sequential values, 1 through 4 say, that the variable
takes on.

for x = 1 to 4 do
printf "%2d %2.4f\ n" x (log2 x)
done

Like Monsieur Jourdain, who discovered he’d been speaking prose

his whole life, you’ve been using abstraction mechanisms without
Figure 1.3: Alan Turing (1912–1954),
realizing it. Without them, programming is impossible. whose Turing machine provided the first
This particular style of programming, imperative programming, is universal model of computation, based
on imperative programming notions
undoubtedly most familiar to you. Its most basic abstraction mecha-
of state and state change. Turing is
nisms are the state variable and the loop. It is the style seen in some of rightfully credited with fundamental
the earliest, most influential programming languages, from F O RT R A N contributions to essentially all areas of
computer science: the theory of com-
to the A L G O L family of languages, to C, to Python, and beyond. And puting, hardware, software, artificial
it is the style of programming captured by the first universal model of intelligence, computational biology, and
much more. His premature death by
computation, the T U R I N G M A C H I N E of Alan Turing (Figure 1.3).
suicide at 41 after undergoing “therapy”
But there are many other abstraction mechanisms than state vari- at the hands of the British government
ables and loops, underpinning many other programming paradigms following his conviction for the “crime”
of homosexuality is certainly one of the
than imperative programming, and allowing many other ways of de- great intellectual tragedies of the twen-
signing computations. It is the goal of this book to introduce several tieth century. (The British government
got around to apologizing for his treat-
such abstraction mechanisms, provide practice in their use and appli-
ment some 50 years later.) The highest
cation, and thereby open up a broad range of programming possibili- award in computing, the Turing Award,
ties not otherwise available. is appropriately named after him.

An especially important abstraction mechanism is the function,

a mapping from inputs to outputs. The idea of the function gives its
name to the paradigm of functional programming, and we will begin
with functions and functional programming ideas. But functional
programming is only one of several paradigms that we will discuss.

1.1 An extended example: greatest common divisor

By way of example of the distinction between imperative and func-

tional programming, consider the very practical question of tiling a
bathroom floor of size 28 by 20 units. We can tile such a floor with tiles
18 PROGRAMMING WELL

that are 2 by 2, since both 28 and 20 are evenly divisible by 2, but 3 by 3

tiles don’t work, since neither 28 nor 20 are divisible by 3. If we want to
use the fewest tiles, it would be useful to know the largest number that
divides both dimensions evenly, their G R E AT E S T C O M M O N D I V I S O R
(GCD).
Here is how we might program a calculation of GCD in an impera-
tive style:
let gcd_down a b =
let guess = ref (min a b) in
while (a mod !guess <> 0) || (b mod !guess <> 0) do
guess := !guess - 1
done;
!guess ;;

This procedure works by counting down from the smaller of the two
numbers, one by one, until a common divisor is found. Since the
search for the common divisor is from the largest to the smallest possi-
bility, the greatest common divisor is found.
In the functional style, this same “countdown” algorithm might be
coded like this:
let gcd_func a b =
let rec downfrom guess =
if (a mod guess <> 0) || (b mod guess <> 0) then
downfrom (guess - 1)
else guess in
downfrom (min a b) ;;

Here, in the context of calculating the GCD of a and b, a new function

downfrom is introduced to check a particular guess of the GCD of the
two numbers. The downfrom function takes an input guess and checks
whether it is the GCD of a and b. If so, the output value of the function
is the guess guess itself, but if not, a one-smaller guess is tried. Having
defined this counting-down function, the calculation of the GCD itself
proceeds just by guessing the minimum of the two numbers.
You may find unusual some of the properties of this latter imple-
mentation of what is essentially the identical algorithm – counting
down one by one from the minimum of the two numbers until a
common divisor is found. First, there are no overt loops, and no as-
signments to variables that change the state of the computation by
changing the value of a variable. It’s just functions and their applica-
tion. Second, the function downfrom defined in the code appeals to
downfrom itself as part of the calculation of its output. It is defined by
R E C U R S I O N , that is, in terms of itself. Such functions are recursive, and
when they invoke themselves for a computation are said to recur.1 You 1
Not recurse please. To recurse is to
curse again, not the kind of thing a
may wonder whether this is quite kosher. Isn’t defining something in
program – or a person – should be
terms of itself a bad idea? But in this case at least, the definition works doing.
 INTRODUCTION 19

fine, because the value of downfrom guess depends not on the value
of downfrom guess itself but of downfrom (guess - 1), a different
value. This may itself depend on downfrom (guess - 2), and so on,
but eventually one of the inputs to downfrom will be a common divisor,
and in that case, the output value of downfrom does not depend on
downfrom itself. The recursion “bottoms out” and the GCD is returned.
This style of programming – by defining and applying functions –
has a certain elegance, which can be seen already in the distinction
between the two versions of the GCD computation already provided.
But as it turns out, the algorithm underlying both of these implemen-
tations is a truly bad one. Counting down is just not the right way to
calculate the GCD of two numbers. As far back as 300 B C E , Euclid of
Alexandria provided a far better algorithm in Proposition 1 of Book
7 (Figure 1.4) of his treatise on mathematics, Elements. Euclid’s algo-
rithm for GCD is based on the following insight: Any square tiling of a
Figure 1.4: Proposition 1 of Book 7
20 by 28 area will tile both a 20 by 20 square and the 8 by 20 remainder. of Euclid’s Elements, providing his
More generally, any square tiling of an a by b area (where a is greater algorithm for calculating the greatest
common divisor of two numbers.
than b) will tile both a b by b square and the b by a − b remainder.
Thus, to calculate the GCD of a and b, it suffices to calculate the GCD
of b and a − b. Eventually, we’ll be looking for the GCD of two instances
of the same number (that is, a and b will be the same; we’ll be looking
to tile a square area) in which case we know the GCD; it is a (or b) it-
self. Figure 1.5 shows the succession of smaller and smaller rectangles
explored by Euclid’s algorithm for the 20 by 28 case.
An initial presentation of Euclid’s algorithm is this:
let rec gcd_euclid a b =
if a < b then gcd_euclid b a
else if a = b then a
else gcd_euclid b (a - b) ;;

Now, in the case that a = b, were we to continue on one more round of

checking the GCD of b and a − b, the difference a − b would simply be
0. Thus, we can check for this condition instead.
let rec gcd_euclid a b =
if a < b then gcd_euclid b a
else if b = 0 then a
else gcd_euclid b (a - b) ;;

We can simplify further. When subtracting off b from a, the remainder

may still be greater than b, in which case, we’ll want to subtract b
again, continuing to subtract b until, eventually, the remainder is less
than b. Thus, instead of using the difference a − b as the new second
argument of the recursive call, we can use the remainder a mod b.
let rec gcd_euclid a b =
if a < b then gcd_euclid b a
20 PROGRAMMING WELL

28
28

8 8
4
20
20 20 4
4
(a) (b) (c) (d) (e)

Figure 1.5: Euclid’s algorithm for GCD

else if b = 0 then a starting (a) with a 20 × 28 rectangle to
be tiled. Removing the 20 × 20 square
else gcd_euclid b (a mod b) ;;
(b) leaves a 20 × 8 remainder to be
tiled. From that rectangle, we remove,
Finally, notice that if a < b, then the values b and a mod b are just b successively, two 8 × 8 squares (c),
and a, respectively – exactly the values we want to use for the recursive leaving a 4 × 8 remainder. Finally,
call in that case. We can therefore drop the test for a < b entirely. removing a 4 × 4 square (d) leaves a 4 × 4
square, the largest square that can tile
let rec gcd_euclid a b = the whole (e).
if b = 0 then a
else gcd_euclid b (a mod b) ;;

This is E U C L I D ’ S A L G O R I T H M . Compare it to the countdown algo-

rithm above. The difference is stark. Euclid’s method is beautiful in its
simplicity.
It is also, as it turns out, much more efficient. This can be deter-
mined analytically or experienced empirically.

1.2 Programming as design

Euclid’s algorithm for GCD shows us that there is more than one way
to solve a problem, and some ways are better than others. The dimen-
sions along which programmed solutions can be better or worse are
manifold. They include

• succinctness,

• efficiency,

• readability,

• maintainability,

• provability,

• testability,

and, most importantly but ineffably,

 INTRODUCTION 21

• beauty.

Computer programming is not the only practice where practitioners

may generate multiple ways of satisfying a goal, which can be eval-
uated along multiple independent and perhaps conflicting metrics.
Architects, engineers, illustrators, industrial designers may generate
wildly different plans in response to a client’s constraints and desires.
All live in a space of possibilities from which they choose solutions that
vary along multiple, often competing, criteria.
What all of these practices have in common is D E S I G N – the navi-
gation of a space of options, generated by applicable tools, in search of
the good, as measured along multiple dimensions. In the case of com-
puter programming, the tools are exactly the abstraction mechanisms
provided by a programming language.
A crucial consideration in teaching programming from this perspec-
tive is what abstraction mechanisms to concentrate on, as these define
the space of options within which we can navigate. As discussed above,
the most important of these abstraction mechanisms is the function.
In addition to being a fantastic method for abstracting computation
(which will become clear some time around Chapter 8), functions
also serve as a platform upon which many other abstraction mecha-
nisms can be deployed and combined. It may be difficult at first to see
the incredible utility of the function as a unifying abstraction mecha-
nism, but hopefully, as you see more and more examples of their use
in combination with other techniques, you will come to appreciate the
function’s centrality in the design of programs.
Indeed, functions and their application are such a powerful compu-
tational tool that they constitute, by themselves, a complete universal
computational mechanism. The Princeton mathematician and logi-
cian Alonzo Church (1936) developed a “calculus” of functions alone, Figure 1.6: Princeton professor Alonzo
the so-called L A M B D A C A L C U L U S (see Section A.1.4), a logical system Church (1903–1995), inventor of the
lambda calculus, the foundation of all
that included functions and their application and literally nothing
functional programming languages;
else – no data objects or data structures of any kind, neither atomic PhD adviser of Alan Turing.
(like integers) nor composite (like lists); no mutable state (like vari-
able assignment); no control structures (like conditionals or loops).
Astoundingly, Turing (1937) was then able to show that anything that
can be computed by his universal model of computation, the Turing
machine, can also be computed in Church’s lambda calculus. Thus, the
lambda calculus – comprised only of functions and their applications
remember – is itself a universal model of computation. This argument
for the universality of Turing’s and Church’s computation models is
now known as the C H U R C H - T U R I N G T H E S I S . (The close connection
between the lambda calculus and the Turing machine mirrors the close
relationship between Church and Turing; Church was Turing’s PhD
22 PROGRAMMING WELL

adviser at Princeton.)
In this book, we concentrate on the following abstraction mecha-
nisms, listed with the style of programming they are associated with:

Table 1.1: Some abstraction mecha-

Abstraction Programming paradigm
nisms and the programming paradigms
they allow.
functions functional programming
algebraic data types structure-driven programming
polymorphism generic programming
abstract data types modular programming
mutable state imperative programming
loops procedural programming
lazy evaluation programming with infinite data structures
object dispatch object-oriented programming
concurrency concurrent programming

Of course, there are many other abstraction mechanisms and pro-

gramming paradigms, but these should both give you a good sense of
the importance of a variety of abstractions and provide an excellent
base on which to build.
As with any design practice, computer programming is best learned
by seeing a range of examples of the space of options – examples that
are better or worse along one dimension or another – with attention
paid to the process of developing, modifying, and improving such
solutions. For that reason, we will often show computer programs be-
ing built up in stages and being modified to demonstrate alternative
designs (as we did with the GCD example above), and programming
problems will be revisited as new abstraction mechanisms open fur-
ther parts of the design space. You may find the multiple variations on
a theme redundant – as indeed they are – but we know of no better way
to get across the idea of programming as a design practice than the
careful development and exploration of a significant program design
space.

1.3 The OCaml programming language

In order that we can introduce multiple abstraction mechanisms

and programming paradigms with a minimum of programming lan- Figure 1.7: Robin Milner (1934–2010),
guage detail, we use a multi-paradigm programming language called developer of the ML programming
language, from which OCaml derives,
O C A M L . (The examples above were written in OCaml.) OCaml is a the first functional language with type
member of the ML family of programming languages first developed inference. He received the Turing Award
in 1991 for his work on ML and other
at University of Edinburgh by Robin Milner (Figure 1.7) in the 1970’s.
innovations.
The OCaml dialect of ML itself was developed at the French national
 INTRODUCTION 23

research lab Institut National de Recherche en Informatique et en

Automatique (INRIA), where it continues to be developed and main-
tained. OCaml is a multi-paradigm programming language in that it
provides support not only for functional programming, but also imper-
ative programming, object-oriented programming, and all the other
mechanisms and paradigms listed in Table 1.1.
OCaml is especially attractive from a pedagogical standpoint be-
cause it provides these capabilities on the basis of a relatively small
foundation of well-designed orthogonal primitive language constructs,
so that programming concepts can be introduced and experimented
with, without the need for learning a huge set of syntactic idiosyn-
crasies. Nonetheless, as with learning any new programming language,
it will take a bit of getting used to the ideas and notations of OCaml,
and in fact getting practice with learning new notations is a useful skill
in its own right.
I emphasize that this book is not a book about OCaml program-
ming. (For instance, this book doesn’t pretend to present the language
comprehensively, instead covering only those parts of the language
needed to present the principles being taught. For that reason, you will
want to get at least a bit familiar with the reference documentation of
the language.) Rather, it is a book about the role of abstraction in the
design of software, which uses the OCaml language as the medium in
which to express these ideas. But in order to get these ideas across, we
need some language, and it turns out that OCaml is an ideal language
for this pedagogical purpose. Of course, we’ll have to spend some time
going over the particularities of the OCaml language, which may seem
odd mostly because of their unfamiliarity. The text may have a bit of
a disjointed quality to it, bouncing back and forth between details
of OCaml and higher-level concepts. But the time spent learning the
details of the language isn’t time wasted. It pays off in lessons that gen-
eralize to any programming you will do in the future. You may discover,
like many do, that once you’ve gained some proficiency with OCaml,
you find its charms irresistible, and continue to use it (or its close
derivatives like Microsoft’s F#, Facebook’s Reason, Apple’s Swift, or
Mozilla’s Rust) when appropriate – as many companies including those
mentioned do. But whether you continue to program in OCaml or not,
the patterns of thinking and the sophistication of your understanding
will be the payoff of this process, translatable to any programming
you’ll do in the future. In fact, the market for software developers re-
flects this payoff as well. As shown in Figure 1.8, the market rewards
software developers fluent in the kinds of technologies and ideas fea-
tured in this book; their salaries are substantially higher on average.
Although such pecuniary benefits aren’t the point of this book, they
24 PROGRAMMING WELL

Table 1

Erlang
certainly don’t hurt. Erlang 115000 Scala
OCaml
OCaml
Scala 115000
Clojure
OCaml 114000 Go
1.4 Tools and skills for design
Clojure 110000 Groovy
Objective-C
Go 110000 F#
The space of design options available
Groovy to you is enabled
110000 by the palette Hack
Perl
of abstraction mechanisms thatObjective-C 110000
you can fluently deploy. Navigating Kotlin
F# 108000 Rust
the design space to find the best solutions is facilitated by a set of skills
108000
Swift
Hack
TypeScript
and analytic tools, which we will
Perlalso introduce throughout
106000 the follow- Bash/Shell
Kotlin 105000 CoffeeScript
ing chapters as they become pertinent. These include more precise ObjectPascal
Rust 105000
notions of the syntax and semantics of programming languages, fa- Haskell
Swift 102000 Java
cility with notations, sensitivityTypeScript
to programming style (see especially
102000
Lua
Ruby
Appendix B), programming interface
Bash/Shelldesign, unit testing,
100000 tools (big- Julia
CoffeeScript 100000 C
O notation, recurrence equations) for analyzing efficiency of code. JavaScript
ObjectPascal 100000
Python
Having these tools and skills at Haskell
your disposal will add to your computa-
100000 90K 95K 100K 105K 110K 115K 120K
tional tool-box and stretch yourJava
thinking about what it means to write
100000
Lua
good code. I expect, based on my own experiences,100000
that learning to Figure 1.8: United States average salary
Ruby 100000 by technology, from StackOverflow
develop, analyze, and express your software ideas with precision will Developer Survey 2018. Highlighted
Julia 98500
also benefit your abilities to develop,
C analyze, and express
98000 ideas more bars correspond to technologies in the
JavaScript 98000 typed functional family.
generally.
Python 98000

1
2
A Cook’s tour of OCaml

To give a flavor of working with the OCaml programming language,

we introduce OCaml through an I N T E R P R E T E R of the language, called
ocaml, which is invoked from the command line thus:1 1
We assume that you’ve already in-
stalled the OCaml tools, as described at
% ocaml the ocaml.org web site.

Upon running ocaml, you will see a P R O M P T (“#”) allowing you to type
an OCaml expression.

% ocaml
OCaml version 4.11.1
#

Exercise 1
The startup of the ocaml interpreter indicates that this is version 4.11.1 of the software.
What version of ocaml are you running?
Once the OCaml prompt is available, you can enter a series of
OCaml expressions to calculate the values that they specify. Numeric
(integer) expressions are a particularly simple case, so we’ll start with
those. The integer L I T E R A L S – like 3 or 42 or -100 – specify integer
values directly, but more complex expressions built by applying arith-
metic functions to other values do as well. Consequently, the OCaml
interpreter can be used as a kind of calculator.

# 42 ;;
- : int = 42
# 3 + 4 * 5 ;;
- : int = 23
# (3 + 4) * 5 ;;
- : int = 35

Since this is the first example we’ve seen of interaction with the
OCaml interpreter, some glossing may be useful. The OCaml interac-
tive prompt, ‘#’, indicates that the user can enter an OCaml expression,
such as ‘3 + 4 * 5’. A double semicolon ‘;;’ demarcates the end of
the expression. The system reads the expression, evaluates it (that
26 PROGRAMMING WELL

is, calculates its value), and prints an indication of the result, then
loops back to provide another prompt for the next expression. For
this reason, the OCaml interactive system is referred to as the “ R E A D -
2
E VA L - P R I N T L O O P ” or R E P L . Whenever we show the results of an 2
To exit the R E P L , just enter the end-
of-file character, ^d, typed by holding
interaction with the R E P L , the interpreter’s output will be shown in a
down the control key while pressing the
slanted font to distinguish it from the input. d key.
You’ll notice that the R E P L obeys the standard order of operations,
with multiplication before addition for instance. This precedence can
be overwritten in the normal manner using parentheses.

Exercise 2
Try entering some integer expressions into the OCaml interpreter and verify that appro-
priate values are returned.
Although we’ll introduce the aspects of the OCaml language in-
crementally over the next few chapters, to get a general idea of using
the language, we demonstrate its use with the GCD algorithm from
Chapter 1. We type the definition of the gcd_euclid function into the
R E P L:

# let rec gcd_euclid a b =

# if b = 0 then a
# else gcd_euclid b (a mod b) ;;
val gcd_euclid : int -> int -> int = <fun>

Now we can make use of that definition to calculate the greatest com-
mon divisor of 20 and 28

# gcd_euclid 20 28 ;;
- : int = 4

But we’re getting ahead of ourselves.

3
Expressions and the linguistics of programming lan-
guages

Programming is an expressive activity: We express our intentions to a

computer using a language – a programming language – that is in some
ways similar to the natural languages with which we communicate
with each other.
One of the deep truths of linguistics, known since the time of the
great Sanskrit grammarian Pān.ini in the fourth century B C E , is that the
expressive units of natural languages, or E X P R E S S I O N S as we will call
them, have hierarchical structure. (The recovery of that structure used
to be a typical subject matter taught to students in “grammar school”
through the exercise of sentence diagramming.) Characterizing what
are the well-formed and -structured phrases of a language constitutes
the realm of S Y N TA X .

3.1 Specifying syntactic structure with rules

The expressions of English (and other natural languages) are formed as

sequences of words to form expressions of various types. By way of ex-
ample, noun phrases can be formed in various ways: as a single noun
(party or drinker or tea), or by putting together (in sequential order) a
noun phrase and a noun (as in tea party), or by putting together (again
in order) an adjective (iced or mad) and another noun phrase as in
(iced tea). We can codify these rules by defining classes of expressions
like ⟨noun ⟩ or ⟨nounphrase ⟩ or ⟨adjective ⟩. We’ll write the rule that
allows forming a noun phrase from a single noun as

⟨nounphrase ⟩ ∶∶= ⟨noun ⟩

28 PROGRAMMING WELL

The rules that form a noun phrase from an adjective and a noun
phrase or from a noun phrase and a noun are, respectively,

⟨nounphrase ⟩ ∶∶= ⟨adjective ⟩⟨nounphrase ⟩

⟨nounphrase ⟩ ∶∶= ⟨nounphrase ⟩⟨noun ⟩

In these rules, we write ⟨noun ⟩ to indicate the class of noun expres-

sions, ⟨nounphrase ⟩ to indicate the class of noun phrases, and in
general, put the names of classes of expressions in angle brackets to
represent elements of that class. The notation ∶∶= should be read as
“can be composed from”, so that expressions of the class on the left of
the ∶∶= can be composed by putting together expressions of the classes
listed on the right of the ∶∶=, in the order indicated.
This rule notation for presenting the syntax of languages is called
B A C K U S - N AU R F O R M (BNF), named after John Backus and Peter
Naur, who proposed it for specifying the syntax of the A L G O L family of
programming languages. But as noted above, the idea goes back much
further, at least to Pān.ini.
Putting these rules together, the BNF specification for noun phrases
is

⟨nounphrase ⟩ ∶∶= ⟨noun ⟩

∣ ⟨adjective ⟩⟨nounphrase ⟩
∣ ⟨nounphrase ⟩⟨noun ⟩

Here, we’ve rephrased the three rules as a single rule with three alter-
native right-hand sides. The BNF notation allows separating alterna-
tive right-hand sides with the vertical bar (∣) as we have done here.
A specification of a language using rules of this sort is referred to as
a G R A M M A R . According to this grammar, we can build noun phrases
like mad tea party

⟨nounphrase ⟩

⟨adjective ⟩ ⟨nounphrase ⟩

mad ⟨nounphrase ⟩ ⟨noun ⟩

⟨noun ⟩ party

tea

or iced tea drinker

 E X P R E S S I O N S A N D T H E L I N G U I S T I C S O F P R O G R A M M I N G L A N G UA G E S 29

⟨nounphrase ⟩

⟨nounphrase ⟩ ⟨noun ⟩

⟨adjective ⟩ ⟨nounphrase ⟩ drinker

iced ⟨noun ⟩

tea
Notice the difference in structure. In mad tea party, the adjective mad
is combined with the phrase tea party, but in iced tea drinker, the
adjective iced does not combine with tea drinker. The drinker isn’t
iced; the tea is!
But these same rules can also be used to build an alternative tree for
“iced tea drinker”:
⟨nounphrase ⟩

⟨adjective ⟩ ⟨nounphrase ⟩

iced ⟨nounphrase ⟩ ⟨noun ⟩

⟨noun ⟩ drinker

tea
The expression iced tea drinker is A M B I G U O U S (as is mad tea party);
the trees make clear the two syntactic analyses.
Importantly, as shown by these examples, it is the syntactic tree
structures that dictate what the expression means. The first tree seems
to describe a drinker of cold beverages, the second a cold drinker
of beverages. The syntactic structure of an utterance thus plays a
crucial role in its meaning. The characterization of the meanings of
expressions on the basis of their structure is the realm of S E M A N T I C S ,
pertinent to both natural and programming languages. We’ll come
back to the issue of semantics in detail in Chapters 13 and 19.

Exercise 3
Draw a second tree structure for the phrase mad tea party, thereby demonstrating that it
is also ambiguous.

Exercise 4
How many trees can you draw for the noun phrase flying purple people eater? Keep in
mind that flying and purple are adjectives and people and eater are nouns.
The English language, and all natural languages, are ambiguous that
way. Fortunately, context, intonation, and other clues disambiguate
30 PROGRAMMING WELL

these ambiguous constructions so that we are mostly unaware of

the ambiguities.1 In the case of the mad tea party, we understand 1
The rare exceptions where ambiguities
are brought to our attention account
the phrase as having the syntactic structure as displayed above (as
for the humor (of a sort) found in
opposed to the one referred to in Exercise 3). syntactically ambiguous sentences, as
in the old joke that begins “I shot an
elephant in my pajamas.”
3.2 Disambiguating ambiguous expressions

Programming language expressions, like the utterances of natural

language, have syntactic structure as well. Without some care, pro-
gramming languages might be ambiguous too. Consider the following
BNF rules for simple arithmetic expressions built out of numbers and
B I N A RY O P E R AT O R S (operators, like +, -, *, and /, that take two argu-
ments).2 2
In defining expression classes using
this notation, we use subscripts to dif-
ferentiate among different occurrences
⟨expr ⟩ ∶∶= ⟨exprleft ⟩⟨binop ⟩⟨exprright ⟩ of the same expression class, such as
∣ ⟨number ⟩ the two ⟨expr ⟩ instances ⟨exprleft ⟩ and
⟨exprright ⟩ in the first BNF rule.
⟨binop ⟩ ∶∶= + ∣ - ∣ * ∣ /
⟨number ⟩ ∶∶= 0 ∣ 1 ∣ 2 ∣ 3 ∣ ⋯

Using these rules, we can build two trees for the expression 3 + 4 *
5:

⟨expr ⟩

⟨expr ⟩ ⟨binop ⟩ ⟨expr ⟩

⟨expr ⟩ ⟨binop ⟩ ⟨expr ⟩ * ⟨number ⟩

⟨number ⟩ + ⟨number ⟩ 5

3 4

or

⟨expr ⟩

⟨expr ⟩ ⟨binop ⟩ ⟨expr ⟩

⟨number ⟩ + ⟨expr ⟩ ⟨binop ⟩ ⟨expr ⟩

3 ⟨number ⟩ * ⟨number ⟩

4 5
 E X P R E S S I O N S A N D T H E L I N G U I S T I C S O F P R O G R A M M I N G L A N G UA G E S 31

But in the case of programming languages, we don’t have the luxury

of access to intonation or shared context to disambiguate expressions.
Instead, we rely on other tools – conventions and annotations.
In the way of conventions, we rely on a conventional O R D E R O F
O P E R AT I O N S that dictates which operations we tend to do “first”, that
is, lower in the tree. We refer to this kind of priority of operators as
their P R E C E D E N C E , with higher precedence operators appearing lower
in the tree than lower precedence operators. By convention, we take
the additive operators (+ and -) to have lower precedence than the
multiplicative operators (*, /). Thus, the expression 3 + 4 * 5 has the
structure shown in the second tree, not the one shown in the first. For
that reason, it expresses the value 23 and not 35.
Precedence is not sufficient to disambiguate, for instance, expres-
sions with two binary operators of the same precedence. Precedence
alone doesn’t disambiguate the structure of 5 - 4 - 1: Is it (5 - 4)
- 1, that is, 0, or 5 - (4 - 1), that is, 2. Here, we rely on the A S S O -
C I AT I V I T Y of an operator. We say that subtraction, by convention, is
L E F T A S S O C I AT I V E , so that the operations are applied starting with
the left one. The grouping is (5 - 4) - 1. Other operators, such as
OCaml’s exponentiation operator ** are R I G H T A S S O C I AT I V E , so that
2. ** 2. ** 3. is disambiguated as 2. ** (2. ** 3.). Its value is
256., not 64..3 3
The ** operator applies to and returns
floating point values, hence the decimal
Associativity and precedence conventions go a long way in picking
point dots in the arguments and return
out the abstract structure of concrete expressions. But what if we want values.
to override those conventions? What if, say, we want to express the For a more complete presentation of
the precedences and associativities of
left-branching tree for 3 + 4 * 5? We can use annotations, as indeed, all of the built-in operators of OCaml,
we already have, to enforce a particular structure. This is the role of see the documentation on OCaml’s
operators.
PA R E N T H E S E S , to override conventional rules for disambiguating
expressions. In the case at hand, we write (3 + 4) * 5 to obtain the
left-branching tree.

Exercise 5
What is the structure of the following OCaml expressions? Draw the corresponding
tree so that it reflects the actual precedences and associativities of OCaml. Then, try
typing the expressions into the R E P L to verify that they are interpreted according to the
structure you drew.

1. 10 / 5 / 2
2. 5. +. 4. ** 3. /. 2.
3. (5. +. 4.) ** (3. /. 2.)
4. 1 - 2 - 3 - 4

You may have been taught this kind of rule under the mnemonic
P E M D A S. But the ideas of precedence, associativity, and annotation
are quite a bit broader than the particulars of the P E M D A S convention.
They are useful in thinking more generally about the relationship
between what we will call concrete syntax and abstract syntax.
32 PROGRAMMING WELL

3.3 Abstract and concrete syntax

The right way to think of expressions, then, is as hierarchically struc-

tured objects, which we have been depicting with trees as specified
by BNF grammar rules. From a practical perspective, however, when
programming, we are forced to notate these expressions in an un-
structured linear form as a sequence of characters, in order to enter
them into a computer. We use the term A B S T R A C T S Y N TA X for ex-
pressions qua structured objects, and C O N C R E T E S Y N TA X for their
linear-notated manifestations.
In order to more directly present the abstract syntax that corre-
sponds to a concrete expression, we draw trees as above that depict the
structure. So, for instance, the concrete syntax expression 3 + 4 * 5
corresponds to the A B S T R A C T S Y N TA X T R E E

⟨expr ⟩

⟨expr ⟩ ⟨binop ⟩ ⟨expr ⟩

⟨number ⟩ + ⟨expr ⟩ ⟨binop ⟩ ⟨expr ⟩

3 ⟨number ⟩ * ⟨number ⟩

4 5
We might abbreviate the tree structure to highlight the important
aspects by eliding the expression classes as

3 *

4 5
Then the alternative abstract syntax tree

+ 5

3 4
would correspond to the concrete syntax (3 + 4) * 5. Parentheses as
used for grouping are therefore notions of concrete syntax, not abstract
syntax. Similarly, conventions of precedence and associativity have to
do with the interpretation of concrete syntax, as opposed to abstract
syntax.
 E X P R E S S I O N S A N D T H E L I N G U I S T I C S O F P R O G R A M M I N G L A N G UA G E S 33

In fact, there are multiple concrete syntax expressions for this ab-
stract syntax, such as (3 + 4) * 5, ((3 + 4) * 5), (3 + ((4))) *
5. But certain expressions that may seem related do not have this same
abstract syntax: 5 * (3 + 4) or ((4 + 3) * 5) or (3 + 4 + 0) *
5. Although these expressions specify the same value, they do so in
syntactically distinct ways. The fact that multiplication and addition
are commutative, or that 0 is an additive identity – these are semantic
properties, not syntactic.

Exercise 6
Draw the (abbreviated) abstract syntax tree for each of the following concrete syntax
expressions. Assume the further BNF rule
⟨expr ⟩ ∶∶= ⟨unop ⟩⟨expr ⟩

for unary operators like ~-, the unary negation operator.

1. (~- 4) + 6
2. ~- (4 + 6)
3. 20 / ~- 4 + 6
4. 5 * (3 + 4)
5. ((4 + 3) * 5)
6. (3 + 4 + 0) * 5

Exercise 7
What concrete syntax corresponds to the following abstract syntax trees? Show as many
as you’d like.
1. ~-

1 42

2. /

84 +

0 42
3. +

84 /

0 42

3.4 Expressing your intentions

It is through the expressions of a programming language – structured

as abstract syntax and notated through concrete syntax – that pro-
34 PROGRAMMING WELL

grammers express their intentions to a computer. The computer inter-

prets the expressions in order to carry out those intentions.
Programming is an expressive activity with multiple audiences. Of
course, the computer is one audience; a program allows for program-
mers to express their computational intentions to the computer. But
there are human audiences as well. Programs can be used to commu-
nicate to other people – those who might be interested in an algorithm
for its own sake, or those who are tasked with testing, deploying, or
maintaining the programs. One of these latter programmers might
even be the future self of the author of the original code. Weeks or even
days after writing some code, you might well have already forgotten
why you wrote the code a certain way. The following fundamental
principle thus follows:

Edict of intention:
Make your intentions clear.
Programmers make mistakes. If their intentions are well expressed,
other programmers reviewing the code can notice that those inten-
tions are inconsistent with the code. Even the computer interpreting
the program can itself take appropriate action, notifying the program-
mer with a useful error or warning before the code is executed and the
unintended behavior can manifest itself.
Over the next chapters, we’ll see many ways that the edict of inten-
tion is applied. One of the most fundamental is through documenta-
tion of code.

3.4.1 Commenting

One of the most valuable aspects of the concrete syntax of any pro-
gramming language is the facility to provide elements in a concrete
program that have no correspondence whatsoever in the abstract syn-
tax, and therefore no effect on the computation expressed by the pro-
gram. The audience for such C O M M E N T S is the population of human
readers of the program. Comments serve the crucial expressive pur-
pose of documenting the intended workings of a program for those
human readers.
In OCaml, comments are marked by surrounding them with special
delimiters: (* ⟨⟩ *).4 The primary purpose of comments is satisfying 4
We use the symbol ⟨⟩ here and
throughout the later chapters as a con-
the edict of intention. Comments should therefore describe the why
venient notation to indicate unspecified
rather than the how of a program. Section B.2 presents some useful text of some sort, a textual anonymous
stylistic considerations in providing comments for documenting pro- variable of a sort. Here, it stands in
for the text that forms the comment.
grams. In other contexts it stands in for the
There are other aspects of concrete syntax that can be freely de- arguments of an operator, constructor,
or subexpression, for instance, in ⟨⟩ + ⟨⟩
ployed because they have no affect on the computation that a program
or ⟨⟩ list or let ⟨⟩ in ⟨⟩ .
 E X P R E S S I O N S A N D T H E L I N G U I S T I C S O F P R O G R A M M I N G L A N G UA G E S 35

carries out. These too can be judiciously deployed to help express your
intentions. For instance, the particular spacing used in laying out the
elements of a program doesn’t affect the computation that the program
expresses. Spaces, newlines, and indentations can therefore be used to
make your intentions clearer to a reader of the code, by laying out the
code in a way that emphasizes its structure or internal patterns. Simi-
larly, the choice of variable names is completely up to the programmer.
Names can be consistently renamed without affecting the computa-
tion. Programmers can take advantage of this fact by choosing names
that make clear their intended use.

Having clarified these aspects of the syntactic structure of program-

ming languages (and OCaml in particular) – distinguishing concrete
and abstract syntax; presenting precedence, associativity, and paren-
thesization for disambiguation – we turn now to begin the discussion
of OCaml as a language of types and values.
4
Values and types

OCaml is a

• value-based,

• strongly, statically, implicitly typed,

• functional

programming language. In this chapter, we introduce these aspects of

the language.

4.1 OCaml expressions have values

The OCaml language is, at its heart, a language for calculating values.
The expressions of the language specify these values, and the process
of calculating the value of an expression is termed E VA L U AT I O N . We’ve
already seen examples of OCaml evaluating some simple expressions
in Chapter 2:

# 3 + 4 * 5 ;;
- : int = 23
# (3 + 4) * 5 ;;
- : int = 35

The results of these evaluations are integers, and the output printed by
the R E P L indicates this by the int, about which we’ll have more to say
shortly.

4.1.1 Integer values and expressions

Integer values are built using a variety of operators and functions.

We’ve seen the standard arithmetic operators for integer addition (+),
subtraction (-), multiplication (*), and division (/). Integer negation
is with the ~- operator (a tilde followed by a hyphen), which is kept
distinct from the subtraction operator for clarity.
38 PROGRAMMING WELL

A full set of built-in operators is provided in OCaml’s Stdlib mod-

ule, one of a large set of OCaml library modules that provide a range
of functions. The Stdlib module is OCaml’s “standard library” in the
sense that the values it provides can be referred to anywhere with-
out any additional qualification, whereas values from other modules
require a prefix, for example, List.length or Hashtbl.create.
1
You’ll want to look over the Stdlib module documentation to get 1
There is nothing special going on with
Stdlib. It’s just that by default, the
a sense of what is available.
Stdlib module is “opened”, whereas
Here are some examples of integer expressions using these opera- other library modules like List and
tors: Hashtbl are not. The behavior of
modules will become clear when they
# 1001 / 365 ;; (* # of years in 1001 nights *) are fully introduced in Chapter 12.
- : int = 2
# 1001 mod 365 ;; (* # of nights left over *)
- : int = 271
# 1001 - (1001 / 365) * 365 ;; (* ...or alternatively *)
- : int = 271

Notice the use of comments to document the intentions behind the

calculations.

4.1.2 Floating point values and expressions

In addition to integers, OCaml provides other kinds of values. Real

numbers can be represented using a floating point approximation.
Floating point literals can be expressed in several ways, using decimal
notation (3.14), with an exponent (314e-2), and even in hexadecimal
(0x1.91eb851eb851fp+1).
# 3.14 ;;
- : float = 3.14
# 314e-2 ;;
- : float = 3.14
# 0x1.91eb851eb851fp+1 ;;
- : float = 3.14

Floating point expressions can be built up with a variety of oper-

ators, including addition (+.), subtraction (-.), multiplication (*.),
division (/.), and negation (~-.). Again, the Stdlib module provides a
fuller set, including operators for square root (sqrt) and various kinds
of rounding (floor and ceil).
# 3.14 *. 2. *. 2. ;; (* area of circle of radius 2 *)
- : float = 12.56
# ~-. 5e10 /. 2.718 ;;
- : float = -18395879323.0316429

Notice that the floating point operators are distinct from those for
integers. Though this will take some getting used to, the reason for this
design decision in the language will become apparent shortly.
 VA L U E S A N D T Y P E S 39

Exercise 8
Use the OCaml R E P L to calculate the value of the G O L D E N R AT I O , a proportion thought
to be especially pleasing to the eye (Figure 4.1).
√
1+ 5
.
2
You’ll want to use the built in sqrt function for floating point numbers. Be careful to use
floating point literals and operators. If you find yourself confronted with errors in solving
this exercise, come back to it after reading Section 4.2.

4.1.3 Character and string values

As in many programming languages, text is represented as strings Figure 4.1: A rectangle with width and
of C H A R A C T E R S . Character literals are given in single quotes, for in- height in the golden ratio.

stance, ’a’, ’X’, ’3’. Certain special characters can be specified only
by escaping them with a backslash, for instance, the single-quote char-
acter itself ’\’’ and the backslash ’\\’, as well as certain whitespace
characters like newline ’\n’ or tab ’\t’.
String literals are given in double quotes (with special characters
similarly escaped), for instance, "", "first", " and second". They
can be concatenated with the ^ operator.2 2
A useful trick is to use the escape
sequence of a backslash, a newline, and
# "" ^ "first" ^ " and second" ;; any amount of whitespace, all of which
- : string = "first and second" will be ignored, so as to split a string
over multiple lines. For instance,
# "First, " ^ "second, \
4.1.4 Truth values and expressions # third, \
# and fourth." ;;
There are two T RU T H VA L U E S , indicated in OCaml by the literals true - : string = "First, second, third, and fourth."
and false. Logical reasoning based on truth values was codified by the
British mathematician George Boole (1815–1864), leading to the use of
the term boolean for such values, and the type name bool for them in
OCaml.
Just as arithmetic values can be operated on with arithmetic oper-
ators, the truth values can be operated on with logical operators, such
as operators for conjunction (&&), disjunction (||), and negation (not).
(See Section A.2 for definitions of these operators.)

# false ;;
- : bool = false
# true || false ;;
- : bool = true
# true && false ;;
- : bool = false
# true && not false ;;
- : bool = true

The equality operator = tests two values3 for equality, returning 3

This is the first example of a function
that can apply to values of different
true if they are equal and false otherwise. There are other C O M PA R I -
types, a powerful idea that we will
S O N O P E R AT O R S as well: < (less than), > (greater than), <= (less than or explore in detail in Chapter 9.
equal), >= (greater than or equal), <> (not equal).
40 PROGRAMMING WELL

# 3 = 3 ;;
- : bool = true
# 3 > 4 ;;
- : bool = false
# 1 + 1 = 2 ;;
- : bool = true
# 3.1416 = 314.16 /. 100. ;;
- : bool = false
# true = false ;;
- : bool = false
# true = not false ;;
- : bool = true
# false < true ;;
- : bool = true

Exercise 9
Are any of the results of these comparisons surprising? See if you can figure out why the
results are that way.
Of course, the paradigmatic use of truth values is in the ability
to compute different values depending on the truth or falsity of a
condition. The OCaml C O N D I T I O N A L expression follows the template4 4
We describe the syntax of the construct
using the angle bracket convention
if ⟨exprtest ⟩ then ⟨exprtrue ⟩ else ⟨exprfalse ⟩ for classes of expression used in BNF
rules as introduced in Chapter 3. We
whose effect is to return the value of the ⟨exprtrue ⟩ if the value of the will continue to do so throughout as
test expression ⟨exprtest ⟩ is true and the value of the ⟨exprfalse ⟩ if the we introduce new constructs of the
language.
value of ⟨exprtest ⟩ is false. As mentioned in footnote 2 on
# if 3 = 3 then 0 else 1 ;; page 30, in defining expression classes
using this notation, we use subscripts
- : int = 0
to differentiate among different occur-
# 2 * if 3 > 4 then 3 else 4 + 5 ;;
rences of the same expression class,
- : int = 18
as we have done here with the three
# 2 * (if 3 > 4 then 3 else 4) + 5 ;; instances of the ⟨expr ⟩ class – ⟨exprtest ⟩,
- : int = 13 ⟨exprtrue ⟩, and ⟨exprfalse ⟩.

4.2 OCaml expressions have types

We’ve introduced these additional values grouped according to their

use. Integers are the type of things that integer operations are appro-
priate for; floating point numbers are the type of things that floating
point operations are appropriate for; truth values are the type of things
that logical operations are appropriate for. And conversely, it makes
no sense to apply operations to values for which they are not appro-
priate. Therefore, OCaml is a T Y P E D language. Every expression of the
language is associated with a type.
Figure 4.2: Small inconsistencies can
Using values in ways inconsistent with their type is perilous. The lead to major problems: The explosion
maiden flight of the Ariane 5 rocket on June 4, 1996 ended spectac- of the Ariane 5 on June 4, 1996.

ularly 37 seconds after launch when the rocket self-destructed. The

reason? A floating-point value was used as an integer, causing an im-
plicit conversion that overflowed. Using values in inappropriate ways
 VA L U E S A N D T Y P E S 41

is a frequent source of bugs in code, even if not with the dramatic after-
math of the Ariane 5 explosion. As we will see, associating values with
types can often prevent these kinds of bugs.
The OCaml language is S TAT I C A L LY T Y P E D , in that the type of an
expression can be determined just by examining the expression in
its context. It is not necessary to run the code in which an expression
occurs in order to determine the type of an expression, as might be
necessary in a DY N A M I C A L LY T Y P E D language (Python or JavaScript,
for instance).
Types are themselves a powerful abstraction mechanism. Types are
essentially abstract values. By reasoning about the types of expressions,
we can convince ourselves of the correctness of code without having to
run it.
Furthermore, OCaml is S T R O N G LY T Y P E D ; values may not be
used in ways inappropriate for their type. One of the ramifications of
OCaml’s strong typing is that functions only apply to values of certain
types and only return values of certain types. For instance, the addition
function specified by the + operator expects integer arguments and
returns an integer result.
By virtue of strong, static typing, the programming system (com-
piler or interpreter) can tell the programmer when type constraints are
violated even before the program is run, thereby preventing bugs before
they happen. If you attempt to use a value in a manner inconsistent
with its type, OCaml will complain with a typing error. For instance,
integer multiplication can’t be performed on floating point numbers or
strings:
# 5 * 3 ;;
- : int = 15
# 5 * 3.1416 ;;
Line 1, characters 4-10:
1 | 5 * 3.1416 ;;
^^^^^^
Error: This expression has type float but an expression was
expected of type
int
# "five" * 3 ;;
Line 1, characters 0-6:
1 | "five" * 3 ;;
^^^^^^
Error: This expression has type string but an expression was
expected of type
int

Programmers using a language with strong static typing for the first
time often find the frequent type errors limiting and even annoying.
Furthermore, there are some computations that can’t be expressed well
with such strict limitations, especially low-level systems computations
42 PROGRAMMING WELL

Type Type expression Example values An example expression

integers int 1 -2 42 (3 + 4) * 5
floating point numbers float 3.14 -2. 2e12 (3.0 +. 4.) *. 5e0
characters char ’a’ ’&’ '\n' char_of_int (int_of_char ’s’)
strings string "a" "3 + 4" "re" ^ "bus"
truth values bool true false true && not false
unit unit () ignore (3 + 4)

Table 4.1: Some of the atomic OCaml

that need access to the underlying memory representation of values. types with example values and an
But a type error found at compile time is a warning that data use er- example expression.

rors could show up at run time after the code has been deployed – and
when it’s far too late to repair it. Strong static type constraints are thus
an example of a language restraint that frees programmers from verify-
ing that their code does not contain “bad” operations by empowering
the language interpreter to do so itself. (Looking ahead to the edict of
prevention in Chapter 11, it makes the illegal inexpressible.)

4.2.1 Type expressions and typings

In OCaml, every type has a “name”. These names are given as T Y P E

E X P R E S S I O N S , a kind of little language for naming types. Just as there
are value expressions for specifying values, there are type expressions
for specifying types.
In this language of type expressions, each AT O M I C T Y P E has its
own name. We’ve already seen the names of the integer, floating point,
and truth value types – int, float, and bool, respectively – in the
examples earlier in this chapter, because the R E P L prints out a type
expression for a value’s type along with the value itself, for instance,

# 42 ;;
- : int = 42
# 3.1416 ;;
- : float = 3.1416
# false ;;
- : bool = false

Notice that the R E P L presents the type of each computed value after a
colon (:). (Why a colon? You’ll see shortly.)
Table 4.1 provides a more complete list of some of the atomic types
in OCaml (some not yet introduced), along with their type names,
some example values, and an example expression that specifies a value
of that type using some functions that return values of the given type.
(We’ll get to non-atomic (composite) types in Chapter 7.)
It is often useful to notate that a certain expression is of a certain
 VA L U E S A N D T Y P E S 43

type. Such a T Y P I N G is notated in OCaml using the : operator, placing

the value to the left of the operator and its type to the right. So, for
instance, the following typings hold:

• 42 : int

• true : bool

• 3.14 *. 2. *. 2. : float

• if 3 > 4 then 3 else 4 : int

The first states that the expression 42 specifies an integer value, the
second that true specifies a boolean truth value, and so forth. The :
operator is sometimes read as “the”, thus “42, the integer” or “true, the
bool”. The typing operator is special in that it combines an expression
from the value language (to its left) with an expression from the type
language (to its right).
We can test out these typings right in the R E P L . (The parentheses
are necessary.)

# (42 : int) ;;
- : int = 42
# (true : bool) ;;
- : bool = true
# (3.14 *. 2. *. 2. : float) ;;
- : float = 12.56
# (if 3 > 4 then 3 else 4 : int) ;;
- : int = 4

The R E P L generates an error when a value is claimed to be of an

inappropriate type.

# (42 : float) ;;
Line 1, characters 1-3:
1 | (42 : float) ;;
^^
Error: This expression has type int but an expression was expected
of type
float
Hint: Did you mean `42.'?

Exercise 10
Which of the following typings hold?
1. 3 + 5 : float
2. 3. + 5. : float
3. 3. +. 5. : float
4. 3 : bool
5. 3 || 5 : bool
6. 3 || 5 : int
44 PROGRAMMING WELL

Try typing these into the R E P L to see what happens. (Remember to surround them with
parentheses.)
Finally, in OCaml, expressions are I M P L I C I T LY T Y P E D . Although all
expressions have types, and the types of expressions can be annotated
using typings, the programmer doesn’t need to specify those types in
general. Rather, the OCaml interpreter can typically deduce the types
of expressions at compile time using a process called T Y P E I N F E R -
E N C E. In fact, the examples shown so far depict this inference. The
REPL prints not only the value calculated for each expression but also
the type that it inferred for the expression.

4.3 The unit type

In OCaml, the phrases of the language are expressions, expressing

values. In many other programming languages, the phrases of the lan-
guage are not always used to express values. Rather, they are used as
commands. They are of interest because of what they do, not what they
are. This approach is especially prevalent in imperative programming,
the term ‘imperative’ deriving from the Latin ‘imperativus’, meaning
‘pertaining to a command’. But OCaml, like other functional languages,
is uniform in privileging expressions over commands.
Occasionally, we have an expression that really need compute
no value. But since every expression has to have a value in OCaml,
we need to assign a value to such expressions as well. In this case,
we use the value (), spelled with an open and close parenthesis and
pronounced “unit”. This value is the only value of the type unit. Since
the unit type has only one value, that value conveys no information,
which is just what we want as the value of an expression whose value
is irrelevant. The unit type will feature more prominently once we
explore imperative programming within OCaml in Chapter 15.

Exercise 11
Give a typing for a value of the unit type.

4.4 Functions are themselves values

OCaml is a functional programming language, by which we mean that

functions are F I R S T- C L A S S VA L U E S – they can be passed as arguments
to functions or returned as the value of functions. Functions that take
functions as arguments or return functions as values are referred to
as H I G H E R - O R D E R F U N C T I O N S , and the powerful programming
paradigm that makes full use of this capability, which we will introduce
in Chapter 8, is H I G H E R - O R D E R F U N C T I O N A L P R O G R A M M I N G .
Related to the idea that functions are values is that they have types
 VA L U E S A N D T Y P E S 45

as well. In Exercise 8, you used the sqrt function to take the square
root of a floating point number. This function, sqrt, is itself a value
and has a type. The type of a function expresses both the type of its
argument (in this case, float) and the type of its output (again float).
The type expression for a function (the type’s “name”) is formed by
placing the symbol -> (read “arrow” or “to”) between the argument
type and the output type. Thus the type for sqrt is float -> float
(read “float arrow float” or “float to float”), or, expressed as a typing,
sqrt : float -> float.
You can verify this typing yourself, just by evaluating sqrt:

# sqrt ;;
- : float -> float = <fun>

Since functions are themselves values, they can be evaluated, and the
REPL performs type inference and provides the type float -> float
along with a printed representation of the value itself <fun>, indicating
that the value is a function of some sort.5 5
The actual value of a function is a
complex data object whose internal
Because the argument type of sqrt is float, it can only be applied
structure is not useful to print, so this
to values of that type. And since the result type of sqrt is float, only abstract presentation <fun> is printed
functions that take float arguments can apply to expressions like instead.

sqrt 42..

Exercise 12
Try applying the sqrt function to an argument of some type other than float, for
instance, a value of type bool. What happens?
Of course, the real power in functional programming comes from
defining your own functions. We’ll move to this central topic in Chap-
ter 6, but first, it is useful to provide a means of naming values (includ-
ing functions), to which we turn in the next chapter.
5
Naming and scope

5.1 Variables are names for values

We introduced the concept of a variable in Chapter 1 as seen in the

imperative programming paradigm – the variable as a locus of mutable
state, which takes on different values over time. But in the functional
paradigm, variables are better thought of simply as names for values.
To introduce a name for a value for use in some other expression,
OCaml provides the local naming construct:
1
Variables in OCaml are required
let ⟨var ⟩ : ⟨type ⟩ = ⟨exprdef ⟩ in ⟨exprbody ⟩ to be sequences of alphabetic and
numeric characters along with the
In this construct, ⟨var ⟩ is a variable,1 which will be the name of a underscore character (_) and the prime
value of the given ⟨type ⟩; ⟨exprdef ⟩ is an expression defining a value character (’). The first character in the
of the given ⟨type ⟩; and ⟨exprbody ⟩ is an expression within which the variable name must be alphabetic or an
underscore. The special role of the latter
variable can be used as the name for the defined value. We say that case is discussed later in Section 7.2.
the construction B I N D S the name ⟨var ⟩ to the value ⟨exprdef ⟩ for use
in ⟨exprbody ⟩.2 For this reason, the let construction is referred to as 2
The name being defined is sometimes
referred to as the D E F I N I E N D U M ,
a B I N D I N G C O N S T RU C T . We’ll introduce other binding constructs in
the expression it names being the
Chapters 6 and 7. D E F I N I E N S.

As an example,3 we might provide a name for the important con- 3

In these examples, we follow the stylis-
tic guidelines described in Section B.1.7
stant π in the context of calculating the area of a circle of radius 2:
in indenting the body of a let to the
# let pi : float = 3.1416 in same level as the let keyword itself. The
# pi *. 2. *. 2. ;; rationale is provided there.
- : float = 12.5664

Informally speaking (and we’ll provide a more rigorous description in

Chapter 13), the construct operates as follows: The ⟨exprdef ⟩ expression
is evaluated to a value, and then the ⟨exprbody ⟩ is evaluated, but as if
occurrences of the definiendum ⟨var ⟩ were first replaced by the value
of the definiens ⟨exprdef ⟩.

5.2 The type of a let-bound variable can be inferred

It may seem obvious to you that in an expression like

48 PROGRAMMING WELL

let pi : float = 3.1416 in

pi *. 2. *. 2. ;;

the variable pi is of type float. What else could it be, given that its
value is a float literal, and it is used as an argument of the *. oper-
ator, which takes float arguments? You would be right, and OCaml
itself can make this determination, inferring the type of pi without the
explicit typing being present. For that reason, the type information in
the let construct is optional. We can simply write

let pi = 3.1416 in
pi *. 2. *. 2. ;;

and the calculation proceeds as usual. This ability to infer types is what
we mean when we say (as in Section 4.2.1) that OCaml is implicitly
typed.
Although these typings when introducing variables are optional,
nonetheless, it can still be useful to provide explicit type information
when naming a value. First, following the edict of intention, it allows
the programmer to make clear the intended types, so that the OCaml
interpreter can verify that the programmer’s intention was followed
and so that readers of the code are aware of that intention. Second,
there are certain (relatively rare) cases (Section 9.7) in which OCaml
cannot infer a type for an expression in context; in such cases, the
explicit typing is necessary.

5.3 let expressions are expressions

Remember that all expressions in OCaml have values, even let expres-
sions. Thus we can use them as subexpressions of larger expressions.

# 3.1416 *. (let radius = 2.

# in radius *. radius) ;;
- : float = 12.5664

Exercise 13
Are the parentheses necessary in this example? Try out the expression without the
parentheses and see what happens.
A particularly useful application of the fact that let expressions
can be used as first-class values is that they may be embedded in other
let expressions to get the effect of defining multiple names. Here, we
define both the constant π and a radius to calculate the area of a circle
of radius 4:

# let pi = 3.1416 in
# let radius = 4. in
# pi *. radius ** 2. ;;
- : float = 50.2656
 NAMING AND SCOPE 49

Exercise 14
Use the let construct to improve the readability of the following code to calculate the
length of the hypotenuse of a particular right triangle:
# sqrt (1.88496 *. 1.88496 +. 2.51328 *. 2.51328) ;;
- : float = 3.1416

5.4 Scope

The name defined in the let expression is available only in the body
of the expression. The name is L O C A L to the body, and unavailable
outside of the body. We say that the S C O P E of the variable – that is, the
code within which the variable is available as a name of the defined
value – is the body of the let expression. This explains the following
behavior:
# (let s = "hi ho " in
# s ^ s) ^ s ;;
Line 2, characters 9-10:
2 | s ^ s) ^ s ;;
^
Error: Unbound value s

The body of the let expression in this example ends at the closing
parenthesis, and thus the variable s defined by that construct is un-
available (“unbound”) thereafter.

Exercise 15
Correct the example to provide the triple concatenation of the defined string.

Exercise 16
What type do you expect is inferred for s in the example?
In particular, the scope of a local let naming does not include the
definition itself (the ⟨exprdef ⟩ part between the = and the in). Thus the
following expression is ill-formed:
# let x = x + 1 in
# x * 2 ;;
Line 1, characters 8-9:
1 | let x = x + 1 in
^
Error: Unbound value x

And a good thing too, for what would such an expression mean? This
kind of recursive definition isn’t well founded. Nonetheless, there are
useful recursive definitions, as we will see in Section 6.4.
What if we define the same name twice? There are several cases to
consider. Perhaps the two uses are disjoint, as in this example:
# sqrt ((let x = 3. in x *. x)
# +. (let x = 4. in x *. x)) ;;
- : float = 5.
50 PROGRAMMING WELL

Since each x is introduced with its own let and has its own body, the
scopes are disjoint. The occurrences of x in the first expression name
the number 3. and in the second name the number 4.. But in the
following case, the scopes are not disjoint:
# sqrt (let x = 3. in
# x *. x +. (let x = 4. in x *. x )) ;;
- : float = 5.

The scope of the first let encompasses the entire second let. Do the
highlighted occurrences of x in the body of the second let name 3.
or 4.? The rule used in OCaml (and most modern languages) is that
the occurrences are bound by the nearest enclosing binding construct
for the variable. The same binding relations hold as if the inner let-
bound variable x and the occurrences of x in its body were uniformly
renamed, for instance, as y:
# sqrt (let x = 3. in
# x *. x +. (let y = 4. in y *. y)) ;;
- : float = 5.

By virtue of this convention that variables are bound by the closest

binder, when an inner binder for a variable falls within the scope of an
outer binder for the same variable, the outer variable is inaccessible
in the inner scope. We say that the outer variable is S H A D OW E D by the
inner variable. For instance, in
# let x = 1 in
# x + let x = 2 in
# x + let x = 4 in
# x ;;
- : int = 7

the innermost x (naming 4) shadows the outer two, and the middle x
(naming 2) shadows the outer x (naming 1). Thus the three highlighted
occurrences of x name 1, 2, and 4, respectively, which the expression as
a whole sums to 7.
Since the scope of a let-bound variable is the body of the construct,
but not the definition, occurrences of the same variable in the defi-
nition must be bound outside of the let. Consider the highlighted
occurrence of x on the second line:
let x = 3 in
let x = x * 2 in
x + 1 ;;

This occurrence is bound by the let in line 1, not the one in line 2.
That is, it is equivalent to the renaming
let x = 3 in
let y = x * 2 in
y + 1 ;;
 NAMING AND SCOPE 51

Exercise 17
For each occurrence of the variable x in the following examples, which let construct
binds it? Rewrite the expressions by renaming the variables to make them distinct while
preserving the bindings.
1. let x = 3 in
let x = 4 in
x * x ;;

2. let x = 3 in
let x = x + 2 in
x * x ;;

3. let x = 3 in
let x = 4 + (let x = 5 in x) + x in
x * x ;;

5.5 Global naming and top-level let

The let construct introduced above introduces a local name, local

in the sense that its scope is just the body of the let. OCaml pro-
vides a global naming construct as well. By simply leaving off the ‘in
⟨exprbody ⟩’ part of the let construct, the name can continue to be used
thereafter; the scope of the naming extends all the way through the
remainder of the R E P L session or to the end of the program file.

# let pi = 3.1416 ;;
val pi : float = 3.1416
# let radius = 4.0 ;;
val radius : float = 4.
# pi *. radius *. radius ;;
- : float = 50.2656
# 2. *. pi *. radius ;;
- : float = 25.1328

The R E P L indicates that new names have been introduced by present-

ing typings for the names (pi : float or radius : float) as well as
displaying their values.
This global naming may look a bit like assignment in imperative
languages. We can have, for instance,

# let x = 3 ;;
val x : int = 3
# let x = x + 1 ;;
val x : int = 4
# x + x ;;
- : int = 8

The second line may look like it is assigning a new value to x. But no,
all that is happening is that there is a new name (coincidentally the
same as a previous name) for a new value. The old name x for the value
3 is still around; it’s just inaccessible, shadowed by the new name x. (In
Chapter 15, we provide a demonstration that this is so.)
52 PROGRAMMING WELL

Exercise 18
In the sequence of expressions
let tax_rate = 0.05 ;;
let price = 5. ;;
let price = price * (1. +. tax_rate) ;;
price ;;

what is the value of the final expression? (You can use the R E P L to verify your answer.)
Global naming is available only at the top level. A global name
cannot be defined from within another expression, for instance, the
body of a local let. The following is thus not well-formed:

# let radius = 4. in
# let pi = 3.1416 in
# let area = pi *. radius ** 2. ;;
Line 3, characters 30-32:
3 | let area = pi *. radius ** 2. ;;
^^
Error: Syntax error

Exercise 19
How might you get the effect of this definition of a global variable area by making use of
local variables for pi and radius?

We alluded to the fact that in OCaml, functions are first-class values,

and as such they can be named as well. In fact, the ability to name val-
ues becomes most powerful when the named values are functions. In
the next chapter, we introduce functions and function application in
OCaml, and start to demonstrate the power of functions as an abstrac-
tion mechanism.
 NAMING AND SCOPE 53

s
6
Functions

A function is a mapping from an input (called the function’s A R G U -

M E N T ) to an output (the function’s VA L U E ). For instance, consider the
1
mapping from integers to their successors: 1
It turns out that this function is already
available within OCaml as the built-in
⋮ function succ.

-2 → -1

-1 → 0
0→1

1→2
2→3

or the function that maps integers to a truth value that specifies

whether the integer is or isn’t an even number:

⋮
-2 → true
-1 → false

0 → true
1 → false

2 → true
⋮

We can make use of the function by A P P LY I N G it to its argument.

You’ll be most familiar with the traditional and ubiquitous mathemat-
ical notation for function application, in which a symbol naming the
function precedes a parenthesized, comma-separated list of the ar-
guments, as, for instance, f (1, 2, 3).2 It is thus perhaps surprising that 2
Some historical background on this
notation is provided in Section A.1.2.
OCaml doesn’t use this notation for function application. Instead, it
follows the notational convention proposed by Church in his lambda
56 PROGRAMMING WELL

calculus. (See Section 1.2.) In the lambda calculus, functions and their
application are so central (indeed, there’s basically nothing else in the
logic) that the addition of the parentheses in the function application
notation becomes onerous. Instead, Church proposed merely prefix-
ing the function to its argument. Instead of f (1), Church’s notation
would have f 1. Instead of f (g (1)), he would have f (g 1), using the
parentheses for grouping, but not for demarcating the arguments.
Similarly, in OCaml, the function merely precedes its argument. The
successor of 41 is simply succ 41. The square root of two is sqrt 2.0.
# succ 41 ;;
- : int = 42
# sqrt 2.0 ;;
- : float = 1.41421356237309515

Recall from Section 4.4 that functions (as all values) have types,
which can be expressed as type expressions using the -> operator.
For instance, the successor function has the type given by the type Figure 6.1: Moses Schönfinkel (1889–
expression int -> int and the “evenness” function the type int -> 1942), Russian logician and math-
ematician, first specified the use of
bool. higher-order functions to mimic the
effect of multiple-argument functions.

6.1 Multiple arguments and currying

The simple prefix notation for function application is only appropriate

when functions take exactly one argument. But it turns out that this
is not a substantial limitation in a system (like the lambda calculus
and like OCaml) in which functions are themselves values. Suppose
we have a function that we think of as taking multiple arguments
simultaneously (like f (1, 2, 3)). We can reconceptualize f as taking
only one argument (in this case, the argument 1), returning a function
that takes the second argument 2, again returning a function that takes
the third and final argument 3, returning the final value. The type of
such a function, which takes three integers returning an integer result,
say, is thus
int -> (int -> (int -> int))

In essence, the function takes its three arguments one at a time, return-
ing a function after each argument before the last. Although this trick Figure 6.2: Haskell Curry (1900–1982),
American logician, promulgator of
was first discussed by Schönfinkel (1924), it is referred to as C U R RY I N G the use of higher-order functions
a function, the resulting function being curried, so named after Haskell to simulate functions of multiple
arguments, which is referred to as
Curry who popularized the approach. currying in his honor.
Because in OCaml functions take one argument, the language
makes extensive use of currying, and language constructs facilitate
its use. For instance, the -> type expression operator is right associa-
tive (see Section 3.2) in OCaml, so that the type of the curried three-
argument function above can be expressed as
 FUNCTIONS 57

int -> int -> int -> int

Application, conversely, is left associative, so that applying a curried

function f to its arguments can be notated f 1 2 3 instead of ((f 1)
2) 3.
We’ve already used some curried functions without noticing. The
two-argument arithmetic and boolean operators, like +, /., and &&, are
curried. As usual, the R E P L reveals their type:

# (+) ;;
- : int -> int -> int = <fun>
# (/.) ;;
- : float -> float -> float = <fun>
# (&&) ;;
- : bool -> bool -> bool = <fun>

Normally, we write these operators I N F I X , placing the operator be-

tween its two arguments, but by placing the operator in parentheses3 3
Care must be taken when parenthesiz-
ing the multiplication operators * and
as we’ve done, the OCaml R E P L interprets them as regular P R E F I X
*. to convert them to prefix functions.
functions, in which the function appears before its argument. Making Since OCaml comments are provided
use of this ability, they can even be applied in the one-by-one manner, as (* ⟨⟩ *), parenthesizing as (*) will
be misinterpreted as the beginning of a
as we’ve done here both parenthesized and unparenthesized: comment. To avoid this problem, place
spaces between the parentheses and the
# ((+) 3) 4 ;;
operator: ( * ).
- : int = 7
# (+) 3 4 ;;
- : int = 7

Exercise 20
What (if anything) are the types and values of the following expressions? Try to figure
them out yourself before typing them into the R E P L to verify your answer.
1. (-) 5 3
2. 5 - 3
3. - 5 3
4. "O" ^ "Caml"
5. (^) "O" "Caml"
6. (^) "O"
7. ( ** ) – See footnote 3.

6.2 Defining anonymous functions

Now we get to the whole point of functional programming: defining

your own functions. Suppose we want to specify a function that maps a
certain input, call it x, to an output, say the doubling of x. The follow-
ing expression does the trick: fun x : int -> 2 * x.

# fun x : int -> 2 * x ;;

- : int -> int = <fun>
58 PROGRAMMING WELL

The arrow -> separates the typing of a variable that represents the
input, the integer x, from an expression that represents the output, 2
* x. The output expression can, of course, make free use of the input
variable as part of the computation.
We can apply this function to an argument (21, say). We use the
usual OCaml prefix function application syntax, placing the function
before its argument:
# (fun x : int -> 2 * x) 21 ;;
- : int = 42

In general, we construct such a “function without a name”, an

4
A N O N Y M O U S F U N C T I O N , with the OCaml fun construct: 4
Warning: The same arrow symbol ->
is used in defining both function values
fun ⟨var ⟩ : ⟨type ⟩ -> ⟨expr ⟩ and function types. This sometimes
leads to confusion. Be aware that
Here, ⟨var ⟩ is a variable, the name of the argument of the function, though the same symbol is used for
which will name a value of the given ⟨type ⟩; ⟨expr ⟩ is an expression both, the two are quite distinct.

defining the output of the function.

The fun construct, like the let construct, is a binding construct.
The fun construct introduces a variable and binds occurrences of that
variable in its scope. The scope of the variable is the body of the fun,
the expression ⟨expr ⟩ after the arrow.
As was the case for let expressions, when the type of the variable
can be inferred from how it is used in the definition part, as is typically
the case, the typing part can be left off. So, for instance, the doubling
function could be written
# fun x -> 2 * x ;;
- : int -> int = <fun>

and the same type int -> int still inferred.

Exercise 21
Try defining your own functions, perhaps one that squares a floating point number, or
one that repeats a string.

6.3 Named functions

Now that we have the ability to define functions (with fun) and the
ability to name values (with let), we can put them together to name
newly-defined functions. Here, we give a global naming of the dou-
bling function and use it:
# let double = fun x -> 2 * x ;;
val double : int -> int = <fun>
# double 21 ;;
- : int = 42

Here are functions for the circumference and area of circles of given
radius:
 FUNCTIONS 59

# let pi = 3.1416 ;;
val pi : float = 3.1416

# let area =
# fun radius ->
# pi *. radius ** 2. ;;
val area : float -> float = <fun>

# let circumference =
# fun radius ->
# 2. *. pi *. radius ;;
val circumference : float -> float = <fun>

# area 4. ;;
- : float = 50.2656
# circumference 4. ;;
- : float = 25.1328

6.3.1 Compact function definitions

This method for defining named functions, though effective, is a bit

cumbersome. For that reason, OCaml provides a simpler syntax for
defining functions, in which a definition for the calling pattern itself is
provided. Instead of the phrasing

let ⟨varfunc ⟩ = fun ⟨vararg ⟩ -> ⟨expr ⟩

OCaml allows the following equivalent phrasing

let ⟨varfunc ⟩ ⟨vararg ⟩ = ⟨expr ⟩

This syntax for defining functions may be more familiar from other
languages. It is also consistent with a more general pattern-matching
syntax that we will come to in Section 7.2.
This compact syntax for function definition is an example of S Y N -
5
TA C T I C S U G A R , a bit of additional syntax that serves to abbreviate 5
The term “syntactic sugar” was first
used by Landin (1964) (Figure 17.7)
a more complex construction. By adding some syntactic sugar, the
to describe just such abbreviatory
language can provide simpler expressions without adding underlying constructs.
constructs to the language; a language with a small core set of con-
structs can still have a sufficiently expressive concrete syntax that it
is pleasant to program in. As we introduce additional syntactic sugar
constructs, notice how they allow for idiomatic programming without
increasing the core language.
We can use this more compact function definition notation to
provide a more elegant definition of the doubling function:

# let double x = 2 * x ;;
val double : int -> int = <fun>
# double (double 3) ;;
- : int = 12
60 PROGRAMMING WELL

This compact notation applies to local definitions as well.

# let double x = 2 * x in
# double (double 3) ;;
- : int = 12

It even extends to multiple-argument curried functions. The defini-

tion
# let hypotenuse x y =
# sqrt (x ** 2. +. y ** 2.) ;;
val hypotenuse : float -> float -> float = <fun>

is syntactic sugar for (and hence completely equivalent to) the defini-
tion
# let hypotenuse =
# fun x ->
# fun y ->
# sqrt (x ** 2. +. y ** 2.) ;;
val hypotenuse : float -> float -> float = <fun>

6.3.2 Providing typings for function arguments and outputs

As in all definitions, you can provide a typing for the variable being
defined, as in
# let hypotenuse : float -> float -> float =
# fun x ->
# fun y ->
# sqrt (x ** 2. +. y ** 2.) ;;
val hypotenuse : float -> float -> float = <fun>

and it is good practice to do so for top-level definitions. That way,

you are registering your intentions as to the types – remember the
edict of intention? – and the language interpreter can verify that those
intentions are satisfied. (See Section B.3.4.)
In the compact notation, typings can and should be provided for
the application of the function to its arguments, as well as for the ar-
guments itself. In the hypotenuse function definition, the application
hypotenuse x y is of type float, which can be recorded as

# let hypotenuse x y : float =

# sqrt (x ** 2. +. y ** 2.) ;;
val hypotenuse : float -> float -> float = <fun>

Each of the arguments can be explicitly typed as well.

# let hypotenuse (x : float) (y : float) : float =
# sqrt (x ** 2. +. y ** 2.) ;;
val hypotenuse : float -> float -> float = <fun>

Here, we have recorded that x and y are each of float type, and the re-
sult of an application hypotenuse x y is also a float, which together
 FUNCTIONS 61

capture the full information about the type of hypotenuse itself. Con-
sequently, the type inferred for the hypotenuse function itself is, as
before, float -> float -> float, that is, a curried binary function
from floats to floats.

Exercise 22
Consider the following beginnings of function declarations. How would these appear
using the compact notation?

1. let foo : bool -> int -> bool = ...

2. let foo : bool -> (int -> bool) = ...

3. let foo : (float -> int) -> float -> bool = ...

Exercise 23
What are the types for the following expressions?

1. let greet y = "Hello" ˆ y in greet "World!" ;;

2. fun x -> let exp = 3. in x ** exp ;;

Exercise 24
Define a function square that squares a floating point number. For instance,
# square 3.14 ;;
- : float = 9.8596
# square 1234567. ;;
- : float = 1524155677489.

Exercise 25
Define a function abs : int -> int that computes the absolute value of an integer.
# abs (-5) ;;
- : int = 5
# abs 0 ;;
- : int = 0
# abs (3 + 4) ;;
- : int = 7

Exercise 26
The Stdlib.string_of_bool function returns a string representation of a boolean.
Here it is in operation:
# string_of_bool (3 = 3) ;;
- : string = "true"
# string_of_bool (0 = 3) ;;
- : string = "false"

What is the type of string_of_bool? Provide your own function definition for it.

Exercise 27
Define a function even : int -> bool that determines whether its integer argument
is an even number. It should return true if so, and false otherwise. Try using both the
compact notation for the definition and the full desugared notation. Try versions with
and without typing information for the function name.

Exercise 28
Define a function circle_area : float -> float that returns the area of a circle of a
given radius specified by its argument. Try all of the variants described in Exercise 27.

Figure 6.3: The frustrum of a cone,

62 PROGRAMMING WELL

Exercise 29
A frustrum (Figure 6.3) is a three-dimensional solid formed by slicing off the top of a
cone parallel to its base. The volume V of a frustrum with radii r 1 and r 2 and height h is
given by the formula
πh
V = (r 12 + r 1 r 2 + r 22 ) .
3
Implement a function to calculate the volume of a frustrum given the radii and height.
Problem 30
The calculation of the date of Easter, a calculation so important to early Christianity
that it was referred to simply as C O M P U T U S (“the computation”), has been the subject
of innumerable algorithms since the early history of the Christian church. An especially
simple method, published in Nature in 1876 and attributed to “A New York correspon-
dent” (1876), proceeds by sequentially calculating the following values on the basis of the
year Y :

a = Y mod 19 h = (19a + b − d − g + 15) mod 30

Y c
b= i=
100 4
c = Y mod 100 k = c mod 4
b
d= l = (32 + 2e + 2i − h − k ) mod 7
4
a + 11h + 22l
e = b mod 4 m=
451
b +8 h + l − 7m + 114
f = month =
25 31
b − f +1
g= day = ((h + l − 7m + 114) mod 31) + 1
3
Write two functions, computus_month and computus_day, which take an integer year
argument and return, respectively, the month and day of Easter as calculated by the
method above. Use them to verify that the date of Easter in 2018 was April 1.

6.4 Defining recursive functions

Consider the F A C T O R I A L function, which maps its integer input onto

the product of all the positive integers that are no larger. Thus, the
factorial of 3, traditionally notated with a suffixed exclamation mark
as 3!, is the product of 1, 2, and 3, that is, 6; and 4! is 24. Notice that 4!
is 4 ⋅ 3!, which makes sense because 3! has already incorporated all the
integers up to 3, so the only remaining integer to multiply in is 4 itself.
Indeed, in general,

n ! = n ⋅ (n − 1)!

for all integers n greater than 1, and if we take the value of 0! to be 1,

the equation even holds for n = 1. This serves to completely define
the factorial function. We can take its definition to be given by the two
equations6 6
See Section A.1.1 for more background
on defining mathematical functions by
0! = 1 equations.

n ! = n ⋅ (n − 1)! for n > 0

 FUNCTIONS 63

We can implement the factorial function directly from this defini-

tion. The first line of the definition, setting up the name of the function
(fact), its single integer argument (n), and its output type (int) is
straightforward.
let fact (n : int) : int =
...

The body of the function starts by distinguishing the two cases, when n
is zero and when n is positive.
let fact (n : int) : int =
if n = 0 then ...
else ...

The zero case is simple; the output value is 1.

let fact (n : int) : int =
if n = 0 then 1
else ...

The non-zero case involves multiplying n by the factorial of n - 1.

let fact (n : int) : int =
if n = 0 then 1
else n * fact (n - 1) ;;

Let’s try it.

# let fact (n : int) : int =
# if n = 0 then 1
# else n * fact (n - 1) ;;
Line 3, characters 9-13:
3 | else n * fact (n - 1) ;;
^^^^
Error: Unbound value fact
Hint: If this is a recursive definition,
you should add the 'rec' keyword on line 1

There seems to be a problem. Recall from Section 5.4 that the scope
of a let is the body of the let (or the code following a global let),
but not the definition part of the let. Yet we’ve referred to the name
fact in the definition of the fact function. The scope rules for the let
constructs (both local and global) disallow this.
In order to extend the scope of the naming to the definition itself, to
allow a recursive definition, we add the rec keyword after the let.
# let rec fact (n : int) : int =
# if n = 0 then 1
# else n * fact (n - 1) ;;
val fact : int -> int = <fun>

The rec keyword means that the scope of the let includes not only its
body but also its definition part. With this change, the definition goes
through, and in fact, the function works well:
64 PROGRAMMING WELL

# fact 0 ;;
- : int = 1
# fact 1 ;;
- : int = 1
# fact 4 ;;
- : int = 24
# fact 20 ;;
- : int = 2432902008176640000

You may in the past have been admonished against defining some-
thing in terms of itself, such as “comb: an object used to comb one’s
hair; to comb: to run a comb through.” You may therefore find some-
thing mysterious about recursive definitions. How can we make use of
a function in its own definition? We seem to be using it before it’s even
fully defined. Isn’t that problematic?
Of course, recursive definition can be problematic. For instance, 7
In fact, the computer scientist C. A. R.
Hoare in his 1981 Turing Award lecture
consider this recursive definition of a function to add “just one more”
described his own introduction to
to a recursive invocation of itself: recursion this way:

# let rec just_one_more (x : int) : int = Around Easter 1961, a course

# 1 + just_one_more x ;; on A L G O L 60 was offered in
val just_one_more : int -> int = <fun> Brighton, England, with Peter
Naur, Edsger W. Dijkstra, and
The definition works just fine, but any attempt to use it fails impres- Peter Landin as tutors. . . . It
sively: was there that I first learned
about recursive procedures
# just_one_more 42 ;; and saw how to program the
Stack overflow during evaluation (looping recursion?). sorting method which I had
earlier found such difficulty
The error message “Stack overflow during evaluation (looping recur- in explaining. It was there
sion?)” gives a hint as to what’s gone wrong; there is indeed a looping that I wrote the procedure,
recursion that would go on forever if the computer didn’t run out of immodestly named QU I C K -
memory (“stack overflow”) first. S O RT , on which my career

But a recursion that is well founded can be quite useful.7 In the case as a computer scientist is
founded. Due credit must
of factorial, each recursive invocation of fact is given an argument
be paid to the genius of the
that is one smaller than the previous invocation, so that eventually designers of A L G O L 60 who
an invocation on argument 0 will occur and the recursion will end. included recursion in their
Because there are branches of computation (namely, the first arm language and enabled me
of the conditional) without recursive invocations of fact, and those to describe my invention
branches will eventually be taken, all is well. so elegantly to the world.
I have regarded it as the
But will those branches always be eventually taken? Unfortunately
highest goal of programming
not. language design to enable
# fact (~-5) ;;
good ideas to be elegantly
Stack overflow during evaluation (looping recursion?). expressed. (Hoare, 1981)

This looks familiar. Counting down from any non-negative integer will
eventually get us to zero. But counting down from a negative integer
won’t. We intended the factorial function to apply only to non-negative
 FUNCTIONS 65

integers, the values for which it’s defined, but we didn’t express that
intention – the edict of intention again – with this unfortunate result.
You might think that we could solve this problem with types. In-
stead of specifying the argument as having integer type, perhaps we
could specify it as of non-negative integer type. Unfortunately, OCaml
does not provide for this more fine-grained type, and in any case, other
examples might require different constraints on the type, perhaps odd
integers only, or integers larger than 7, or integers within a certain
range.

Exercise 31
For each of the following cases, define a recursive function of a single argument for
which the definition is well founded (and the computation terminates) only when the
argument is
8
1. an odd integer; If you are interested in the issue, you
might explore the literature on so-called
2. an integer less than or equal to 5;
D E P E N D E N T T Y P E S Y S T E M S , which
3. the integer 0; provide this expanded expressivity at
4. the truth value true. the cost of much more complex type
inference computations.
OCaml’s type system isn’t expressive enough to capture these fine-
grained distinctions.8 Instead, we’ll have to deal with such anomalous
conditions using different techniques, which will be the subject of
Chapter 10.

Exercise 32
Imagine tiling a floor with square tiles of ever-increasing size, each one abutting the
previous two, as in Figure 6.4. The sides of the tiles grow according to the F I B O N A C C I
S E QU E N C E , in which each number is the sum of the previous two. By convention, the
first two numbers in the sequence are 0 and 1. Thus, the third number in the sequence is
0 + 1 = 1, the fourth is 1 + 1 = 2, and so forth.
The first 10 numbers in the Fibonacci sequence are Figure 6.4: A Fibonacci tiling.
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, . . .
The Fibonacci sequence has connections to many natural phenomena, from the
spiral structure of seashells (as alluded to in the figure) to the arrangement of seeds in a
sunflower to the growth rate of rabbits. It even relates to the golden ratio: the tiled area
depicted in the figure tends toward a golden rectangle (see Exercise 8) as more tiles are
added. (Exercise 187 explores this fact.)
Define a recursive function fib : integer -> integer that given an index into the
Fibonacci sequence returns the integer at that index. For instance,
# fib 1 ;;
- : int = 0
# fib 2 ;;
- : int = 1
# fib 8 ;;
- : int = 13

Exercise 33
Define a function fewer_divisors : int -> int -> bool, which takes two integers,
n and bound, and returns true if n has fewer than bound divisors (including 1 and n). For
example:
# fewer_divisors 17 3 ;;
- : bool = true
# fewer_divisors 4 3 ;;
- : bool = false
# fewer_divisors 4 4 ;;
- : bool = true
66 PROGRAMMING WELL

Do not worry about zero or negative arguments or divisors. Hint: You may find it useful
to define an auxiliary function to simplify the definition of fewer_divisors.

6.5 Unit testing

Having written some functions, how can we have some assurance

that our code is correct? Best might be a mathematical proof that the
code does what it’s supposed to do. Such a proof would guarantee that
the code generates the appropriate values regardless of what inputs
it is given. This is the domain of F O R M A L V E R I F I C AT I O N of software.
Unfortunately, the difficulty of providing formal specifications that can
be verified, along with the arduousness of carrying out the necessary
proofs, means that this approach to program correctness is used only
in rare circumstances.
But if we can’t have a proof that a program generates the appro-
priate values on all input values, perhaps we can at least verify that it
generates the appropriate values on some of them – even better if the
values we verify are representative of a full range of cases. This leads us
to the approach of U N I T T E S T I N G , the systematic evaluation of code
on known inputs, comparing the behavior to the expected.
Consider the fact function defined above. It exhibits the following
(correct) behavior:

# fact 1 ;;
- : int = 1
# fact 2 ;;
- : int = 2
# fact 5 ;;
- : int = 120
# fact 10 ;;
- : int = 3628800

We can describe the correctness conditions for these inputs as a series

of boolean expressions.

# fact 1 = 1 ;;
- : bool = true
# fact 2 = 2 ;;
- : bool = true
# fact 5 = 120 ;;
- : bool = true
# fact 10 = 3628800 ;;
- : bool = true

A unit testing function for fact, call it fact_test, verifies that fact
calculates the correct values for representative examples. (Let’s start
with these.) One approach is to simply evaluate each of the conditions
and make sure that they are all true.
 FUNCTIONS 67

# let fact_test () =
# fact 1 = 1
# && fact 2 = 2
# && fact 5 = 120
# && fact 10 = 3628800 ;;
val fact_test : unit -> bool = <fun>

We run the tests by calling the function:

# fact_test () ;;
- : bool = true

If all of the tests pass (as they do in this case), the testing function
returns true. If any test fails, it returns false. Unfortunately, in the
latter case it provides no help in tracking down the tests that fail.
In order to provide information about which tests have failed, we’ll
print an indicative message associated with the test. An auxiliary
function to handle the printing will be helpful:9 9
We’re making use here of two language
constructs that, strictly speaking, belong
# let unit_test (test : bool) (msg : string) : unit = in later chapters, as they involve side
# if test then effects, computational artifacts that
# Printf.printf "%s passed\ n" msg don’t affect the value expressed: the
# else sequencing operator (;) discussed in
# Printf.printf "%s FAILED\ n" msg ;; Section 15.3, and the printf function in
the Printf library module. Side effects
val unit_test : bool -> string -> unit = <fun>
in general are introduced in Chapter 15.

Now the fact_test function can call unit_test to verify each of the
conditions.
# let fact_test () =
# unit_test (fact 1 = 1) "fact 1";
# unit_test (fact 2 = 2) "fact 2";
# unit_test (fact 5 = 120) "fact 5";
# unit_test (fact 10 = 3628800) "fact 10" ;;
val fact_test : unit -> unit = <fun>

Running fact_test provides a report on the performance of fact on

each of the unit tests.
# fact_test () ;;
fact 1 passed
fact 2 passed
fact 5 passed
fact 10 passed
- : unit = ()

We’ll want to unit test fact as completely as is practicable. We can’t

test every possible input, but we can at least try examples representing
as wide a range of cases as possible. We’re missing an especially impor-
tant case, the base case for the recursion, fact 0. We’ll add a unit test
for that case:
# let fact_test () =
# unit_test (fact 0 = 1) "fact 0 (base case)";
68 PROGRAMMING WELL

# unit_test (fact 1 = 1) "fact 1";

# unit_test (fact 2 = 2) "fact 2";
# unit_test (fact 5 = 120) "fact 5";
# unit_test (fact 10 = 3628800) "fact 10" ;;
val fact_test : unit -> unit = <fun>

We haven’t tested the function on negative numbers, and probably

should. But fact as currently written wasn’t intended to handle those
cases. We postpone discussion about unit testing in such cases to
Section 10.5, when we’ll have further tools at hand. (See Exercise 83.)
Testing the hypotenuse function presents further issues. We might
want to check the simple case of the hypotenuse of a unit triangle,
whose hypotenuse ought to be about 1.41421356, as well as the limit-
ing case of a “triangle” with zero-length sides.

# let hypotenuse_test () =
# unit_test (hypotenuse 0. 0. = 0.) "hyp 0 0";
# unit_test (hypotenuse 1. 1. = 1.41421356) "hyp 1 1" ;;
val hypotenuse_test : unit -> unit = <fun>

# hypotenuse_test () ;;
hyp 0 0 passed
hyp 1 1 FAILED
- : unit = ()

The test reveals a problem. The unit triangle test has failed, not
because the hypotenuse function is wrong but because the value we’ve
proposed isn’t exactly the floating point number calculated. The float
type has a fixed capacity for representing numbers, and can’t therefore
represent all numbers exactly. The best we can do is check that floating
point calculations are approximately correct, within some tolerance.
Rather than checking the condition as above, instead we can check
that the value is within, say, 0.0001 of the value in the test, a condition
like this:

# hypotenuse 1. 1. -. 1.41421356 < 0.0001 ;;

- : bool = true

Instead of writing out these more complex conditions each time

they’re needed, we’ll devise another unit testing function for approxi-
mate floating point calculations:

# let unit_test_within (tolerance : float)

# (test_value : float)
# (expected : float)
# (msg : string)
# : unit =
# unit_test (abs_float (test_value -. expected) < tolerance) msg ;;
val unit_test_within : float -> float -> float -> string -> unit =
<fun>
 FUNCTIONS 69

We can restate the hypotenuse_test function to make use of these

approximate tests. (We’ve added a few more for other conditions.)

# let hypotenuse_test () =
# unit_test_within 0.0001 (hypotenuse 0. 0.) 0. "hyp 0 0";
# unit_test_within 0.0001 (hypotenuse 1. 1.) 1.4142 "hyp 1 1";
# unit_test_within 0.0001 (hypotenuse ~-.1. 1.) 1.4142 "hyp -1 1";
# unit_test_within 0.0001 (hypotenuse 2. 2.) 2.8284 "hyp 2 2" ;;
val hypotenuse_test : unit -> unit = <fun>

Calling the function demonstrates that all of the calculations hold

within the required tolerance.

# hypotenuse_test () ;;
hyp 0 0 passed
hyp 1 1 passed
hyp -1 1 passed
hyp 2 2 passed
- : unit = ()

We’ll return to the question of unit testing in Sections 10.5 and 17.6,
when we have more advanced tools to use.
7
Structured data and composite types

The kinds of data that we’ve introduced so far have been unstructured.
The values are separate atoms,1 discrete undecomposable units. Each 1
The term “atom” is used here in its
sense from Democritus and other clas-
integer is separate and atomic, each floating point number, each truth
sical philosophers, the indivisible units
value. But the power of data comes from the ability to build new data making up the physical universe. Now,
from old by putting together data structures. of course, we know that though chemi-
cal elements are made of atoms, those
In this chapter, we’ll introduce three quite general ways built into atoms themselves have substructure
OCaml to structure data: tuples, lists, and records. For each such way, and are not indivisible. Unlike the phys-
ical world, the world of discrete data can
we describe how to construct structures from their parts using value
be well thought of as being built from
constructors; what the associated type of the structures is and how to indivisible atoms.
construct a type expression for them using type constructors; and how
to decompose the structures into their component parts using pattern-
matching. (We turn to methods for generating your own composite
data structures in Chapter 11.) We start with tuples.

7.1 Tuples

The first structured data type is the T U P L E , a fixed length sequence

of elements. The smallest tuples are pairs, containing two elements,
then triples, quadruples, quintuples, sextuples, septuples, and so forth.
(The etymology of the term “tuple” derives from this semi-productive
suffix.)
In OCaml, a tuple value is formed using the VA L U E C O N S T RU C T O R
for tuples, an infix comma. A pair containing the integer 3 and the
truth value true, for instance, is given by 3, true. The order is crucial;
the pair true, 3 is a different pair entirely. (Indeed, as we will see,
these two pairs are not even of the same type.)
The type of a pair is determined by the types of its parts. We name
the type by forming a type expression giving the types of the parts
combined using the infix T Y P E C O N S T RU C T O R *, read “cross” (for
“cross product”). For instance, the pair 3, true is of type int * bool
72 PROGRAMMING WELL

(read, “int cross bool”).

Exercise 34
What are the types for the following pair expressions?
1. false, 5
2. false, true
3. 3, 5
4. 3.0, 5
5. 5.0, 3
6. 5, 3
7. succ, pred

Triples are formed similarly. A triple of the elements 1, 2, and

"three" would be 1, 2, "three"; its type is int * int * string.
This triple should not be confused with the pair consisting of the inte-
ger 1 and the int * string pair 2, "three". Such a pair containing
a pair is also constructable, as 1, (2, "three"), and is of type int *
(int * string). The parentheses in both the value expression and
the type expression make clear that this datum is structured as a pair,
not a triple.

Exercise 35
Construct a value for each of the following types.
1. bool * bool
2. bool * int * float
3. (bool * int) * float
4. (int * int) * int * int
5. (int -> int) * int * int
6. (int -> int) * int -> int

Exercise 36
Integer division leaves a remainder. It is sometimes useful to calculate both the result of
the quotient and the remainder. Define a function div_mod : int -> int -> (int *
int) that takes two integers and returns a pair of their division and the remainder. For
instance,
# div_mod 40 20 ;;
- : int * int = (2, 0)
# div_mod 40 13 ;;
- : int * int = (3, 1)
# div_mod 0 12 ;;
- : int * int = (0, 0)

Using this technique of returning a pair, we can get the effect of a function that returns
multiple values.

Exercise 37
In Exercise 30, you are asked to implement the computus to calculate the month and
day of Easter for a given year by defining two functions, one for the day and one for the
year. A more natural approach is to define a single function that returns both the month
and the day. Use the technique from Exercise 36 to implement a single function for
computus.
 S T RU C T U R E D D ATA A N D C O M P O S I T E T Y P E S 73

7.2 Pattern matching for decomposing data structures

The value constructor is used to construct composite values from

parts. How can we do the inverse, extracting the parts from the com-
posite structure? Perhaps surprisingly, we make use of the value con-
structor for this purpose as well, by matching a template pattern con-
taining the constructor against the structure being decomposed. The
match construction is used to perform this matching and decomposi-
tion. The general form of a match is2 2
The first vertical bar is, strictly speak-
ing, optional. We uniformly use it for
match ⟨expr ⟩ with consistency of demarcating the patterns
| ⟨pattern1 ⟩ -> ⟨expr1 ⟩ appearing on consecutive lines, as
discussed in Section B.1.7.
| ⟨pattern2 ⟩ -> ⟨expr2 ⟩
...

The structured value given by the ⟨expr ⟩ is pattern-matched against

each of the patterns ⟨pattern1 ⟩, ⟨pattern2 ⟩, and so on, in that order.
These patterns may contain variables. Whichever pattern matches
first, the variables therein name the corresponding parts of the ⟨expr ⟩
being matched against. The corresponding ⟨expri ⟩, which may use the
variables just bound by the pattern, is evaluated to provide the value of
the match construction as a whole. The variables in patterns are newly
introduced names, just like those in let and fun expressions, and like
those variables, they also have a scope, namely, the corresponding
⟨expri ⟩.
For example, suppose we want to add the integers in an integer pair.
We need to extract the integers in order to operate on them. Here is a
function that extracts the two parts of the pair and returns their sum. Figure 7.1: Computer scientist Marianne
Baudinet’s (1985) work with David
# let addpair (pair : int * int) : int = MacQueen on compiling ML-style
# match pair with pattern matching constructs to efficient
matching code proved to be the break-
# | x, y -> x + y ;;
through that made the extensive use of
val addpair : int * int -> int = <fun>
pattern matching in ML-style languages
# addpair (3, 4) ;;
practical.
- : int = 7

In the pattern x, y, the variables x and y are names that can be used
for the two components of the pair, as they have been in the expression
x + y. There is nothing special about the names x and y; any variables
could be used.
The match used here is especially simple in having just a single
pattern/result pair. Only one is needed because there is only one value
constructor for pairs. We’ll shortly see examples where more than one
pattern is used.
Notice how the match construction allows us to deconstruct a struc-
tured datum into its component parts simply by matching against
a template that uses the very same value constructor that is used to
74 PROGRAMMING WELL

build such data in the first place. This method for decomposition is
extremely general. It allows extracting the component parts from arbi-
trarily structured data.
You might think, for instance, that it would be useful to have a
function that directly extracts the first or second element of a pair. But
these can be written in terms of the match construct.3 3
The functions fst and snd are avail-
able as part of the Stdlib module, but
# let fst (pair : int * int) : int = it’s useful to see how they can be writ-
# match pair with ten in terms of the core of the OCaml
# | x, y -> x ;; language.
Line 3, characters 5-6:
3 | | x, y -> x ;;
^
Warning 27: unused variable y.
val fst : int * int -> int = <fun>
# fst (3, 4) ;;
- : int = 3

The warning message arises because the variable y appears in the

pattern, but is never used in the corresponding action. Often this is a
sign that something is awry in one’s code: Why would you establish a
variable only to ignore its value? For that reason, this warning message
can be quite useful in catching subtle bugs. But in cases like this, where
the value of the variable is really irrelevant, the warning is misleading.
To eliminate it, an A N O N Y M O U S VA R I A B L E – a variable starting with
the underscore character – can be used instead. This codifies the
programmer’s intention that the variable not be used, and disables the
warning message. This is a good example of the edict of intention: by
clearly and uniformly expressing our intention not to use a variable,
the language interpreter can help find latent bugs where we intended
to use a variable but did not (as when a variable name is misspelled).

# let fst (pair : int * int) : int =

# match pair with
# | x, _y -> x ;;
val fst : int * int -> int = <fun>
# fst (3, 4) ;;
- : int = 3

Exercise 38
Define an analogous function snd : int * int -> int that extracts the second
element of an integer pair. For instance,
# snd (3, 4) ;;
- : int = 4

As another example, consider the problem of calculating the dis-

tance between two points, where the points are given as pairs of
floats. First, we need to extract the coordinates in each dimension
by pattern matching:
 S T RU C T U R E D D ATA A N D C O M P O S I T E T Y P E S 75

let distance p1 p2 =
match p1 with
| x1, y1 ->
match p2 with
| x2, y2 -> ...calculate the distance... ;;

Rather than use two separate pattern matches, one for each argument,
we can perform both matches at once using a pattern that matches
against the pair of points p1, p2.

let distance p1 p2 =
match p1, p2 with
| (x1, y1), (x2, y2) -> ...calculate the distance... ;;

Once the separate components of the points are in hand, the distance
can be calculated:

# let distance p1 p2 =
# match p1, p2 with
# | (x1, y1), (x2, y2) ->
# sqrt ((x2 -. x1) ** 2. +. (y2 -. y1) ** 2.) ;;
val distance : float * float -> float * float -> float = <fun>

The ability to pattern match to extract and name data components

is so useful that OCaml provides syntactic sugar to integrate it into
other binding constructs, such as the let and fun constructs. In cases
where there is only a single pattern to be matched (as in the examples
above), the matching can be performed directly in the let. That is, an
expression of the form
let ⟨var ⟩ = ⟨expr ⟩ in
match ⟨var ⟩ with
| ⟨pattern1 ⟩ -> ⟨expr1 ⟩

can be “sugared” to4 4

Anonymous functions can benefit
from this syntactic sugar as well, for
let ⟨pattern1 ⟩ = ⟨expr ⟩ in instance, as in
⟨expr1 ⟩ # (fun (x, y) -> x + y) (3, 4) ;;
- : int = 7
Using this sugared form further simplifies the distance function.

let distance p1 p2 =
let (x1, y1), (x2, y2) = p1, p2 in
sqrt ((x2 -. x1) ** 2. +. (y2 -. y1) ** 2.) ;;

Finally, pattern matching can even be used in global let constructs,

to further simplify.

# let distance (x1, y1) (x2, y2) =

# sqrt ((x2 -. x1) ** 2. +. (y2 -. y1) ** 2.) ;;
val distance : float * float -> float * float -> float = <fun>

# distance (1., 1.) (2., 2.) ;;

- : float = 1.41421356237309515
76 PROGRAMMING WELL

As usual, it is useful to add typings in the global definition to make

clear the intended types of the arguments and the result:5 5
This example provides a good oppor-
tunity to mention that for readability
# let distance (x1, y1 : float * float) code lines should be kept short. We
# (x2, y2 : float * float) use a convention described in the style
# : float = guide (Section B.3.4) for breaking up
# sqrt ((x2 -. x1) ** 2. +. (y2 -. y1) ** 2.) ;; long function definition introductions.
val distance : float * float -> float * float -> float = <fun>

# distance (1., 1.) (2., 2.) ;;

- : float = 1.41421356237309515

Exercise 39
Simplify the definitions of addpair and fst above by taking advantage of this syntactic
sugar.
Using this shorthand can make code much more readable, and
is thus recommended. See the style guide (Section B.4.2) for further
discussion.

Exercise 40
Define a function slope : float * float -> float * float -> float that returns
the slope between two points.

7.2.1 Advanced pattern matching

Not only composite types can be the object of pattern matching. Pat-
terns can match particular values of atomic types as well, such as int
or bool. One could, for instance, write

# let int_of_bool (cond : bool) : int =

# match cond with
# | true -> 1
# | false -> 0 ;;
val int_of_bool : bool -> int = <fun>

# int_of_bool true ;;
- : int = 1
# int_of_bool false ;;
- : int = 0

For booleans, however, the use of a conditional is considered a better

approach:6 sing cond = true as the test part of the conditional is 6
U
redundant and stylistically poor. See Section B.5.2.

# let int_of_bool (cond : bool) : int =

# if cond then 1 else 0 ;;
val int_of_bool : bool -> int = <fun>

Integers can also be matched against:

# let is_small_int (x : int) : bool =

# match abs x with
# | 0 -> true
 S T RU C T U R E D D ATA A N D C O M P O S I T E T Y P E S 77

# | 1 -> true
# | 2 -> true
# | _ -> false ;;
val is_small_int : int -> bool = <fun>

# is_small_int ~-1 ;;
- : bool = true
# is_small_int 2 ;;
- : bool = true
# is_small_int 7 ;;
- : bool = false

Notice here the use of an anonymous variable _ as a W I L D - C A R D

pattern that matches any value.
In the is_small_int function, the same result is appropriate for
multiple patterns. Rather, than repeat the result expression in each
case, multiple patterns can be associated with a single result, by listing
the patterns interspersed with vertical bars (|).

# let is_small_int (x : int) : bool =

# match abs x with
# | 0 | 1 | 2 -> true
# | _ -> false ;;
val is_small_int : int -> bool = <fun>

7.3 Lists

Tuples are used for packaging together fixed-length sequences of

elements of perhaps differing type. L I S T S , conversely, are used for
packaging together varied-length sequences of elements all of the same
type. The type constructor list for lists thus operates on a single type,
the type of the list elements, and is written in P O S T F I X position – that
is, following its argument. For instance, the type corresponding to a list
of integers is given by the type expression int list, a list of booleans
as bool list, a list of points (pairs of floats) as (float * float)
list.
There are two value constructors for lists. The first value con-
structor, written [] and conventionally read as “ N I L ”, specifies the
empty list, that is, the list containing no elements at all. The second
value constructor, written with an infix :: and conventionally read
as “ C O N S ”,7 takes two arguments – a first element and a further list of 7
The term “cons” for this constructor
derives from the cons function in one of
elements – and specifies the list whose first element is its first argu-
the earliest and most influential func-
ment and whose remaining elements are the second. (The two parts tional programming languages, Lisp. It
of a non-empty list, the first element and the remaining elements, are derives from the idea of constructing a
list by adding a new element.
called the H E A D and the TA I L of the list, respectively.)
Suppose we want a list of integers containing just the integer 4.
Such a list can be constructed by starting with the empty list [], and
78 PROGRAMMING WELL

“consing” 4 to it as 4 :: []. The list containing, in sequence, 2 and 4

is constructed by consing 2 onto the list containing 4, that is, 2 :: (4
:: []). The list of the integers 1, 2, and 4 is analogously 1 :: (2 ::
(4 :: [])).
As usual, some notational cleanup is in order. First, we can take
advantage of the fact that the :: operator is right associative, so that
the parentheses in the lists above are not needed. We can simply write
1 :: 2 :: 4 :: []. Second, OCaml provides a more familiar alterna-
tive notation – more sugar – for lists, writing the elements of the list in
order within brackets and separated by semicolons, as [1; 2; 4]. We
can think of all of these as alternative concrete syntaxes for the same
underlying abstract structure, given by

::

1 ::

2 ::

4 []

You can verify the equivalence of these notations by entering them into
OCaml:
# 1 :: (2 :: (4 :: [])) ;;
- : int list = [1; 2; 4]
# 1 :: 2 :: 4 :: [] ;;
- : int list = [1; 2; 4]
# [1; 2; 4] ;;
- : int list = [1; 2; 4]

Notice that in all three cases, OCaml provides the inferred type int
list and reports the value using the sugared list notation.8 8
The list containing elements, say, 1
and 2 – written [1; 2] – should not
Exercise 41 be confused with the pair of those
Which of the following expressions are well-formed, and for those that are, what are their same elements – written (1, 2). The
types and how would their values be written using the sugared notation? concrete syntactic differences may
be subtle (semicolon versus comma;
1. 3 :: []
brackets versus parentheses) but their
2. true :: false respective types make the distinction
3. true :: [false] quite clear.
4. [true] :: [false]
5. [1; 2; 3.1416]
6. [4; 2; -1; 1_000_000]
7. ([true], false)

Using the :: and bracketing notations, we can construct lists from

their elements. How can we extract those elements from lists? As al-
ways in OCaml, decomposing structured data is done with pattern-
matching; no new constructs are needed. We’ll see examples shortly.
 S T RU C T U R E D D ATA A N D C O M P O S I T E T Y P E S 79

7.3.1 Some useful list functions

To provide some intuition with list processing, we’ll construct a few

useful functions, starting with a function to determine if an integer list
is empty or not. We start with considering the type of the function. Its
argument should be an integer list (of type int list) and its result a
truth value (of type bool), so the type of the function itself is int list
-> bool. This type information is just what we need in order to write
the first line of the function definition, naming the function’s argument
and incorporating the typing information:

let is_empty (lst : int list) : bool = ...

Now we need to determine whether lst is empty or not, that is, what
value constructor was used to construct it. We can do so by pattern
matching lst against a series of patterns. Since lists have only two
value constructors, two patterns will be sufficient.

let is_empty (lst : int list) : bool =

match lst with
| [] -> ...
| head :: tail -> ...

What should we do in these two cases? In the first case, we can con-
clude that lst is empty, and hence, the value of the function should 9
We’ve used alignment of the arrows
be true. In the second case, lst must have at least one element (now in the pattern match to emphasize the
named head by the pattern match), and is thus non-empty; the value parallelism between these two cases.
See the discussion in the style guide
of the function should be false.9 (Section B.1.7) for differing views on this
practice.
# let is_empty (lst : int list) : bool =
# match lst with
# | [] -> true
# | head :: tail -> false ;;
Line 4, characters 2-6:
4 | | head :: tail -> false ;;
^^^^
Warning 27: unused variable head.
Line 4, characters 10-14:
4 | | head :: tail -> false ;;
^^^^
Warning 27: unused variable tail.
val is_empty : int list -> bool = <fun>
10
The “wild card” anonymous variable
Since neither head nor tail are used in the second pattern match, _ is special in not serving as a name

they should be made anonymous variables to codify that intention that can be later referred to, and is thus
10 allowed to be used more than once in a
(and avoid a warning message). pattern.
# let is_empty (lst : int list) : bool =
# match lst with
# | [] -> true
# | _ :: _ -> false ;;
val is_empty : int list -> bool = <fun>
80 PROGRAMMING WELL

# is_empty [] ;;
- : bool = true
# is_empty [1; 2; 3] ;;
- : bool = false
# is_empty (4 :: []) ;;
- : bool = false

Sure enough, the function works well on the test cases.

Let’s try another example: calculating the L E N G T H of a list, the
count of its elements. We use the same approach, starting with the
type of the function. The argument is an int list as before, but the
result type is an int providing the count of the elements; overall, the
function is of type int list -> int. The type of the function in
hand, the first line writes itself.
let length (lst : int list) : int = ...

And again, a pattern match on the sole argument is a natural first step
to decide how to proceed in the calculation.
let length (lst : int list) : int =
match lst with
| [] -> ...
| hd :: tl -> ...

In the first match case, the list is empty; hence its length is 0.
let length (lst : int list) : int =
match lst with
| [] -> 0
| hd :: tl -> ...

The second case is more subtle, however. The length must be at

least 1 (since the list at least has the single element hd). But the length
11
of the list overall depends on tl, and in particular, the length of tl. If As with the definition of the recursive
factorial function in Section 6.4, the
only we had a method for calculating the length of tl. well-founded basis of this recursive
But we do; the length function itself can be used for this purpose! definition depends on the recursive
calls heading in the direction of the base
Indeed, the whole point of length is to calculate lengths of int lists
case. In this case, the recursive applica-
like tl. We can call length recursively on tl, and add 1 to the result to tion of the function is to a smaller data
calculate the length of the full list lst. 11 structure, the tail of the original argu-
ment, and all further applications will
# let rec length (lst : int list) : int = similarly be to smaller and smaller data
# match lst with structures. This process can’t continue
# | [] -> 0 indefinitely. Inevitably, it will bottom
out in application to the empty list, at
# | _hd :: tl -> 1 + length tl ;;
which point the computation is non-
val length : int list -> int = <fun>
recursive and terminates. Recursive
computation may seem a bit magical
(We’ve made _hd an anonymous variable for the same reasons as when you first confront it, but over time
above, and also inserted the rec keyword to allow the recursive refer- it becomes a powerful tool natural to
deploy.
ence to length within the definition.)
We can test the function on a few examples to demonstrate it.
 S T RU C T U R E D D ATA A N D C O M P O S I T E T Y P E S 81

# length [1; 2; 4] ;;
- : int = 3
# length [] ;;
- : int = 0
# length [[1; 2; 4]] ;;
Line 1, characters 8-17:
1 | length [[1; 2; 4]] ;;
^^^^^^^^^
Error: This expression has type 'a list
but an expression was expected of type int

Exercise 42
Why does this last example cause an error, given that its input is a list of length one?
Chapter 9 addresses this problem more thoroughly.
As a final example, we’ll implement a function that, given a list of
pairs of integers, returns the list of products of the pairs. For example,
the following behaviors should hold.
# prods [2,3; 4,5; 0,10] ;;
- : int list = [6; 20; 0]
# prods [] ;;
- : int list = []

By now the process should be familiar. Start with the type of the
function: (int * int) list -> int list. Use the type to write the
function introduction:
let rec prods (lst : (int * int) list) : int list = ...

Use pattern-matching to decompose the argument:

let rec prods (lst : (int * int) list) : int list =
match lst with
| [] -> ...
| hd :: tl -> ...

In the first pattern match, the list is empty; we should thus return the
empty list of products.
let rec prods (lst : (int * int) list) : int list =
match lst with
| [] -> []
| hd :: tl -> ...

Finally, we get to the tricky bit. If the list is nonempty, the head will be
a pair of integers, which we’ll want access to. We could pattern match
against hd to extract the parts:
let rec prods (lst : (int * int) list) : int list =
match lst with
| [] -> []
| hd :: tl ->
match hd with
| (x, y) -> ...
82 PROGRAMMING WELL

but it’s simpler to fold that pattern match into the list pattern match
itself:

let rec prods (lst : (int * int) list) : int list =

match lst with
| [] -> []
| (x, y) :: tl -> ...

Now, the result in the second pattern match should be a list of integers,
the first of which is x * y and the remaining elements of which are the
products of the pairs in tl. The latter can be computed recursively as
prods tl. (It’s a good thing we thought ahead to use the rec keyword.)
Finally, the list whose first element is x * y and whose remaining
elements are prods tl can be constructed as x * y :: prods tl.

# let rec prods (lst : (int * int) list) : int list =

# match lst with
# | [] -> []
# | (x, y) :: tl -> x * y :: prods tl ;;
val prods : (int * int) list -> int list = <fun>
# prods [2,3; 4,5; 0,10] ;;
- : int list = [6; 20; 0]
# prods [] ;;
- : int list = []

You’ll have noticed a common pattern to writing these functions,

one that is widely applicable.

1. Write down some examples of the function’s use.

2. Write down the type of the function.

3. Write down the first line of the function definition, based on the
type of the function, which provides the argument and result types.

4. Using information about the argument types, decompose one or

more of the arguments.

5. Solve each of the subcases, paying attention to the types, to con-

struct the output value.

6. Test the examples from Step 1.

Using this S T RU C T U R E - D R I V E N P R O G R A M M I N G pattern can make

it so that simple functions of this sort almost write themselves. No-
tice the importance of types in the process. The types constrain so
many aspects of the function that they provide a guide to writing the
function itself.

Exercise 43
Define a function sum : int list -> int that computes the sum of the integers in its
list argument.
 S T RU C T U R E D D ATA A N D C O M P O S I T E T Y P E S 83

# sum [1; 2; 4; 8] ;;
- : int = 15

What should this function return when applied to the empty list?

Exercise 44
Define a function prod : int list -> int that computes the product of the integers
in its list argument.
# prod [1; 2; 4; 8] ;;
- : int = 64

What should this function return when applied to the empty list?

Exercise 45
Define a function sums : (int * int) list -> int list, analogous to prods
above, which computes the list each of whose elements is the sum of the elements of the
corresponding pair of integers in the argument list. For example,
# sums [2,3; 4,5; 0,10] ;;
- : int list = [5; 9; 10]
# sums [] ;;
- : int list = []

Exercise 46
Define a function inc_all : int list -> int list, which increments all of the
elements in a list.
# inc_all [1; 2; 4; 8] ;;
- : int list = [2; 3; 5; 9]

Exercise 47
Define a function square_all : int list -> int list, which squares all of the
elements in a list.
# square_all [1; 2; 4; 8] ;;
- : int list = [1; 4; 16; 64]

Exercise 48
Define a function append : int list -> int list -> int list to append two
integer lists. Some examples:
# append [1; 2; 3] [4; 5; 6] ;;
- : int list = [1; 2; 3; 4; 5; 6]
# append [] [4; 5; 6] ;;
- : int list = [4; 5; 6]
# append [1; 2; 3] [] ;;
- : int list = [1; 2; 3]

Exercise 49
Define a function concat : string -> string list -> string, which takes a
string sep and a string list lst, and returns one string with all the elements of lst
concatenated together but separated by the string sep.12 Some examples: 12
The OCaml library module String
# concat ", " ["first"; "second"; "third"] ;; already provides just this function under
- : string = "first, second, third" the same name.
# concat "..." ["Moo"; "Baa"; "Lalala"] ;;
- : string = "Moo...Baa...Lalala"
# concat ", " [] ;;
- : string = ""
# concat ", " ["Moo"] ;;
- : string = "Moo"

We’ve gone through the valuable exercise of writing a bunch of

useful list functions. But list processing is so ubiquitous that OCaml
provides a library module for just such functions. We’ll discuss the
List module further in Section 9.4.
84 PROGRAMMING WELL

7.4 Records

Tuples and lists use the order within a sequence to individuate their
elements. An alternative, R E C O R D S , name the elements, providing
each with a unique label. The type constructor specifies the labels and
the type of element associated with each. For instance, suppose we’d
like to store information about people: first and last name and year of
birth. An appropriate record type would be

{lastname : string; firstname : string; birthyear : int}

Each of the elements in a record is referred to as a F I E L D . Since the

fields are individuated by their label, the order in which they occur is
immaterial; the same type could have been specified reordering the
fields as

{firstname : string; birthyear : int; lastname : string}

with no difference (except perhaps to add a bit of confusion to a reader

expecting a more systematic ordering).
Unlike lists and tuples, which are built-in types in OCaml, particular
record types are user-defined. OCaml needs to know about the type –
its fields, their labels and types – in order to make use of them. Records
are the first of the user-defined types we’ll explore in detail in Chap-
ter 11. To define a new type, we use the type construction to give the
type a name:

type ⟨typename ⟩ = ⟨typeexpr ⟩

13
We might name the type above person: A common confusion when first using
record types concerns when to use :
# type person = {lastname : string; and when to use = within fields. Here’s a
# firstname : string; way to think about the usages: The use
of : in record type expressions evokes
# birthyear : int} ;;
the use of : in a typing. In a sense, the
type person = { lastname : string; firstname : string; birthyear :
type constructor provides a typing for
int; } each of the fields. The use of = in record
value expressions evokes the use of = in
Now that the type is defined and OCaml is aware of its fields’ labels naming constructs.
and types, we can start constructing values of that type. To construct a
record value, we use the strikingly similar notation of placing the fields,
separated by semicolons, within braces. In record value expressions,
the label of a field is separated from its value by an =.13 We define a
value of the record type above:

# let ac =
# {firstname = "Alonzo";
# lastname = "Church";
# birthyear = 1903} ;;
val ac : person =
{lastname = "Church"; firstname = "Alonzo"; birthyear = 1903}
 S T RU C T U R E D D ATA A N D C O M P O S I T E T Y P E S 85

Notice that the type inferred for ac is person, the defined name for the
record type.
As usual, we use pattern matching to decompose a record into its
constituent parts. A simple example decomposes the ac value just
created to extract the birth year.
# match ac with
# | {lastname = lname;
# firstname = fname;
# birthyear = byear} -> byear ;;
Line 2, characters 15-20:
2 | | {lastname = lname;
^^^^^
Warning 27: unused variable lname.
Line 3, characters 12-17:
3 | firstname = fname;
^^^^^
Warning 27: unused variable fname.
- : int = 1903

The warnings remind us that there are unused variables that should be
explicitly marked as such:
# match ac with
# | {lastname = _lname;
# firstname = _fname;
# birthyear = byear} -> byear ;;
- : int = 1903

By way of example, we can define a function that takes a value

of type person and returns the person’s full name by extracting and
concatenating the first and last names.
# let fullname (p : person) : string =
# match p with
# | {firstname = fname;
# lastname = lname;
14
In fact, the _ notation isn’t necessary,
# birthyear = _byear} -> but it performs the useful role of
# fname ^ " " ^ lname ;;
capturing the programmer’s intention
that the set of fields is not complete.
val fullname : person -> string = <fun>
In fact, OCaml will provide a warning
(when properly set up) if an incomplete
This function can be used to generate the full name:
record match isn’t marked with this
# fullname ac ;; notation.
- : string = "Alonzo Church"

It’s a bit cumbersome to have to mention every field in a record

pattern match when we are interested in only a subset of the fields.
Fortunately, patterns need only specify a subset of the fields, using the
notation _ to stand for any remaining fields.14
let fullname (p : person) : string =
match p with
| {firstname = fname; lastname = lname; _} ->
fname ^ " " ^ lname ;;
86 PROGRAMMING WELL

Another simplification in record patterns, called F I E L D P U N N I N G ,

is allowed for fields in which the label and the variable name are iden-
tical. In that case, the label alone is all that is required. We can use field
punning to simplify fullname:

let fullname (p : person) : string =

match p with
| {firstname; lastname; _} ->
firstname ^ " " ^ lastname ;;

As a final simplification, the syntactic sugar allowing single-pattern

matches in let constructs allows us to eliminate the explicit match
entirely:

# let fullname ({firstname; lastname; _} : person) : string =

# firstname ^ " " ^ lastname ;;
val fullname : person -> string = <fun>

# fullname ac ;;
- : string = "Alonzo Church"

7.4.1 Field selection

Pattern matching permits extracting the values of all of the fields of a

record (or any subset). When only one field value is needed, however, a
more succinct technique suffices. The familiar dot notation from many
programming languages allows selection of a single field.

# ac.firstname ;;
- : string = "Alonzo"
# ac.birthyear ;;
- : int = 1903

Thus, the fullname function could have been written as

# let fullname (p : person) : string =

# p.firstname ^ " " ^ p.lastname ;;
val fullname : person -> string = <fun>
# fullname ac ;;
- : string = "Alonzo Church"

Which notation to use is again a design matter, which will depend

on the individual case.

7.5 Comparative summary

These three data structuring mechanisms provide three different ap-

proaches to the same idea – agglomerating a collection of elements
into a single unit. The differences arise in how the elements are indi-
viduated. In tuples and lists, an element is individuated by its index
 S T RU C T U R E D D ATA A N D C O M P O S I T E T Y P E S 87

in an ordered collection. In records, an element is individuated by its

label in a labeled but unordered collection.
Tuples and records collect a fixed number of elements. Because
the number of elements is fixed, they can be of differing type. The
type of the tuple or record indicates what type each element has. Lists,
on the other hand, collect an arbitrary number of elements. In order
to be able to operate on any arbitrary element, the types of all the
elements must be indicated in the type of the list itself. This constraint
is facilitated by having all elements have the same type, so that they
can be operated on uniformly.
Table 7.1 provides a summary of the differing structuring mecha-
nisms.

Tuples Records Lists

element types differing differing uniform

selected by order label order
type constructors ⟨⟩ * ⟨⟩ ⟨⟩ * ⟨⟩ * ⟨⟩ ⋯ {a : ⟨⟩ ; b : ⟨⟩ ; c : ⟨⟩ ; ...} ⟨⟩ list
value constructors ⟨⟩ , ⟨⟩ ⟨⟩ , ⟨⟩ , ⟨⟩ ⋯ {a = ⟨⟩ ; b = ⟨⟩ ; c = ⟨⟩ ; ...} [] ⟨⟩ :: ⟨⟩

Table 7.1: Comparison of three structur-

ing mechanisms: tuples, records, and
7.6 Problem set: The prisoners’ dilemma lists.

This section provides substantive background for one of the course prob-
lem sets. For common background on logistics and procedures for prob-
lem sets, including such topics as accessing the problem set code, compil-
ing and testing your code, reflecting on the process, and submitting your
solution, please review the course document “Problem set procedures”.

7.6.1 Background

I’m an apple farmer who hates apples but loves broccoli. You’re a
broccoli farmer who hates broccoli but loves apples. The obvious
solution to this sad state of affairs is for us to trade – I ship you a box of
my apples and you ship me a box of your broccoli. Win-win.
But I might try to get clever by shipping an empty box. Instead of
cooperating, I “defect”. I still get my broccoli from you (assuming you
don’t defect) and get to keep my apples. You, thinking through this
scenario, realize that you’re better off defecting as well; at least you’ll
get to keep your broccoli. But then, nobody gets what we want; we’re
both worse off. The best thing to do in this D O N AT I O N G A M E seems to
be to defect.
It’s a bit of a mystery, then, why people cooperate at all. The answer
may lie in the fact that we engage in many rounds of the game. If you
get a reputation for cooperating, others may be willing to cooperate as
well, leading to overall better outcomes for all involved.
88 PROGRAMMING WELL

The donation game is an instance of a classic game-theory thought

experiment called the P R I S O N E R ’ S D I L E M M A . A prisoner’s dilemma is
Player 2
a type of game involving two players in which each player is individ-
Cooperate Defect
ually incentivized to choose a particular action, even though it may
Cooperate (3, 3) (−2, 5)
not result in the best global outcome for both players. The outcomes Player 1
Defect (5, −2) (0, 0)
are commonly specified through a payoff matrix, such as the one in
Table 7.2. Table 7.2: Example payoff matrix for
To read the matrix, Player 1’s actions are outlined at the left and a prisoner’s dilemma. This particular
payoff matrix corresponds to a donation
Player 2’s actions at the top. The entry in each box corresponds to a game in which providing the donation
payoff to each player, depending on their respective actions. For in- (of apples or broccoli, say) costs 2 unit
and receiving the donation provides a
stance, the top-right box indicates the payoff when Player 1 cooperates
benefit of 5 units.
and Player 2 defects. Player 1 receives a payoff of −2 and Player 2 re-
ceives a payoff of 5 in that case.
To see why a dilemma arises, consider the possible actions taken
by Player 1. If Player 2 cooperates, then Player 1 should defect rather
than cooperating, since the payoff from defecting is higher (5 > 3).
If Player 2 defects, then Player 1 should again defect since the payoff
from defecting is higher (0 > −2). The same analysis applies to Player
2. Therefore, both players are incentivized to defect. However, the
payoff from both players defecting (each getting 0) is objectively worse
for both players than the payoff from both players cooperating (each
getting 3).
An I T E R AT E D P R I S O N E R ’ S D I L E M M A is a multi-round prisoner’s
dilemma, where the number of rounds is not known.15 A S T R AT E G Y 15
If the number of rounds is known
by the players ahead of time, players
specifies what action to take based on a history of past rounds of a
are again incentivized to defect for
game. We can (and will) represent a history as a list of pairs of actions all rounds. We will not delve into
(cooperate or defect) taken in the past, and a strategy as a function the reasoning here, as that is outside
the scope of this course, but it is an
from histories to actions. interesting result!
For example, a simple strategy is to ignore the histories and always
defect. We call that the “nasty” strategy. More optimistic is the “patsy”
strategy, which always cooperates.
Whereas the above analysis showed both players are incentivized
to defect in a single-round prisoner’s dilemma (leading to the nasty
strategy), that is no longer necessarily the case if there are multiple
rounds. Instead, more complicated strategies can emerge as players
can take into account the history of their opponent’s plays and their
own. A particularly effective strategy – effective because it leads to
cooperation, with its larger payoffs – is T I T- F O R - TAT . In the tit-for-tat
strategy, the player starts off by cooperating in the first round, and then
in later rounds chooses the action that the other player just played,
rewarding the other player’s cooperation by cooperating and punishing
the other player’s defection by defecting.
In this problem set, you’ll complete a simulation of the iterated
 S T RU C T U R E D D ATA A N D C O M P O S I T E T Y P E S 89

prisoner’s dilemma that allows for testing different payoff matrices and
strategies.

7.6.2 Some practice functions

To get started, you’ll write a series of functions that perform simple

manipulations over lists, strings, numbers, and booleans. (Some of
these may be useful later in implementing the iterated prisoner’s
dilemma.) See the comments in ps1.ml for the specifications. Give the
functions the names listed in the comments, as they must be named as
specified in order to compile against our automated unit tests.
The best way to learn about the core language is to work directly
with the core language features. Consequently, you should not use any
library functions in implementing your solutions to this problem set.

7.6.3 Unit testing

Thorough testing is important in all your work. Testing will help you
find bugs, avoid mistakes, and teach you the value of short, clear func-
tions. In the file ps1_tests.ml, we’ve put some prewritten tests for
one of the practice functions using the testing method of Section 6.5.
Spend some time understanding how the testing function works and
why these tests are comprehensive. Then, for each function in Prob-
lem 1, write tests that thoroughly test the functionality of each of the
remaining sections of Problem 1, covering all code paths and corner
cases.
To run your tests, run the shell command

% make ps1_tests

in the directory where your ps1.ml is located and then run the com-
piled program by executing ps1_tests.byte:

% ./ps1_tests.byte

The program should then generate a full report with all of the unit tests
for all of the functions in the problem set.

7.6.4 The prisoner’s dilemma

Having implemented some practice functions, you will apply func-

tional programming concepts to complete a model of the iterated pris-
oner’s dilemma, including the implementation of various strategies,
including one of your own devising.
Follow the comments in ps1.ml for the specifications. Feel free to
use any of the functions that you implemented in the earlier parts of
the problem set.
90 PROGRAMMING WELL

All of the programming concepts needed to do the problem set

have already been introduced. There’s no need to use later constructs,
and you should refrain from doing so. In particular, you’ll want to
avoid imperative programming, and you should not use any library
functions.

7.6.5 Tournament

To allow you to see how your custom strategy fares, we will run a
course-wide round-robin tournament in which your strategy will be
run against every other student’s strategy for a random number of
rounds. To keep things interesting, we will add a small amount of noise
to each strategy: 5% of the time, we will use the opposite action of the
one a strategy specifies.
The tournament is pseudonymized and optional. If you want to
participate, you’ll provide a pseudonym for your entrant. If you specify
an empty pseudonym, we won’t enter you in the tournament, though
we encourage you to come up with clever strategies to compete against
your fellow classmates! We’ll post and regularly update a leaderboard,
displaying each student’s pseudonym and their strategy’s average
payoff. The top score will win a rare and wonderful prize!

7.6.6 Testing

You should again provide unit tests in ps1_tests.ml for each function
that you implement in Problem 2.
In addition, you can choose to uncomment the main function in
ps1.ml and then recompile the file by running make ps1.byte in
your shell. Then, run the command ./ps1.byte and you should see
via printed output that Nasty earns a payoff of 500 and Patsy earns
a payoff of −200. You can change the strategies played to see how
different strategies perform against each other.
8
Higher-order functions and functional programming

Having laid the groundwork for programming with functions, in this

chapter we present the edict of irredundancy, and show how higher-
order functions serve as a mechanism to satisfy the edict.
Recall that abstraction is the process of viewing a set of apparently
dissimilar things as instantiating an underlying identity. Plato in his
Phaedrus has Socrates adduce two rhetorical principles. The first
Socrates describes as

That of perceiving and bringing together in one idea the scattered

particulars, that one may make clear by definition the particular thing
which he wishes to explain. (Plato, 1927)

that is, a principle of abstraction. (Socrates’s second principle shows

up at the end of this chapter.)
Abstraction in programming is this process applied to code, and
can be enabled by appropriate language constructs. Programming
abstraction is important because it enables programmers to satisfy
perhaps the most important edict of programming:

Edict of irredundancy:
Never write the same code twice.

A standard technique that beginning programmers use is “cut and

paste” programming – you find some code that does more or less
what you need, perhaps code you’ve written before, and you cut and
paste it into your program, adjusting as necessary for the context the
code now appears in. There is a high but mostly hidden cost to the
cut and paste approach. If you find a bug in one of the copies, it needs
to be fixed in all of the copies. If some functionality changes in one
of the copies, the other copies don’t benefit unless they are modified
too. As documentation is added to clarify one of the copies, it must
be maintained for all of them. When one of the copies is tested, no
assurance is thereby gained for the other copies. There’s a theme here.
92 PROGRAMMING WELL

Having written the same code twice, all of the problems of debugging,
maintaining, documenting, and testing code have been similarly
multiplied.
The edict of irredundancy is the principle of avoiding the problems
introduced by duplicative code. Rather than write the same code twice,
the edict calls for viewing the apparently dissimilar pieces of code as
instantiating an underlying identity, and factoring out the common
parts using an appropriate abstraction mechanism.
Given the emphasis in the previous chapters, it will be unsurprising
to see that the abstraction mechanism we turn to first is the function
itself. By examining some cases of similar code, we will present the
use of higher-order functions to achieve the abstraction, in so doing
presenting some of the most well known abstractions of higher-order
functional programming on lists – map, fold, and filter.

8.1 The map abstraction

In Exercises 46 and 47, you wrote functions to increment and to square

all of the elements of a list. After solving the first of these exercises with
# let rec inc_all (xs : int list) : int list =
# match xs with
# | [] -> []
# | hd :: tl -> (1 + hd) :: (inc_all tl) ;;
val inc_all : int list -> int list = <fun>

you may have thought to cut and paste the solution, modifying it
slightly to solve the second:
# let rec square_all (xs : int list) : int list =
# match xs with
# | [] -> []
# | hd :: tl -> (hd * hd) :: (square_all tl) ;;
val square_all : int list -> int list = <fun>

These “apparently dissimilar” pieces of code bear a striking resem-

blance, a result of the cutting and pasting. And to the extent that they
echo the same idea, we’ve written the same code twice, violating the
edict of irredundancy. Can we view them abstractly as “instantiating
an underlying identity”?
The differences between these functions are localized in their last
lines, where they compute the head of the output list from the head of
the input list – in inc_all as 1 + hd, in square_all as hd * hd. Do
we have a tool to characterize what is done to the head of the input list
in each case? Yes, the function! In inc_all, we are essentially applying
the function fun x -> 1 + x to the head, and in square_all, the
function fun x -> x * x. We can make this clearer by rewriting the
two snippets of code as explicit applications of a function.
 H I G H E R- O R D E R F U N C T I O N S A N D F U N C T I O N A L P RO G R A M M I N G 93

let rec inc_all (xs : int list) : int list =

match xs with
| [] -> []
| hd :: tl -> (fun x -> 1 + x) hd :: (inc_all tl) ;;

let rec square_all (xs : int list) : int list =

match xs with
| [] -> []
| hd :: tl -> (fun x -> x * x) hd :: (square_all tl) ;;

Now, we can take advantage of the fact that in OCaml functions

are first-class values, which can be used as arguments or outputs of
functions, to construct a single function that performs this general
task of applying a function, call it f, to each element of a list. We add f
as a new argument and replace the different functions being applied
to hd with this f. Historically, this abstract pattern of computation –
performing an operation on all elements of a list – is called a M A P . We
capture it in a function map that abstracts both inc_all and square_-
all.

# let rec map (f : int -> int) (xs : int list) : int list =
# match xs with
# | [] -> []
# | hd :: tl -> f hd :: (map f tl) ;;
val map : (int -> int) -> int list -> int list = <fun>

The map function takes two arguments (curried, that is, one at a time),
the first of which is itself a function, to be applied to all elements of its
second integer list argument. Its type is thus (int -> int) -> int
list -> int list. With map in hand, we can perform the equivalent
of inc_all and square_all directly.
# map (fun x -> 1 + x) [1; 2; 4; 8] ;;
- : int list = [2; 3; 5; 9]
# map (fun x -> x * x) [1; 2; 4; 8] ;;
- : int list = [1; 4; 16; 64]

In fact, map can even be used to define the functions inc_all and
square_all.

# let inc_all (xs : int list) : int list =

# map (fun x -> 1 + x) xs ;;
val inc_all : int list -> int list = <fun>
# let square_all (xs : int list) : int list =
# map (fun x -> x * x) xs ;;
val square_all : int list -> int list = <fun>

These definitions of inc_all and square_all don’t suffer from the

violation of the edict of irredundancy exhibited by our earlier ones.
By abstracting out the differences in those functions and capturing
them in a single higher-order function map, we’ve simplified each of the
definitions considerably.
94 PROGRAMMING WELL

But making full use of higher-order functions as an abstraction

mechanism allows even further simplification, via partial application.

8.2 Partial application

Although we traditionally think of functions as being able to take

more than one argument, in OCaml functions always take exactly one
argument. Here, for instance, is the power function, which appears to
take two arguments, an exponent n and a base x, and returns x n :

# let rec power (n, x) =

# if n = 0 then 1
# else x * power ((n - 1), x) ;;
val power : int * int -> int = <fun>
# power (3, 4) ;;
- : int = 64

Though it appears to be a function of two arguments, “desugaring”

makes clear that there is really only one argument. First, we desugar
the let:

let rec power =

fun (n, x) ->
if n = 0 then 1
else x * power ((n - 1), x) ;;

and then desugar the pattern match in the fun:

let rec power =

fun arg ->
match arg with
| (n, x) -> if n = 0 then 1
else x * power ((n - 1), x) ;;

demonstrating that all along, power was a function (defined with fun)
of one argument (now called arg).
How about this definition of power?

# let rec power n x =

# if n = 0 then 1
# else x * power (n - 1) x ;;
val power : int -> int -> int = <fun>
# power 3 4 ;;
- : int = 64

Again, desugaring reveals that all of the functions in the definition take
a single argument.

let rec power =

fun n ->
fun x ->
if n = 0 then 1
else x * power (n - 1) x ;;
 H I G H E R- O R D E R F U N C T I O N S A N D F U N C T I O N A L P RO G R A M M I N G 95

As described in Section 6.1, we use the term “currying” for encoding

a multi-argument function using nested, higher-order functions,
as this latter definition of power. In OCaml, we tend to use curried
functions, rather than uncurried definitions like the first definition of
power above; the whole language is set up to make that easy to do.
We can use the power function to define a function to cube num-
bers (take numbers to the third power):
# let cube x = power 3 x ;;
val cube : int -> int = <fun>
# cube 4 ;;
- : int = 64

But since power is curried, we can define the cube function even more
simply, by applying the power function to its “first” argument only.
# let cube = power 3 ;;
val cube : int -> int = <fun>
# cube 4 ;;
- : int = 64

A perennial source of confusion is that in this definition of cube by par-

tial application, no overt argument appears in the definition. There’s
no let cube x = ... here. The expression power 3 is already a func-
tion (of type int -> int). It is the cubing function, not just the result
of applying the cubing function. This is PA RT I A L A P P L I C AT I O N : the
applying of a curried function to only some of its arguments, resulting
in a function that takes the remaining arguments.
The order in which a curried function takes its arguments thus
becomes an important design consideration, as it determines what
partial applications are possible. With partial application at hand, we
can define other functions for powers of numbers. Here’s a version of
square:
# let square = power 2 ;;
val square : int -> int = <fun>
# square 4 ;;
- : int = 16

Understanding what’s going on in these examples is a good indica-

tion that you “get” higher-order functional programming. So we pause
for some exercises to provide a little practice with partial application.

Exercise 50
A T E S S E R A C T is the four-dimensional analog of a cube, so fourth powers of numbers are
sometimes referred to as T E S S E R A C T I C N U M B E R S . Use the power function to define a
function tesseract that takes its integer argument to the fourth power.
Now, map is itself a curried function and therefore can itself be par-
tially applied to its first argument. It takes its function argument and
its list argument one at a time, and applying it only to its first argu-
ment generates a function that applies that argument function to all
96 PROGRAMMING WELL

of the elements of a list. We can partially apply map to the increment

function to generate the inc_all function we had before.

# let inc_all = map (fun x -> 1 + x) ;;

val inc_all : int list -> int list = <fun>

But there are even further opportunities for partial application.1 1

Partial application takes full advantage
of the first-class nature of functions to
The addition function itself is curried, as we noted in Section 6.1. It
enable compact and elegant definitions
can thus be partially applied to one argument to form the increment of functions. However, you should be
function: (+) 1. (Recall the use of parentheses around the + operator aware that it does make type infer-
ence more difficult in the presence of
in order to allow it to be used as a normal prefix function.) Notice polymorphism, a topic we’ll discuss in
how the types work out: Both fun x -> 1 + x and (+) 1 have the Section 9.7.
same type, namely, int -> int. So the definition of inc_all can be
expressed simply is as

# let inc_all = map ((+) 1) ;;

val inc_all : int list -> int list = <fun>

# inc_all [1; 2; 4; 8] ;;
- : int list = [2; 3; 5; 9]

Similarly, square_all can be written as the mapping of the square

function:

# let square_all = map square ;;

val square_all : int list -> int list = <fun>

# square_all [1; 2; 4; 8] ;;
- : int list = [1; 4; 16; 64]

Compare this to the original definition of square_all:

# let rec square_all (xs : int list) : int list =

# match xs with
# | [] -> []
# | hd :: tl -> (hd * hd) :: (square_all tl) ;;
val square_all : int list -> int list = <fun>

Exercise 51
Use the map function to define a function double_all that takes an int list argument
and returns a list with the elements doubled.

8.3 The fold abstraction

Let’s take a look at some other functions that bear a striking resem-
blance. Exercises 43 and 44 asked for definitions of functions that took,
respectively, the sum and the product of the elements in a list. Here are
some possible solutions, written in the recursive style of Chapter 7:

# let rec sum (xs : int list) : int =

# match xs with
 H I G H E R- O R D E R F U N C T I O N S A N D F U N C T I O N A L P RO G R A M M I N G 97

# | [] -> 0
# | hd :: tl -> hd + (sum tl) ;;
val sum : int list -> int = <fun>

# let rec prod (xs : int list) : int =

# match xs with
# | [] -> 1
# | hd :: tl -> hd * (prod tl) ;;
val prod : int list -> int = <fun>

As before, note the striking similarity of these two definitions. They

differ in just two places (highlighted above): an initial value to return
on the empty list and the operation to apply to the next element of the
list and the recursively processed suffix of the list.
This abstract pattern of computation – combining all of the ele-
ments of a list one at a time with a binary function, starting with an
initial value – is called a F O L D . We repeat the abstraction process from
the previous section, defining a function called fold to capture the
abstraction.
# let rec fold (f : int -> int -> int)
# (xs : int list)
# (init : int)
# : int =
# match xs with
# | [] -> init
# | hd :: tl -> f hd (fold f tl init) ;;
val fold : (int -> int -> int) -> int list -> int -> int = <fun>

Notice the two additional arguments – f and init – which correspond

exactly to the two places that sum and prod differed.2 In summary, the 2
Ideally, these two arguments – f and
init – would be placed as the first two
type of fold is (int -> int -> int) -> int list -> int -> int.
arguments of fold so that they could be
The fold abstraction is simply the repeated embedded application conveniently partially applied. (In fact,
of a binary function, starting with an initial value, to all of the elements the Haskell functional programming
language uses this argument order for
of a list. That is, given a list of n elements [x_1, x_2, x_3, ..., their fold functions.) By convention,
x_n], the fold of a binary function f with initial value init is however, the argument order for this
fold operation in OCaml is as provided
f x_1 (f x_2 (f x_3 ( ⋯ (f x_n init)⋯))) . here, allowing for partially applying
the f argument but not init. The init
Now sum can be defined using fold: argument is placed at the end to reflect
its use as the rightmost element being
# let sum lst =
operated on during the fold. As you’ll
# fold (fun x y -> x + y) lst 0 ;;
see later, the alternative fold_left
val sum : int list -> int = <fun> function uses the Haskell argument
order.
or, noting that + is itself the curried addition function we need as the
first argument to fold:
# let sum lst = fold (+) lst 0 ;;
val sum : int list -> int = <fun>

The prod function, similarly, is a kind of fold, this time of the prod-
uct function starting with the multiplicative identity 1.
98 PROGRAMMING WELL

# let prod lst = fold ( * ) lst 1 ;;

val prod : int list -> int = <fun>

A wide variety of list functions follow this pattern. Consider taking

the length of a list, a function from Section 7.3.1.

let rec length (lst : int list) : int =

match lst with
| [] -> 0
| _hd :: tl -> 1 + length tl ;;

This function matches the fold structure as well. The initial value, the
length of an empty list, is 0, and the operation to apply to the head of
the list and the recursively processed tail is to simply ignore the head
and increment the value for the tail.

# let length lst = fold (fun _hd tlval -> 1 + tlval) lst 0 ;;
val length : int list -> int = <fun>
# length [1; 2; 4; 8] ;;
- : int = 4

The function that we’ve called fold operates “right-to-left” produc-

ing

f x_1 (f x_2 (f x_3 ( ⋯ (f x_n init)⋯))) .

For this reason, it is sometimes referred to as fold_right; in fact, that

is the name of the corresponding function in OCaml’s List module.
The symmetrical function fold_left operates left-to-right, calculat-
ing

(f ⋯ (f (f (f init x_1) x_2) x_3) x_n) .

where init is as before an initial value, and f is a binary function

taking as arguments the recursively processed prefix and the next
element in the list.

Exercise 52
Define the higher-order function fold_left : (int -> int -> int) -> int -> int
list -> int, which performs this left-to-right fold.

Because addition is associative, a list can be summed by either a

fold_right as above or a fold_left. The definition analogous to the
one using fold_right is

# let sum lst = fold_left (+) 0 lst ;;

val sum : int list -> int = <fun>

but (because the list argument of fold_left is the final argument) this
can be further simplified by partial application:

# let sum = fold_left (+) 0 ;;

val sum : int list -> int = <fun>
 H I G H E R- O R D E R F U N C T I O N S A N D F U N C T I O N A L P RO G R A M M I N G 99

Exercise 53
Define the length function that returns the length of a list, using fold_left.

Exercise 54
A cousin of the fold_left function is the function reduce,3 which is like fold_left 3
The higher-order functional program-
except that it uses the first element of the list as the initial value, calculating ming paradigm founded on functions
like map and reduce inspired the wildly
(f ⋯ (f (f x_1 x_2) x_3) x_n) . popular Google framework for parallel
Define the higher-order function reduce : (int -> int -> int) -> int list -> processing of large data sets called, not
int, which works in this way. You might define reduce recursively as we did with fold surprisingly, MapReduce (Dean and
and fold_left or nonrecursively by using fold_left itself. (By its definition reduce is Ghemawat, 2004).
undefined when applied to an empty list, but you needn’t deal with this case where it’s
applied to an invalid argument.)

8.4 The filter abstraction

The final list-processing abstraction we look at is the F I LT E R , which

serves as an abstract version of functions that return a subset of ele-
ments of a list, such as the following examples, which return the even,
odd, positive, and negative elements of an integer list.

# let rec evens xs =

# match xs with
# | [] -> []
# | hd :: tl -> if hd mod 2 = 0 then hd :: evens tl
# else evens tl ;;
val evens : int list -> int list = <fun>

# let rec odds xs =

# match xs with
# | [] -> []
# | hd :: tl -> if hd mod 2 <> 0 then hd :: odds tl
# else odds tl ;;
val odds : int list -> int list = <fun>

# let rec positives xs =

# match xs with
# | [] -> []
# | hd :: tl -> if hd > 0 then hd :: positives tl
# else positives tl ;;
val positives : int list -> int list = <fun>

# let rec negatives xs =

# match xs with
# | [] -> []
# | hd :: tl -> if hd < 0 then hd :: negatives tl
# else negatives tl ;;
val negatives : int list -> int list = <fun>

We leave the definition of an appropriate abstracted function filter

: (int -> bool) -> int list -> int list as an exercise.

Exercise 55
100 PROGRAMMING WELL

Define a function filter : (int -> bool) -> int list -> int list that returns
a list containing all of the elements of its second argument for which its first argument
returns true.

Exercise 56
Provide definitions of evens, odds, positives, and negatives in terms of filter.

Exercise 57
Define a function reverse : int list -> int list, which returns the reversal of its
argument list. Instead of using explicit recursion, define reverse by mapping, folding, or
filtering.

Exercise 58
Define a function append : int list -> int list -> int list (as described in
Exercise 48) to calculate the concatenation of two integer lists. Again, avoid explicit
recursion, using map, fold, or filter functions instead.

We’ve used the same technique three times in this chapter – notic-
ing redundancies in code and carving out the differing bits to find
the underlying commonality. Determining the best place to carve is
an important skill, the basis for R E F A C T O R I N G of code, which is the
name given to exactly this practice. And it turns out to match Socrates’s
second principle in Phaedrus:

P H A E D RU S :
And what is the other principle, Socrates?
S O C R AT E S :
That of dividing things again by classes, where the natural
joints are, and not trying to break any part after the manner of a bad
carver. (Plato, 1927)

This principle deserves its own name:

Edict of decomposition:
Carve software at its joints.

The edict of decomposition arises throughout programming prac-

tice, but plays an especial role in Chapter 18, where it motivates the
programming paradigm of object-oriented programming. For now,
however, we continue in the next chapter our pursuit of mechanisms
for capturing more abstractions, by allowing generic programs that
operate over various types, a technique called polymorphism.

8.5 Problem set 2: Higher-order functional programming

This section provides substantive background for one of the course prob-
lem sets. For common background on logistics and procedures for prob-
lem sets, including such topics as accessing the problem set code, compil-
ing and testing your code, reflecting on the process, and submitting your
solution, please review the course document “Problem set procedures”.
 H I G H E R- O R D E R F U N C T I O N S A N D F U N C T I O N A L P RO G R A M M I N G 101

This assignment focuses on programming in the functional pro-

gramming paradigm, with special attention to the idiomatic use of
higher-order functions like map, fold, and filter. In doing so, you will
exercise important features of functional languages, such as recursion,
pattern matching, and list processing.

8.5.1 Higher order functional programming

Mapping, folding, and filtering are important techniques in functional

languages that allow the programmer to abstract out the details of
traversing and manipulating lists. Each can be used to accomplish a
great deal with very little code. In this problem set, you will create a
number of functions using the higher-order functions map, filter,
and fold. In OCaml, these functions are available as List.map,
List.filter, List.fold_right, and List.fold_left in the List
module.
The file mapfold.ml contains starter code for a set of functions that
operate on lists. For each one, you are to provide the implementation
of that function using the higher-order (mapping, folding, filtering)
functions directly. The point is to use the higher-order functional
programming paradigm idiomatically. The problems will be graded
accordingly: a solution, even a working one, that does not use the
higher-order functional paradigm, deploying these higher-order func-
tions properly, will receive little or no credit. For instance, solutions
that require you to change the let to a let rec indicate that you
probably haven’t assimilated the higher-order functional paradigm.
However, you should feel free to use functions you’ve defined earlier in
the problem set to implement others later where appropriate.
Remember to provide unit tests for all of the functions in
mapfold.ml in a file mapfold_tests.ml. We have included an ex-
ample of its use in the starter code for mapfold_tests.ml. You should
provide at least one test per code path for every function that you write
on this problem set.
9
Polymorphism and generic programming

What happens when the edict of intention runs up against the edict
of irredundancy? The edict of intention calls for expressing clearly the
intended types over which functions operate, so that the language can
provide help by checking that the types are used consistently. We’ve
heeded that edict, for example, in our definition of the higher-order
function map from the previous chapter, repeated here:
# let rec map (f : int -> int) (xs : int list) : int list =
# match xs with
# | [] -> []
# | hd :: tl -> f hd :: (map f tl) ;;
val map : (int -> int) -> int list -> int list = <fun>

The map function is tremendously useful for a wide variety of opera-

tions over integer lists. It seems natural to apply the same idea to other
kinds of lists as well. For instance, we may want to define a function to
double all of the elements of a float list or implement the prods
function from Section 7.3.1 to take the products of pairs of integers in a
list of such pairs. Using map we can try
# let double = map (fun x -> 2. *. x) ;;
Line 1, characters 33-34:
1 | let double = map (fun x -> 2. *. x) ;;
^
Error: This expression has type int but an expression was expected
of type
float
# let prods = map (fun (x, y) -> x * y) ;;
Line 1, characters 21-27:
1 | let prods = map (fun (x, y) -> x * y) ;;
^^^^^^
Error: This pattern matches values of type 'a * 'b
but a pattern was expected which matches values of type int

but we run afoul of the typing constraints on map, which can only apply
functions of type int -> int, and not float -> float or int * int
-> int.
104 PROGRAMMING WELL

Of course, we can implement a version of map for lists of these types

as well:

# let rec map_float_float (f : float -> float)

# (xs : float list)
# : float list =
# match xs with
# | [] -> []
# | hd :: tl -> f hd :: (map_float_float f tl) ;;
val map_float_float : (float -> float) -> float list -> float list
= <fun>

# let rec map_intpair_int (f : int * int -> int)

# (xs : (int * int) list)
# : int list =
# match xs with
# | [] -> []
# | hd :: tl -> f hd :: (map_intpair_int f tl) ;;
val map_intpair_int : (int * int -> int) -> (int * int) list -> int
list =
<fun>

This is where we run up against the edict of irredundancy: we’ve writ-

ten the same code three times now, once for each set of argument
types.
What we’d like is a way to map functions over lists generically, while
still obeying the constraint that whatever type the list elements are,
they are appropriate to apply the function to; and whatever type the
function returns, the map returns a list of elements of that type.

9.1 Type inference and type variables

The solution to this quandary is found in type inference. In a language

with type inference, like OCaml, the type inference process combines
all of the type constraints implicit in the use of typed functions to-
gether with all of the constraints in explicit typings to compute the
types for all of the expressions in a program. For instance, in the defini-
tion

# let succ x = x + 1 ;;
val succ : int -> int = <fun>

it follows from the fact that the + function is applied to x that x must
have the same type as the argument type for +, that is, int. Similarly,
since succ x is calculated as the output of the + function, it must have
the same type as +’s output type, again int. Since succ’s argument is of
type int and output is of type int, its type must be int -> int. And
in fact that is the type OCaml reports for it, even though no explicit
typings were provided.
 P O LY M O R P H I S M A N D G E N E R I C P R O G R A M M I N G 105

Propagating type information in this way results in a fully instanti-

ated type int -> int. But what if there aren’t enough constraints in
the code to yield a fully instantiated type? The I D E N T I T Y F U N C T I O N
id, which just returns its argument unchanged, is an example:

# let id x = x ;;
val id : 'a -> 'a = <fun>

Since x is never involved in any applications in the definition of id,

there are no type constraints on it. All that we can conclude is that
whatever type x is – call it α – the id function must take values of type
α as argument and return values of type α as output. That is, id must
be of type α -> α.
The id function doesn’t have a fully instantiated type. It is a P O LY -
M O R P H I C F U N C T I O N , with a P O LY M O R P H I C T Y P E . The term poly-
morphic means many forms; the id function can take arguments of
many forms and operate on them similarly.
To express polymorphic types, we need to extend the type expres-
sion language. We use T Y P E VA R I A B L E S to specify that any type can
be used. We write type variables as identifiers with a prefixed quote
mark – ’a, ’b, ’c, and so forth – and conventionally read them as their
corresponding Greek letter – α (alpha), β (beta), γ (gamma) – as we’ve
done above. Notice that OCaml has reported a polymorphic type for
id, namely, ’a -> ’a (read, “α to α”). This type makes the claim, “for
any type α, if id is applied to an argument of type α it returns a value
of type α.”

9.2 Polymorphic map

Returning to the map function, we wanted a way to map functions over

lists generically. If we just remove the typings in the definition of map,
it would seem that we could have just such a function, a polymorphic
version of map.
# let rec map f xs =
# match xs with
# | [] -> []
# | hd :: tl -> f hd :: (map f tl) ;;
val map : ('a -> 'b) -> 'a list -> 'b list = <fun>

This function performs the same computation as the previous version

of map, just without any of the explicit type constraints enforced. The
function f is applied to elements of xs and returns elements that
appear in the result list, so the type of the argument of f must be the Figure 9.1: J. Roger Hindley (1939–
), codeveloper with Robin Milner
type of the elements of xs and the type of the result of f must be the (Figure 1.7) of the Hindley-Milner
type of the elements of the returned list simply as a consequence of the type inference algorithm that OCaml
structure of the code. relies on for inferring the most general
polymorphic types for expressions.
106 PROGRAMMING WELL

Happily, the type inference process that OCaml uses – developed by

Roger Hindley (Figure 9.1) and Robin Milner (Figure 1.7) – infers these
constraints automatically, concluding that map, like id, has a poly-
morphic type, which the OCaml type inference system has inferred
and reported as (’a -> ’b) -> ’a list -> ’b list. This type ex-
presses the constraint that “for any types α and β, if map is applied to a
function from α values to β values, it will return a function that when
given a list of α values returns a list of β values.”
This polymorphic version of map can be used to implement double
and prods as above. In each case, the types for these functions are
themselves properly inferred by instantiating the type variables of the
polymorphic map type. 1 1
Note the use of partial application in
these examples.
# let double = map (fun x -> 2. *. x) ;;
val double : float list -> float list = <fun>
# let prods = map (fun (x, y) -> x * y) ;;
val prods : (int * int) list -> int list = <fun>

As inferred by OCaml, double takes a float list argument and re-

turns a float list, and prods takes an (int * int) list argument
and returns an int list.

9.3 Regaining explicit types

By taking advantage of polymorphism in OCaml, we’ve satisfied the

edict of irredundancy by defining a polymorphic version of map. Unfor-
tunately, we seem to have forgone the edict of intention, since we are
no longer explicitly providing information about the intended type for
map.
But by using the additional expressivity provided by type variables,
we can express the intended typing for map explicitly.

# let rec map (f : 'a -> 'b) (xs : 'a list) : 'b list =
# match xs with
# | [] -> []
# | hd :: tl -> f hd :: (map f tl) ;;
val map : ('a -> 'b) -> 'a list -> 'b list = <fun>

The type variables make clear the intended constraints among f, xs,
and the return value f xs.
Problem 59
For each of the following types construct an expression for which OCaml would infer
that type. (No cheating by using explicit typing annotations with the : operator!) The ex-
pressions need not be practical or do anything useful; they need only have the requested
type. For example, for the type bool * bool, the expression true, true would be a
possible answer.

1. bool * bool -> bool

2. ’a list -> bool list
 P O LY M O R P H I S M A N D G E N E R I C P R O G R A M M I N G 107

3. (’a * ’b -> ’a) -> ’a -> ’b -> ’a

4. int * ’a * ’b -> ’a list -> ’b list
5. bool -> unit
6. ’a -> (’a -> ’b) -> ’b
7. ’a -> ’a -> ’b

Exercise 60
Define polymorphic versions of fold and filter, providing explicit polymorphic typing
information.

Exercise 61
Perhaps surprisingly, the map function can itself be written in terms of fold. Provide a
definition for map that involves just a single call to fold.
Problem 62
For each of the following definitions of a function f, give its most general type (as would
be inferred by OCaml) or explain briefly why no type exists for the function.

1. let f x =
x +. 42. ;;

2. let f g x =
g (x + 1) ;;

3. let f x =
match x with
| [] -> x
| h :: t -> h ;;

4. let rec f x a =
match x with
| [] -> a
| h :: t -> h (f t a) ;;

5. let f x y =
match x with
| (w, z) -> if w then y z else w ;;

6. let f x y =
x y y ;;

7. let f x y =
x (y y) ;;

8. let rec f x =
match x with
| None
| Some 0 -> None
| Some y -> f (Some (y - 1)) ;;

9. let f x y =
if x then [x]
else [not x; y] ;;

9.4 The List library

One way, perhaps the best, for satisfying the edict of irredundancy
is to avoid writing the same code twice by not writing the code even
once, instead taking advantage of code that someone else has already
written. OCaml, like many modern languages, comes with a large set
of libraries (packaged as modules, which we’ll cover in Chapter 12)
that provide a wide range of functions. The List module in particular
provides exactly the higher-order list processing functions presented
108 PROGRAMMING WELL

in this and the previous chapter as polymorphic functions. The docu-

mentation for the List module gives typings and descriptions for lots
of useful list processing functions. For instance, the module provides
the map, fold, and filter abstractions of Chapter 8, described in the
documentation as

• map : (’a -> ’b) -> ’a list -> ’b list

map f [a1; ...; an] applies function f to a1, . . . , an, and builds
the list [f a1; ...; f an] with the results returned by f. Not
tail-recursive.2 2
We’ll come back to the issue of tail
recursion in Section 16.2.2.
• fold_left : (’a -> ’b -> ’a) -> ’a -> ’b list -> ’a
fold_left f a [b1; ...; bn] is f (... (f (f a b1) b2)
...) bn.

• fold_right : (’a -> ’b -> ’b) -> ’a list -> ’b -> ’b

fold_right f [a1; ...; an] b is f a1 (f a2 (... (f an b)
...)). Not tail-recursive.

• filter : (’a -> bool) -> ’a list -> ’a list

filter p l returns all the elements of the list l that satisfy the
predicate p. The order of the elements in the input list is preserved.

They can be invoked as List.map, List.fold_left, and so forth. The

library provides many other useful functions, including

• append : ’a list -> ’a list -> ’a list

Concatenate two lists. Same as the infix operator @.. . .

• partition : (’a -> bool) -> ’a list -> ’a list * ’a

list

partition p l returns a pair of lists (l1, l2), where l1 is the

list of all the elements of l that satisfy the predicate p, and l2 is the
list of all the elements of l that do not satisfy p. The order of the
elements in the input list is preserved.

The List library has further functions for sorting, combining, and
transforming lists in all kinds of ways.
Although these functions are built into OCaml through the List
library, it’s still useful to have seen how they are implemented and
why they have the types they have. In particular, it makes clear that
the power of list processing via higher-order functional programming
doesn’t require special language constructs; they arise from the in-
teractions of simple language primitives like first-class functions and
structured data types.
 P O LY M O R P H I S M A N D G E N E R I C P R O G R A M M I N G 109

Problem 63
Provide an implementation of the List.map function over a list using only a call to
List.fold_right over the same list, or provide an argument for why it’s not possible to
do so.

Problem 64
Provide an implementation of the List.fold_right function using only a call to
List.map over the same list, or provide an argument for why it’s not possible to do so.

Problem 65
In the list module, OCaml provides a function partition : (’a -> bool) -> ’a
list -> ’a list * ’a list. According to the OCaml documentation, “partition p
l returns a pair of lists (l1, l2), where l1 is the list of all the elements of 1 that satisfy
the predicate p, and 12 is the list of all the elements of l that do not satisfy p. The order of
the elements in the input list is preserved.”
For example, we can use this to divide a list into two new ones, one containing the
even numbers and one containing the odd numbers:
# List.partition (fun n -> n mod 2 = 0)
# [1; 2; 3; 4; 5; 6; 7] ;;
- : int list * int list = ([2; 4; 6], [1; 3; 5; 7])

As described above, the List module provides the partition function of type (’a ->
bool) -> ’a list -> ’a list * ’a list. Give your own definition of partition,
implemented directly without the use of any library functions except for those in the
Stdlib module.

Exercise 66
Define a function permutations : ’a list -> ’a list list, which takes a list of
values and returns a list containing every permutation of the original list. For example,
# permutations [1; 2; 3] ;;
- : int list list =
[[1; 2; 3]; [2; 1; 3]; [2; 3; 1]; [1; 3; 2]; [3; 1; 2]; [3; 2; 1]]

It doesn’t matter what order the permutations appear in the returned list. Note that if
the input list is of length n, then the answer should be of length n ! (that is, the factorial
of n). Hint: One way to do this is to write an auxiliary function, interleave : int ->
int list -> int list list, that yields all interleavings of its first argument into its
second. For example:
# interleave 1 [2; 3] ;;
- : int list list = [[1; 2; 3]; [2; 1; 3]; [2; 3; 1]]

9.5 Problem section: Function composition

The C O M P O S I T I O N of two unary functions f and g is the function that

applies f to the result of applying g to its argument.
For example, suppose you’re given a list of pairs of integers, where
we think of each pair as containing a number and a corresponding
weight. We’d like to compute the W E I G H T E D S U M of the numbers, that
is, the sum of the numbers where each has been weighted according
to (that is, multiplied by) its weight. Recall the sum function from
Exercise 43 and the prods function from Section 7.3.1. The weighted
average of a pair-list can be computed by applying the sum function to
the result of applying the prods function to the list. Thus, weighted_-
sum is just the composition of sum and prods.
110 PROGRAMMING WELL

Problem 67
Provide an OCaml definition for a higher-order function @+ that takes two functions
as arguments and returns their composition. The function should have the following
behavior:
# let weighted_sum = sum @+ prods ;;
val weighted_sum : (int * int) list -> int = <fun>
# weighted_sum [(1, 3); (2, 4); (3, 5)] ;;
- : int = 26

Notice that by naming the function @+, it is used as an infix, right-associative operator.
See the operator table in the OCaml documentation for further information about the
syntactic properties of operators. When defining the function itself, though, you’ll want
to use it as a prefix operator by wrapping it in parentheses, as (@+).
Problem 68
What is the type of the @+ function?

9.6 Problem section: Credit card numbers and the Luhn

check

Here’s an interesting bit of trivia: Not all credit card numbers are well-
formed. The final digit in a 16-digit credit card number is in fact a
C H E C K S U M , a digit that is computed from the previous 15 by a simple
algorithm.
The algorithm used to generate the checksum is called the L U H N
C H E C K. To calculate the correct final checksum digit used in a 16-digit
credit card number, you perform the following computation on the
first 15 digits of the credit card number:

1. Take all of the digits in an odd-numbered position (the leftmost

digit being the first, not the zero-th digit, hence an odd-numbered
one) and double them, subtracting nine if the doubling is greater Figure 9.2: A sample credit card

than nine (called “casting out nines”).

As an example, we’ll use the (partial) credit card number from the
card in Figure 9.2:

4275 3156 0372 549x

Here, the odd-numbered digits (4, 7, 3, 5, 0, 7, 5, and 9) have been

underlined. We double them and cast out nines to get 8, 5, 6, 1, 0, 5,
1, and 9.

2. Add all of the even position digits and the doubled odd position
digits together. For the example above, the sum would be

(2 + 5 + 1 + 6 + 3 + 2 + 4) + (8 + 5 + 6 + 1 + 0 + 5 + 1 + 9) = 23 + 35 = 58 .

3. The checksum is then the digit that when added to this sum makes
it a multiple of ten. In the example above the checksum would be
2, since adding 2 to 58 generates 60, which is a multiple of 10. Thus,
 P O LY M O R P H I S M A N D G E N E R I C P R O G R A M M I N G 111

the sequence 4275 3156 0372 5492 is a valid credit card number, but
changing the last digit to any other makes it invalid. (In particular,
the final 3 in the card in Figure 9.2 is not the correct checksum!)

Problem 69
Define an explicitly recursive polymorphic function odds to extract the elements at
odd-numbered indices in a list, where the indices are counted starting with 1, so that
# let cc = [4; 2; 7; 5; 3; 1; 5; 6; 0; 3; 7; 2; 5; 4; 9] ;;
val cc : int list = [4; 2; 7; 5; 3; 1; 5; 6; 0; 3; 7; 2; 5; 4; 9]
# odds cc ;;
- : int list = [4; 7; 3; 5; 0; 7; 5; 9]

Exercise 70
What is the type of the odds function?
In addition to odds, it will be useful to have a function evens that
extracts the elements at even-numbered indices in a list.
Problem 71
Define evens to extract the elements at even-numbered indices in a list, where the
indices are counted starting with 1, so that
# let cc = [4; 2; 7; 5; 3; 1; 5; 6; 0; 3; 7; 2; 5; 4; 9] ;;
val cc : int list = [4; 2; 7; 5; 3; 1; 5; 6; 0; 3; 7; 2; 5; 4; 9]
# evens cc ;;
- : int list = [2; 5; 1; 6; 3; 2; 4]

The process of doubling a number and “casting out nines” is easy to

implement as well. Here is some code to do that:
# let doublemod9 (n : int) : int =
# (n * 2 - 1) mod 9 + 1 ;;
val doublemod9 : int -> int = <fun>

Finally, it will be useful to have a function to sum a list of integers.

Problem 72
Implement the function sum using the tail-recursive List module function fold_left.

All the parts are now in place to implement the Luhn check algo-
rithm.
Problem 73
Implement a function luhn that takes a list of integers and returns the check digit
for that digit sequence. (You can assume that it is called with a list of 15 integers.) For
instance, for the example above
# luhn cc ;;
- : int = 2

You should feel free to use the functions evens, odds, doublemod9, sum, and any other
OCaml library functions that you find useful and idiomatic.
Problem 74
Now that you know how to generate valid credit card numbers not your own, do you
think it would be legal for you to use these numbers on an e-commerce web site to test
whether your implementation is correct? Would it be ethical for you to do so?

9.7 Weak type variables

The List module provides polymorphic hd and tl functions for ex-

tracting the head and tail of a list.
112 PROGRAMMING WELL

Exercise 75
What are the types of the hd and tl functions? See if you can determine them without
looking them up.
These can be composed to allow, for instance, extracting the head of
the tail of a list, that is, the list’s second item.
# let second = List.hd @+ List.tl ;;
val second : '_weak1 list -> '_weak1 = <fun>

This definition works,

# second [1; 2; 3] ;;
- : int = 2

but why did the typing of second have those oddly named type vari-
ables?
Type variables like ’_weak1 (with the initial underscore) are W E A K
T Y P E VA R I A B L E S , not true type variables. They maintain their poly-
morphism only temporarily, until the first time they are applied. Weak
type variables arise because in certain situations OCaml’s type infer-
ence can’t figure out how to express the most general types and must
resort to this fallback approach.
When a function with these weak type variables is applied to argu-
ments with a specific type, the polymorphism of the function disap-
pears. Having applied second to an int list, OCaml further instanti-
ates the type of second to only apply to int list arguments, losing its
polymorphism. We can see this in two ways, first by checking its type
directly,
# second ;;
- : int list -> int = <fun>

and second by attempting to apply it to a list of another type,

# second [1.0; 2.1; 3.2] ;;
Line 1, characters 8-11:
1 | second [1.0; 2.1; 3.2] ;;
^^^
Error: This expression has type float but an expression was
expected of type
int

To correct the problem, you can of course add in specific typing

information
# let second : float list -> float =
# List.hd @+ List.tl ;;
val second : float list -> float = <fun>

but this provides no polymorphism. Alternatively, you can provide a

full specification of the call pattern in the definition rather than the
partial application that was used above:
 P O LY M O R P H I S M A N D G E N E R I C P R O G R A M M I N G 113

# let second x = (List.hd @+ List.tl) x ;;

val second : 'a list -> 'a = <fun>

which gives OCaml sufficient hints to infer types more generally. Of

course, in this case, the composition operator isn’t really helping. We
might as well have defined second more directly as

# let second x = List.hd (List.tl x) ;;

val second : 'a list -> 'a = <fun>

For the curious, if you want to see what’s going on in detail, you can
check out the discussion in the section “A function obtained through
partial application is not polymorphic enough” in the OCaml FAQ.
10
Handling anomalous conditions

Despite best efforts, on occasion a condition arises – let’s call it an

A N O M A LY – that a function can’t handle. What to do? In this chap-
ter, we present two approaches. The function can return a value that
indicates the anomaly, thereby handling the anomaly explicitly. Alter-
natively, the function can stop normal execution altogether, throwing
control to some handler of the anomaly. In OCaml, the first approach
involves option types, the second exceptions.
As a concrete example, consider a function to calculate the M E D I A N
number in a list of integer values, that is, the value that has an equal
number of smaller and larger values. The median can be calculated by
sorting all of the values in the list and taking the middle element of the
sorted list. Taking advantage of a few functions from the List module
(sort, length, and nth) and the Stdlib module (compare), 1 we can 1
Since we make heavy use of the List
module functions in this chapter, we
define
will open the module (but preserve
compare as the Stdlib version)
# let median (lst: 'a list) : 'a =
# open List ;;
# nth (sort compare lst) (length lst / 2) ;; # let compare = Stdlib.compare ;;
val median : 'a list -> 'a = <fun> val compare : 'a -> 'a -> int = <fun>

so as to avoid having to prefix each use

of the functions with the List. module
We can test it out on a few lists:
qualifier. The issue will become clearer
when modules are fully introduced in
# median [1; 5; 9; 7; 3] ;; Chapter 12.
- : int = 5
# median [1; 2; 3; 4; 3; 2; 1] ;;
- : int = 2
# median [1; 1; 1; 1; 1] ;;
- : int = 1
# median [7] ;;
- : int = 7

The function works fine most of the time, but there is one anoma-
lous condition to consider, where the median isn’t well defined: What
should the median function do on the empty list?
116 PROGRAMMING WELL

10.1 A non-solution: Error values

You might have thought to return a special E R R O R VA L U E in the

anomalous case. Perhaps 0 or -1 or MAX_INT come to mind as pos-
sible error values. Augmenting the code to return a globally defined
error value might look like this:
# let cERROR = -1 ;;
val cERROR : int = -1

# let median (lst: 'a list) : 'a =

# if lst = [] then cERROR
# else nth (sort compare lst) (length lst / 2) ;;
val median : int list -> int = <fun>

There are two problems. First, the method can lead to gratuitous type
instantiation; second, and more critically, it manifests in-band signal-
ing.
Check the types inferred for the two versions of median above. The
original is appropriately polymorphic, of type ’a list -> ’a. But
because the error value cERROR used in the second version is of type
int, median becomes instantiated to int list -> int. The code
no longer applies outside the type of the error value, restricting its
generality and utility. And there is a deeper problem.
Consider the sad fate of poor Christopher Null, a technology jour-
nalist with a rather inopportune name. Apparently, there is a fair
amount of software that uses the string "null" as an error value for
cases in which no last name was provided. Errors can then be checked
for using code like
if last_name = "null" then ...

Maybe you see the problem. Poor Mr. Null reports that

I’ve been embroiled in a cordial email battle with Bank of Amer-

ica, literally for years, over my email address, which is simply
null@nullmedia.com. Using null as a mailbox name simply does not
work at B of A. The system will not accept it, period. (Null, 2015)

These kinds of problems confront poor Mr. Null on a regular basis.

Null has fallen afoul of I N - B A N D S I G N A L I N G of errors, in which an
otherwise valid value is used to indicate an error. The string "null"
is, of course, a valid string that, for all the programmer knows, might
be someone’s name, yet it is used to indicate a failure condition in
which no name was provided. (The solution is not to use a string,
dpfnzzlwrpf say,2 that is less likely to be someone’s last name as the 2
In fact, “Dpfnzzlwrpf” is the name
of a fictitious corporation in Jonathan
error value. That merely postpones the problem.)
Caws-Elwitt’s “Letter to a Customer”.
Similarly, 0 or -1 or MAX_INT are all possible values for the median (Conley, 2009) Could it also be a last
of an integer list. Using one of them as an in-band error value means name? Why not? For a while, it was my
username on Skype. True story.
 HANDLING ANOMALOUS CONDITIONS 117

that users of the median function can’t tell the difference between the
value being the true median or the median being undefined.
Having dismissed the in-band error signaling approach, we turn to
better solutions.

10.2 Option types

The first approach, like the in-band error value approach, still handles
the problem explicitly, right in the return value of the function. How-
ever, rather than returning an in-band value, an int (or whatever the
type of the list elements is), the function will return an out-of-band
None value, that has been added to the int type to form an optional
int, a value of type int option.
Option types are another kind of structured type, beyond the lists,
tuples, and records from Chapter 7. The postfix type constructor
option creates an option type from a base type, just as the postfix
type constructor list does. There are two value constructors for op-
tion type values: None (connoting an anomalous value), and the prefix
value constructor Some. The argument to Some is a value of the base
type.
For the median function, we’ll use an int option as the return
value, or, more generically, an ’a option. In the anomalous condition,
we return None, and in the normal condition in which a well-defined
median v can be computed, we return Some v.

# let median (lst: 'a list) : ('a option) =

# if lst = [] then None
# else Some (nth (sort compare lst) (length lst / 2)) ;;
val median : 'a list -> 'a option = <fun>

# median [1; 2; 3; 4; 42] ;;

- : int option = Some 3
# median [] ;;
- : 'a option = None

This version of the median function when applied to an int list

does not return an int, even when the median is well defined. It re-
turns an int option, which is a distinct type altogether. Nonetheless,
a caller of this function might want access to the int wrapped inside
the int option value. As with all structured types, we access the com-
ponent elements of an option value via pattern matching, as in this
example function, which replicates the (deprecated) in-band value so-
lution, returning the median of the list, or the error value if no median
exists:

# let median_or_error (lst : int list) : int =

# match median lst with
118 PROGRAMMING WELL

# | None -> cERROR

# | Some v -> v ;;
val median_or_error : int list -> int = <fun>

In implementing median above, we used the polymorphic function

nth : ’a list -> int -> ’a provided by the List module, which
given a list lst and an integer index returns the element of lst at the
given index (numbered starting with 0).
# List.nth [1; 2; 4; 8] 2 ;;
- : int = 4
# List.nth [true; false; false] 0 ;;
- : bool = true

Exercise 76
Why do you think nth was designed so as to take its list argument before its index
argument?
If we were to reimplement this function, it might look something
like this:
# let rec nth (lst : 'a list) (n : int) : 'a =
# match lst with
# | hd :: tl ->
# if n = 0 then hd
# else nth tl (n - 1) ;;
Lines 2-5, characters 0-19:
2 | match lst with
3 | | hd :: tl ->
4 | if n = 0 then hd
5 | else nth tl (n - 1)...
Warning 8: this pattern-matching is not exhaustive.
Here is an example of a case that is not matched:
[]
val nth : 'a list -> int -> 'a = <fun>

This definition works, as shown in the following examples:

# nth [1; 2; 3] 1 ;;
- : int = 2
# nth [0; 1; 2] (nth [1; 2; 3] 1) ;;
- : int = 2 3
We use the suffix _opt to mark func-
tions that return an optional value, as
However, OCaml has warned us that the pattern match in the def- is conventional in OCaml library func-
inition of nth is not exhaustive – there are possible values that will tions. In fact, as noted below, the List
module provides an nth_opt function
match none of the provided patterns – and helpfully provides the miss- in addition to its nth function.
ing case, the empty list. Of course, if we ask to take the n-th element of
an empty list, there is no element to take; this represents an anomalous
condition.
Leaving the handling of this case implicit violates the edict of inten-
tion; we should clearly express what happens in all cases. Once again,
we can use option types to explicitly mark the condition in the return
value. We do so in a function called nth_opt.3
 HANDLING ANOMALOUS CONDITIONS 119

# let rec nth_opt (lst : 'a list) (n : int) : 'a option =

# match lst with
# | [] -> None
# | hd :: tl ->
# if n = 0 then Some hd
# else nth_opt tl (n - 1) ;;
val nth_opt : 'a list -> int -> 'a option = <fun>

# nth_opt [1; 2; 3] 1 ;;
- : int option = Some 2
# nth_opt [1; 2; 3] 5 ;;
- : int option = None

Exercise 77
Another anomalous condition for nth and nth_opt is the use of a negative index. What
currently is the behavior of nth_opt with negative indices? Revise the definition of
nth_opt to appropriately handle this case as well.

Exercise 78
Define a function last_opt : ’a list -> ’a option that returns the last element in
a list (as an element of the option type) if there is one, and None otherwise.
# last_opt [] ;;
- : 'a option = None
# last_opt [1; 2; 3; 4; 5] ;;
- : int option = Some 5

Exercise 79
The variance of a sequence of n numbers x 1 , . . . , x n is given by the following equation:

∑ni=1 (x i − m )2
n −1
where n is the number of elements in the sequence, m is the arithmetic mean (or
average) of the elements in the sequence, and x i is the i -th element in the sequence.
The variance is only well defined for sequences with two or more elements. (Do you see
why?)
Define a function variance : float list -> float option that returns None
if the list has fewer than two elements. Otherwise, it should return the variance of the
numbers in its list argument, wrapped appropriately for its return type.4 For example: 4
If you want to compare your output
# variance [1.0; 2.0; 3.0; 4.0; 5.0] ;; with an online calculator, make sure you
- : float option = Some 2.5 find one that calculates the (unbiased)
# variance [1.0] ;; sample variance.
- : float option = None

Remember to use the floating point version of the arithmetic operators when operating
on floats (+., *., etc). The function float can convert (“cast”) an int to a float.

10.2.1 Option poisoning

There is a problem with using option types to handle anomalies, as in

nth_opt. Whenever we want to use the value of an nth_opt element in
a further computation, we need to carefully extract the value from the
option type. We can’t, for instance, merely write

# nth_opt [0; 1; 2] (nth_opt [1; 2; 3] 1) ;;

Line 1, characters 18-39:
1 | nth_opt [0; 1; 2] (nth_opt [1; 2; 3] 1) ;;
120 PROGRAMMING WELL

^^^^^^^^^^^^^^^^^^^^^
Error: This expression has type int option
but an expression was expected of type int

Instead we must work inside out, painstakingly extracting values and

passing on Nones:
# match (nth_opt [1; 2; 3] 1) with
# | None -> None
# | Some v -> nth_opt [0; 1; 2] v ;;
- : int option = Some 2

And if that result is part of a further computation, even something as

simple as adding 1 to it, we have to resort to
# match (nth_opt [1; 2; 3] 1) with
# | None -> None
# | Some v ->
# match nth_opt [0; 1; 2] v with
# | None -> None
# | Some v -> Some (v + 1) ;;
- : int option = Some 3

Much of the elegance of the functional programming paradigm, the

ability to simply embed function applications with other functional
applications, is lost. We call this phenomenon O P T I O N P O I S O N I N G :
The introduction of an option type in an embedded computation
requires verbose extraction of values and reinjecting them into option
types as the computation continues. (Option poisoning is a particular
instance of the dreaded programming phenomenon of the P Y R A M I D
O F D O O M .)
Functions that regularly display anomalous conditions that ought
to be directly handled by the caller are well suited for use of option
5
types. But where an anomalous condition is rare and isn’t the kind of Newer techniques, such as O P T I O N A L
CHAINING in the Swift programming
thing that the caller should handle, an alternative approach is useful, language, deal with option poisoning
to avoid the pyramid of doom. Rather than explicitly marking the in a more elegant way, providing a
middle ground between the verbose
occurrence of an anomaly in the return value, it can be implicitly dealt
option handling of OCaml and the use
with by changing the flow of control of the program entirely. This is the of exceptions. For the programming-
approach based on exceptions, to which we now turn.5 language-theory-inclined, the M O N A D
concept from category theory, first
imported into programming languages
10.3 Exceptions with Haskell, generalizes the concept.
The lesson here is that continuing
progress is being made in the design of
Instead of modifying the return type of nth to allow for returning a programming languages to deal with
None marker of an anomaly, we can leave the return type unchanged, new and recurring programming issues.

and in case of anomaly, raise an E XC E P T I O N .

When an exception is raised, execution of the function stops. Of
course, if execution stops, the function can’t return a value, which is
appropriate given that the existence of the anomaly means that there’s
no appropriate value to return.
 HANDLING ANOMALOUS CONDITIONS 121

What about the function that called the one that raised the excep-
tion? It is expecting a value of a certain type to be returned, but in this
case, no such value is supplied. The calling function thus can’t return
either. It stops too. And so on and so forth.
We can write a version of nth that raises an exception when the
index is too large.
# let rec nth (lst : 'a list) (n : int) : 'a =
# match lst with
# | [] -> raise Exit
# | hd :: tl ->
# if n = 0 then hd
# else nth tl (n - 1) ;;
val nth : 'a list -> int -> 'a = <fun>

# nth [1; 2; 3] 1 ;;
- : int = 2
# nth [1; 2; 3] 5 ;;
Exception: Stdlib.Exit.
# (nth [0; 1; 2] (nth [1; 2; 3] 1)) + 1 ;;
- : int = 3

There are several things to notice here. First, the return type of nth re-
mains ’a, not ’a option. Under normal conditions, it returns the n-th
element itself, not an option-wrapped version thereof. This allows its
use in embedded applications (as in the third example above) without
leading to the dreaded option poisoning. When an error does occur, as
in the second example, execution stops and a message is printed by the
OCaml R E P L (“Exception: Stdlib.Exit.”) describing the exception
that was raised, namely, the Exit exception defined in the Stdlib li-
brary module. No value is returned from the computation at all, so no
value is ever printed by the R E P L .
The code that actually raises the Exit exception is in the third line
of nth: raise Exit. The built-in raise function takes as argument an
expression of type exn, the type for exceptions. As it turns out, Exit is
a value of that type, as can be verified directly:
# Exit ;;
- : exn = Stdlib.Exit

The Exit exception is provided in the Stdlib module as a kind of

catch-all exception, but other exceptions are more appropriate to raise
in different circumstances.

• The value constructor Invalid_argument : string -> exn, is

intended for use when an argument to a function is inappropriate.
It would be appropriate to use when the index of nth is negative.

• The value constructor Failure : string -> exn, is intended

for use when a function isn’t well-defined as called. It would be
122 PROGRAMMING WELL

appropriate to use when the index of nth is too large for the given
list.

Both of these constructors take a string argument, typically used to

provide an explanation of what went wrong. The explanation can be
used when the exception information is handled, for instance, by the
REPL printing its error message.
Taking advantage of these exceptions, nth can be rewritten as

# let rec nth (lst : 'a list) (n : int) : 'a =

# if n < 0 then
# raise (Invalid_argument "nth: negative index")
# else
# match lst with
# | [] -> raise (Failure "nth: index too large")
# | hd :: tl ->
# if n = 0 then hd
# else nth tl (n - 1) ;;
val nth : 'a list -> int -> 'a = <fun>

# nth [1; 2; 4; 8] ~-3 ;;

Exception: Invalid_argument "nth: negative index".
# nth [1; 2; 4; 8] 1 ;;
- : int = 2
# nth [1; 2; 4; 8] 42 ;;
Exception: Failure "nth: index too large".

We’ve dealt with both of the anomalous conditions by raising appropri-

ate exceptions.
Not coincidentally, the List.nth function (in the List library mod-
ule) works exactly this way, raising Invalid_argument and Failure
exceptions under just these circumstances. But a List.nth_opt func-
tion is also provided, for cases in which the explicit marking of anoma-
lies with an option type is more appropriate.

10.3.1 Handling exceptions

Returning to the median example above, and repeated here for refer-
ence (but this time using our own implementation of nth),

# let median (lst : 'a list) : 'a =

# nth (sort compare lst) (length lst / 2) ;;
val median : 'a list -> 'a = <fun>

this code doesn’t use option types and doesn’t use the raise func-
tion to raise any exceptions. What does happen when the anomalous
condition occurs?

# median [] ;;
Exception: Failure "nth: index too large".
 HANDLING ANOMALOUS CONDITIONS 123

An exception was raised, not by the median function, but by our nth
function that it calls, which raises a Failure exception when it is called
to take an element of the empty list. The exception propagates from
the nth call to the median call to the top level of the R E P L .
Perhaps you, as the writer of some code, have an idea about how
to handle particular anomalies that might otherwise raise an excep-
tion. Rather than allow the exception to propagate to the top level,
you might want to handle the exception yourself. The try...with
construct allows for this.
The syntax of the construction is

try ⟨expr ⟩
with ⟨match ⟩

where ⟨expr ⟩ is an expression that may raise an exception, and

⟨match ⟩ is a pattern match with one or more branches, which is used
when an exception is raised in the course of evaluating ⟨expr ⟩.
For example, we can implement nth_opt in terms of nth by embed-
ding the call to nth within a try...with:6 6
We’ve taken advantage of the ability
to use the same result expression for
multiple patterns, as described in
# let nth_opt (lst : 'a list) (index : int) : 'a option = Section 7.2.1.
# try
# Some (nth lst index)
# with
# | Failure _
# | Invalid_argument _ -> None ;;
val nth_opt : 'a list -> int -> 'a option = <fun>

# nth_opt [1; 2; 3] 0 ;;
- : int option = Some 1
# nth_opt [1; 2; 3] (-1) ;;
- : int option = None
# nth_opt [1; 2; 3] 4 ;;
- : int option = None

This implementation of nth_opt attempts to evaluate Some (nth lst

index). Under normal conditions, the call to nth returns a value v, in
which case Some v is the result of the try and of the function itself.
But if an exception is raised in the evaluation of the try – presumably
by an anomalous condition in the call to nth – the exception raised
will be matched against the two patterns and the result of that pattern
match will be used. If nth raises either a Failure exception or an
Invalid_argument exception, the result of the try...with will be
None (as is appropriate for an implementation of nth_opt). If any
other exception is raised, no pattern will match and the exception will
continue to propagate.
124 PROGRAMMING WELL

10.3.2 Zipping lists

As another example of handling anomalous conditions, we consider

a function for “zipping” lists. The result of zipping two lists together
is a list of corresponding pairs of elements of the original lists. A zip
function in OCaml ought to have the following behavior:

# zip ['a'; 'b'; 'c']

# [ 1 ; 2 ; 3 ] ;;
- : (char * int) list = [('a', 1); ('b', 2); ('c', 3)]

Let’s try to define the function, starting with its type. The zip func-
tion takes two lists, with types, say, ’a list and ’b list, and returns
a list of pairs each of which has an element from the first list (of type
’a) and an element from the second (of type ’b). The pairs are thus of
type ’a * ’b and the return value of type (’a * ’b) list. The type
of the whole function, then, is ’a list -> ’b list -> (’a * ’b)
list. From this, the header follows directly.

let rec zip (xs : 'a list)

(ys : 'b list)
: ('a * 'b) list =
...

We’ll need the first elements of each of the lists, so we match on

both lists (as a pair) to extract their parts

let rec zip (xs : 'a list)

(ys : 'b list)
: ('a * 'b) list =
match xs, ys with
| [], [] -> ...
| xhd :: xtl, yhd :: ytl -> ...

If the lists are empty, the list of pairs of their elements is empty too.

let rec zip (xs : 'a list)

(ys : 'b list)
: ('a * 'b) list =
match xs, ys with
| [], [] -> []
| xhd :: xtl, yhd :: ytl -> ...

Otherwise, the zip of the non-empty lists starts with the two heads
paired. The remaining elements are the zip of the tails.

let rec zip (xs : 'a list)

(ys : 'b list)
: ('a * 'b) list =
match xs, ys with
| [], [] -> []
| xhd :: xtl, yhd :: ytl ->
(xhd, yhd) :: (zip xtl ytl) ;;
 HANDLING ANOMALOUS CONDITIONS 125

You’ll notice that there’s an issue. And if you don’t notice, the inter-
preter will, as soon as we enter this definition:

# let rec zip (xs : 'a list)

# (ys : 'b list)
# : ('a * 'b) list =
# match xs, ys with
# | [], [] -> []
# | xhd :: xtl, yhd :: ytl ->
# (xhd, yhd) :: (zip xtl ytl) ;;
Lines 4-7, characters 0-27:
4 | match xs, ys with
5 | | [], [] -> []
6 | | xhd :: xtl, yhd :: ytl ->
7 | (xhd, yhd) :: (zip xtl ytl)...
Warning 8: this pattern-matching is not exhaustive.
Here is an example of a case that is not matched:
((_::_, [])|([], _::_))
val zip : 'a list -> 'b list -> ('a * 'b) list = <fun>

There are missing match cases, in particular, when one of the lists is
empty and the other isn’t. This can arise whenever the two lists are of
different lengths. In such a case, the zip of two lists is not well defined.
As usual, we have two approaches to addressing the anomaly, with
options and with exceptions. We’ll pursue them in order.
We can make explicit the possibility of error values by returning an
option type.

let rec zip_opt (xs : 'a list)

(ys : 'b list)
: ('a * 'b) list option = ...

The normal match cases can return their corresponding option type
value using the Some constructor.

let rec zip_opt (xs : 'a list)

(ys : 'b list)
: ('a * 'b) list option =
match xs, ys with
| [], [] -> Some []
| xhd :: xtl, yhd :: ytl ->
Some ((xhd, yhd) :: (zip_opt xtl ytl)) ;;

Finally, we can add a wild-card match pattern for the remaining cases.

# let rec zip_opt (xs : 'a list)

# (ys : 'b list)
# : ('a * 'b) list option =
# match xs, ys with
# | [], [] -> Some []
# | xhd :: xtl, yhd :: ytl ->
# Some ((xhd, yhd) :: (zip_opt xtl ytl))
# | _, _ -> None ;;
126 PROGRAMMING WELL

Line 7, characters 20-37:

7 | Some ((xhd, yhd) :: (zip_opt xtl ytl))
^^^^^^^^^^^^^^^^^
Error: This expression has type ('c * 'd) list option
but an expression was expected of type ('a * 'b) list

The interpreter tells us that there’s a type problem. The recursive

call zip_opt xtl ytl is of type (’c * ’d) list option but the
cons requires an (’a * ’b) list. What we have here is a bad case
of option poisoning. We’ll have to decompose the return value of the
recursive call to extract the list within, handling the None case at the
same time.

# let rec zip_opt (xs : 'a list)

# (ys : 'b list)
# : ('a * 'b) list option =
# match xs, ys with
# | [], [] -> Some []
# | xhd :: xtl, yhd :: ytl ->
# match zip_opt xtl ytl with
# | None -> None
# | Some ztl -> Some ((xhd, yhd) :: ztl)
# | _, _ -> None ;;
Line 10, characters 2-6:
10 | | _, _ -> None ;;
^^^^
Error: This pattern matches values of type 'a * 'b
but a pattern was expected which matches values of type
('c * 'd) list option

Now what!? The interpreter complains of another type mismatch, this

time in the final pattern, which is of type ’a * ’b, but which, for
some reason, the interpreter thinks should be of type (’c * ’d) list
option. This kind of error is one of the most confusing for beginning
OCaml programmers.

Exercise 80
Try to see if you can diagnose the problem before reading on.
The indentation of this code notwithstanding, the final pattern
match is associated with the inner match, not the outer one. The inner
match is, indeed, for list options. The intention was that only the lines
beginning | None... and | Some ... be part of that match, but the
next line has been caught up in it as well.
One simple solution is to use parentheses to make explicit the
intended structure of the code.

# let rec zip_opt (xs : 'a list)

# (ys : 'b list)
# : ('a * 'b) list option =
# match xs, ys with
# | [], [] -> Some []
 HANDLING ANOMALOUS CONDITIONS 127

# | xhd :: xtl, yhd :: ytl ->

# (match zip_opt xtl ytl with
# | None -> None
# | Some ztl -> Some ((xhd, yhd) :: ztl))
# | _, _ -> None ;;
val zip_opt : 'a list -> 'b list -> ('a * 'b) list option = <fun>

Better yet is to make explicit the patterns that fall under the wildcard
allowing them to move up in the ordering.

# let rec zip_opt (xs : 'a list)

# (ys : 'b list)
# : ('a * 'b) list option =
# match xs, ys with
# | [], [] -> Some []
# | [], _
# | _, [] -> None
# | xhd :: xtl, yhd :: ytl ->
# match zip_opt xtl ytl with
# | None -> None
# | Some ztl -> Some ((xhd, yhd) :: ztl) ;;
val zip_opt : 'a list -> 'b list -> ('a * 'b) list option = <fun>

Exercise 81

Why is it necessary to make the patterns explicit before moving them up in the ordering?
What goes wrong if we leave the pattern as _, _?
As an alternative, we can implement zip to raise an exception on
lists of unequal length. Doing so simplifies the matches, since there’s
no issue of option poisoning.

# let rec zip (xs : 'a list)

# (ys : 'b list)
# : ('a * 'b) list =
# match xs, ys with
# | [], [] -> []
# | [], _
# | _, [] -> raise (Invalid_argument
# "zip: unequal length lists")
# | xhd :: xtl, yhd :: ytl ->
# (xhd, yhd) :: (zip xtl ytl) ;;
val zip : 'a list -> 'b list -> ('a * 'b) list = <fun>

Exercise 82

Define a function zip_safe that returns the zip of two equal-length lists, returning the
empty list if the arguments are of unequal length. The implementation should call zip.

# zip_safe [1; 2; 3] [3; 2; 1] ;;

- : (int * int) list = [(1, 3); (2, 2); (3, 1)]
# zip_safe [1; 2; 3] [3; 2] ;;
- : (int * int) list = []

What problems do you see in this function?

128 PROGRAMMING WELL

10.3.3 Declaring new exceptions

Exceptions are first-class values, of the type exn. Like lists and options,
exceptions have multiple value constructors. We’ve seen some already:
Exit, Failure, Invalid_argument. (It’s for that reason that we can
pattern match against them in the try...with construct.)
Exceptions are exceptional in that new value constructors can be
added dynamically. Here we define a new exception value constructor:

# exception Timeout ;;
exception Timeout

It turns out that this exception will be used in Chapter 17.

Exception constructors can take arguments. We define an
UnboundVariable constructor that takes a string argument, used in
Chapter 13, as

# exception UnboundVariable of string ;;

exception UnboundVariable of string

Exercise 83
In Section 6.4, we noted a problem with the definition of fact for computing the
factorial function; it fails on negative inputs. Modify the definition of fact to raise an
exception to make that limitation explicit.

Exercise 84
What are the types of the following expressions (or the values they define)?

1. Some 42
2. [Some 42; None]
3. [None]
4. Exit
5. Failure "nth"
6. raise (Failure "nth")
7. raise
8. fun _ -> raise Exit
9. let failwith s =
raise (Failure s)

10. let sample x =

failwith "not implemented"

11. let sample (x : int) (b : bool) : int list option =

failwith "not implemented"

Problem 85
As in Problem 59, for each of the following OCaml function types define a function f
(with no explicit typing annotations, that is, no uses of the : operator) for which OCaml
would infer that type. (The functions need not be practical or do anything useful; they
need only have the requested type.)
1. int -> int -> int option
2. (int -> int) -> int option
3. ’a -> (’a -> ’b) -> ’b
4. ’a option list -> ’b option list -> (’a * ’b) list
 HANDLING ANOMALOUS CONDITIONS 129

Problem 86
As in Problem 62, for each of the following function definitions of a function f, give
a typing for the function that provides its most general type (as would be inferred by
OCaml) or explain briefly why no type exists for the function.
1. let rec f x =
match x with
| [] -> f
| h :: t -> raise Exit ;;

2. let f x =
if x then (x, true)
else (true, not x) ;;

Problem 87
Provide a more succinct definition of the function f from Problem 86(2), with the same
type and behavior.

10.4 Options or exceptions?

Which should you use when writing code to handle anomalous con-
ditions? Options or exceptions? This is a design decision. There is no
universal right answer.
Options are explicit: The type gives an indication that an anomaly
might occur, and the compiler can make sure that such anomalies are
handled. Exceptions are implicit: You (and the compiler) can’t tell if an
exception might be raised while executing a function. But exceptions
are therefore more concise. The error handling doesn’t impinge on the
data and so doesn’t poison every downstream use of the data. Code to
handle the anomaly doesn’t have to exist everywhere between where
the problem occurs and where it’s dealt with.
Which is more important, explicitness or concision? It depends.

• If the anomaly is a standard part of the computation, a frequent

occurrence, that argues for making it explicit in an option type.

• If the anomaly is a rare occurrence, that argues for hiding it implic-

itly in the code.

• If the anomaly is localized to a small part of the code within which it

can be handled, it makes sense to use an option type in that region.

• If the anomaly is ubiquitous, with the possibility of occurring any-

where in the code, the overhead of explicitly handling it everywhere
in the code with an option type is likely too cumbersome. For ex-
ample, a computation may run out of memory at more or less any
point. It makes no sense to have a function return an option type,
with None reserved for the case where the computation happened to
run out of memory in the function. Rather, running out of memory
is a natural use for an exception (and in fact, OCaml raises excep-
tions when it runs out of memory).
130 PROGRAMMING WELL

Is the anomalous occurrence a frequent case? Use options. A rare

event? Use exceptions. Is the anomalous occurrence intrinsic to the
conception? Use options. Extrinsic? Use exceptions.
Design decisions like this are ubiquitous. They are the bread and
butter of the programming process. The precursor to making these
decisions is possessing the tools that allow the alternative designs, the
understanding of what the ramifications are, and the judgement to
make a reasonable choice. The importance of having the choice is why,
for instance, the List module provides both nth and nth_opt.

10.5 Unit testing with exceptions

In Section 6.5, we called for unit testing of functions to verify their

correctness on representative inputs. Using the methodology of that
section, we might write a unit testing function for nth, call it nth_test:

# let nth_test () =
# unit_test (nth [5] 0 = 5) "nth singleton";
# unit_test (nth [1; 2; 3] 0 = 1) "nth start";
# unit_test (nth [1; 2; 3] 1 = 2) "nth middle" ;;
val nth_test : unit -> unit = <fun>

We run the tests by calling the function:

# nth_test () ;;
nth singleton passed
nth start passed
nth middle passed
- : unit = ()

The test function provides a report of the performance on all of the

tests, showing that all tests are passed.
As mentioned in Section 6.5, we’ll want to unit test nth as com-
pletely as is practicable, trying examples representing as wide a range
of cases as possible. For instance, we might be interested in whether
nth works in selecting the first, a middle, and the last element of a list.
We’ve checked the first two of these conditions, but not the third. We
can adjust the testing function accordingly:

# let nth_test () =
# unit_test (nth [5] 0 = 5) "nth singleton";
# unit_test (nth [1; 2; 3] 0 = 1) "nth start";
# unit_test (nth [1; 2; 3] 1 = 2) "nth middle";
# unit_test (nth [1; 2; 3] 2 = 3) "nth last" ;;
val nth_test : unit -> unit = <fun>

What about selecting at an index that is too large, as in the example

nth [1; 2; 3] 4? We should make sure that nth works properly in
this case as well. But what does “works properly” mean? According
 HANDLING ANOMALOUS CONDITIONS 131

to the specification in the List module, nth should raise a Failure

exception in this case. So we’ll need a boolean expression that is true
just in case evaluating the expression nth [1; 2; 3] 4 raises the
proper exception. We can achieve this by using a try ⟨⟩ with ⟨⟩ to trap
any exception raised and verifying that it is the correct one. We might
start with
# try nth [1; 2; 3] 4
# with
# | Failure _ -> true
# | _ -> false ;;
Line 3, characters 15-19:
3 | | Failure _ -> true
^^^^
Error: This expression has type bool but an expression was expected
of type
int

but this fails to type-check, since the type of the nth expression is int
(since it was applied to an int list), whereas the with clauses return
a bool. We’ll need to return a bool in the try as well. In fact, we should
return false; if nth [1; 2; 3] 4 manages to return a value and not
raise an exception, that’s a sign that nth has a bug! We revise the test
condition to be
# try let _ = nth [1; 2; 3] 4 in
# false
# with
# | Failure _ -> true
# | _ -> false ;;
- : bool = true

Adding this unit test to the unit testing function gives us

# let nth_test () =
# unit_test (nth [5] 0 = 5) "nth singleton";
# unit_test (nth [1; 2; 3] 0 = 1) "nth start";
# unit_test (nth [1; 2; 3] 1 = 2) "nth middle";
# unit_test (nth [1; 2; 3] 2 = 3) "nth last";
# unit_test (try let _ = nth [1; 2; 3] 4 in
# false
# with
# | Failure _ -> true
# | _ -> false) "nth index too big";;
val nth_test : unit -> unit = <fun>

# nth_test () ;;
nth singleton passed
nth start passed
nth middle passed
nth last passed
nth index too big passed
- : unit = ()
132 PROGRAMMING WELL

We’ll later see more elegant ways to put together unit tests (Sec-
tion 17.6).

Exercise 88
Augment nth_test to verify that nth works properly under additional conditions: on the
empty list, with negative indexes, with lists other than integer lists, and so forth.

With options and exceptions and their corresponding types, we’ve

completed the introduction of the major compound data types that are
built into the OCaml language. Table 10.1 provides a full list of these
compound types, with their type constructors and value constructors.
The advantages of compound types shouldn’t be limited to built-ins
though. In the next chapter, we’ll extend the type system to allow user-
defined compound types.

Table 10.1: Built-in compound data

Type Type constructor Value constructors
types.
functions ⟨⟩ -> ⟨⟩ fun ⟨⟩ -> ⟨⟩

tuples ⟨⟩ * ⟨⟩ ⟨⟩ , ⟨⟩
⟨⟩ * ⟨⟩ * ⟨⟩ ⟨⟩ , ⟨⟩ , ⟨⟩
⋯
lists ⟨⟩ list []
⟨⟩ :: ⟨⟩
[ ⟨⟩ ; ⟨⟩ ; ...]

records { ⟨⟩ : ⟨⟩ ; ⟨⟩ : ⟨⟩ ; ...} { ⟨⟩ = ⟨⟩ ; ⟨⟩ = ⟨⟩ ; ...}

options ⟨⟩ option None

Some ⟨⟩

exceptions exn Exit

Failure ⟨⟩
⋯
user-defined See Chapter 11

10.6 Problem set 3: Bignums and RSA encryption

This section provides substantive background for one of the course prob-
lem sets. For common background on logistics and procedures for prob-
lem sets, including such topics as accessing the problem set code, compil-
ing and testing your code, reflecting on the process, and submitting your
solution, please review the course document “Problem set procedures”.

Cryptography is the science of methods for storing or transmitting

messages securely and privately.
 HANDLING ANOMALOUS CONDITIONS 133

Cryptographic systems typically use keys for encryption and decryp-

tion. An encryption key is used to convert the original message (the
plaintext) to coded form (the ciphertext). A corresponding decryption
key is used to convert the ciphertext back to the original plaintext.
In traditional cryptographic systems, the same key is used for both
encryption and decryption, which must be kept secret. Two parties
can exchange coded messages only if they share the secret key. Since
anyone who learned that key would be able to decode the messages,
keys must be carefully guarded and transmitted only under tight se-
curity, for example, couriers handcuffed to locked, tamper-resistant
briefcases!
In 1976, Diffie and Hellman initiated a new era in cryptography with
their discovery of a new approach: public-key cryptography. In this Figure 10.1: Whitfield Diffie (1944–) and
approach, the encryption and decryption keys are different from each Martin Hellman (1948–), co-inventors of
public-key cryptography, for which they
other. Knowing the encryption key cannot help you find the decryp-
received the Turing Award in 2015.
tion key. Thus, you can publish your encryption key publicly – on the
web, say – and anyone who wants to send you a secret message can use
it to encode a message to send to you. You do not have to worry about
key security at all, for even if everyone in the world knew your encryp-
tion key, no one could decrypt messages sent to you without knowing
your decryption key, which you keep private to yourself. You used
public-key encryption when you set up your CS51 git repositories: the
command ssh-keygen generated a public encryption key and private
decryption key for you. You uploaded the public key and (hopefully)
kept the private key to yourself.
The best known public-key cryptosystem is due to computer sci-
entists Rivest, Shamir, and Adelman, and is known by their initials,
RSA. The security of your web browsing probably depends on RSA en-
cryption. The system relies on the fact that there are fast algorithms
for exponentiation and for testing prime numbers, but no known fast
algorithms for factoring extremely large numbers. In this problem set
you will complete an implementation of a version of the RSA system.
(If you’re interested in some of the mathematics behind RSA, see Sec-
tion 10.6.3. However, an understanding of that material is not needed
to complete the problem set.)
Crucially, RSA requires manipulation of very large integers, much
larger than can be stored, for instance, as an OCaml int value. OCaml’s
int type has a size of 63 bits, and therefore can represent integers
between −262 and 262 − 1. These limits are available as OCaml constants
min_int and max_int:

# min_int, max_int ;;
- : int * int = (-4611686018427387904, 4611686018427387903)

The int type can then represent integers with up to 18 or so digits, that
134 PROGRAMMING WELL

is, integers in the quintillions, but RSA needs integers with hundreds of
digits.
Computer representations for arbitrary size integers are tradition-
ally referred to as B I G N U M S . In this assignment, you will be imple-
menting bignums, along with several operations on bignums, includ-
ing addition and multiplication. We provide code that will use your
bignum implementation to implement the RSA cryptosystem. Once
you complete your bignum implementation, you’ll be able to encrypt
and decrypt messages using this public-key cryptosystem, and dis-
cover a hidden message that we’ve provided encoded in this way.

10.6.1 Big numbers

To handle arbitrarily large integers (as large as the computer’s memory

allows), we define a new algebraic data type to represent bignums,
along with the standard arithmetic operations plus, times, compari-
son operators, and so on.
In this bignum implementation, an integer n will be represented
as a list of int values, namely the coefficients of the expansion of n in
some base. For example, suppose the base is 1000. Then the number
123456789 can be written as:

(123 ⋅ 10002 ) + (456 ⋅ 10001 ) + (789 ⋅ 10000 ) ,

7
which we will represent by the list [123; 456; 789]. Notice that the We name the base using this distinc-
tive naming convention, with initial
least significant coefficient (789) appears last in the list. As another ‘c’ (for constant) and upper-case
example, the number 12000000000000 is represented by the list [12; mnemonic to emphasize that it is a
global constant. Such global constants
0; 0; 0; 0].
should be rare, and thus typographically
The base used by your bignum implementation is defined at the top distinctive.
of the file bignum.ml:7

let cBASE = 1000 ;;

To make it easier to implement some of the functions below, you may

want to use a base of 1000 while debugging and testing. However,
you’ll want to make sure that the code works for any value of cBASE, 8
For reasons of simplicity in imple-
always referring to that variable rather than hard-coding a value of menting functions to move between
1000, so that if cBASE is changed to a different value, your code should bignums and strings, we’ll assume
that cBASE is a power of 10. Similarly,
still work.8 This is good programming practice in general, and makes it implementing certain operations on
easier to modify code later. bignums here is simplified by requiring
that cBASE * cBASE < int_max, that
Here is the actual type definition for bignums:
is cBASE can be safely squared without
# type bignum = {neg : bool; coeffs : int list} ;; overflowing the integer representation.
You can assume that both of these
type bignum = { neg : bool; coeffs : int list; }
conditions hold as invariants.

The neg field specifies whether the number is positive (if neg is false)
or negative (if neg is true). The coeffs field is the list of coefficients
 HANDLING ANOMALOUS CONDITIONS 135

in some base, where each coefficient is between 0 and cBASE - 1,

inclusive. Assuming a base of 1000, to represent the integer 123456789,
we would use:

# {neg = false; coeffs = [123; 456; 789]} ;;

- : bignum = {neg = false; coeffs = [123; 456; 789]}

and to represent −9999 we would use:

# {neg = true; coeffs = [9; 999]} ;;

- : bignum = {neg = true; coeffs = [9; 999]}

In other words, the record

{neg = false, coeffs = [a n ; a n −1 ; a n −2 ; ...; a 0 ]}

represents the integer

a n ⋅ base n + a n −1 ⋅ base n −1 + a n −2 ⋅ base n −2 + ⋯ + a 0

An empty list thus represents 0. This defines the correspondance

between bignum and the integers.

Representation invariants The implementation of bignums will be

simpler if we impose certain representation invariants. A R E P R E S E N -
TAT I O N I N VA R I A N T is a property of values in your representation that
you enforce and that you can thus assume is true when writing func-
tions to handle these values. If the representation invariant is violated,
the value is not a valid value for the type. Any function you write that
produces a value of the type (for example, from_int) should produce a
value satisfying the representation invariant.
The invariants for the bignum representation are as follows:

• Zero will always be represented as {neg = false; coeffs = []},

and never as {neg = true; coeffs = []}.

• There will be no leading zeroes on the list of coefficients. The value

{neg = false; coeffs = [0; 0; 125]} violates this invariant.

• Coefficients are never negative and are always strictly less than
cBASE.

Functions that consume bignums may assume that they satisfy the
invariant. We will not test your code using bignums that violate the
invariant. Functions that return bignums should preserve these in-
variants. For example, your functions should never return a bignum
representing zero with the neg flag set to true or a bignum with a
coefficients list with leading zeros.
136 PROGRAMMING WELL

Using your solution Using the functions from_string and to_-

string, you will be able to test your functions by converting between
the bignum representation and a string representation. (These func-
tions further assume that cBASE is a power of 10.) Here is a sample
interaction with a completed implementation of bignums, in which we
multiply 123456789 by 987654321:

# let answer = times (from_string "123456789")

# (from_string "987654321") ;;
val answer : bignum = {neg = false; coeffs = [121; 932; 631; 112;
635; 269]}
# to_string answer ;;
- : string = "121932631112635269"

What you need to do You will implement several functions in

bignum.ml that operate on bignums. As usual, feel free to change
the header of the function definition, for instance by adding a rec key-
word, if you think it helpful, but do not alter the types of any functions,
as we will be unit testing assuming those type signatures.
Problem 89
Implement the function negate : bignum -> bignum, which gives the negation of a
bignum (the bignum times -1).
Problem 90
Implement the functions equal : bignum -> bignum -> bool, less : bignum
-> bignum -> bool, and greater : bignum -> bignum -> bool that compare
two numbers b1 and b2 and return a boolean indicating whether b1 is equal to, less
than, or greater than b2, respectively. You can assume that the arguments satisfy the
representation invariants.
Problem 91
Implement conversion functions to_int : bignum -> int option and from_int :
int -> bignum between integers and bignums. The function to_int : bignum ->
int option should return None if the number is too large to fit in an OCaml integer.
You’ll want to be careful when checking whether values fit within OCaml integers.
In particular, you shouldn’t assume that max_int is the negative of min_int; in fact, it
isn’t, as seen above. Instead, you may use the fact that max_int = abs(min_int + 1),
though by careful design choices you can avoid even that assumption.
Problem 92
We have provided you with a function, plus_pos : bignum -> bignum -> bignum,
which adds two bignums and provides the result as a bignum. However, it has a limita-
tion: this function only works if the resulting sum is positive. (This might not be the case,
for example if b2 is negative and larger in absolute value than b1.) Write the function
plus : bignum -> bignum -> bignum, which can add arbitrary bignums, without this
limitation. It should call plus_pos and shouldn’t be too complex. Hints: How can you
use the functions you’ve written so far to check, without doing the addition, whether the
resulting sum will be negative? If the sum will be negative, can you adjust the numbers to
find a different way of generating the sum using only additions that obey the limitation?
And a hint on your unit tests for this problem: you’ll definitely want to test a case where
the result comes out negative.
Problem 93
Implement the function times : bignum -> bignum -> bignum, which multiplies two
bignums. Use the traditional algorithm you learned in grade school (as in Figure 10.2),
but remember that we are representing numbers in base 1000 (say), not 10. The main
 HANDLING ANOMALOUS CONDITIONS 137

goal is correctness, so keep your code as simple as possible. Make sure your code works
with positive numbers, negative numbers, and zero. Assume that the arguments satisfy
5 4 3
the invariant. Hint: You may want to write a helper function that multiplies a bignum by
× 2 2 4
a single int (which might be one coefficient of a bignum).
2 1 7 2
+ 1 0 8 6 0
Using bignums to implement RSA We’ve provided an implementation + 1 0 8 6 0 0
of the RSA cryptosystem in the file rsa.ml. It uses the module Bignum, = 1 2 1 6 3 2
that is, the bignum implementation in bignum.ml that you’ve just
Figure 10.2: Multiplication of 543 and
completed. 224 using the grade school algorithm.
In the file rsa_puzzle.ml, we’ve placed some keys and a ciphertext First, you multiply the first number
(543 in this example) by each of the
with a secret message. If your implementation of bignums is correct, digits of the second number, from least
you should be able to compile and run the file: significant to most significant (4, then
2, then 2), adding an increasing number
% ocamlbuild rsa_puzzle.byte of zeros at the end (shown in italics),
no zeros for the least significant digit
% ./rsa_puzzle.byte (resulting in 2172), one for the next
(1086 plus one zero yielding 10860),
two for the next, and so forth. Then
to reveal the secret message! the partial products 2172, 10860, and
108600 are summed to generate the
final result 121632.
10.6.2 Challenge problem: Multiply faster

As on previous problem sets and several in the future, we are providing

an additional problem or two for those who would like an extra chal-
lenge. These problems are for your karmic edification only, and will
not affect your grade. We encourage you to attempt this problem only
once you have done your best work on the rest of the problem set.
The multiplication algorithm you implemented in Section 10.6.1
will work just fine for most integers of reasonable sizes, including the
ones needed by the RSA implementation. However, some exceedingly
smart people have devised algorithms which, on very large numbers
(think thousands of digits), are considerably faster than the multiplica-
tion algorithm you learned in grade school.
Problem 94
Challenge See if you can implement such an algorithm in times_faster. You may want
to start by looking up the Karatsuba algorithm. This algorithm recursively multiplies
smaller and smaller numbers. Note that, when the numbers become small enough (2-4
digits), you can and probably should simply call the times function you implemented
earlier. However, don’t just call this function on any numbers you are given; that’s not
any faster!

10.6.3 More background: How the RSA cryptosystem works

This section is intended for those who are interested in more math-
ematical details on how the RSA system uses bignum calculations to
achieve public-key encryption and decryption. Nothing in this section
is needed to complete the pset.
We want to encrypt messages that are strings of characters, but
the RSA system does not work with characters, but with integers. To
138 PROGRAMMING WELL

encrypt a piece of text, we first convert it to a number (a bignum in

fact) by combining the ASCII codes of the characters; we can then
encrypt the resulting number.
To generate public and private keys in the RSA system, you select
two very large prime numbers p and q. (Recall that a prime number is
a positive integer greater than 1 with no divisors other than itself and
1.) How to find large primes is a subject in itself, but beyond the scope
of these notes.
You then compute numbers n and m:

n=pq
m = (p − 1)(q − 1)

It turns out that

• With very few exceptions, for almost all numbers e < n, e m

(mod n ) = 1.

• No one knows how to compute m, p, or q efficiently, even knowing

n.

(Notation: We write a (mod n ) for the remainder obtained when

dividing a by b using ordinary integer division. We use the notation
[a = b ] (mod n ) to mean that a (mod n ) = b (mod n ). Equivalently,
[a = b ] (mod n ) if a − b is divisible by n. For example, [17 = 32]
(mod 5).)
Now you pick a number e < m relatively prime to m; that is, such
that e and m have no factors in common except 1. The significance
of relative primality is that e is relatively prime to m if and only if e
is invertible mod m, that is, if and only if there exists a d such that
[d e = 1] (mod m ). Moreover, it is possible to compute d from e and m
using Euclid’s algorithm, described below. Your public key, which you
can advertise to the world, is the pair (n, e ). Your private key is (n, d ).
Anyone who wants to send you a secret message s (represented by
an integer remember) encrypts it by computing E (s ), where

E (s ) = s e (mod n )

That is, if the plaintext is represented by the number s which is less

than n, the ciphertext E (s ) is obtained by raising s to the power e, then
taking the remainder modulo n.
The decryption process is exactly the same, except that d is used
instead of e:

D (s ) = s d (mod n )
 HANDLING ANOMALOUS CONDITIONS 139

The operations E and D are inverses:

D (E (s )) = (s e )d (mod n )
= s d e (mod n )
= s 1+km (mod n )
= s (s m )k (mod n )
= s 1k (mod n )
= s (mod n )
=s

For the last step to hold, the integer s representing the plaintext
must be less than n. That’s why we break the message up into chunks.
Also, this only works if s is relatively prime to n, that is, it has no fac-
tors in common with n other than 1. If n is the product of two large
primes, then all but negligibly few messages s < n satisfy this property.
If by some freak chance s and n turned out not to be relatively prime,
then the code would be broken; but the chances of this happening by
accident are insignificantly small.
In summary, to use the RSA system:

1. Pick large primes p and q.

2. Compute n = pq and m = (p − 1)(q − 1).

3. Choose e relatively prime to m and use this to compute d such that

[d e = 1] (mod m ).

4. Publish the pair (n, e ) as your public key, but keep d , p, and q
secret.

How secure is RSA? At present, the only known way to obtain d from
e and n is to factor n into its prime factors p and q, then compute m
and proceed as above. But despite centuries of effort by number theo-
rists, factoring large integers efficiently is still an open problem. Until
someone comes up with an efficient way to factor numbers, or dis-
covers some other way to compute d from e and n, the cryptosystem
appears to be secure for large numbers n.
11
Algebraic data types

Data types can be divided into the atomic types (with atomic type
constructors like int and bool) and the composite types (with parame-
terized type constructors like ⟨⟩ * ⟨⟩ , ⟨⟩ list, and ⟨⟩ option).
What is common to all of the built-in composite types introduced
so far1 is that they allow building data structures through the combina- 1
The exception is the composite type
of functions. Functions are the rare
tion of just two methods.
case of a composite type in OCaml not
structured as an algebraic data type as
1. Conjunction: Multiple components can be conjoined to form a defined below.
composite value containing all of the components.
For instance, values of pair type, int * float say, are formed as
the conjunction of two components, the first component an int
and the second a float.

2. Alternation: Multiple components can be disjoined, serving as

alternatives to form a composite value containing one of the values.
For instance, values of type int list are formed as the alternation 2
Algebra is the mathematical study of
of two components. One alternative is []; the other is the pair (itself structures that obey certain laws. Typi-
a conjunction) of a component of type int and a component of type cal of algebras is to form such structures
by operations that have exactly the
int list. duality of conjunction and alternation
found here. For instance, arithmetic
Data types built by conjunction and disjunction are called A L G E B R A I C algebras have multiplication and ad-
2 dition as, respectively, the conjunction
D ATA T Y P E S . As mentioned, we’ve seen several examples already, as
and alternation operators. Boolean
built-in composite data types. But why should the power of algebraic algebras have logical conjunction (‘and’)
data types be restricted to built-in types? Such a simple and elegant and disjunction (‘or’). Set algebras have
cross-product and union. The term
construction like algebraic types could well be a foundational con-
algebraic data type derives from this
struct of the language, not only to empower programmers using the connection to these structured algebras.
language but also to provide a foundation for the built-in constructs
themselves.
OCaml inherits from its antecedents (especially, the Hope program-
ming language developed at the University of Edinburgh, the univer-
sity that brought us ML as well) the ability to define new algebraic data
types as user code.
142 PROGRAMMING WELL

Let’s start with a simple example based on genome processing, ex-

emplifying the use of alternation. DNA sequences are long sequences
composed of only four base amino acids: guanine (G), cytosine (C),
adenine (A), and thymine (T).
We can define an algebraic data type for the DNA bases via alter-
nation. The type, called base, will have four value constructors cor- Figure 11.1: DNA carries information
encoded as sequences of four amino
responding to the four base letters. The alternatives are separated by acids.
vertical bars (|). Here is the definition of the base type, introduced by
the keyword type:

# type base = G | C | A | T ;;
type base = G | C | A | T

This kind of type declaration defines a VA R I A N T T Y P E , which lists a set

of alternatives, variant ways of building elements of the type: A or T
or C or G.3 Having defined the base type, we can refer to values of that 3
Using argumentless variants in this
way serves the purpose of enumerated
type.
types in other languages – enum in C, C
# A ;;
derivatives, Java, and Perl, for instance.
Variants thus generalize enumerated
- : base = A
types.
# G ;;
- : base = G

As with all composite types, computations that depend on the

particular values of the type use pattern-matching to structure the
cases. For instance, each DNA base has a complementary base: A
and T are complementary, as are G and C. A function to return the
complement of a base uses pattern-matching to individuate the cases:

# let comp_base bse =

# match bse with
# | A -> T
# | T -> A
# | G -> C
# | C -> G ;;
val comp_base : base -> base = <fun>
# comp_base G ;;
- : base = C

Variants correspond to the alternation approach to building com-

posite values. The conjunction approach is enabled by allowing the
alternative value constructors to take an argument of a specified type.
That argument itself can conjoin components by tupling.
As an example, DNA sequences themselves can be implemented as
an algebraic data type that we’ll call dna. Taking inspiration from the
list type for sequences, DNA sequences can be categorized into two
alternatives, two variants – the empty sequence, for which we will use
the value constructor Nil; and non-empty sequences, for which we
will use the value constructor Cons. The Cons constructor will take two
 A L G E B R A I C D ATA T Y P E S 143

arguments (uncurried), one for the first base in the sequence and one
for the rest of the dna sequence.4 4
There is a subtle distinction concern-
ing when type constructors take a single
# type dna = tuple argument or multiple arguments
# | Nil written with tuple notation. For the
# | Cons of base * dna ;; most part, the issue can be ignored,
type dna = Nil | Cons of base * dna so long as the type definition doesn’t
place the argument sequence within
The Cons constructor takes two arguments (using tuple notation), the parentheses. For the curious, see the
“Note on tupled constructors” in the
first of type base and the second of type dna. It thus serves to conjoin a OCaml documentation.
base element and another dna sequence.
Having defined this new type, we can construct values of that type:
# let seq = Cons (A, Cons (G, Cons (T, Cons (C, Nil)))) ;;
val seq : dna = Cons (A, Cons (G, Cons (T, Cons (C, Nil))))

and pattern-match against them:

# let first_base =
# match seq with
# | Cons (x, _) -> x
# | Nil -> failwith "empty sequence" ;;
val first_base : base = A

The dna type is defined recursively,5 as one of its variants (Cons) 5

In value definitions (with let), recur-
sion must be marked explicitly with
includes another value of the same type. By using recursion, we can
the rec keyword. In type definitions,
define data types whose values can be of arbitrary size. no such explicit marking is required,
To process data values of arbitrary size, recursive functions are an and in fact nonrecursive definitions
can only be formed using distinct type
ideal match. A function to construct the complement of an entire DNA names. This design decision was pre-
sequence is naturally recursive. sumably motivated by the ubiquity of
recursive type definitions as compared
# let rec complement seq = to recursive value definitions. It’s a
# match seq with contentious matter as to whether this
# | Nil -> Nil quirk of OCaml is a feature or a bug.
# | Cons (b, seq) -> Cons (comp_base b, complement seq) ;;
val complement : dna -> dna = <fun>

# complement seq ;;
- : dna = Cons (T, Cons (C, Cons (A, Cons (G, Nil))))

11.1 Built-in composite types as algebraic types

The dna type looks for all the world just like the list type built into
OCaml, except for the fact that its elements are always of type base.
6
We name the type bool_ so as not to
shadow the built-in type bool. Similarly
Indeed, our choice of names of the value constructors (Nil and Cons) for the underscore versions list_ and
emphasizes the connection. option_ below.
Value constructors in defined alge-
In fact, many of the built-in composite types can be implemented as
braic types are restricted to starting with
algebraic data types in this way. Boolean values are essentially a kind of capital letters in OCaml. The built-in
enumerated type, hence algebraic.6 type differs only in using lower case
constructors true and false.
# type bool_ = True | False ;;
type bool_ = True | False
144 PROGRAMMING WELL

We’ve already seen an algebraic type implementation of base lists.

Similar implementations could be generated for lists of other types.

# type int_list = INil | ICons of int * int_list ;;

type int_list = INil | ICons of int * int_list
# type float_list = FNil | FCons of float * float_list ;;
type float_list = FNil | FCons of float * float_list

Following the edict of irredundancy, we’d prefer not to write this same
code repeatedly, differing only in the type of the list elements. Fortu-
nately, variant type declarations can be polymorphic.

# type 'a list_ = Nil | Cons of 'a * 'a list_ ;;

type 'a list_ = Nil | Cons of 'a * 'a list_

In polymorphic variant data type declarations like this, a new type

constructor (list_ in this case) is defined that takes a type argument
(here, the type variable ’a). The type constructor is always postfix, like
the built-in constructors list and option that you’ve already seen.7 7
If we need a type constructor that takes
more than one type as an argument, we
Option types can be viewed as a polymorphic variant type with two
use the cross-product type notation, as
constructors for the None and Some cases. in the (’key, ’value) dictionary
type defined in Section 11.3.
# type 'a option_ = None | Some of 'a ;;
type 'a option_ = None | Some of 'a

The point of seeing these alternative implementations of the built-

in composite types (booleans, lists, options) is not that one would
actually use the seimplementations. That would flout the edict of
irredundancy. And the reimplementations of lists and options don’t
benefit from the concrete syntax niceties of the built-in versions; no
infix :: for instance, or bracketed lists. Rather than defining a dna type
in this way, in a real application we’d just use the base list type. If a
name for this type is desired the type name dna can be defined by

type dna = base list ;;

The point instead is to demonstrate the power of algebraic data type

definitions and show that even more of the language can be viewed
as syntactic sugar for pre-provided user code. Thus, the language can
again be seen as deploying a small core of basic notions to build up a
highly expressive medium.

11.2 Example: Boolean document search

The variant type definitions in this chapter aren’t the first examples of
algebraic type definitions you’ve seen. In Section 7.4, we noted that
record types were user-defined types, defined with the type keyword,
as well.
 A L G E B R A I C D ATA T Y P E S 145

Record types are a kind of dual to variant types. Instead of starting

with alternation – this or that or the other – record types start with
8
conjunction – this and that and the other. As an aid in building a document
corpus, it will be useful to have a func-
As an example, consider a data type for documents. A document tion tokenize : string -> string
will be made up of a list of words (each a string), as well as some meta- list that splits up a string into its
component words (here defined as any
data about the document, perhaps its title, author, and so forth. For
characters separated by whitespace).
this example, we’ll stick just to titles, so an appropriate type definition We use some functions from the Str
would be library module, made available using
the #load directive to the R E P L , to split
# type document = { title : string; up the string.
# words : string list } ;; # #load "str.cma" ;;
type document = { title : string; words : string list; } # let tokenize : string -> string list =
# Str.split (Str.regexp "[ \ t\ n]+") ;;
A corpus of such documents can be implemented as a document val tokenize : string -> string list = <fun>
list. We build a small corpus of first lines of novels.8 Did you notice the use of partial appli-
cation?
# let first_lines : document list = (* output suppressed *)
We’ve also suppressed the output for
# [ {title = "Moby Dick";
this R E P L input to save space, as indi-
# words = tokenize cated by the (* output suppressed
# "Call me Ishmael ."}; *) comment here and elsewhere.
# {title = "Pride and Prejudice";
# words = tokenize
# "It is a truth universally acknowledged , \
# that a single man in possession of a good \
# fortune must be in want of a wife ."};
# {title = "1984";
# words = tokenize
# "It was a bright cold day in April , and \
# the clocks were striking thirteen ."};
# {title = "Great Gatsby";
# words = tokenize
# "In my younger and more vulnerable years \
# my father gave me some advice that I've \
# been turning over in my mind ever since ."}
# ] ;;

We might want to query for documents with particular patterns of

words. A boolean query allows for different query types: requesting
documents in which a particular word occurs; or (inductively) docu-
ments that satisfy both one query and another query; or documents
that satisfy either one query or another query. We instantiate the idea
in a variant type definition.
# type query =
# | Word of string
# | And of query * query
# | Or of query * query ;;
type query = Word of string | And of query * query | Or of query *
query

To evaluate such queries against a document, we’ll write a function

eval : document -> query -> bool, which should return true just
in case the document satisfies the query.
146 PROGRAMMING WELL

let rec eval ({title; words} : document)

(q : query)
: bool = ...

Note the use of pattern-matching right in the header line, as well as the
use of field punning to simplify the pattern.
The evaluation of the query depends on its structure, so we’ll want
to match on that.
let rec eval ({title; words} : document)
(q : query)
: bool =
match q with
| Word word -> ...
| And (q1, q2) -> ...
| Or (q1, q2) -> ...

For the first variant, we merely check that the word occurs in the list of
words:

let rec eval ({title; words} : document)

(q : query)
: bool =
match q with
| Word word -> List.mem word words
| And (q1, q2) -> ...
| Or (q1, q2) -> ...

(The function List.mem : ’a -> ’a list -> bool is useful here,

a good reason to familiarize yourself with the rest of the List library
module.)
What about the other variants? In these cases, we’ll want to recur-
sively evaluate the subparts of the query (q1 and q2) against the same
document. We’ve already decomposed the document into its compo-
nents title and words. We could reconstruct the document as needed
for the recursive evaluations:
let rec eval ({title; words} : document)
(q : query)
: bool =
match q with
| Word word -> List.mem word words
| And (q1, q2) -> (eval {title; words} q1)
&& (eval {title; words} q2)
| Or (q1, q2) -> (eval {title; words} q1)
|| (eval {title; words} q2) ;;

but this seems awfully verbose. We refer to {title; words} four

different times. It would be helpful if we could both pattern match
against the document argument and name it as a whole as well. OCaml
provides a special pattern constructed as

⟨pattern ⟩ as ⟨variable ⟩
 A L G E B R A I C D ATA T Y P E S 147

for just such cases. Such a pattern both pattern matches against the
⟨pattern ⟩ as well as binding the ⟨variable ⟩ to the expression being
matched against as a whole. We use this technique both to provide
a name for the document as a whole (doc) and to extract its compo-
nents. (Once we have a variable doc for the document as a whole, we
no longer need to refer to title, so we use an anonymous variable
instead.)
let rec eval ({words; _} as doc : document)
(q : query)
: bool =
match q with
| Word word -> List.mem word words
| And (q1, q2) -> (eval doc q1) && (eval doc q2)
| Or (q1, q2) -> (eval doc q1) || (eval doc q2) ;;

That’s better. But we’re still calling eval doc four times on different
subqueries. We can abstract that function and reuse it; call it eval’:
let eval ({words; _} as doc : document)
(q : query)
: bool =
let rec eval' (q : query) : bool =
match q with
| Word word -> List.mem word words
| And (q1, q2) -> (eval' q1) && (eval' q2)
| Or (q1, q2) -> (eval' q1) || (eval' q2) in
... ;;

There’s an important idea hidden here, which follows from the scoping
rules of OCaml. Because the eval’ definition falls within the scope
of the definition of eval and the associated variables words and q,
those variables are available in the body of the eval’ definition. And in
fact, we make use of that fact by referring to words in the first pattern-
match. (The outer q is actually shadowed by the inner q, so it isn’t
referred to in the body of the eval’ definition. The occurrence of q in
the match q is a reference to the q argument of eval’.)
Now that we have eval’ defined it suffices to call it on the main
query and let the recursion do the rest. At this point, however, the
alternative variable name doc is no longer referenced, and can be
eliminated.
# let eval ({words; _} : document)
# (q : query)
# : bool =
# let rec eval' (q : query) : bool =
# match q with
# | Word word -> List.mem word words
# | And (q1, q2) -> (eval' q1) && (eval' q2)
# | Or (q1, q2) -> (eval' q1) || (eval' q2) in
# eval' q ;;
val eval : document -> query -> bool = <fun>
148 PROGRAMMING WELL

Let’s try it on some sample queries. We’ll use the first line of The Great
Gatsby.

# let gg = nth first_lines 3 ;; (* output suppressed *)

# eval gg (Word "the") ;;

- : bool = false
# eval gg (Word "and") ;;
- : bool = true
# eval gg (And ((Word "the"), (Word "and"))) ;;
- : bool = false
# eval gg (Or ((Word "the"), (Word "and"))) ;;
- : bool = true

Now, we return to the original goal, to search among a whole corpus

of documents for those satisfying a query. The function eval_all :
document list -> query -> string list will return the titles of all
documents in the document list that satisfy the query.
The eval_all function should be straightforward to write, as it
involves filtering the document list for those satisfying the query, then
extracting their titles. The filter and map list-processing functions are
ideal for this.

# let eval_all (docs : document list)

# (q : query)
# : string list =
# List.map (fun doc -> doc.title)
# (List.filter (fun doc -> (eval doc q))
# docs) ;;
val eval_all : document list -> query -> string list = <fun>

We start with the docs, filter them with a function that applies eval to
select only those that satisfy the query, and then map a function over
them to extract their titles.
From a readability perspective, it is unfortunate that the description
of what the code is doing – start with the corpus, then filter, then map
– is “inside out” with respect to how the code reads. This follows from
the fact that in OCaml, functions come before their arguments in
applications, whereas in this case, we like to think about a data object
followed by a set of functions that are applied to it. A language with
backwards application would be able to structure the code in the more
readable manner.
Happily, the Stdlib module provides a B A C K WA R D S A P P L I C AT I O N
infix operator |> for just such occasions.

# succ 3 ;;
- : int = 4
# 3 |> succ ;; (* start with 3; increment *)
- : int = 4
# 3 |> succ (* start with 3; increment; ... *)
 A L G E B R A I C D ATA T Y P E S 149

# |> (( * ) 2) ;; (* ... and double *)

- : int = 8

Exercise 95
What do you expect the type of |> is?

Exercise 96
How could you define the backwards application operator |> as user code?
Taking advantage of the backwards application operator can make
the code considerably more readable. Instead of

List.filter (fun doc -> (eval doc q))

docs

we can start with docs and then filter it:

docs
|> List.filter (fun doc -> (eval doc q))

Then we can map the title extraction function over the result:

docs
|> List.filter (fun doc -> (eval doc q))
|> List.map (fun doc -> doc.title)

The final definition of eval_all is then

# let eval_all (docs : document list)

# (q : query)
# : string list =
# docs
# |> List.filter (fun doc -> (eval doc q))
# |> List.map (fun doc -> doc.title) ;;
9
val eval_all : document list -> query -> string list = <fun> Not coincidentally, natural languages
often allow alternative orders for
phrases for just this same goal of
Some examples:
moving “heavier” phrases to the right.
For example, the normal order for verb
# eval_all first_lines (Word "and") ;;
phrases with the verb “give” places
- : string list = ["1984"; "Great Gatsby"]
the object before the recipient, as in
# eval_all first_lines (Word "me") ;; “Arden gave the book to Bellamy”. But
- : string list = ["Moby Dick"; "Great Gatsby"] when the object is very “heavy” (long
# eval_all first_lines (And (Word "and", Word "me")) ;; and complicated), it sounds better to
- : string list = ["Great Gatsby"] place the object later, as in “Arden gave
# eval_all first_lines (Or (Word "and", Word "me")) ;; to Bellamy every last book in the P. G.
- : string list = ["Moby Dick"; "1984"; "Great Gatsby"]
Wodehouse collection.” Backwards
application gives us this same flexibility,
to move “heavy” expressions (like
The change in readability from using backwards application has complicated functions) later in the
a moral. Concrete syntax can make a big difference in the human code.

usability of a programming language. The addition of a backwards

application adds not a jot to the expressive power of the language, but
when used appropriately it can dramatically reduce the cognitive load
on a human reader.9
150 PROGRAMMING WELL

11.3 Example: Dictionaries

A dictionary is a data structure that manifests a relationship between

a set of keys and their associated values. In an English dictionary, for
instance, the keys are the words of the language and the associated
values are their definitions. But dictionaries can be used in a huge
variety of applications.
A dictionary data type will depend on the types of the keys and the
values. We’ll want to define the type, then, as polymorphic – a (’key,
’value) dictionary.10 One approach (an exceptionally poor one 10
Names of type variables are arbitrary,
so we might as well use that ability to
as it will turn out, but bear with us) is to store the keys and values as
give good mnemonic names to them
separate equal-length lists in two record fields. – ’key and ’value instead of’a and
’b – following the edict of intention in
# type ('key, 'value) dictionary = { keys : 'key list; making our intentions clear to readers
# values : 'value list } ;; of the code.
type ('key, 'value) dictionary = { keys : 'key list; values :
'value list; }

Looking up an entry in the dictionary by key, returning the correspond-

ing value, can be performed in a few ways. Here’s one:

# let rec lookup ({keys; values} : ('key, 'value) dictionary)

# (request : 'key)
# : 'value option =
# match keys, values with
# | [], [] -> None
# | key :: keys, value :: values ->
# if key = request then Some value
# else lookup {keys; values} request ;;
Lines 4-8, characters 0-34:
4 | match keys, values with
5 | | [], [] -> None
6 | | key :: keys, value :: values ->
7 | if key = request then Some value
8 | else lookup {keys; values} request...
Warning 8: this pattern-matching is not exhaustive.
Here is an example of a case that is not matched:
((_::_, [])|([], _::_))
val lookup : ('key, 'value) dictionary -> 'key -> 'value option =
<fun>

The problem with this dictionary representation is obvious. The

entire notion of a dictionary assumes that for each key there is a single
value. But this approach to implementing dictionaries provides no
such guarantee. An illegal dictionary – like {keys = [1; 2; 3];
values = ["first"; "second"]}, in which one of the keys has no
value – is representable. In such cases, the lookup function will raise
an exception.

# let bad_dict = {keys = [1; 2; 3];

# values = ["first"; "second"]} ;;
 A L G E B R A I C D ATA T Y P E S 151

val bad_dict : (int, string) dictionary =

{keys = [1; 2; 3]; values = ["first"; "second"]}
# lookup bad_dict 4 ;;
Exception: Match_failure ("//toplevel//", 4, 0).
# lookup bad_dict 3 ;;
Exception: Match_failure ("//toplevel//", 4, 0).

Adding additional match cases merely postpones the problem.

# let rec lookup ({keys; values} : ('key, 'value) dictionary)
# (request : 'key)
# : 'value option =
# match keys, values with
# | [], _
# | _, [] -> None
# | key :: keys, value :: values ->
# if key = request then Some value
# else lookup {keys; values} request ;;
val lookup : ('key, 'value) dictionary -> 'key -> 'value option =
<fun>
# lookup bad_dict 4 ;;
11
This idea has a long history in func-
- : string option = None
tional programming with algebraic
# lookup bad_dict 3 ;;
data types, but seen in its crispest form
- : string option = None is likely due to Yaron Minsky, who
phrases it as “Make illegal states unrep-
The function still allows data structures that do not express legal dic- resentable.” Ben Feldman uses “Make
tionaries to be used. Indeed, we can no longer even distinguish be- impossible states impossible.” But the
idea dates back to at least the begin-
tween simple cases of lookup of a missing key and problematic cases of
nings of statically typed programming
lookup in an ill-formed dictionary structure. languages. By referring to inexpressibil-
A better dictionary design would make such illegal structures im- ity, rather than unrepresentability, we
generalize the notion to include cases
possible to even represent. This idea is important enough for its own we consider in Chapter 12.
edict.

Edict of prevention:
12
An idiosyncrasy of OCaml requires
Make the illegal inexpressible. that the dictionary type be defined in
stages in this way, rather than all at once
We’ve seen this idea before in the small. It’s the basis of type checking as
itself, which allows the use of certain values only with functions that # type ('key, 'value) dictionary =
are appropriate to apply to them – integers with integer functions, # { key : 'key; value : 'value } list ;;
Line 2, characters 31-35:
booleans with boolean functions – preventing all other uses. In a 2 | { key : 'key; value : 'value } list ;;
strongly typed language like OCaml, illegal operations, like applying ^^^^
Error: Syntax error
an integer function to a boolean value, simply can’t be expressed as
The use of and to combine multiple type
valid well-typed code.
definitions into a single simultaneous
The edict of prevention11 challenges us to find an alternative struc- definition isn’t required here, but is
ture in which this kind of mismatch between the keys and values can’t when the type definitions are mutually
recursive.
occur. Such a structure may already have occurred to you. Instead
of thinking of a dictionary as a pair of lists of keys and values, we can
think of it as a list of pairs of keys and values.12
# type ('key, 'value) dict_entry =
# { key : 'key; value : 'value }
152 PROGRAMMING WELL

# and ('key, 'value) dictionary =

# ('key, 'value) dict_entry list ;;
type ('key, 'value) dict_entry = { key : 'key; value : 'value; }
and ('key, 'value) dictionary = ('key, 'value) dict_entry list

The type system will now guarantee that every dictionary is a list
whose elements each have a key and a value. A dictionary with un-
equal numbers of keys and values is not even expressible. The lookup
function can still recur through the pairs, looking for the match:

# let rec lookup (dict : ('key, 'value) dictionary)

# (request : 'key)
# : 'value option =
# match dict with
# | [] -> None
# | {key; value} :: tl ->
# if key = request then Some value
# else lookup tl request ;;
val lookup : ('key, 'value) dictionary -> 'key -> 'value option =
<fun>

# let good_dict = [{key = 1; value = "one"};

# {key = 2; value = "two"};
# {key = 3; value = "three"}] ;;
val good_dict : (int, string) dict_entry list =
[{key = 1; value = "one"}; {key = 2; value = "two"};
{key = 3; value = "three"}]
# lookup good_dict 3 ;;
- : string option = Some "three"
# lookup good_dict 4 ;;
- : string option = None

In this particular case, changing the structure of dictionaries to make

the illegal inexpressible also very slightly simplifies the lookup code
as well. But even if pursuing the edict of prevention makes code a bit
more complex, it can be well worth the trouble in preventing bugs
from arising in the first place.
Not all illegal states can be prevented by making them inexpressible
through the structuring of the types. For instance, this updated dictio-
nary structure still allows dictionaries that are ill-formed in allowing
the same key to occur more than once. We’ll return to this issue when
we further apply the edict of prevention in Chapter 12.
Problem 97
The game of mini-poker is played with just six playing cards: You use only the face cards
(king, queen, jack) of the two suits spades and diamonds. There is a ranking on the cards:
Any spade is better than any diamond, and within a suit, the cards from best to worst are
king, queen, jack.
In this two-player game, each player picks a single card at random, and the player
with the better card wins.
For the record, it’s a terrible game.
Provide appropriate type definitions to represent the cards used in the game. It
should contain structured information about the suit and value of the cards.
 A L G E B R A I C D ATA T Y P E S 153

Figure 11.2: The cards of mini-poker,

depicted in order from best to worst.

Problem 98
What is an appropriate type for a function better that determines which of two cards is
“better” in the context of mini-poker, returning true if and only if the first card is better
than the second?

Problem 99
Provide a definition of the function better.

11.4 Example: Arithmetic expressions

One of the elegancies admitted by the generality of algebraic data types

is their use in capturing languages.
By way of example, a language of simple integer arithmetic expres-
sions can be defined by the following grammar, written in Backus-Naur
form as described in Section 3.1.

⟨expr ⟩ ::= ⟨integer ⟩

| ⟨expr1 ⟩ + ⟨expr2 ⟩
| ⟨expr1 ⟩ - ⟨expr2 ⟩
| ⟨expr1 ⟩ * ⟨expr2 ⟩
| ⟨expr1 ⟩ / ⟨expr2 ⟩
| ~- ⟨expr ⟩

(We’ll take this to define the abstract syntax of the language. Concrete
syntax notions like precedence and associativity of the operators and
parentheses for disambiguating structure will be left implicit in the
usual way.)
We can define a type for abstract syntax trees for these arithmetic
expressions as an algebraic data type. The definition follows the gram-
mar almost trivially, one variant for each line of the grammar.

# type expr =
# | Int of int
# | Plus of expr * expr
# | Minus of expr * expr
# | Times of expr * expr
# | Div of expr * expr
# | Neg of expr ;;
type expr =
Int of int
| Plus of expr * expr
| Minus of expr * expr
154 PROGRAMMING WELL

| Times of expr * expr

| Div of expr * expr
| Neg of expr

The arithmetic expression given in OCaml concrete syntax as (3 + 4)

* ~- 5 corresponds to the following value of type expr:
# Times (Plus (Int 3, Int 4), Neg (Int 5)) ;;
- : expr = Times (Plus (Int 3, Int 4), Neg (Int 5))

A natural thing to do with expressions is to evaluate them. The

recursive definition of the expr type lends itself to recursive evaluation
of values of that type, as in this definition of a function eval : expr
-> int.

# let rec eval (exp : expr) : int =

# match exp with
# | Int v -> v
# | Plus (x, y) -> (eval x) + (eval y)
# | Minus (x, y) -> (eval x) - (eval y)
# | Times (x, y) -> (eval x) * (eval y)
# | Neg x -> ~- (eval x) ;;
Lines 2-7, characters 0-29:
2 | match exp with
3 | | Int v -> v
4 | | Plus (x, y) -> (eval x) + (eval y)
5 | | Minus (x, y) -> (eval x) - (eval y)
6 | | Times (x, y) -> (eval x) * (eval y)
7 | | Neg x -> ~- (eval x)...
Warning 8: this pattern-matching is not exhaustive.
Here is an example of a case that is not matched:
Div (_, _)
val eval : expr -> int = <fun>

Helpfully, the interpreter warns us of a missing case in the match.

One of the variants in the algebraic type definition, division, is not
covered by the match. A key feature of defining variant types is that
the interpreter can perform these kinds of checks on your behalf. The
oversight is easily corrected.

# let rec eval (exp : expr) : int =

# match exp with
# | Int v -> v
# | Plus (x, y) -> eval x + eval y
# | Minus (x, y) -> eval x - eval y
# | Times (x, y) -> eval x * eval y
# | Div (x, y) -> eval x / eval y
# | Neg x -> ~- (eval x) ;;
val eval : expr -> int = <fun>

We can test the evaluator with examples like the one above.

# eval (Times (Plus (Int 3, Int 4), Neg (Int 5))) ;;

- : int = -35
 A L G E B R A I C D ATA T Y P E S 155

# eval (Int 42) ;;

- : int = 42
# eval (Div (Int 5, Int 0)) ;;
Exception: Division_by_zero.

Of course, we already have a way of doing these arithmetic calcula-

tions in OCaml. We can just type the expressions into OCaml directly
using OCaml’s concrete syntax.

# (3 + 4) * ~- 5 ;;
- : int = -35
# 42 ;;
- : int = 42
# 5 / 0 ;;
Exception: Division_by_zero.

So what use is this kind of thing?

This evaluator is not trivial. By making the evaluation of this lan-
guage explicit, we have the power to change the language to diverge
from the language it is implemented in. For instance, OCaml’s inte-
ger division truncates the result towards zero. But maybe we’d rather
round to the nearest integer? We can implement the evaluator to do
that instead.

Exercise 100
Define a version of eval that implements a different semantics for the expression
language, for instance, by rounding rather than truncating integer divisions.

Exercise 101
Define a function e2s : expr -> string that returns a string that represents the fully
parenthesized concrete syntax for the argument expression. For instance,
# e2s (Times (Plus (Int 3, Int 4), Neg (Int 5))) ;;
- : string = "((3 + 4) * (~- 5))"
# e2s (Int 42) ;;
- : string = "42"
# e2s (Div (Int 5, Int 0)) ;;
- : string = "(5 / 0)"

The opposite process, recovering abstract syntax from concrete syntax, is called parsing.
More on this in the final project (Chapter 21).

(a)
11.5 Example: Binary trees

Trees are a class of data structures that store values of a certain type
in a hierarchically structured manner. They constitute a fundamen-
tal data structure, second only perhaps to lists in their repurposing
flexibility. Indeed, the arithmetic expressions of Section 11.4 are a
kind of tree structure, as are the binary search trees mentioned in
Section 11.6.1.
In this section, we concentrate on a certain kind of polymorphic
B I N A RY T R E E , a kind of tree whose nodes have distinct left and right
subtrees, possibly empty. Some examples can be seen in Figure 11.3.
(b)

Figure 11.3: Two trees: (a) an integer

156 PROGRAMMING WELL

A binary tree can be an empty tree (depicted with a bullet symbol (●)
in the diagrams), or a node that stores a single value (of type ’a, say)
along with two subtrees, referred to as the left and right subtrees.
A polymorphic binary tree type can thus be defined by the following
algebraic data type definition:

# type 'a bintree =

# | Empty
# | Node of 'a * 'a bintree * 'a bintree ;;
type 'a bintree = Empty | Node of 'a * 'a bintree * 'a bintree

For instance, the tree of Figure 11.3(a) can be encoded as an instance

of an int bintree as

# let int_bintree =
# Node (16, Node (93, Empty, Empty),
# Node (3, Node (42, Empty, Empty),
# Empty)) ;;
val int_bintree : int bintree =
Node (16, Node (93, Empty, Empty),
Node (3, Node (42, Empty, Empty), Empty))

Exercise 102
Construct a value str_bintree of type string bintree that encodes the tree of
Figure 11.3(b).
Now let’s write a function to sum up all of the elements stored in an
integer tree. The natural approach to carrying out the function is to
follow the recursive structure of its tree argument.

# let rec sum_bintree (t : int bintree) : int =

# match t with
# | Empty -> 0
# | Node (n, left, right) -> n + sum_bintree left
# + sum_bintree right ;;
val sum_bintree : int bintree -> int = <fun>

Exercise 103
Define a function preorder of type ’a bintree -> ’a list that returns a list of all of
the values stored in a tree in P R E O R D E R , that is, placing values stored at a node before
the values in the left subtree, in turn before the values in the right subtree. For instance,
# preorder int_bintree ;;
- : int list = [16; 93; 3; 42]

You’ll notice a certain commonality between the sum_bintree and

preorder functions. Both operate by “walking” the tree, traversing it
from its root down, recursively operating on the subtrees, and then
combining the value stored at a node and the recursively computed
values for the subtrees into the value for the tree as a whole. What
differs among them is what value to return for empty trees and what
function to apply to compute the overall value from the subparts. We
can abstract this tree walk functionality with a function that takes three
 A L G E B R A I C D ATA T Y P E S 157

arguments: (i) the value to use for empty trees, (ii) the function to ap-
ply at nodes to the value stored at the node and the values associated
with the two subtrees, along with (iii) a tree to walk; it carries out the
recursive process on that tree. Since this is a kind of “fold” operation
over binary trees, we’ll name the function foldbt.

Exercise 104
What is the appropriate type for the function foldbt just described?

Exercise 105
Define the function foldbt just described.

Exercise 106
Redefine the function sum_bintree using foldbt.

Exercise 107
Redefine the function preorder using foldbt.

Exercise 108
Define a function find : ’a bintree -> ’a -> bool in terms of foldbt, such that
find t v is true just in case the value v is found somewhere in the tree t.
# find int_bintree 3 ;;
- : bool = true
# find int_bintree 7 ;;
- : bool = false

11.6 Supplementary material

11.6.1 Lab 6: Variants, algebraic types, and pattern matching

In this lab, you will define and use algebraic data types to model res-
idential address information, and work with a specific type of binary
tree, the B I N A RY S E A R C H T R E E , which allows for efficient storage and
search of ordered information. A particular application is the use of
Figure 11.4: Saul Gorn with the
G O R N A D D R E S S E S , named after the early computer pioneer Saul Gorn
UNIVAC-1 computer at the Univer-
of University of Pennsylvania (Figure 11.4), who invented the tech- sity of Pennsylvania, 1950s.
nique.

11.7 Problem set 4: Symbolic differentiation

This section provides substantive background for one of the course prob-
lem sets. For common background on logistics and procedures for prob-
lem sets, including such topics as accessing the problem set code, compil-
ing and testing your code, reflecting on the process, and submitting your
solution, please review the course document “Problem set procedures”.

Solving an equation like x 2 = x + 1 N U M E R I C A L LY yields a partic-

ular number as an approximation to the solution for x, for instance,
1.618. Solving the equation S Y M B O L I C A L LY √
yields an expression repre-
senting the solution exactly, for instance, 1+2 5 . (The golden ratio! See
158 PROGRAMMING WELL

Exercise 8.) The earliest computing devices were used to calculate nu-
merically. Charles Babbage envisioned his analytical engine as a device
for calculating numeric tables, and Ada Lovelace’s famous program for
Babbage’s analytical engine numerically calculated Bernoulli numbers.
But Lovelace (Figure 11.5) was perhaps the first computer scientist
to have the revolutionary idea that computers could be used for much
more than numerical calculations.
The operating mechanism. . . might act upon other things besides num-
ber, were objects found whose mutual fundamental relations could be
expressed by those of the abstract science of operations, and which
should be also susceptible of adaptations to the action of the operating
notation and mechanism of the engine. Supposing, for instance, that
the fundamental relations of pitched sounds in the science of harmony
and of musical composition were susceptible of such expression and
adaptations, the engine might compose elaborate and scientific pieces Figure 11.5: A rare daguerrotype of Ada
Lovelace (Augusta Ada King, Countess
of music of any degree of complexity or extent. (Menabrea and Lovelace,
of Lovelace, 1815–1852) by Antoine
1843, page 694) Claudet, taken c. 1843, around the
time she was engaged in writing her
One of the applications of the power of computers to transcend nu- notes on the Babbage analytical engine.
merical calculation, which Lovelace immediately saw, was to engage in (Menabrea and Lovelace, 1843)

mathematics symbolically rather than numerically.

It seems to us obvious, however, that where operations are so indepen-
dent in their mode of acting, it must be easy by means of a few simple
provisions and additions in arranging the mechanism, to bring out a
double set of results, viz. – 1st, the numerical magnitudes which are the
results of operations performed on numerical data. (These results are
the primary object of the engine). 2ndly, the symbolical results to be
attached to those numerical results, which symbolical results are not
less the necessary and logical consequences of operations performed
upon symbolical data, than are numerical results when the data are
numerical. (Menabrea and Lovelace, 1843, page 694–5)

The first carrying out of symbolic mathematics by computer arose

over a hundred years later, in the work of Turing-Award-winning com-
puter scientist John McCarthy (Figure 11.6). In the summer of 1958,
McCarthy made a major contribution to the field of programming
languages. With the objective of writing a program that performed Figure 11.6: John McCarthy (1927–2011),
symbolic differentiation (that is, the process of finding the derivative one of the founders of (and coiner
of the term) artificial intelligence.
of a function) of algebraic expressions in an effective way, he noticed His LISP programming language was
that some features that would have helped him to accomplish this task widely influential in the history of
programming languages. He was
were absent in the programming languages of that time. This led him
awarded the Turing Award in 1971.
to the invention of the programming language LISP (McCarthy, 1960)
and other ideas, such as the concept of list processing (from which
LISP derives its name), recursion, and garbage collection, which are
essential to modern programming languages.
McCarthy saw that the power of higher-order functional program-
ming, together with the ability to manipulate structured data, make
 A L G E B R A I C D ATA T Y P E S 159

it possible to carry out such symbolic mathematics in an especially

elegant manner. However, it was Jean Sammet (Figure 11.7) who first
envisioned a full system devoted to symbolic mathematics more gen-
erally. Her FORMAC system (Sammet, 1993) ushered in a wave of
symbolic mathematics systems that have made good on Lovelace’s
original observation. Nowadays, symbolic differentiation of algebraic
expressions is a task that can be conveniently accomplished on mod-
ern mathematical packages, such as Mathematica and Maple.
This assignment focuses on using abstract data types to design
your own mini-language – a mathematical expression language over
which you’ll perform symbolic mathematics by computing derivatives
symbolically.
Figure 11.7: Jean Sammet (1928–
2017), head of the FORMAC project
to build “the first widely available
11.7.1 A language for symbolic mathematics programming language for symbolic
mathematical computation to have
In this section, your mission is to define a language of mathematical significant practical usage” (Sammet,
expressions representing functions of a single variable, and write code 1993). She was awarded the Augusta
Ada Lovelace Award in 1999 and the
that can differentiate (that is, take the derivative of) and evaluate those Computer Pioneer Award in 2009 for her
expressions. Symbolic expressions consist of numbers, variables, and work on FORMAC and (with Admiral
Grace Hopper) the programming
standard numeric functions (addition, subtraction, multiplication,
language COBOL.
division, exponentiation, negation, trigonometric functions, and so
forth) applied to them.
It may have been a long time since you thought about differential
calculus or trigonometric functions or taking a derivative. In fact,
maybe you’ve never studied any of that. Have no fear! This problem
set isn’t really about calculus or trig. Rather, it’s about manipulating
representations of expressions. We give you all of the formulas you need
to do the symbolic manipulations; all you need to do is represent them
in OCaml.

Conceptual Overview We want to be able to manipulate symbolic

expressions such as x 2 + sin(−x ), so we’ll need a way of representing
expressions as data in OCaml. For that purpose, we use OCaml types
to define the appropriate data structures. The expression data type
allows for four different kinds of expressions: numbers, variables, and
unary and binary operator expressions. For our purposes, only one
variable (call it x) is needed, and it will be represented by the Var con-
structor for the expression type. Numbers are represented with the
Num constructor, which takes a single float argument to specify which
number is being denoted. Binary operator expressions, in which a
binary operator like addition or division is applied to two subexpres-
sions, is represented by the Binop constructor, and similarly for unary
operators like sine or negation, which take only a single subexpression.
160 PROGRAMMING WELL

The expression data type can therefore be defined as follows (and

as provided to you in the file expressionLibrary.ml):
(* Binary operators. *)
type binop = Add | Sub | Mul | Div | Pow ;;

(* Unary operators. *)
type unop = Sin | Cos | Ln | Neg ;;

(* Expressions *)
type expression =
| Num of float
| Var
| Binop of binop * expression * expression
| Unop of unop * expression ;;

For instance, the mathematical expression x −2 would be represented

by this OCaml value:
Binop (Pow, Var, Unop (Neg, Num 2))

You can think of the data objects of this expression type as defin-
ing trees where nodes are the type constructors and the children of
each node are the specific operator to use and the arguments of that
constructor. These are just the abstract syntax trees of Section 3.3.
Although numeric expressions frequently make use of parentheses
– and sometimes necessarily so, as in the case of the expression (x +
3)(x − 1) – the expression data type definition has no provision for
parenthesization. Why isn’t that needed? It might be helpful to think
about how this example would be represented.

Provided code We have provided some functions to create and ma-

nipulate expression values. The checkexp function is contained in
expression.ml. The others are contained in expressionLibrary.ml.
Here, we provide a brief description of them and some example evalu-
ations.

• parse: Translates a string in infix form (such as "x^2 + sin(~x)")

into an expression (treating "x" as the variable). (The function
uses “~” for unary negation rather than “–”, to make it distinct
from binary subtraction.) The parse function parses according
to the standard order of operations – so "5+x*8" will be read as
"5+(x*8)".

# parse ("5+x*8") ;;
- : ExpressionLibrary.expression =
Binop (Add, Num 5., Binop (Mul, Var, Num 8.))

• to_string: Returns a string representation of an expression in a

readable form, using infix notation. This function adds parentheses
 A L G E B R A I C D ATA T Y P E S 161

around every binary operation so that the output is completely

unambiguous.

# let exp = Binop (Add,

# Binop (Pow, Var, Num 2.0),
# Unop (Sin, Binop (Div, Var, Num 5.0))) ;;
val exp : ExpressionLibrary.expression =
Binop (Add, Binop (Pow, Var, Num 2.), Unop (Sin, Binop (Div, Var,
Num 5.)))
# to_string exp ;;
- : string = "((x^2.)+(sin((x/5.))))"

• to_string_smart : Returns a string representation of an expression

in an even more readable form, only adding parentheses where
needed to override associativity and precedence.

# to_string_smart exp ;;
- : string = "x^2.+sin(x/5.)"

• rand_exp : Takes a length l and returns a randomly generated

expression of length at most 2l . Useful for generating expressions
for debugging purposes.

# let () = Random.init 2 (* for consistency *) ;;

# rand_exp 5 ;;
- : ExpressionLibrary.expression =
Binop (Mul, Unop (Ln, Num (-11.)), Binop (Sub, Var, Num (-17.)))
# rand_exp 5 ;;
- : ExpressionLibrary.expression = Binop (Mul, Num 4., Var)

• rand_exp_str : Takes a length l and returns a string representation

of length at most 2l .

# let () = Random.init 2 (* for consistency *) ;;

# rand_exp_str 5 ;;
- : string = "ln(~11.)*(x-~17.)"
# rand_exp_str 5 ;;
- : string = "4.*x"

• checkexp : Takes a string and a value and prints the results of call-
ing every function to be tested except find_zero.

Simple expression manipulation Start by implementing two func-

tions that perform simple expression manipulation. The function
contains_var : expression -> bool returns true when its argu-
ment contains a variable (that is, the constructor Var). The function
evaluate : expression -> float -> float takes an expression
and a numeric value for the variable in the expression and returns the
numerical evaluation of the expression at that value.
162 PROGRAMMING WELL

As a helpful note, in testing evaluate, rather than testing that

you get a particular float value, you may want to use unit_test_-
within to verify that the value is within a certain tolerance of the
answer you expect. This is necessary to avoid small differences due to
the imprecision of float arithmetic.

Symbolic differentation Next, we want to develop a function that

takes an expression e as its argument and returns an expression e’
representing the derivative of the expression with respect to x. This
process is referred to as symbolic differentiation.
When implementing this function, recall the chain rule from your
calculus course:

( f (g (x )))′ = f ′ (g (x )) ⋅ g ′ (x )

Using the chain rule, we can write the derivatives for the other func-
tions in our language, as shown in Figure 11.8.13 13
If the kinds of notation used here
are unfamiliar, the discussion in Sec-
tion A.1.3 may be helpful.

Figure 11.8: Rules for taking derivatives

( f (x ) + g (x ))′ = f ′ (x ) + g ′ (x ) for a variety of expression types.

( f (x ) − g (x ))′ = f ′ (x ) − g ′ (x )
( f (x ) ⋅ g (x ))′ = f ′ (x ) ⋅ g (x ) + f (x ) ⋅ g ′ (x )
f (x ) ( f ′ (x ) ⋅ g (x ) − f (x ) ⋅ g ′ (x ))
′
( ) =
g (x ) g (x )2
(sin f (x ))′ = f ′ (x ) ⋅ cos f (x )
(cos f (x ))′ = f ′ (x ) ⋅ ~ sin f (x )
f ′ (x )
(ln f (x ))′ =
f (x )
( f (x ) ) = h ⋅ f ′ (x ) ⋅ f (x )h −1
h ′

where h contains no variables

f ′ (x ) ⋅ g (x )
( f (x )g (x ) )′ = f (x )g (x ) ⋅ (g ′ (x ) ⋅ ln f (x ) + )
f (x )
(n )′ = 0 where n is any constant
(x ) = 1
′

We’ve provided two cases for calculating the derivative of f (x )g (x ) ,

one for where g (x ) is an expression (h) that contains no variables, and
one for the general case. The first is a special case of the second, but it
 A L G E B R A I C D ATA T Y P E S 163

is useful to treat them separately, because when the first case applies,
the second case produces unnecessarily complicated expressions.
Your task is to implement the derivative function whose type is
expression -> expression. The result of your function must be cor-
rect, but need not be expressed in the simplest form. Take advantage of
this in order to keep the code in this part as short as possible.
Once you’ve implemented the derivative function, you should be
able to calculate the symbolic derivative of various functions. Here’s an
example, calculating the derivative of x 2 + 5:
# to_string_smart (derivative (parse "x^2 + 5")) ;;
- : string = "2.*1.*x^(2.-1.)+0."

The result generated, 2 ⋅ 1 ⋅ x 2−1 + 0, isn’t in its simplest form, but it does
correctly capture the derivative, 2 ⋅ x.
To make your task easier, we have provided an outline of the func-
tion with many of the cases already filled in. We also provide a func-
tion, checkexp, which checks the functions you write in Problems
1–3 for a given input. The portions of the function that require your
attention currently read failwith "not implemented".

Zero-finding An application of the derivative of a function is to nu-

merically calculate the Z E R O S of a function, the values of its argument
that the function maps to zero. One way to do so numerically is New-
ton’s method.
Newton’s method to find a zero of a function f works by starting
with a guess x 0 and repeatedly calculating better and better estimates
of the zero x 1 , x 2 , x 3 , . . .. Starting with the initial guess x 0 , Newton’s
method proceeds by calculating the x n values according to the recur-
rence
f (x n )
x n +1 = x n − ′ ,
f (x n )
continuing until a good enough x is found. In determining if an x is
“good enough”, we make use of a small threshold ² below which we are
satisfied with an estimate’s closeness to a zero; in particular, we seek a
value x such that ∣ f (x )∣ < ², that is, the value that the expression eval-
uates to at x is “within ² of 0”. (We are not using the more demanding
requirement of seeking an x such that ∣x − x̂ ∣ < ² for some x̂ for which
f (x̂ ) = 0.)
Notice that the reestimation process in Newton’s method crucially
relies on the ability to evaluate both the function f itself and its deriva-
tive f ′ . Fortunately, you’ve already implemented functions to evaluate
expressions and to calculate their derivatives.
Your remaining task is then to implement the function find_zero
: expression -> float -> float -> int -> float option.
164 PROGRAMMING WELL

This function should take an expression (representing f ), a starting

guess for the zero (x 0 ), a precision requirement (²), and a limit on the
number of times to repeat the reestimation process. It should return
None if no zero was found within the desired precision by the time
the limit was reached, and Some x if a zero was found at x within
the desired precision. (You should return the first such value; there’s
no need to exhaust the maximum number of reestimations.) As an
example of the use of find_zero, we can find the zero of 3x − 1, starting
the search around 0 and looking for an estimate accurate to 0.0001
with a maximum of 100 reestimations:
# find_zero (parse "3 * x - 1") 0. 0.00001 100 ;;
- : float option = Some 0.333333333333333315

Note that there are cases where Newton’s method will fail to pro-
duce a zero, such as for the function x 1/3 . You are not responsible for
finding a zero in those cases, but just for the correct implementation of
Newton’s method.

Challenge problem: Symbolic zero-finding If you find yourself with

plenty of time on your hands after completing the problem set to this
point and submitting it successfully, feel free to try this week’s extra
challenge problem, which is completely optional but good for your
karma.
The function you wrote above allows you to find the zero (or a zero)
of most functions that can be represented with our expression lan-
guage. This makes it quite powerful. However, in addition to numeric
solving like this, Mathematica and many similar programs can per-
form symbolic algebra. These programs can solve equations using
techniques similar to those you learned in middle and high school (as
well as more advanced techniques for more complex equations) to
get exact, rather than approximate answers. For example, given the
expression representing 3x − 1, your find_zero function might return
something like 0.33333, depending on your value of ². The exact so-
lution, however is given by the expression 1/3, and this answer can be
found by a program that solves equations symbolically.
Performing the symbolic manipulations on complex expressions
necessary to solve equations is quite difficult in general, and we do not
expect you to handle the general case. However, there is one type of
expression for which symbolic zero-finding is not so difficult. These
are expressions of degree one, those that can be simplified to the form
a ⋅ x + b, where the highest exponent of the variable is 1. You likely
learned how to solve equations of the form a ⋅ x + b = 0 years ago, and
can apply the same skills in writing a program to solve these.
Write a function, find_zero_exact which will exactly find the
 A L G E B R A I C D ATA T Y P E S 165

zero of degree one expressions. Your function should, given a valid

input expression that has a zero, return Some exp where exp is an
expression that contains no variables, evaluates to the zero of the given
expression, and is exact. If the expression is not degree one or has no
zero, it should return None.
You need not return the simplest expression, though it could be
instructive to think about how to simplify results. For example, find_-
zero_exact (parse "3*x-1") might return Binop (Div, Num
1., Num 3.) or Unop(Neg, Binop (Div, Num -1., Num 3.))
but should not return Num 0.333333333 as this is not exact.
Note that degree-one expressions need not be as simple as ax + b.
Something like 5x − 3 + 2(x − 8) is also a degree-one expression, since
it can be simplified to ax + b by distributing and simplifying. You
may want to think about how to handle these types of expressions as
well, and think more generally about how to determine whether an
expression is of degree one.
Hint: You may want to start by writing a function that will crawl over
an expression and distribute any multiplications or divisions, resulting
in something of a form like ax + b (or maybe ax + bx + c x + d + e + f or
similar).
12
Abstract data types and modular programming

The algebraic data types introduced in the last chapter are an expres-
sive tool for defining sophisticated data structures. But with great
power comes great responsibility.
As an example, consider one of the most fundamental of all data
structures, the QU E U E . A queue is a collection of elements that admits
of operations like creating an empty queue, adding elements one by
one (called E N QU E U E I N G ), and removing them one-by-one (called
D E QU E U I N G ), where crucially the first element enqueued is the first to
be dequeued. The common terminology for this regimen is F I R S T- I N -
F I R S T- O U T or FIFO.
We can provide a concrete implementation of the queue data type
using the list data type, along with functions for enqueueing and de-
queueing. An empty queue will be implemented as the empty list, with
non-empty queues storing elements in order of their enqueueing, so
newly enqueued elements are added at the end of the list.

# (* empty_queue -- An empty queue *)

# let empty_queue = [] ;;
val empty_queue : 'a list = []

# (* enqueue elt q -- Returns a queue resulting from

# enqueuing a new elt onto q. *)
# let enqueue (elt : 'a) (q : 'a list) : 'a list =
# q @ [elt] ;;
val enqueue : 'a -> 'a list -> 'a list = <fun>

# (* dequeue q -- Returns a pair of the next element

# in q and the queue resulting from dequeueing
# that element. *)
# let dequeue (q : 'a list) : 'a * 'a list =
# match q with
# | [] -> raise (Invalid_argument
# "dequeue: empty queue")
# | hd :: tl -> hd, tl ;;
val dequeue : 'a list -> 'a * 'a list = <fun>
168 PROGRAMMING WELL

We can use these functions to enqueue and then dequeue a series

of integers. Notice how the first element enqueued (the 1) is the first
element dequeued.

# let q = empty_queue
# |> enqueue 1 (* enqueue 1, 2, and 4 *)
# |> enqueue 2
# |> enqueue 4 ;;
val q : int list = [1; 2; 4]
# let next, q = dequeue q ;; (* dequeue 1 *)
val next : int = 1
val q : int list = [2; 4]
# let next, q = dequeue q ;; (* dequeue 2 *)
val next : int = 2
val q : int list = [4]
# let next, q = dequeue q ;; (* dequeue 4 *)
val next : int = 4
val q : int list = []

Data structures built in this way can be used as intended, as they

were above. (You’ll note the FIFO behavior.) But if used in unexpected
ways, things can go wrong quickly. Here, for instance, we enqueue
some integers, then reverse the queue before dequeuing the elements
in a last-in-first-out (LIFO) order. That’s not supposed to happen.

# let q = empty_queue
# |> enqueue 1 (* enqueue 1, 2, and 4 *)
# |> enqueue 2
# |> enqueue 4
# |> List.rev ;; (* yikes! *)
val q : int list = [4; 2; 1]
# let next, q = dequeue q ;; (* dequeue 4 *)
val next : int = 4
val q : int list = [2; 1]
# let next, q = dequeue q ;; (* dequeue 2 *)
val next : int = 2
val q : int list = [1]
# let next, q = dequeue q ;; (* dequeue 1 *)
val next : int = 1
val q : int list = []

Of course, reversing the elements is not an operation that ought to be

possible on a queue. Queues, like other data structures, are defined
by what operations can be performed on them, namely, enqueue
and dequeue. These operations obey an I N VA R I A N T , that the order
in which elements appear when dequeued is the same as the order
in which they were enqueued. Performing inappropriate operations
on data structures is the path to violating such invariants, leading to
software errors. Our implementation of queues as lists allows all sorts
of inappropriate operations, like reversal of the enqueued elements,
or taking the n-th element, or mapping over the elements, or any
 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 169

other operation appropriate for lists but not queues. What we need is
the ability to enforce restraint on the operations applicable to a data
structure so as to preserve the invariants.
An analogy: The lights and heating in hotel rooms are intended to
be on when the room is occupied, but they should be lowered when
the room is empty. We can think of this as an invariant: If the room is
unoccupied, the lights and heating are off. One approach to increasing
compliance with this invariant is through documentation, placing a (a) (b)
sign at the door “Please turn off the lights when you leave.” But many Figure 12.1: Two approaches to pre-
serving the invariant that the lights
hotels now use a key card switch, a receptacle near the door in which are off when the room is vacant: (a) an
you insert the key card for the hotel room when you enter, in order exhortation documenting the invariant;
(b) a key card switch that disables the
to enable the lights and heating. (See Figure 12.1.) Since you have
lights when the key is removed.
to bring your key card with you when you leave the room, thereby
disabling the lights and heating, there is literally no way to violate
the invariant. The state of California estimates that widespread use
of hotel key card switches saves tens of millions of dollars per year
(California Utilities Statewide Codes and Standards Team, 2011, page
6). Preventing violation of an invariant beats documenting it.
We’ve seen this idea of avoiding illegal states before in the edict of
prevention. But in the queue example, type checking doesn’t stop us
from representing a bad state, and simple alternative representations
for queues that prevent inappropriate operations don’t come to mind.
We need a way to implement new data types and operations such that
the values of those types can only be used with the intended opera-
tions. We can’t make the bad queues unrepresentable, but perhaps we
can make them inexpressible, which should be sufficient for gaining
the benefit of the edict of prevention.
The key idea is to provide an A B S T R A C T D ATA T Y P E (ADT), a data
type definition that provides not only a concrete I M P L E M E N TAT I O N
of the data type values and operations on them, but also enforces that
only those operations can be applied, making it impossible to express
the application of other operations. This influential idea, the basis for
modular programming, was pioneered by Barbara Liskov (Figure 12.2
in her CLU programming language.
The allowed operations are specified in a S I G N AT U R E ; no other
Figure 12.2: The idea of abstract data
aspects of the implementation of the data type can be seen other types – grouping some functionality
than those specified by the signature. Users of the abstract data type over types and hiding the implementa-
tion of that functionality behind a strict
can avail themselves of the functionality specified in the signature,
interface – is due to computer scientist
while remaining oblivious of the particularities of the implementa- Barbara Liskov, and is first seen in her
tion. The signature specifies an interface to using the data structure, influential CLU programming language
from 1974. Her work on data abstraction
which serves as an A B S T R A C T I O N B A R R I E R ; only the aspects of the and object-oriented programming led
implementation specified in the signature may be made use of. to her being awarded the 2008 Turing
Award, computer science’s highest
The idea of hiding aspects of the implementation from those who
honor.
170 PROGRAMMING WELL

shouldn’t need access to those aspects is fundamental enough for an

edict of its own, the edict of compartmentalization:

Edict of compartmentalization:
Limit information to those with a need to know.

In the case of the queue abstract data type, all that users of the
implementation have a need to know is the types for the operations
involving queues, viz., the creation of queues and the enqueueing and
dequeueing of elements; that’s all the signature should specify. The im-
plementation may be in terms of lists (or any of a wide variety of other
methods) but the users of the abstract data type should not be able to
avail themselves of the further aspects of the implementation. By pre-
venting them from using aspects of the implementation, the invariants
implicit in the signature can be maintained. A further advantage of
hiding the details of the implementation of a data structure behind the
abstraction barrier (in addition to making illegal operations inexpress-
ible) is that it becomes possible to modify the implementation without
affecting its use. This aspect of abstract data types is tremendously
powerful.
We’ve seen other applications of the edict of compartmentaliza-
tion before, for instance, in the use of helper functions local to (and
therefore only accessible to) a function being defined. The alternative,
defining the helper function globally could lead to unintended use of
and reliance on that function, which had been intended only for its
more focused purpose.

12.1 Modules

In OCaml, abstract data types are implemented using M O D U L E S . Mod-

ules provide a way of packaging together several components – types
and values involving those types, including functions manipulating
values of those types – subject to constraints of a signature. A module
is specified by placing the definitions of its components between the
keywords struct and end:
struct
⟨definition1 ⟩
⟨definition2 ⟩
⟨definition3 ⟩
...
end

Each ⟨definition ⟩ is a definition of a type or value (including functions,

and even exceptions).
 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 171

Just as values can be named using the let construct, modules can
be named using the module construct:

module ⟨modulename ⟩ =
⟨moduledefinition ⟩

12.2 A queue module

As a first example of the use of modules to provide for abstract data

types, we return to the queue data type that we started with, which
provides a type for, say, integer queues, int_queue, together with func-
tions enqueue : int -> int_queue -> int_queue and dequeue :
int_queue -> int * int_queue. (Even better would be to general-
ize queues as polymorphically allowing for elements of any base type.
We’ll do so in Section 12.4.)
A module IntQueue1 implementing the queue abstract data type is 1
Module names are required to begin
with an uppercase letter. You’ve seen
# (* IntQueue -- An implementation of integer queues as examples before in the Stdlib and List
# int lists, where the elements are kept with older module names.
# elements closer to the head of the list. *)
# module IntQueue =
# struct
# type int_queue = int list
# let empty_queue : int_queue = []
# let enqueue (elt : int) (q : int_queue)
# : int_queue =
# q @ [elt]
# let dequeue (q : int_queue) : int * int_queue =
# match q with
# | [] -> raise (Invalid_argument
# "dequeue: empty queue")
# | hd :: tl -> hd, tl
# end ;;
module IntQueue :
sig
type int_queue = int list
val empty_queue : int_queue
val enqueue : int -> int_queue -> int_queue
val dequeue : int_queue -> int * int_queue
end

Exercise 109
Define a different implementation of integer queues as int lists where the elements
are kept with older elements farther from the head of the list. What are the advantages
and disadvantages of this implementation?
Components of a module are referenced using the already fa-
miliar notation of prefixing the module name and a dot before the
component. We’ve seen this already in examples like List.nth or
Str.split. Similarly, users of the IntQueue module can refer to
172 PROGRAMMING WELL

IntQueue.empty_queue or IntQueue.enqueue. Let’s use this mod-

ule to perform various queue operations:

# let q = IntQueue.empty_queue
# |> IntQueue.enqueue 1 (* enqueue 1, 2, and 4 *)
# |> IntQueue.enqueue 2
# |> IntQueue.enqueue 4 ;;
val q : IntQueue.int_queue = [1; 2; 4]

All of this module prefixing gets cumbersome quickly. We can instead

just “open” the module to gain access to all of its components.2

# open IntQueue ;;
# let q = empty_queue
# |> enqueue 1 (* enqueue 1, 2, and 4 *)
# |> enqueue 2
# |> enqueue 4 ;;
val q : IntQueue.int_queue = [1; 2; 4]
# let next, q = dequeue q ;; (* dequeue 1 *)
val next : int = 1
val q : IntQueue.int_queue = [2; 4]
# let next, q = dequeue q ;; (* dequeue 2 *)
val next : int = 2
val q : IntQueue.int_queue = [4] 2
A useful technique to simplify access
# let next, q = dequeue q ;; (* dequeue 4 *) to a module without opening it (and
val next : int = 4 thereby shadowing any existing names)
val q : IntQueue.int_queue = [] is to provide a short alternative name for
the module.

Unfortunately, nothing restricts us from using arbitrary aspects of # module IQ = IntQueue ;;

module IQ = IntQueue
the module’s implementation, for instance, reversing the elements of # let q = IQ.empty_queue
the queue. # |> IQ.enqueue 1
# |> IQ.enqueue 2
# open IntQueue ;; # |> IQ.enqueue 4 ;;
# let q = empty_queue val q : IQ.int_queue = [1; 2; 4]

# |> enqueue 1 (* enqueue 1, 2, and 4 *) Also of great utility is to open a module

# |> enqueue 2 just within a particular local scope.
# |> enqueue 4 OCaml provides for this with its L O C A L
# |> List.rev ;; O P E N construct:

val q : int list = [4; 2; 1] # let q =

# let next, q = dequeue q ;; (* dequeue 1 *) # let open IntQueue in

# empty_queue
val next : int = 4
# |> enqueue 1
val q : IntQueue.int_queue = [2; 1]
# |> enqueue 2
# let next, q = dequeue q ;; (* dequeue 2 *)
# |> enqueue 4 ;;
val next : int = 2 val q : IntQueue.int_queue = [1; 2; 4]
val q : IntQueue.int_queue = [1]
# let next, q = dequeue q ;; (* dequeue 4 *)
val next : int = 1
val q : IntQueue.int_queue = []

What we need is a signature that restricts the use of the compo-

nents of a module, just as a type restricts use of a value. This signa-
ture/module pairing carefully separates what the caller of code sees
– the module signature, which provides the abstract type structure
 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 173

of the components, that is, how they are used – from what the imple-
menter or developer sees – the module implementation, including the
concrete types and values for the components, that is, how they are
implemented.
The notation for specifying signatures is similar to that for modules,
except for the use of sig instead of struct; and naming signatures is
like naming modules with the addition of the evocative type keyword.

module type ⟨moduletype ⟩ =

sig
⟨definition1 ⟩
⟨definition2 ⟩
⟨definition3 ⟩
...
end

We can define a signature INT_QUEUE3 for an integer queue module: 3

Signature names must also begin
with an uppercase letter. We follow
# module type INT_QUEUE = the stylistic convention of using all
# sig uppercase for signature names.
# type int_queue
# val empty_queue : int_queue
# val enqueue : int -> int_queue -> int_queue
# val dequeue : int_queue -> int * int_queue
# end ;;
module type INT_QUEUE =
sig
type int_queue
val empty_queue : int_queue
val enqueue : int -> int_queue -> int_queue
val dequeue : int_queue -> int * int_queue
end

The signature provides a full listing of all the aspects of a module that
are visible to users of the module. In particular, the module provides a
type called int_queue, but since the concrete implementation of that
type is not provided in the signature, it is unavailable to users of mod-
ules satisfying the signature. The signature states that the module must
provide a value empty_queue but what the concrete implementation of
that value is is again hidden. And so on.
Notice that where the module implementation defines named
values using the let construct, the signature uses the val construct,
which provides a name and a type, but no definition of what is named.
Extending the analogy between signatures and types further, we can
specify that a module satisfies and is constrained by a signature with a
notation almost identical to that constraining a value to a certain type.

module ⟨modulename ⟩ : ⟨signature ⟩ =

⟨moduledefinition ⟩
174 PROGRAMMING WELL

We could define IntQueue as satisfying the INT_QUEUE signature by

adding this kind of “typing” as in the highlighted addition below:

# (* IntQueue -- An implementation of integer queues as

# int lists, where the elements are kept with older
# elements closer to the head of the list. *)
# module IntQueue : INT_QUEUE =
# struct
# type int_queue = int list
# let empty_queue : int_queue = []
# let enqueue (elt : int) (q : int_queue)
# : int_queue =
# q @ [elt]
# let dequeue (q : int_queue) : int * int_queue =
# match q with
# | [] -> raise (Invalid_argument
# "dequeue: empty queue")
# | hd :: tl -> hd, tl
# end ;;
module IntQueue : INT_QUEUE

This module implements integer queues abstractly, allowing access

only as specified by the INT_QUEUE signature. For instance, after build-
ing a queue, we no longer have access to its concrete implementation.

# open IntQueue ;;
# let q = empty_queue
# |> enqueue 1 (* enqueue 1, 2, and 4 *)
# |> enqueue 2
# |> enqueue 4 ;;
val q : IntQueue.int_queue = <abstr>

The value of q is reported simply as <abstr> connoting an abstract

value hidden behind the abstraction barrier. We can’t “see inside”.
Similarly, application of an operation not sanctioned by the signature,
like list reversal, now fails.

# List.rev q ;;
Line 1, characters 9-10:
1 | List.rev q ;;
^
Error: This expression has type IntQueue.int_queue
but an expression was expected of type 'a list

OCaml reports a type error. The function List.rev requires an ar-

gument of type ’a list, but it is being applied to a queue, of type
IntQueue.int_queue. True, the type IntQueue.int_queue is im-
plemented as an ’a list, but that fact is hidden from users of the
module by the signature, hidden behind the abstraction barrier.
 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 175

12.3 Signatures hide extra components

What happens when a module defines more components than its sig-
nature provides for? As a trivial example, we will define an O R D E R E D
TYPE as a type that has an associated comparison function that pro-
vides an ordering on elements of the type. The definition of such a
module provides for these two components: a type, call it t, and a
function that takes two elements x and y of type t and returns an inte-
ger indicating the ordering of the two, -1 if x is smaller, +1 if x is larger,
and 0 if the two are equal in the ordering.4 4
We use this arcane approach for
the compare function to mimic the
This specification of what constitutes an ordered type can be cap-
Stdlib.compare library function.
tured in a signature ORDERED_TYPE: Frankly, a better approach would be
to take the result of the comparison to
# module type ORDERED_TYPE = be a value in an enumerated type type
# sig order = Less | Equal | Greater.
# type t
# val compare : t -> t -> int
# end ;;
module type ORDERED_TYPE = sig type t val compare : t -> t -> int
end

A simple implementation of an ordered type is based on the string

type. Notice that we explicitly specify the signature for the module:

# module StringOrderedType : ORDERED_TYPE =

# struct
# type t = string
# let compare = Stdlib.compare
# end ;;
module StringOrderedType : ORDERED_TYPE

We take advantage of the built in compare function in the Stdlib

module,5 which is a general purpose comparison function that uses 5
Although the Stdlib prefix isn’t
needed – the components of the Stdlib
the same return value convention of -1, 0, +1 for elements that are less
module are always available – we add it
than, equal, and greater than, respectively. A more interesting example here for clarity.
is an ordered type for points (pairs of floats) where the ordering on
points is based on which is closer to the origin. This time, however, we
don’t specify a signature for the module:

# module PointOrderedType =
# struct
# type t = float * float
# let norm (x, y) =
# x ** 2. +. y ** 2.
# let compare p1 p2 =
# Stdlib.compare (norm p1) (norm p2)
# end ;;
module PointOrderedType :
sig
type t = float * float
val norm : float * float -> float
176 PROGRAMMING WELL

val compare : float * float -> float * float -> int

end

We can make use of the module to see how this ordering works on
some examples.
# let open PointOrderedType in
# compare (1., 1.) (5., 0.),
# compare (1., 1.) (-1., -1.),
# compare (1., 1.) (0., 1.1) ;;
- : int * int * int = (-1, 0, 1)

Note that the PointOrderedType module contains three compo-

nents: the type t, and functions norm and compare. It goes beyond the
ORDERED_TYPE signature in providing an extra function,

# PointOrderedType.norm ;;
- : float * float -> float = <fun>
# PointOrderedType.norm (1., 1.) ;;
- : float = 2.

since we did not explicitly restrict it to that signature. If instead we

restrict PointOrderedType to the ORDERED_TYPE signature, only the
components in that signature are made available.
# module PointOrderedType : ORDERED_TYPE =
# struct
# type t = float * float
# let norm (x, y) =
# x ** 2. +. y ** 2.
# let compare p1 p2 =
# Stdlib.compare (norm p1) (norm p2)
# end ;;
module PointOrderedType : ORDERED_TYPE

The norm function is no longer defined:

# PointOrderedType.norm ;;
Line 1, characters 0-21:
1 | PointOrderedType.norm ;;
^^^^^^^^^^^^^^^^^^^^^
Error: Unbound value PointOrderedType.norm

In general, only the aspects of a module consistent with its signature are
visible outside of its implementation to users of the module. All other
aspects are hidden behind the abstraction barrier. In particular, the
norm function is not available, and the identity of the type t is hidden
as well. We can tell, because we no longer can compare two points.
# PointOrderedType.compare (1., 1.) (5., 0.) ;;
Line 1, characters 25-33:
1 | PointOrderedType.compare (1., 1.) (5., 0.) ;;
^^^^^^^^
Error: This expression has type 'a * 'b
but an expression was expected of type PointOrderedType.t
 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 177

The arguments we are providing are expected to be of type t but we

are providing arguments of type float * float. Although the im-
plementation equates these types, outside of the abstraction barrier
their equality isn’t known. (Yes, this is a problem. We’ll address it using
sharing constraints later in Section 12.5.2.)
A fundamental role of modules and their signatures is to establish
these abstraction barriers so that information about how data types
happen to be implemented can’t leak out and be taken advantage of.

12.4 Modules with polymorphic components

Returning to the queue example, there’s no reason to restrict queues to

integer elements. We can make the components of the module poly-
morphic, using type variables as usual to capture the places where
arbitrary types can appear. We start with a polymorphic queue signa-
ture:

# module type QUEUE = sig

# type 'a queue
# val empty_queue : 'a queue
# val enqueue : 'a -> 'a queue -> 'a queue
# val dequeue : 'a queue -> 'a * 'a queue
# end ;;
module type QUEUE =
sig
type 'a queue
val empty_queue : 'a queue
val enqueue : 'a -> 'a queue -> 'a queue
val dequeue : 'a queue -> 'a * 'a queue
end

and define a queue module satisfying the signature:

# (* Queue -- An implementation of polymorphic queues

# as lists, where the elements are kept with older
# elements closer to the head of the list. *)
# module Queue : QUEUE = struct
# type 'a queue = 'a list
# let empty_queue : 'a queue = []
# let enqueue (elt : 'a) (q : 'a queue) : 'a queue =
# q @ [elt]
# let dequeue (q : 'a queue) : 'a * 'a queue =
# match q with
# | [] -> raise (Invalid_argument
# "dequeue: empty queue")
# | hd :: tl -> hd, tl
# end ;;
module Queue : QUEUE

Now we can avail ourselves of queues of different types:

178 PROGRAMMING WELL

# open Queue ;;
# let intq = empty_queue
# |> enqueue 1
# |> enqueue 2 ;;
val intq : int Queue.queue = <abstr>
# let boolq = empty_queue
# |> enqueue true
# |> enqueue false ;;
val boolq : bool Queue.queue = <abstr>
# dequeue intq ;;
- : int * int Queue.queue = (1, <abstr>)
# dequeue boolq ;;
- : bool * bool Queue.queue = (true, <abstr>)

Exercise 110
In Section 11.3, we provided a data type for dictionaries that makes sure that the keys
and values match up properly. We noted, however, that nothing prevents building a
dictionary with multiple occurrences of the same key.
Define a dictionary module signature and implementation that implements dictio-
naries using the type from Section 11.3, and provides a function

add : (’a, ’b) dictionary -> ’a -> ’b -> (’a, ’b)

dictionary
for adding a key and its value to a dictionary and a function

lookup : (’a, ’b) dictionary -> ’a -> ’b option

for looking keys up in the dictionary. The add function should raise an appropriate
exception if the key being added already appears in the dictionary. The lookup function
should return None if the key being looked up does not appear in the dictionary. The
signature should hide the implementation of the type and the functions so that the only
access to the dictionary is through these two functions.
Can you express a dictionary built using this module that has duplicate keys?

12.5 Abstract data types and programming for change

One of the primary advantages of using abstract data types (as op-
posed to concrete data structures) is that by hiding the data type im-
plementations, the implementations can be changed without affecting
users of the data types.
Recall the query type from Section 11.2.

# type query =
# | Word of string
# | And of query * query
# | Or of query * query ;;
type query = Word of string | And of query * query | Or of query *
query

In that section, a corpus of documents was structured as a list of pairs,

each containing a name and a list of strings, the words in the docu-
ment. Given that we’ll be searching for particular words in documents,
an alternative data structure useful for search is the R E V E R S E I N D E X ,
 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 179

a kind of dictionary with words as the keys and a set of document

identifiers (the title strings, say) as the values.
If we implement this concretely, using a list of pairs for the dictio-
nary and a string list for the set of document titles, we end up with the
following type:

# type index = (string * (string list)) list ;;

type index = (string * string list) list

Using a reverse index, the code for evaluating a query is quite simple:

# let rec eval (q : query)

# (idx : index)
# : string list =
# match q with
# | Word word ->
# let (_key, targets) =
# List.find (fun (w, _lst) -> w = word) idx
# in targets
# | And (q1, q2) ->
# intersection (eval q1 idx) (eval q2 idx)
# | Or (q1, q2) ->
# (eval q1 idx) @ (eval q2 idx) ;;
Line 10, characters 0-12:
10 | intersection (eval q1 idx) (eval q2 idx)
^^^^^^^^^^^^
Error: Unbound value intersection

Of course, we’ll need code for the intersection of two lists. Here’s an
approach, in which the lists are kept sorted to facilitate finding dupli-
cates:

# let rec intersection set1 set2 =

# match set1, set2 with
# | [], _
# | _, [] -> []
# | h1 :: t1, h2 :: t2 ->
# if h1 = h2 then h1 :: intersection t1 t2
# else if h1 < h2 then intersection t1 set2
# else intersection set1 t2 ;;
val intersection : 'a list -> 'a list -> 'a list = <fun>

Now, we might get lucky and notice a problematic clash of assump-

tions in the eval function. The intersection function assumes the
lists are sorted, but the final match in eval just appends two lists to
form the union of their elements. Nothing guarantees that the result of
the union is sorted. We can fix that up by using a sort function from the
List module.

# let rec eval (q : query)

# (idx : index)
# : string list =
# match q with
180 PROGRAMMING WELL

# | Word word ->

# let (_, targets) =
# List.find (fun (w, _lst) -> w = word) idx
# in targets
# | And (q1, q2) ->
# intersection (eval q1 idx) (eval q2 idx)
# | Or (q1, q2) ->
# List.sort_uniq compare
# ((eval q1 idx) @ (eval q2 idx)) ;;
val eval : query -> index -> string list = <fun>

But maybe then we notice that in our application, this List.find

lookup takes too much time. It has to look through the elements of
the list sequentially to find the one for the word we’re looking up. That
takes time proportional to the number of words being indexed. (More
on this kind of issue in Chapter 14.) Maybe you recall from an earlier
course that hash tables allow lookup in constant time, and you think to
use them. Luckily, the Hashtbl library module provides hash tables. To
incorporate hash tables, we have to change the index type:

# type index = (string, string list) Hashtbl.t ;;

type index = (string, string list) Hashtbl.t

as well as the word query lookup:

# let rec eval (q : query)

# (idx : index)
# : string list =
# match q with
# | Word word -> Hashtbl.find idx word
# | And (q1, q2) ->
# intersection (eval q1 idx) (eval q2 idx)
# | Or (q1, q2) ->
# List.sort_uniq compare
# ((eval q1 idx) @ (eval q2 idx)) ;;
val eval : query -> index -> string list = <fun>

There’s a theme here. Every change to the underlying data repre-

sentation requires multiple changes to the code, even though nothing
has changed conceptually in the underlying use of the data. We’re still
searching in the data, taking unions and intersections.
Let’s go back to the original specification of the reverse index: “a
kind of dictionary with words as the keys and a set of document identi-
fiers (the title strings, say) as the values.” This specification talks about
abstract data types like dictionaries and sets, but we’ve been trying to
directly implement them in terms of lists and pairs and hash tables.
By embracing the abstractions, we can hide all of the details from our
indexing code.
Suppose we had modules for string sets and for indexes. The string
set module, call it StringSet, would presumably provide set functions
 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 181

like union and intersection. The index module, call it Index would
provide a lookup function. The eval function using these modules
then becomes

let rec eval (q : query)

(idx : Index.dict)
: StringSet.set =
match q with
| Word word -> (match Index.lookup idx word with
| None -> StringSet.empty
| Some v -> v)
| And (q1, q2) -> StringSet.intersection (eval q1 idx)
(eval q2 idx)
| Or (q1, q2) -> StringSet.union (eval q1 idx)
(eval q2 idx) ;;

This is much nicer. It says what the code does at the right level of ab-
straction, in terms of high-level operations like dictionary lookup, or
set intersection and union. It remains silent, as it should, about exactly
how those operations are implemented.
Now we’ll need module definitions for Index and StringSet. We
start with StringSet first, and in particular, its module signature,
since this specifies how the module can be used.

12.5.1 A string set module

A string set module needs to provide some operations for creating and
manipulating the sets. The requirements can be specified in a module
signature. Here’s a first cut:

# module type STRING_SET =

# sig
# (* Type of string sets *)
# type set
# (* An empty set *)
# val empty : set
# (* Returns true if set is empty, false otherwise *)
# val is_empty : set -> bool
# (* Adds string to existing set (if not already a member) *)
# val add : string -> set -> set
# (* Union of two sets *)
# val union : set -> set -> set
# (* Intersection of two sets *)
# val intersection : set -> set -> set
# (* Returns true iff string is in set *)
# val member: string -> set -> bool
# end ;;
module type STRING_SET =
sig
type set
val empty : set
val is_empty : set -> bool
182 PROGRAMMING WELL

val add : string -> set -> set

val union : set -> set -> set
val intersection : set -> set -> set
val member : string -> set -> bool
end

Any implementation of this signature must provide:

• a type, called set;

• an element of that type called empty;

• a function that maps elements of the type to bool, called is_empty;

• and so forth.

From the point of view of the users (callers) of this abstract data
type, this is all they need to know: The name of the type and the func-
tions that apply to values of that type.
To drive this point home, we’ll make use of an implementation
(StringSet) of this abstract data type before even looking at the im-
plementing code.
# let s = StringSet.add "c"
# (StringSet.add "b"
# (StringSet.add "a" StringSet.empty)) ;;
val s : StringSet.set = <abstr>

Note that the string set we’ve called s is of the abstract type
StringSet.set and the particulars of the value implementing the
set are hidden from us as <abstr>.
The types, values, and functions provided in the signature are nor-
mal OCaml objects that interact with the rest of the language as usual.
We can still avail ourselves of the rest of OCaml. For instance, we can
clean up the definition of s using reverse application and a local open:
# let s =
# let open StringSet in
# empty
# |> add "a"
# |> add "b"
6
You’ll notice that we don’t bother
# |> add "c" ;;
adding types to the definitions of the
val s : StringSet.set = <abstr>
values in this module implementation.
Since the signature already provided
Other operations work as well.
explicit types (satisfying the edict of
# StringSet.member "a" s ;; intention), OCaml can verify that the
- : bool = true implementation respects those types.
# StringSet.member "d" s ;; Nonetheless, it can sometimes be useful
to provide further typing information in
- : bool = false
a module implementation.
Of course, the ADT must have an actual implementation for it to work.
We’ve just been assuming one, but we can provide a possible imple-
mentation (the one we’ve been using as it turns out), obeying the
specific signature we just defined.6
 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 183

module StringSet : STRING_SET =

(* Implementation of STRING_SET as list of strings.
Assumes list may be unsorted but with no duplicates. *)
struct
type set = string list
let empty = []
let is_empty set = (set = [])
let member = List.mem
let add elt set =
if List.mem elt set then set
else elt :: set
let union = List.fold_right add
let rec intersection set1 set2 =
match set1 with
| [] -> []
| hd :: tl -> let tlint = intersection tl set2 in
if member hd set2 then add hd tlint
else tlint
end ;;

In this implementation, sets are implemented as string lists. A com-

ment documents the invariant in the implementation that the lists
have no duplicates, though they might not be sorted. But there’s no
way for a user of this module to know any of that; the signature doesn’t
reveal anything about the implementation type. Even though the sets
are implemented as string lists, if we try to do string-list-like opera-
tions, we’ll be thwarted.
# s @ ["b"; "e"] ;;
Line 1, characters 0-1:
1 | s @ ["b"; "e"] ;;
^
Error: This expression has type StringSet.set
but an expression was expected of type 'a list

And it’s a good thing too, because if we could have added the "b"
to the list, suddenly, the list doesn’t obey the invariant required by
the implementation that there be no duplicates. But because of the
abstraction barrier, there’s no way for a user of the module to break the
invariant, so long as the implementation maintains it.
Because the sets are implemented as unsorted lists, when taking the
union of two sets set1 and set2, we must traverse the entirety of the
set2 list once for each element of set1. For small sets, this is not likely
to be problematic, and worrying about this inefficiency may well be a
premature effort at optimization.7 But for a set implementation likely 7
In the introduction to Chapter 14 you’ll
learn that “premature optimization is
to be used widely and on very large sets, it may be useful to address the
the root of all evil.”
issue.
A better alternative from an efficiency point of view is to implement
sets as sorted lists. This requires a bit more work in adding elements
to a set to place them in the right order, but saves effort for union and
184 PROGRAMMING WELL

intersection. We redefine the StringSet module accordingly, still

satisfying the same STRING_SET signature.

# (* Implementation of STRING_SET as list of strings.

# Assumes list is *sorted* with no duplicates. *)
# module StringSet : STRING_SET =
# struct
# type set = string list
# let empty = []
# let is_empty s = (s = [])
# let rec member elt s =
# match s with
# | [] -> false
# | hd :: tl -> if elt = hd then true
# else if elt < hd then false
# else member elt tl
# let rec add elt s =
# match s with
# | [] -> [elt]
# | hd :: tl -> if elt < hd then elt :: s
# else if elt = hd then s
# else hd :: add elt tl
# let union = List.fold_right add
# let rec intersection set1 set2 =
# match set1, set2 with
# | [], _ -> []
# | _, [] -> []
# | h1::t1, h2::t2 ->
# if h1 = h2 then h1 :: intersection t1 t2
# else if h1 < h2 then intersection t1 set2
# else intersection set1 t2
# end ;;
module StringSet : STRING_SET

Now we can test the revised definition.

# let s =
# let open StringSet in
# empty
# |> add "a"
# |> add "b"
# |> add "c" ;;
val s : StringSet.set = <abstr>
# StringSet.member "a" s ;;
- : bool = true
# StringSet.member "d" s ;;
- : bool = false

And here’s the payoff. Even though we’ve completely changed the
implementation of string sets, even using a data structure obeying a
different invariant, the code for using string sets changes not at all.
 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 185

12.5.2 A generic set signature

For document querying, we needed a string set module. For other

purposes we may need sets of other element types. We could generate
similar modules for, say, integer sets, with an appropriate signature:

# module type INT_SET =

# sig
# (* Type of integer sets *)
# type set
# (* The empty set *)
# val empty : set
# (* Returns true if set is empty; false otherwise *)
# val is_empty : set -> bool
# (* Adds integer to existing set (if not already a member) *)
# val add : int -> set -> set
# (* Union of two sets *)
# val union : set -> set -> set
# (* Intersection of two sets *)
# val intersection : set -> set -> set
# (* Returns true iff integer is in set *)
# val member: int -> set -> bool
# end ;;
module type INT_SET =
sig
type set
val empty : set
val is_empty : set -> bool
val add : int -> set -> set
val union : set -> set -> set
val intersection : set -> set -> set
val member : int -> set -> bool
end

but we’d be violating the edict of irredundancy. Rather, we’d prefer

a generic signature for set modules that provides a set type for any
element type.
Here is such a signature. We’ve added a new type to the module, the
type element for elements of the set, and we use it in the types of the
various functions.

# module type SET =

# sig
# (* Type of sets *)
# type set
# (* and their elements *)
# type element
# (* The empty set *)
# val empty : set
# (* Returns true if set is empty; false otherwise *)
# val is_empty : set -> bool
# (* Adds element to existing set (if not already a member) *)
# val add : element -> set -> set
186 PROGRAMMING WELL

# (* Union of two sets *)

# val union : set -> set -> set
# (* Intersection of two sets *)
# val intersection : set -> set -> set
# (* Returns true iff element is in set *)
# val member: element -> set -> bool
# end ;;
module type SET =
sig
type set
type element
val empty : set
val is_empty : set -> bool
val add : element -> set -> set
val union : set -> set -> set
val intersection : set -> set -> set
val member : element -> set -> bool
end

A string set implementation satisfying this signature defines the

element type as string:

# module StringSet : SET =

# struct
# type element = string
# type set = element list
# let empty = []
# let is_empty s = (s = [])
# let rec member elt s =
# match s with
# | [] -> false
# | hd :: tl -> if elt = hd then true
# else if elt < hd then false
# else member elt tl
# let rec add elt s =
# match s with
# | [] -> [elt]
# | hd :: tl -> if elt < hd then elt :: s
# else if elt = hd then s
# else hd :: add elt tl
# let union = List.fold_right add
# let rec intersection set1 set2 =
# match set1, set2 with
# | [], _ -> []
# | _, [] -> []
# | h1::t1, h2::t2 ->
# if h1 = h2 then h1 :: intersection t1 t2
# else if h1 < h2 then intersection t1 set2
# else intersection set1 t2
# end ;;
module StringSet : SET

We can use this StringSet to, for instance, generate an empty

string set:
 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 187

# StringSet.empty ;;
- : StringSet.set = <abstr>

We run into a major problem, though, in the simple act of checking if a

string is a member of the set.
# StringSet.member "a" StringSet.empty ;;
Line 1, characters 17-20:
1 | StringSet.member "a" StringSet.empty ;;
^^^
Error: This expression has type string but an expression was
expected of type
StringSet.element

What’s the problem? It turns out that the abstraction barrier provided
by the SET signature is doing exactly what it should. The implementa-
tion promises to deliver something that satisfies and reveals SET. And
that’s all. The SET signature reveals types set and element, not string
list and string. Viewed from within the implementation, the types
element and string are the same. But from outside the module im-
plementation, only element is available, leading to the type mismatch
with string.
This is a case in which the abstraction barrier is too strict. (We
saw this before in Section 12.3.) We do want to allow the user of the
module to have access to the implementation of the element type, if
only so that module users can provide elements of that type. Rather
than using the too abstract SET signature, we can define slightly less
abstract signatures using S H A R I N G C O N S T R A I N T S , which augment
a signature with one or more type equalities across the abstraction
barrier, identifying abstract types within the signature (element) with
implementations of those types accessible outside the implementation
(string).8 8
Notice how in printing out the result
of defining the new STRING_SET signa-
# module type STRING_SET = SET with type element = string ;; ture, OCaml specifies that the type of
module type STRING_SET = elements is string. Compare this with
sig the version above without the sharing
type set constraint.
type element = string This example requires only a single
sharing constraint, but multiple con-
val empty : set
straints can be useful as well. They are
val is_empty : set -> bool
combined with the and keyword, for
val add : element -> set -> set
example, the pair of sharing constraints
val union : set -> set -> set with type key = D.key and type
val intersection : set -> set -> set value = D.value used in the definition
val member : element -> set -> bool of the MakeOrderedDict module in
end Section 12.6.

Now we can declare the implementation as satisfying this relaxed

signature.
# module StringSet : STRING_SET =
# struct
188 PROGRAMMING WELL

# type element = string

# type set = element list
# let empty = []
# let is_empty s = (s = [])
# let rec member elt s =
# match s with
# | [] -> false
# | hd :: tl -> if elt = hd then true
# else if elt < hd then false
# else member elt tl
# let rec add elt s =
# match s with
# | [] -> [elt]
# | hd :: tl -> if elt < hd then elt :: s
# else if elt = hd then s
# else hd :: add elt tl
# let union = List.fold_right add
# let rec intersection set1 set2 =
# match set1, set2 with
# | [], _ -> []
# | _, [] -> []
# | h1::t1, h2::t2 ->
# if h1 = h2 then h1 :: intersection t1 t2
# else if h1 < h2 then intersection t1 set2
# else intersection set1 t2
# end ;;
module StringSet : STRING_SET

This implementation now allows us to perform operations involving

particular strings.

# StringSet.empty ;;
- : StringSet.set = <abstr>
# StringSet.member "a" StringSet.empty ;;
- : bool = false
# let s =
# let open StringSet in
# empty
# |> add "first"
# |> add "second"
# |> add "third" ;;
val s : StringSet.set = <abstr>
# StringSet.union s s ;;
- : StringSet.set = <abstr>
# StringSet.member "a" s ;;
- : bool = false

12.5.3 A generic set implementation

Sharing constraints solve the problem of duplicative signatures, be-

cause we can define different signatures by adding different sharing
constraints to the generic SET signature:
 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 189

# module type STRING_SET =

# SET with type element = string ;;
# module type INT_SET =
# SET with type element = int ;;
# module type INTBOOL_SET =
# SET with type element = int * bool ;;

Unfortunately, they do nothing for the problem of duplicative imple-

mentations. To implement a module satisfying the INT_SET signature,
we’d need to build the whole module from scratch, like this:
# module IntSet : INT_SET =
# struct
# type element = int
# type set = element list
# let empty = []
# let is_empty s = (s = [])
# let rec member elt s =
# match s with
# | [] -> false
# | hd :: tl -> if elt = hd then true
# else if elt < hd then false
# else member elt tl
# let rec add elt s =
# match s with
# | [] -> [elt]
# | hd :: tl -> if elt < hd then elt :: s
# else if elt = hd then s
# else hd :: add elt tl
# let union = List.fold_right add
# let rec intersection set1 set2 =
# match set1, set2 with
# | [], _ -> []
# | _, [] -> []
# | h1::t1, h2::t2 ->
# if h1 = h2 then h1 :: intersection t1 t2
# else if h1 < h2 then intersection t1 set2
# else intersection set1 t2
# end ;;

The redundancy is massive; the only differences from the StringSet

implementation are those highlighted in red. To solve this violation of
the edict of irredundancy requires more powerful tools.
What we need is a way of generating implementations that depend
on some stuff. In the case at hand, the stuff is just the implementation
of the element type, and perhaps some functionality involving that
type. For instance, in the implementations of the StringSet and
IntSet modules, we availed ourselves of comparing elements using
the < operator. Any type we build a set from using this implementation
approach needs some way of performing such comparisons, but the
< operator may not always be appropriate for that purpose. More
generally, the implementations may depend not only on a type but on
190 PROGRAMMING WELL

some values of that type or functions over the type, or even multiple
types.
If only we had a way of packaging up some types and related values
and functions. But we do have such a way: the module system itself. In
effect, what we need is something akin to a function that takes as ar-
gument a module defining the parameters of the implementation and
returns the desired module. We call these “functions” from modules to
modules F U N C T O R S .
We can use the StringSet and IntSet implementations as the
basis for a functor MakeOrderedSet, which takes a module as argu-
ment to provide the element type and returns a module satisfying the
SET signature. As described above, the argument module should have
a type (call it t) and a way of comparing elements of the type (call it
compare). We’ll have the compare function take two elements of type
t and return an integer specifying whether the first integer is less than
(-1), equal to (0), or greater than (1) the second integer.
You may recognize this signature. It’s the ORDERED_TYPE signature
from Section 12.3, repeated here for reference.
# module type ORDERED_TYPE =
# sig
# type t
# val compare : t -> t -> int
# end ;;
module type ORDERED_TYPE = sig type t val compare : t -> t -> int
end

The argument to the functor should satisfy this signature.

A functor that takes a module with this signature and delivers a SET
implementation can be constructed just by factoring out the type and
the comparison from our previous implementations of IntSet and
StringSet.

# module MakeOrderedSet (Elements : ORDERED_TYPE) : SET =

# struct
# type element = Elements.t
# type set = element list
# let empty = []
# let is_empty s = (s = [])
# let rec member elt s =
# match s with
# | [] -> false
# | hd :: tl ->
# (match Elements.compare elt hd with
# | 0(* equal *) -> true
# | -1 (* less *) -> false
# | _ (* greater *) -> member elt tl)
# let rec add elt s =
# match s with
# | [] -> [elt]
 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 191

# | hd :: tl ->
# (match Elements.compare elt hd with
# | 0
(* equal *) -> s
# | -1 (* less *) -> elt :: s
# | _ (* greater *) -> hd :: add elt tl)
# let union = List.fold_right add
# let rec intersection set1 set2 =
# match set1, set2 with
# | [], _ -> []
# | _, [] -> []
# | h1::t1, h2::t2 ->
# (match Elements.compare h1 h2 with
# | 0 (* equal *) -> h1 :: intersection t1 t2
# | -1 (* less *) -> intersection t1 set2
# | _ (* greater *) -> intersection set1 t2)
# end ;;
module MakeOrderedSet : functor (Elements : ORDERED_TYPE) -> SET

But this won’t do. The returned module satisfies SET, but we’ve
already seen how this is too strong a requirement. The solution is the
same as before, use sharing constraints to allow access to the element
type.
# module MakeOrderedSet (Elements : ORDERED_TYPE)
# : (SET with type element = Elements.t) =
# struct
# type element = Elements.t
# type set = element list
# let empty = []
# let is_empty s = (s = [])
# let rec member elt s =
# match s with
# | [] -> false
# | hd :: tl ->
# (match Elements.compare elt hd with
# | 0
(* equal *) -> true
# | -1 (* less *) -> false
# | _ (* greater *) -> member elt tl)
# let rec add elt s =
# match s with
# | [] -> [elt]
# | hd :: tl ->
# (match Elements.compare elt hd with
# | 0
(* equal *) -> s
# | -1 (* less *) -> elt :: s
# | _ (* greater *) -> hd :: add elt tl)
# let union = List.fold_right add
# let rec intersection set1 set2 =
# match set1, set2 with
# | [], _ -> []
# | _, [] -> []
# | h1::t1, h2::t2 ->
# (match Elements.compare h1 h2 with
# | 0 (* equal *) -> h1 :: intersection t1 t2
192 PROGRAMMING WELL

# | -1 (* less *) -> intersection t1 set2

# | _ (* greater *) -> intersection set1 t2)
# end ;;
module MakeOrderedSet :
functor (Elements : ORDERED_TYPE) ->
sig
type set
type element = Elements.t
val empty : set
val is_empty : set -> bool
val add : element -> set -> set
val union : set -> set -> set
val intersection : set -> set -> set
val member : element -> set -> bool
end

Here we finally have a functor that can generate a set module for any
type. Let’s generate a few, starting with a string set module, which we
can generate by applying the MakeOrderedSet functor to a module
satisfying ORDERED_TYPE linking the string type to an appropriate
ordering function (here, the default Stdlib.compare function).
# module StringSet = MakeOrderedSet
# (struct
# type t = string
# let compare = compare
# end) ;;
module StringSet :
sig
type set
type element = string
val empty : set
val is_empty : set -> bool
val add : element -> set -> set
val union : set -> set -> set
val intersection : set -> set -> set
val member : element -> set -> bool
end

It works as expected:
# let s =
# let open StringSet in
# empty
# |> add "first"
# |> add "second"
# |> add "third" ;;
val s : StringSet.set = <abstr>
# StringSet.union s s ;;
- : StringSet.set = <abstr>
# StringSet.member "a" s ;;
- : bool = false

How about an integer set module? Again, a couple of lines of code

suffice.
 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 193

# module IntSet = MakeOrderedSet

# (struct
# type t = int
# let compare = compare
# end) ;;
module IntSet :
sig
type set
type element = int
val empty : set
val is_empty : set -> bool
val add : element -> set -> set
val union : set -> set -> set
val intersection : set -> set -> set
val member : element -> set -> bool
end
# let s =
# let open IntSet in
# empty
# |> add 1
# |> add 2
# |> add 3 ;;
val s : IntSet.set = <abstr>
# IntSet.union s s ;;
- : IntSet.set = <abstr>
# IntSet.member 4 s ;;
- : bool = false

12.6 A dictionary module

The query evaluation application we’ve been working on (remember

that?) required not only an implementation of a set ADT, but also a
dictionary ADT. Dictionaries are data structures that associate keys to
values, and allow for insertion and deletion of key-value associations,
and looking up of the value associated with a given key (if one exists).
We now have all the tools to build that as well. An appropriate
signature for a dictionary is

# module type DICT =

# sig
# type key
# type value
# type dict
#
# (* empty -- An empty dictionary *)
# val empty : dict
# (* lookup dict key -- Returns as an option the value
# associated with the provided key. If the key is
# not in the dictionary, returns None. *)
# val lookup : dict -> key -> value option
# (* member dict key -- Returns true if and only if the
# key is in the dictionary. *)
194 PROGRAMMING WELL

# val member : dict -> key -> bool

# (* insert dict key value -- Inserts a key-value pair into
# dict. If the key is already present, updates the key to
# have the new value. *)
# val insert : dict -> key -> value -> dict
# (* remove dict key -- Removes the key and its value from the
# dictionary, if present. If the key is not present,
# returns the original dictionary. *)
# val remove : dict -> key -> dict
# end ;;
module type DICT =
sig
type key
type value
type dict
val empty : dict
val lookup : dict -> key -> value option
val member : dict -> key -> bool
val insert : dict -> key -> value -> dict
val remove : dict -> key -> dict
end

We’ll want a functor that builds dictionaries for all kinds of keys
and values. In order to make sure we can compare the keys properly,
including ordering them, we’ll need a comparison function for keys as
well. While we’re at it, we might as well use a nicer convention for the
comparison function, which will return a value of type
type order = Less | Equal | Greater ;;

The argument to the functor should thus satisfy the following signa-
ture:
# module type DICT_ARG =
# sig
# type key
# type value
# (* We need to reveal the order type so users of the
# module can match against it to implement compare *)
# type order = Less | Equal | Greater
# (* Comparison function on keys compares two elements
# and returns their order *)
# val compare : key -> key -> order
# end ;;
module type DICT_ARG =
sig
type key
type value
type order = Less | Equal | Greater
val compare : key -> key -> order
end

An implementation of such a functor is given here. It takes a module

D satisfying DICT_ARG, providing all the needed information about the
 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 195

key and value types and the ordering of keys. It allows access to the key
and value types via sharing constraints, so users of modules generated
by the functor can provide values of those types. This particular imple-
mentation of dictionaries is a simple list of key-value pairs, sorted by
unique keys.

# module MakeOrderedDict (D : DICT_ARG)

# : (DICT with type key = D.key
# and type value = D.value) =
# struct
# type key = D.key
# type value = D.value
#
# (* Invariant: sorted by key, no duplicate keys *)
# type dict = (key * value) list
#
# let empty = []
#
# let rec lookup d k =
# match d with
# | [] -> None
# | (k1, v1) :: d1 ->
# let open D in
# match compare k k1 with
# | Equal -> Some v1
# | Greater -> lookup d1 k
# | Less -> None
#
# let member d k =
# match lookup d k with
# | None -> false
# | Some _ -> true
#
# let rec insert d k v =
# match d with
# | [] -> [k, v]
# | (k1, v1) :: d1 ->
# let open D in
# match compare k k1 with
# | Less -> (k, v) :: d
# | Equal -> (k, v) :: d1
# | Greater -> (k1, v1) :: (insert d1 k v)
#
# let rec remove d k =
# match d with
# | [] -> []
# | (k1, v1) :: d1 ->
# let open D in
# match compare k k1 with
# | Equal -> d1
# | Greater -> (k1, v1) :: (remove d1 k)
# | Less -> d
# end ;;
196 PROGRAMMING WELL

module MakeOrderedDict :
functor (D : DICT_ARG) ->
sig
type key = D.key
type value = D.value
type dict
val empty : dict
val lookup : dict -> key -> value option
val member : dict -> key -> bool
val insert : dict -> key -> value -> dict
val remove : dict -> key -> dict
end

A reverse index, recall, is just a dictionary for mapping string

keys to string set values. (The latter we’ve already built as the type
StringSet.set.) Let’s build one using the MakeOrderedDict functor.
The argument to the functor should specify the key and value types
and the ordering on keys:
# module StringStringSetDictArg
# : (DICT_ARG with type key = string
# and type value = StringSet.set) =
# struct
# type key = string
# type value = StringSet.set
# type order = Less | Equal | Greater
# let compare x y = if x < y then Less
# else if x = y then Equal
# else Greater
# end ;;
module StringStringSetDictArg :
sig
type key = string
type value = StringSet.set
type order = Less | Equal | Greater
val compare : key -> key -> order
end

Now to generate an index module requires only a single line.

# module Index = MakeOrderedDict (StringStringSetDictArg) ;;
module Index :
sig
type key = StringStringSetDictArg.key
type value = StringStringSetDictArg.value
type dict = MakeOrderedDict(StringStringSetDictArg).dict
val empty : dict
val lookup : dict -> key -> value option
val member : dict -> key -> bool
val insert : dict -> key -> value -> dict
val remove : dict -> key -> dict
end

By making use of these generic constructs for sets and dictionaries,

we can build a reverse index type easily, and implement query evalu-
 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 197

ation in a manner that is oblivious to, hence robust to any changes in,
the implementation of the sets and dictionaries. The code for eval can
be as specified before, and repeated here.

# let rec eval (q : query)

# (idx : Index.dict)
# : StringSet.set =
# match q with
# | Word word -> (match Index.lookup idx word with
# | None -> StringSet.empty
# | Some v -> v)
# | And (q1, q2) -> StringSet.intersection (eval q1 idx)
# (eval q2 idx)
# | Or (q1, q2) -> StringSet.union (eval q1 idx)
# (eval q2 idx) ;;
val eval : query -> Index.dict -> StringSet.set = <fun>

More generally, modules allow separating an interface from its

implementation, the key premise of abstract data types and modular
programming, and OCaml’s functors provide for constructing modules
that operate generically.

12.7 Alternative methods for defining signatures and mod-

ules

We’ve already seen two ways to define a module subject to a particular

signature. First is to name the signature explicitly using module type,
and use that name in defining the module itself.

module type SIG_NAME =

sig
...component declarations...
end ;;

module ModuleName : SIG_NAME =

struct
...component implementations...
end ;;

Second is to place an unnamed signature directly constraining the

module definition

module ModuleName : sig

...component declarations...
end =
struct
...component implementations...
end ;;

useful on occasions where the signature is quite short and will only be
used once, so retaining a name for it isn’t needed.
198 PROGRAMMING WELL

There is a third method, widely used within OCaml’s own imple-

mentation of library modules. All of the components defined in a .ml
file automatically constitute a module, whose name is generated by
converting the first letter of the filename to uppercase. For example, if
we have a file named queue.ml whose contents is

type 'a queue = 'a list

let empty_queue : 'a queue = []
let enqueue (elt : 'a) (q : 'a queue) : 'a queue =
q @ [elt]
let dequeue (q : 'a queue) : 'a * 'a queue =
match q with
| [] -> raise (Invalid_argument
"dequeue: empty queue")
| hd :: tl -> hd, tl

then we can refer in other files to Queue.queue to gain access to the

type defined in that file, to Queue.enqueue to access the enqueueing
function, and so forth. We can even place an open Queue at the top
of another file to have unprefixed access to the components of the
module.
How to define a signature for such a module though? OCaml looks
for a file with the same prefix but the extension .mli (the i is for “in-
terface”), which holds the component declarations for the signature.
Thus, we should place in a file queue.mli these declarations:

type 'a queue

val empty_queue : 'a queue
val enqueue : 'a -> 'a queue -> 'a queue
val dequeue : 'a queue -> 'a * 'a queue

The Queue module will then be constrained by this signature, simply by

virtue of the matching filenames.

12.7.1 Set and dictionary modules

The facilities for generating set modules – including the SET signature
and MakeOrderedSet functor – might well be packaged up into a single
module themselves. A file set.ml providing such a module might look
like the following:

(* A Set Module *)

(*.......................................................
Set interface
*)

module type SET =

sig
type element (* elements of the set *)
type set (* sets formed from the elements *)
 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 199

(* The empty set *)

val empty : set
(* Returns true if set is empty; false otherwise *)
val is_empty : set -> bool
(* Adds element to existing set (if not already a member) *)
val add : element -> set -> set
(* Union of two sets *)
val union : set -> set -> set
(* Intersection of two sets *)
val intersection : set -> set -> set
(* Returns true iff element is in set *)
val member : element -> set -> bool
end ;;

(*.......................................................
An implementation for elements of ordered type
*)

(* Module for types with a comparison function *)

module type COMPARABLE =

sig
(* The type of comparable elements *)
type t
(* We need to reveal the order type so users of the
module can match against it *)
type order = Less | Equal | Greater
(* Comparison function compares two elements of the
type and returns their order *)
val compare : t -> t -> order
end

(* Functor that generates sets for any comparable type *)

module MakeOrderedSet (Elements : COMPARABLE)

: (SET with type element = Elements.t) =
(* Implementation of SET as list of elements. Assumes
list is sorted with no duplicates. *)
struct
type element = Elements.t
type set = element list

let empty = []
let is_empty s = (s = [])
let rec member elt s =
match s with
| [] -> false
| hd :: tl ->
let open Elements in
(* so that Elements.compare, Elements.Less,
etc. are in scope *)
match compare elt hd with
| Equal -> true
200 PROGRAMMING WELL

| Less -> false

| Greater -> member elt tl

let rec add elt s =

(* add the elt in the proper place in the
ordered list *)
match s with
| [] -> [elt]
| hd :: tl ->
let open Elements in
match compare elt hd with
| Less -> elt :: s
| Equal -> s
| Greater -> hd :: add elt tl

let union s1 s2 = List.fold_right add s1 s2

let rec intersection xs ys =

match xs, ys with
| [], _ -> []
| _, [] -> []
| xh :: xt, yh :: yt ->
let open Elements in
match compare xh yh with
| Equal -> xh :: intersection xt yt
| Less -> intersection xt ys
| Greater -> intersection xs yt
end ;;

This file defines a module called set that enables usage like the
following, to define and use a StringSet module:
module StringSet =
let open Set in
MakeOrderedSet
(struct
type t = string
type order = Less | Equal | Greater
let compare s t = if s < t then Less
else if s = t then Equal
else Greater
end) ;;

let s = StringSet.create
|> StringSet.add "a"
|> StringSet.add "b"
|> StringSet.add "a" ;;

12.8 Library Modules

Data structures like sets and dictionaries are so generally useful that
you might think the language ought to provide them so that each indi-
vidual programmer doesn’t need to implement them. In fact, OCaml
 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 201

does provide these and many other data structures – as L I B R A RY M O D -

U L E S.
In particular, the Set library module provides functionality much
like the Set module in the previous section, and the Map library mod-
ule provides functionality much like our dictionary module and its
MakeOrderedDict functor.
In later chapters, we’ll happily avail ourselves of these built-in li-
braries. Nonetheless, it’s still important to see how such simple and
general abstract data structures can be provided as modules, for sev-
eral reasons: to demystify what’s going on in the library-provided
modules, to instantiate the idea that the language itself is sufficient for
implementing these ideas, and as examples to inspire ways to imple-
ment other, more application-specific abstract data structures.

12.9 Problem section: Image manipulation

We define here a signature for modules that deal with images and their
manipulation.

module type IMAGING =

sig
(* types for images, which are composed of pixels *)
type image
type pixel
(* an image size is a pair of ints giving number of
rows and columns *)
type size = int * int
(* converting between integers and pixels *)
val to_pixel : int -> pixel
val from_pixel : pixel -> int
(* apply an image filter, a function over pixels,
to every pixel in an image *)
val filter : (pixel -> pixel) -> image -> image
(* apply an image filter to two images, combining
the images pixel by pixel *)
val filter2 : (pixel -> pixel -> pixel)
-> image -> image -> image
(* return a "constant" image of the specified size
where every pixel has the same value *)
val const : pixel -> size -> image
(* display the image in a graphics window *)
val depict : image -> unit
end ;;

The pixels that make up an image are specified by the following signa-
ture:

module type PIXEL =

sig
type t
202 PROGRAMMING WELL

val to_pixel : int -> t

val from_pixel : t -> int
end

Problem 111
We’d like to implement a functor named MakeImaging for generating implementations
of the IMAGING signature based on modules satisfying the PIXEL signature. How should
such a functor start? Give the header line of such beginning with the keyword module
and ending with the = struct....

Here is a module implementing the PIXEL signature for integer

pixels.
module IntPixel : (PIXEL with type t = int) =
struct
type t = int
let to_pixel x = x
let from_pixel x = x
end ;;

Problem 112
Write code that uses the IntPixel module to define an imaging module called
IntImaging.
Problem 113
Write code to use the IntImaging module that you defined in Problem 112 to display a
100 by 100 pixel image where all of the pixels have the constant integer value 5000.

12.10 Problem section: An abstract data type for intervals

A good candidate for an abstract data type is the I N T E RVA L . Abstractly

speaking, an interval is a region between two points, where all that is
required of points is that we be able to compare them as an ordering
(so that we have a well-defined notion of “between”). That is, points
ought to obey the following signature, which may look familiar, as
you’ve seen it in other contexts:

module type COMPARABLE =

sig
type t
type order = Less | Equal | Greater
val compare : t -> t -> order
end ;;

Intervals come up in many different contexts. As an informal ex-

ample, calendars need to associate events with time intervals, such as
3-4pm or 11:30am-3:30pm; the endpoints in this case would be times.
Natural operations over intervals include: the construction of an inter-
val between two points, the extraction of the endpoints of an interval,
taking the union of two intervals (the smallest interval containing
both) or their intersection, and determining the relation between two
intervals (whether they are disjoint, overlapping, or one contains the
other). Here is a signature that provides for this functionality.
 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 203

module type INTERVAL =

sig
type point
type interval
type relation = Disjoint | Overlaps | Contains
(* Returns the interval between two points *)
val interval : point -> point -> interval
(* Returns the endpoints of an interval as a pair
with the first point less than the second. *)
val endpoints : interval -> point * point
(* Returns the union of two intervals *)
val union : interval -> interval -> interval
(* Returns the relation holding between two intervals *)
val relation : interval -> interval -> relation
end ;; Overlaps

The possible relations between two intervals are depicted in Fig- Contains

ure 12.3. (For the interval arithmetic cognoscenti, we’ve left out
many details, such as whether intervals are open or closed; more Disjoint

fine-grained relations; and many other useful operations on intervals.

These issues are beyond the scope of this problem.)
Figure 12.3: A diagrammatic depiction
Problem 114
of the possible relations holding be-
We’d like to have a functor named MakeInterval for generating implementations of the tween two intervals. In the diagram, the
INTERVAL signature based on modules satisfying the COMPARABLE signature. How should gray intervals in the three groups below
such a functor start? Give the header line of such a functor definition beginning with the the black interval are in the “overlaps”
keyword module and ending with the = struct.... (top 2), “contains” (next 5), and “dis-
Problem 115 joint” (bottom 3) relations, respectively,
with the black interval at top. The verti-
An appropriate module satisfying COMPARABLE for the purpose of generating discrete cal dotted lines depict the endpoints of
time intervals would be one where the type is int, with an appropriate comparison the black interval.
function. Define a module named DiscreteTime satisfying COMPARABLE where the type
is int. Make sure the type is accessible outside the module.
Problem 116
Now use the functor MakeInterval to define a module DiscreteTimeInterval
that provides interval functionality over discrete times as defined by the module
DiscreteTime above.

Problem 117
The intersection of two intervals is only well-defined if the intervals are not disjoint. As-
sume that the DiscreteTimeInterval module has been opened, allowing you to make
use of everything in its signature. Now, define a function intersection : interval
-> interval -> interval option that takes two intervals and returns None if they are
disjoint and otherwise returns their intersection (embedded appropriately in the option
type).
Problem 118
Provide three different unit tests that would be useful in testing the correctness of the
DiscreteTimeInterval module.

12.11 Problem section: Mobiles

The artist Alexander Calder (1898-1976) is well known for his distinc-
Figure 12.4: Alexander Calder’s
tive mobiles, sculptures with different shaped objects hung from a L’empennage (1953).
cascade of connecting metal bars. An example is given in Figure 12.11.
204 PROGRAMMING WELL

His mobiles are made with varying shapes at the ends of the con-
nectors – circles, ovals, fins. The exquisite balance of the mobiles
depends on the weights of the various components. In the next few
exercises of this problem, you will model the structure of mobiles as
binary trees such that one can determine if a Calder-like mobile design
is balanced or not. Let’s start with the objects at the ends of the con-
nectors. For our purposes, the important properties of an object will be
its shape and its weight (in arbitrary units; you can interpret them as
pounds).
Problem 119
Define a weight type consisting of a single floating point weight.
Problem 120
Define a shape type, a variant type that allows for three different shapes: circles, ovals,
and fins.
Problem 121
Define an object type that will be used to store information about the objects at the
ends of the connectors, in particular, their weight and their shape.

A mobile can be modeled as a kind of binary tree, where the leaves

of the tree, representing the objects, are elements of type obj, and
the internal nodes, representing the connectors, have a weight, and
each internal node (connector) connects two submobiles. Rather than
directly writing code for a mobile type, though, we’ll digress to build a
more general binary tree module, and then model mobiles using that.
An appropriate signature BINTREE for a simple binary tree module
might be the following:

module type BINTREE =

sig
type leaft (* the type for the leaves of the tree *)
type nodet (* the type for the internal nodes of the tree *)
type tree (* the type for the trees themselves *)

val make_leaf : leaft -> tree

val make_node : nodet -> tree -> tree -> tree
val walk : (leaft -> 'a)
-> (nodet -> 'a -> 'a -> 'a) -> tree -> 'a
end ;;

This module signature specifies separate types for the leaves of trees
and the internal nodes of trees, along with a type for the trees them-
selves; functions for constructing leaf and node trees; and a single
function to "walk" the tree. (We’ll come back to the walk function
later.) In addition to the signature for binary tree modules, we would
need a way of generating implementations of modules satisfying the
BINTREE signature, which we’ll do with a functor MakeBintree. The
MakeBinTree functor takes an argument module of type BINTREE_ARG
that packages up the particular types for the leaves and nodes, that is,
 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 205

the types to use for leaft and nodet. The following module signature
will work:
module type BINTREE_ARG =
sig
type leaft
type nodet
end ;;

Problem 122
Write down the header of a definition of a functor named MakeBintree taking a
BINTREE_ARG argument, which generates modules satisfying the BINTREE signature.
Keep in mind the need for users of the functor-generated modules to access appropriate
aspects of the generated trees. (You don’t need to fill in the actual implementation of the
functor.)

Using the MakeBintree functor described above, you can now

generate a Mobile module, which has objs at the leaves and weights
at the interior nodes.
Problem 123
Define a module Mobile using the functor MakeBintree.
Problem 124
You’ve just used the MakeBintree functor without ever seeing its implementation. Why
is this possible?

You can now build a representation of a mobile using the functions

that the Mobile module makes available.
Problem 125
Figure 12.5: A simple Calder-style
Define a value mobile1 of type Mobile.tree that represents a mobile structured as the
mobile. The depicted mobile has two
one depicted in Figure 12.5.
connectors and three objects (an oval
The walk function, of type (leaft -> ’a) -> (nodet -> ’a and two fins). The connectors each
weigh 1.0, and the objects’ weights are
-> ’a -> ’a) -> tree -> ’a, is of special interest, since it is the
as given in the figure.
sole method for performing computations over these binary trees.
The function is a kind of fold that works over trees instead of lists. It
takes two functions – one for leaves and one for nodes – and applies
these functions to a tree to generate a single value. The leaf function
takes a leaft and returns some value of type ’a. The node function
takes a nodet, as well as the two ’a values recursively returned by
walking its two subtrees, and computes the value for the node itself.
For example, we can use walk to define a function size that counts
how many objects there are in a mobile. The function uses the fact that
leaves are of size 1 and the size of a non-leaf is the sum of the sizes of
its subtrees.
let size mobile =
Mobile.walk (fun _leaf -> 1)
(fun _node left_size right_size ->
left_size + right_size)
mobile ;;

Problem 126
What is the type of size?
206 PROGRAMMING WELL

Problem 127
Use the fact that the walk function is curried to give a slightly more concise definition for
size.
Problem 128
Use the walk function to implement a function shape_count : shape ->
Mobile.tree -> int that takes a shape and a mobile (in that order), and returns
the number of objects in the mobile that have that particular shape.

A mobile is said to be balanced if every connector has the property

that the total weight of all components (that is, objects and connec-
tors) of its left submobile is the same as the total weight of all com-
ponents of its right submobile. (In actuality, we’d have to worry about
other things like the relative lengths of the arms of the connectors, but
we’ll ignore all that.)
Problem 129
Is the mobile shown balanced? Why or why not?
Problem 130
Implement a function balance : Mobile.tree -> weight option that takes a
mobile, and returns None if the argument mobile is not balanced, and Some w if the
mobile is balanced, where w is the total weight of the mobile.

12.12 Problem set 5: Ordered collections

This section provides substantive background for one of the course prob-
lem sets. For common background on logistics and procedures for prob-
lem sets, including such topics as accessing the problem set code, compil-
ing and testing your code, reflecting on the process, and submitting your
solution, please review the course document “Problem set procedures”.

In this assignment you will use modules to define several useful

abstract data types (ADT). The particular ADTs that you’ll be imple-
menting are ordered collections (as implemented through binary
search trees) and priority queues (as implemented through binary
search trees and binary heaps). These are data structures that can store
multiple elements taken from an ordered domain and provide efficient
operations such as insertion and deletion of elements and access to
the minimum and maximum elements.
In the process you will work with the following signatures, modules,
and functors (marked as to which we provide and which you write or
complete [in italics]):

• Orderings (see order.ml) – A variety of utilities concerning order-

ings, including

– A signature for comparable elements [provided]

– Modules satisfying the signature for integers and integer-string
pairs [provided]

• Ordered collections (see orderedcoll.ml)

 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 207

– An ordered collection signature [provided]

– A functor for generating implementations of the signature based
on binary search trees [you complete]
– A module generated using the functor implementing integer
binary search trees [provided]

• Priority queues (see prioqueue.ml)

– A priority queue signature [provided]

– A functor for generating implementations based on lists [you
complete]
– A functor for generating implementations based on binary search
trees [you write, using the binary search tree functor above]
– A functor for generating implementations based on binary heaps
[you complete]
– Modules generated from all three functors implementing integer
priority queues [provided]
– Sort functions that use these modules [provided]

12.12.1 Ordered collections

An ordered collection is a collection of elements that have an intrinsic

ordering to them. Natural operations on ordered collections include
insertion of an element, deletion of an element, searching for an ele-
ment, and extracting the minimum and maximum elements.
To specify the ordering relation on the elements, we will make use of
the following enumerated type (see the file order.ml), which is useful
as the result of comparing two values:

type order = Equal | Less | Greater ;;

A simple module signature for ordered collections, called

ORDERED_COLLECTION_0, specifies an interface with a data type for
collections, as well as empty, insert, search, delete, getmin, and
getmax operations. You’ll want to read and understand it to familiarize
yourself with the syntax for how to write module signatures.

module type ORDERED_COLLECTION_0 =

sig
exception Empty
exception NotFound

(* The type of collections. What this type actually looks

like is left up to the implementation *)
type 'a collection
208 PROGRAMMING WELL

(* The empty collection *)

val empty : 'a collection
(* Inserts elt into collection *)
val insert : ('a -> 'a -> order)
-> 'a -> 'a collection -> 'a collection
(* Searches a collection for the given value. *)
val search : ('a -> 'a -> order)
-> 'a -> 'a collection -> bool
(* Deletes the given value from a collection. May
raise NotFound exception *)
val delete : ('a -> 'a -> order)
-> 'a -> 'a collection -> 'a collection
(* Returns the minimum value of a collection. May
raise Empty exception. *)
val getmin : ('a -> 'a -> order) -> 'a collection -> 'a
(* Returns the maximum value of a collection. May
raise Empty exception. *)
val getmax : ('a -> 'a -> order) -> 'a collection -> 'a
end ;;

The signature explicitly lists the types and values that any module
implementing this interface must define, as well as the exceptions
the implementation provides, and that functions in the interface may
raise.9 For a function like getmin, we could instead choose to return 9
Because of how OCaml handles excep-
tions, listing exceptions is optional, and
an ’a option, which would avoid the need for an exception. But you
you can’t indicate with code which func-
should get used to exceptions like these in modules, since OCaml mod- tions may cause which exceptions, but
ules tend to use them. Remember, functions are values, so functions it is good style to mention in a function’s
comments what exceptions it may raise
are also listed with the val keyword. and under what conditions.
The interface for ORDERED_COLLECTION_0 is not ideal. Consider the
following questions:

• Is ORDERED_COLLECTION_0 a type?

• How would one use ORDERED_COLLECTION_0 ?

• Why do several of the functions require an argument of type ’a ->

’a -> order?

• Why is ORDERED_COLLECTION_0 not ideal?

• How might a call to delete give you incorrect behavior for a cor-
rectly constructed tree?

An improved signature ORDERED_COLLECTION is provided in the file

orderedcoll.ml. To create this better interface, we need to introduce
another module type – COMPARABLE (provided in the file order.ml).
Take a look at the ORDERED_COLLECTION signature, and consider these
questions:

• Why is ORDERED_COLLECTION a better interface?

• Why did we need to introduce another module type COMPARABLE?

 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 209

12.12.2 Implementing ordered collections with binary search

trees

A simple – but particularly inefficient – implementation of ordered

collections is with a sorted list. Insertion places the element in the
proper position; searching can stop once a larger element is found;
the minimum element is the first in the list; the maximum, the last.
(We’ll look at efficiency issues in more detail later.) Binary search trees
provide a much better implementation.
We introduced binary trees in Section 11.5. Here, we’ll use a particu-
lar variant, the B I N A RY S E A R C H T R E E .
A binary search tree is a binary tree that obeys the following invari-
ant:

For each node in a binary search tree, all values stored in its left subtree
are less than the value stored at the node, and all values stored in its
right subtree are greater than the values stored at the node.

What if there are multiple values in the tree, even distinct ones, that
are equal in the ordering? We’ll store all such elements together at a
single node, say as a list.
For instance, consider the following set of elements of type int *
string, gleaned from my personal bucket list:10 10
I won’t say which ones I’ve completed.

4, "learn the alphabet backwards"

1, "take a selfie with my mother"
5, "climb Mount Kilimanjaro"
1, "walk the Harvard Bridge"
3, "learn how to sail"
1, "visit the Leaning Tower of Pisa"
2, "learn how to crack an egg with one hand"
3, "go on a zipline"
3, "sleep in"
4, "watch the sun rise"

The integers can be interpreted as the priority I place on the activity.

Or not. But in any case, let’s take the ordering on the elements to be
given simply by integer comparison of the first element of the pair.
(Thus, 4, "learn the alphabet backwards" and 4, "watch the
sun rise" would compare equal.)
We might store these elements in a binary search tree that looks like
the one in Figure 12.6. Note how for each node, the elements in the left
subtree precede and the right subtree follow in the ordering.
What happens when there are multiple elements that compare
equal in the ordering over elements? There are multiple possibilities,
but here we take the approach (as we have in the figure) of allowing for
that case, and for the purpose of selecting among them (as in searching
or deleting elements), choose the one that was inserted first.
210 PROGRAMMING WELL

Figure 12.6: A sample binary search tree.

The small gray circles indicate leaves of
the tree.

You will provide a functor, called BinSTree, for generating imple-

mentations of the ORDERED_COLLECTION interface. The BinSTree
functor implements a binary search tree as above where values that
compare equal are compressed into a single node containing a list of
those values. Remember that functors are not yet modules; they must
be applied to an argument module in order to produce a module. In
this case, BinSTree takes a module satisfying the COMPARABLE signa-
ture as an argument and returns an ORDERED_COLLECTION module.
Once you have implemented BinSTree, you can create IntTree – a
binary search tree of integers – by applying BinSTree to an integer
implementation of COMPARABLE.
Problem 131
Implement the insert, search, getmin, and getmax functions for BinSTree. (We’ve
provided the rest.) Don’t forget to test it well.

12.12.3 Priority queues

A priority queue is another data structure that can be considered a

collection of ordered elements, but specialized for a simpler set of
functionality. In particular, elements can be added and the minimum
element extracted. That’s all. Priority queues are widely useful, for
instance, when implementing Dijkstra’s algorithm for efficiently com-
puting shortest paths in a network. We have provided the PRIOQUEUE
interface for priority queues, which supports empty, is_empty, add,
and take operations. The add function inserts an element into the
priority queue and the take function removes the minimum element.
Because priority queues allow for only a subset of the operations
of other ordered collections, they admit of more efficient specialized
implementations. In this section you will be implementing priority
queues in three ways – with lists, with binary search trees, and with
binary heaps.
Problem 132
Complete the ListQueue functor: a naive implementation of a priority queue. In this
 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 211

implementation, the elements in the queue are stored in a simple list in priority order.
This implementation is not ideal because either the take or the add operation is O (n )
complexity. (See Chapter 14.)
Problem 133
Implement TreeQueue, which is less naive than ListQueue (but still not ideal). In this
implementation of the PRIOQUEUE interface, the queue is stored as a binary search tree
using the BinSTree functor that you’ve already implemented.

Consider these questions:

• Why is the TreeQueue implementation not ideal?

• What is the worst case complexity of add and take for a TreeQueue?

Finally, you will implement a priority queue using a binary heap,

which has the attractive property of O (l og (n )) complexity for both
add and take. Binary (min)heaps are a kind of balanced binary tree for
which the following ordering invariant holds:

Ordering invariant: The value stored at each node is smaller than all
values stored in the subtrees below the node.

(Ordered trees thus have the attractive property that the minimum
element is always stored at the root of the tree.)
In the skeleton code for the BinaryHeap functor in prioqueue.ml,
we have defined the tree type for implementing the binary heap,
which provides further clarification:
type tree =
| Leaf of elt
| OneBranch of elt * elt
| TwoBranch of balance * elt * tree * tree

The trees for use in forming binary heaps are of three sorts, corre-
sponding to the three variants in the type definition:

• Leaves store a single value of type elt (like the 17 node in Fig-
ure 12.7).

• Along the bottom edge of the tree, a tree with a single child, each
storing a value of type elt, can appear (like the 9—17 branch in
Figure 12.7).

• Finally, regular nodes of the tree store a value of type elt and have
two subtrees, a left and right subtree. (See for example, the node
with value 3 in Figure 12.7.) The node also stores its “balance” as
described below.

Binary heaps as you will implement them obey a further invariant of

being balanced:11 11
This definition of balance is a bit
different from a traditional variant for
Balance invariant: For each TwoBranch node, its left branch has either balanced binary trees that requires the
last (lowest) level in the tree to be filled
the same number or one more node than its right branch.
strictly from left to right.
212 PROGRAMMING WELL

We will call a balanced tree odd or even. A tree is odd if its left child has
one more node than its right child. A tree is even if its children are of
equal size. The invariant says then that all subtrees must be either odd
or even.
Functions over the type will often need to respect combinations of
the ordering invariant and the balance invariant:

Weak invariant: The tree is balanced.

Strong invariant: The tree is balanced and ordered.

The add and take functions must return trees that respect the strong
invariant, and should assume they will only be passed trees that also
obey the strong invariant. That is, they preserve the strong invariant.
We have provided stubs for helper functions that operate on trees that
are required to preserve only the weak invariant. Hint: Your nodes
should track whether they are odd or even. This will help you keep
your tree balanced at all times.
Notice that we have encoded the difference between odd and even
nodes in the tree type that we’ve provided for BinaryHeap. You
should probably first write a size function for your tree type. This
will help you check your representation invariant. You should not be
calling size in the implementation of take; rather, you should be us-
ing size to test take. We have provided you with the implementation
of add and a partial implementation of take. Below are some guide-
lines when implementing take and its helper functions, as well as in
understanding add.

add The add function inserts a node into a spot that will either turn
the main tree from odd to even or from even to odd. We implement
this function for you, but you should understand how it works.

take The take function removes the root of the tree (the minimum
element) and replaces it by a leaf of the tree that, when removed, turns
the tree from odd to even or from even to odd.
After removing and replacing the root node your tree will respect
the weak invariant. You must “fix” the tree to respect the strong invari-
ant, as depicted in Figure 12.7.
Some questions to consider:

• How do we know that our binary heap stays balanced?

• How might you test your binary heap?

• How might you test the helper functions used in implementing your
binary heap?
 A B S T R A C T D ATA T Y P E S A N D M O D U L A R P R O G R A M M I N G 213

3 25 7 7

9 7 9 7 9 25 9 15

17 25 15 17 25 15 17 15 17 25

(a) (b) (c) (d)

Figure 12.7: Visual depiction of fixing a

• Why is it useful to use ListQueue, TreeQueue, and BinaryHeap
binary heap to rebalance it after taking
behind a PRIOQUEUE interface? an element from it. Starting with a heap
satisfying the strong invariant (a), the
Problem 134 minimum element (3) is at the root
Complete the implementation of the binary heap priority queue by providing definitions of the tree. To “take” the minimum
for get_top, fix, get_last, and run_tests, and completing the definition for take. element, the root node is replaced (b)
with the node with value 25, turning the
Now that you’ve provided three different implementations of pri- tree from odd to even. The tree is then
ority queues, all satisfying the PRIOQUEUE interface, we give you an “fixed” by swapping nodes from the root
down the tree (c) until the value at the
example of how to use them to implement sort functions. You should
root has found its appropriate location
use these for testing (in addition to testing within the modules). (d). Of course, in the implementation
there is no actual changing of trees
– no mutable state. Rather, new tree
12.12.4 Challenge problem: Sort functor nodes are made where necessary that
incorporate existing subtrees.
Write a functor for sorting which takes a COMPARABLE module as an
argument and provides a sort function. You should abide by the follow-
ing interface:

type c
val sort : c list -> c list

You should use your BinaryHeap implementation, and test it.

12.12.5 Challenge problem: Benchmarking

Benchmark the running times of heapsort, treesort, and

selectionsort. Arrive at an algorithmic complexity for each sort-
ing algorithm. Record the results of your tests. Be convincing with your
data and analysis when establishing the algorithmic complexity of
each sort.
13
Semantics: The substitution model

We’ve introduced a broad swath of OCaml, describing both the syntax

of different constructions and their use in constructing programs. But
why the expressions of OCaml actually have the meanings they have
has been dealt with only informally.
Semantics is about what expressions mean. As described so far, ask-
ing what an OCaml expression means is tantamount to asking what it
evaluates to, what value it “means the same” as. Before getting into the
details, however, it bears considering why a formal, rigorous, precise
semantics of a programming language is even useful. Why not stick
to the informal discussion of what the constructs of a programming
language do? After all, such informal discussions, written in a natural
language (like English), seem to work just fine for reference manuals
and training videos.
There are three reasons that formalizing a semantics with mathe-
matical rigor is beneficial.

Mental hygiene Programming is used to communicate our com-

putational intentions to others. But what exactly is being com-
municated? Without a precise meaning to the expressions of the
programming language, there is room for miscommunication from
program author to reader.

Interpreters Computers generate computation by interpreting the

expressions of the programming language. Developers of inter-
preters (or compilers) for a programming language implement their
understanding of the meaning of the constructs of the program-
ming language. Without a precise meaning to the expressions of
the programming language, two interpreters might generate differ-
ent computations for the same expression, even though both were
written in good faith efforts to manifest the interpreter developers’
understandings of the language.

Metaprogramming Programs that operate over expressions of the

216 PROGRAMMING WELL

programming language – such as programs to verify correctness of a

program or transform it for efficiency or analyze it for errors – must
use a precise notion of the meanings of those expressions.

For these reasons, we introduce in this chapter a technique for giving a

semantics to some small subsets of OCaml. We continue this exercise
in Chapter 19. The final project described in Chapter 21 – the imple-
mentation of a small subset of OCaml – relies heavily on the discussion
in these two chapters.
As noted, we’ll cash out the meaning of an expression by gener-
ating a simpler expression that “means the same”. In essence, this is
the notion of evaluation that we’ve seen before. In this chapter we’ll
introduce a first method for providing a rigorous semantics of a pro-
gramming language, based on the substitution of subexpressions,
substituting for particular expressions expressions that “mean the
same” but that are simpler.
The underlying conception of substitution as the basis for seman-
Figure 13.1: Gottfried Wilhelm Leibniz
tics dates from 1677 in Gottfried Leibniz’s statement of the identity of
(1646–1716), German philosopher, (co-
indiscernibles: )inventor of the differential and integral
calculus, and philosopher. His law of
That A is the same as B signifies that the one can be substituted for the the identity of indiscernibles underlies
other, salva veritate, in any proposition whatever. substitution semantics.

Salva veritate – preserving the truth. Leibniz claims that substituting

one expression with another that means the same thing preserves the
truth of expressions.
We’ll see later (Chapters 15 and 16) that a naive interpretation of
Leibniz’s law isn’t sustainable. In particular, in the presence of state
and state change, the province of imperative programming, the law
seems to fail. But for the portion of OCaml we’ve seen so far, Leibniz’s
statement works quite well.
Following Leibniz’s view, in this chapter we provide a semantics
for a language that can be viewed as a (simple and untyped) subset
of OCaml, with constructs like arithmetic and boolean operators,
conditionals, functions (including recursive functions), and local
naming.
We provide these semantic notions in two ways: as formal rule
systems that define the evaluation relation, and as computer programs
to evaluate expressions to their values.
The particular method of providing formal semantics that we in-
troduce in this chapter is called large-step operational semantics and Figure 13.2: Gilles Kahn (1946–2006),
French computer scientist, developer
is based on the N AT U R A L S E M A N T I C S method of computer scientist of the natural semantics approach to
Gilles Kahn (Figure 13.2). programming language semantics.
Kahn was president of the French
The semantics we provide is F O R M A L in the sense that the semantic
research institute INRIA, where OCaml
rules rely only on manipulations based on the forms of the notations was developed.
 SEMANTICS: THE SUBSTITUTION MODEL 217

we introduce. The semantics we provide is an O P E R AT I O N A L S E M A N -

TICS because we provide a formal specification of what programs
evaluate to, rather than what they denote.1 The semantics we provide is
a L A R G E - S T E P semantics because it characterizes directly the relation
between expressions and what they (eventually, after perhaps many
1
The primary alternative method
individual small steps) evaluate to, rather than characterizing the rela- of providing a formal semantics is
tion between expressions and what they lead to after each individual D E N OTAT I O N A L S E M A N T I C S , which
addresses exactly this issue of what
small step. (That would be a S M A L L - S T E P S E M A N T I C S .) Notationally,
expressions denote.
we characterize this relation between an expression P and the value v
it evaluates to with an evaluation J U D G E M E N T notated P ⇓ v, which
can be read as “the expression P evaluates to the value v”.

13.1 Semantics of arithmetic expressions

Recall the language of arithmetic expressions from Section 11.4. We

start by augmenting that language with a local naming construct, the
let ⟨⟩ in ⟨⟩ . We’ll express the abstract syntax of the language using the
following BNF:

⟨binop ⟩ ::= +|-|*|/

⟨var ⟩ ::= x|y|z|⋯
⟨expr ⟩ ::= ⟨integer ⟩
| ⟨var ⟩
| ⟨expr1 ⟩ ⟨binop ⟩ ⟨expr2 ⟩
| let ⟨var ⟩ = ⟨exprdef ⟩ in ⟨exprbody ⟩

Exercise 135

For brevity, we left off unary operators. Extend the grammar to add unary operators
(negation, say).
With this grammar, we can express the abstract syntax of the con-
crete expression

let x = 3 in
let y = 5 in
x * y
218 PROGRAMMING WELL

as the tree

⟨expr ⟩

let ⟨var ⟩ = ⟨expr ⟩ in ⟨expr ⟩

x ⟨integer ⟩ let ⟨var ⟩ = ⟨expr ⟩ in ⟨expr ⟩

3 y ⟨integer ⟩ ⟨expr ⟩

5 ⟨expr ⟩ ⟨binop ⟩ ⟨expr ⟩

⟨var ⟩ * ⟨var ⟩

x y

What rules shall we use for evaluating the expressions of the lan-
guage? Recall that we write a judgement P ⇓ v to mean that the expres-
sion P evaluates to the value v. The VA L U E S , the results of evaluation,
are those expressions that evaluate to themselves. By convention, we’ll
use italic capitals like P , Q, etc. to stand for arbitrary expressions, and
v (possibly subscripted) to stand for expressions that are values. You
should think of P and v as expressions structured as per the abstract
syntax of the language – it is the abstract, structured expressions that
have well-defined meanings by the rules we’ll provide – though we
notate them using the concrete syntax of OCaml, since we need some
linear notation for specifying them.
Certain cases are especially simple. Numeric literal expressions like
3 or 5 are already as simplified as they can be. They evaluate to them-
selves; they are values. We could enumerate a plethora of judgements
that express this self-evaluation, like

1⇓1

2⇓2

3⇓3

4⇓4

5⇓5
⋯

but we’d need an awful lot of them. Instead, we’ll just use a schematic
rule for capturing permissible judgements:

n⇓n (R int )
 SEMANTICS: THE SUBSTITUTION MODEL 219

Here, we use a schematic variable n to stand for any integer, and use
the notation n for the OCaml numeral expression that encodes the
number n.
Using this schematic rule notation we can provide general rules for
evaluating other arithmetic expressions. To evaluate an expression of
the form P + Q, where P and Q are two subexpressions, we first need
to know what values P and Q evaluate to; since they will be numeric
values, we can take them to be m and n, respectively. Then the value
that P + Q evaluates to will be m + n. We’ll write the rule as follows:

P + Q⇓

P ⇓m
∣ (R + )
Q ⇓n

⇓ m +n

In this rule notation, the first line is intended to indicate that we are
evaluating P + Q, the blank space to the right of the ⇓ indicating that
some further evaluation judgements are required. Those are the two
indented judgements provided to the right of the long vertical bar
between the two occurrences of ⇓. The final line provides the value
that the original expression evaluates to.
Thus, this rule can be glossed as “To evaluate an expression of
the form P + Q, first evaluate P to an integer value m and Q to an
integer value n. The value of the full expression is then the integer
literal representing the sum of m and n.”
Using these two rules, we can now show a particular evaluation, like
that of the expression 3 + 5:2 2
Wait, where did that 8 come from
exactly? Since 3 ≡ 3 and 5 ≡ 5, the rule
3 + 5⇓ R int gives the result as 3 + 5 ≡ 8 ≡ 8.

3⇓3
∣
5⇓5

⇓8

or the evaluation of 3 + 5 + 7:

3 + 5 + 7⇓
RRR
RRR 3 + 5 ⇓
RRR 3⇓3
RRR ∣
RRR 5⇓5
RRR
RRR ⇓8
RRR
RRR 7 ⇓ 7
R
⇓ 15
220 PROGRAMMING WELL

Exercise 136
Why is the proof for the value of 3 + 5 + 7 not structured as

3 + 5 + 7⇓
RRR 3 ⇓ 3
RRR
RRR 5 + 7 ⇓
RRR
RRR 5⇓5
RRR ∣
RRR 7⇓7
RRR
RRR ⇓ 12
⇓ 15 ?

We should have similar rules for other arithmetic operators. Here’s a

3
What may be mind-boggling here is
possible rule for division: the role of the mathematical notation
used in the result part of the rule. How
P / Q⇓ is it that we can make use of notations
like ⌊m /n ⌋ in defining the semantics
P ⇓m
∣ (R / ) of the / operator? Doesn’t appeal to
Q ⇓n that kind of mathematical notation
beg the question? Or at least call for its
⇓ ⌊m /n ⌋ own semantics? Yes, it does, but since
we have to write down the semantics
These rules for addition and division may look trivial, but they are of constructs somehow or other, we
use commonly accepted mathematical
not. The division rule specifies that the / operator in OCaml when notation applied in the context of
applied to two numerals specifies the integer portion of their ratio. The natural language (in the case at hand,
English). You may think that this merely
language being specified might have been otherwise.3 The language
postpones the problem of giving OCaml
might have used a different operator (like //) for integer division, semantics by reducing it to the problem
of giving semantics for mathematical
P // Q ⇓ notation and English. You would
be right, and the problem is further
P ⇓m exacerbated when the semantics makes
∣ use of mathematical notation that is not
Q ⇓n
so familiar, for instance, the substitution
⇓ ⌊m /n ⌋ notation to be introduced shortly. But
we have to start somewhere.

(as happens to be used in Python 3 for instance). The example should

make clear the distinction between the O B J E C T L A N G UA G E whose
semantics is being defined and the M E TA L A N G UA G E being used to
define it.
Similarly, the rule could have defined the result differently, say

P / Q⇓

P ⇓m
∣
Q ⇓n ,

⇓ ⌈m /n ⌉

which specifies that the result of the division is the integer resulting
from rounding up, rather than down.
Nonetheless, there is not too much work being done by these rules,
and if that were all there were to defining a semantics, there would be
little reason to go to the trouble. Things get more interesting, however,
 SEMANTICS: THE SUBSTITUTION MODEL 221

when additional constructs such as local naming are considered,

which we turn to next.

Exercise 137

Write evaluation rules for the other binary operators and the unary operators you added
in Exercise 135.

13.2 Semantics of local naming

The ⟨expr ⟩ language defined in the grammar above includes a lo-

cal naming construct, whose concrete syntax is expressed with
let ⟨⟩ in ⟨⟩ . What is the semantics of such an expression? It is here
that substitution starts to play a critical role. We will take the meaning
of this local naming construct to work by substituting the value of the
definition for occurrences of the variable in the body. More precisely, we
use the following evaluation rule:

let x = D in B ⇓

D ⇓ vD
∣ (R let )
B [x ↦ v D ] ⇓ v B

⇓ vB

We’ve introduced a new notation – Q [x ↦ P ] – for substituting the

expression P for occurrences of the variable x in the expression Q. For
instance,

(x * x)[x ↦ 5] = 5 * 5

that is, substituting 5 for x in the expression x * x yields 5 * 5. (It

doesn’t yield 25 though. That would require a further evaluation, which
is what the part of the rule B [x ↦ v D ] ⇓ v B does.)
The evaluation rule R let can be glossed as follows: “To evaluate an
expression of the form let x = D in B , first evaluate the expres-
sion D to a value v D and evaluate the result of substituting v D for
occurrences of x in the expression B to a value v B . The value of the full
expression is then v B .”
Using this rule (and the others), we can now show

let x = 5 in x * x ⇓ 25
222 PROGRAMMING WELL

as per the following derivation:

let x = 5 in x * x ⇓
RRR
RRR 5 ⇓ 5
RRR 5
RRR * 5⇓
RRR 5⇓5
RRR ∣
RRR 5⇓5
RRR
RRR ⇓ 25
R
⇓ 25

Let’s put this first derivation together step by step so the steps are
clear. We want a derivation that demonstrates what let x = 5 in x
* x evaluates to. It will be of the form

let x = 5 in x * x ⇓
∣ ⋮
⇓⋯

This pattern matches rule R let , where x plays the role of the schematic
variable x, 5 plays the role of the schematic expression D, and x *
x plays the role of B . We will plug these into the two subderivations
required. First is the subderivation evaluating D (that is, 5):

let x = 5 in x * x ⇓
RRR 5 ⇓
RRR
RRR ∣ ⋮
RRR
RRR ⇓ ⋯
RRR
RRR ⋯

⇓⋯

This subderivation can be completed using the R int rule, which re-
quires no subderivations itself.

let x = 5 in x * x ⇓
RRR 5 ⇓
RRR
RRR ∣
RRR
RRR ⇓ 5
RRR
RRR ⋯

⇓⋯

Thus, the result of this subderivation, v D is 5.

Second is the subderivation for evaluating B [x ↦ v D ] to its value v B .
 SEMANTICS: THE SUBSTITUTION MODEL 223

Now

B [x ↦ v D ] = (x * x)[x ↦ 5]
= x[x ↦ 5] * x[x ↦ 5]
=5 * 5

(We’ll define this substitution operation carefully in Section 13.3.) So

the second subderivation must evaluate the expression 5 * 5:

let x = 5 in x * x ⇓
RRR
RRR 5 ⇓ 5
RRR 5
RRR * 5⇓
RRR ∣ ⋮
RRR
RRR ⇓⋯
R
⇓⋯

This second subderivation matches a rule R ∗ analogous to R + . (You

would have written it in Exercise 137.) Here, 5 plays the role of both P
and Q:

let x = 5 in x * x ⇓
RRR
RRR 5 ⇓ 5
RRR 5
RRR * 5⇓
RRR 5⇓m
RRR ∣
RRR 5⇓n
RRR
RRR ⇓ m ⋅n
R
⇓⋯

Now, the subderivations of the 5 * 5 subderivation both evaluate to

5. We use the R int rule twice, with 5 for both m and n, so m and n are
both 5, and m ⋅ n is 25. The result for the original expression as a whole
is therefore also 25.

let x = 5 in x * x ⇓
RRR
RRR 5 ⇓ 5
RRR 5
RRR * 5⇓
RRR 5⇓5
RRR ∣
RRR 5⇓5
RRR
RRR ⇓ 25
R
⇓ 25

Exercise 138
Carry out derivations for the following expressions:
1. let x = 3 in let y = 5 in x * y
224 PROGRAMMING WELL

2. let x = 3 in let y = x in x * y

3. let x = 3 in let x = 5 in x * y

4. let x = 3 in let x = x in x * x

5. let x = 3 in let x = y in x * x

Are the values for these expressions according to the semantics consistent with how
OCaml evaluates them?

13.3 Defining substitution

Because of the central place of substitution in providing the semantics

of the language, this approach to semantics is referred to as a S U B S T I -
T U T I O N S E M A N T I C S.
Some care is needed in precisely defining this substitution opera-
tion. A start (which we’ll see in Section 13.3.2 isn’t fully correct) is given
by the following recursive equational definition:

m [x ↦ Q ] = m
x [x ↦ Q ] = Q
y [x ↦ Q ] = y where x ≡
/y
(P + R )[ x ↦ Q ] = P [ x ↦ Q ] + R [ x ↦ Q ]
and similarly for other binary operators
(let y = D in B )[x ↦ Q ] = let y = D [x ↦ Q ] in B [x ↦ Q ]

Exercise 139

Verify using this definition for substitution the derivation above showing that
(x * x)[x ↦ 5] = 5 * 5.

13.3.1 Handling variable scope

You may have noticed in Exercise 138 that some care must be taken
when substituting. Consider the following case:

let x = 3 in let x = 5 in x

Intuitively, given the scope rules of OCaml described informally in Sec-

tion 5.4, this expression should evaluate to 5, since the final occurrence
of x is bound by the inner let (defined to be 5), not the outer one.
 SEMANTICS: THE SUBSTITUTION MODEL 225

However, if we’re not careful, we’ll get a derivation like this:

let x = 3 in let x = 5 in x

⇓
RRR
RRR 3 ⇓ 3
RRR let x = 5 in 3 ⇓
RRR
RRR 5⇓5
RRR ∣
RRR 3⇓3
RRR
RRR ⇓3
R
⇓3

The highlighted expression is supposed to be the result of replacing x

with its value 3 in the body of the definition let x = 5 in x, that is,

(let x = 5 in x)[x ↦ 3] .

Using the equational definition given above, we have

(let x = 5 in x)[x ↦ 3]
= let x = 5[x ↦ 3] in x[x ↦ 3]
= let x = 5 in x[x ↦ 3]
= let x = 5 in 3 .

13.3.2 Free and bound occurrences of variables

It appears we must be very careful in how we define this substitution

operation P [x ↦ Q ]. In particular, we don’t want to replace every
occurrence of the token x in P , only the free occurrences. The variable
being introduced in a let should definitely not be replaced, nor should
any occurrences of x within the body of a let that also introduces x.
A binding construct (a let or a fun) is said to B I N D the variable
that it introduces. A variable occurrence is said to be B O U N D if it falls
within the scope of a construct that binds that variable. Thus, in the
expressions fun x -> x + y or let x = 3 in x + y, the high-
lighted occurrences of x are bound occurrences, bound by the fun or
let, respectively, in the expressions.
A variable occurrence is said to be F R E E if it is not bound. Thus,
in the expressions fun x -> x + y or let x = 3 in x + y , the
occurrences of y are free occurrences.

Exercise 140
In the following expressions, draw a line connecting each bound variable to the binding
construct that binds it. Then circle all of the free occurrences of variables.

1. x
2. x + y
226 PROGRAMMING WELL

3. let x = 3 in x

4. let f = f 3 in x + y

5. (fun x -> x + x) x

6. fun x -> let x = y in x + 3

We can define the set4 of F R E E VA R I A B L E S in an expression P , no- 4

For a review of the set notations that
tated F V (P ), through the recursive definition in Figure 13.3. By way we use, see Section A.4.

of example, the definition says that the free variables in the expres-
sion fun y -> f (x + y) are just f and x, as shown in the following
derivation:

F V (fun y -> f (x + y)) = F V (f (x + y)) − {y}

= F V (f) ∪ F V (x + y) − {y}
= {f} ∪ F V (x) ∪ F V (y) − {y}
= {f} ∪ {x} ∪ {y} − {y}
= {f, x, y} − {y}
= {f, x}

Exercise 141

Use the definition of F V to derive the set of free variables in the expressions below.
Circle all of the free occurrences of the variables.

1. let x = 3 in let y = x in f x y

2. let x = x in let y = x in f x y

3. let x = y in let y = x in f x y

4. let x = fun y -> x in x

Exercise 142

The definition of F V in Figure 13.3 is incomplete, in that it doesn’t specify the free
variables in a let rec expression. Add appropriate rules for this construct of the
language, being careful to note that in an expression like let rec x = fun y -> x in
x, the variable x is not free. (Compare with Exercise 141(4).)

Now that we have formalized the idea of free and bound variables,
it may be clearer what is going wrong in the previous substitution
example. The substitution rule for substituting into a let expression

(let y = D in B )[x ↦ Q ] = let y = D [x ↦ Q ] in B [x ↦ Q ]

shouldn’t apply when x and y are the same variable. In such a case, the
occurrences of x in D or B are not free occurrences, but are bound by
the let. We modify the definition of substitution accordingly:
 SEMANTICS: THE SUBSTITUTION MODEL 227

m [x ↦ Q ] = m
x [x ↦ Q ] = Q
y [x ↦ Q ] = y where x ≡
/y
(P + R )[ x ↦ Q ] = P [ x ↦ Q ] + R [ x ↦ Q ] and similarly for other binary operators
(let y = D in B )[x ↦ Q ] = let y = D [x ↦ Q ] in B [x ↦ Q ] where x ≡
/y
(let x = D in B )[x ↦ Q ] = let x = D [x ↦ Q ] in B

Exercise 143
Use the definition of the substitution operation above to give the expressions (in con-
crete syntax) specified by the following substitutions:
1. (x + x)[x ↦ 3]
2. (x + x)[y ↦ 3]
3. (x * x)[x ↦ 3 + 4]
4. (let x = y in y + x)[y ↦ z]
5. (let x = y in y + x)[x ↦ z]

Exercise 144
Use the semantic rules developed so far (see Figure 13.5) to reduce the following expres-
sions to their values. Show the derivations.
1. let x = 3 * 4 in
x + x

2. let y = let x = 5
in x + 1
in y + 2

13.4 Implementing a substitution semantics

Given a grammar and appropriate semantic evaluation rules and def-

initions for substitution, it turns out to be quite simple to implement
the corresponding semantics, as a function that evaluates expressions
to their values.
The grammar defining the abstract syntax of the language (repeated
here for reference)

⟨binop ⟩ ::= +|-|*|/

⟨var ⟩ ::= x|y|z|⋯
⟨expr ⟩ ::= ⟨integer ⟩
| ⟨var ⟩
| ⟨expr1 ⟩ ⟨binop ⟩ ⟨expr2 ⟩
| let ⟨var ⟩ = ⟨exprdef ⟩ in ⟨exprbody ⟩

can be implemented, as we have done before (Section 11.4), with an

algebraic type definition
228 PROGRAMMING WELL

# type binop = Plus | Divide ;;

type binop = Plus | Divide

# type varspec = string ;;

type varspec = string

# type expr =
# | Int of int
# | Var of varspec
# | Binop of binop * expr * expr
# | Let of varspec * expr * expr ;;
type expr =
Int of int
| Var of varspec
| Binop of binop * expr * expr
| Let of varspec * expr * expr

The varspec type specifies strings as a means to differentiate distinct

variables. The binop type enumerates the various binary operators.
(For brevity, in this example, we’ve only included two binary operators,
for addition and division.) The expr type provides the alternative
methods for building expressions recursively.
Then, the abstract syntax for the concrete expression

let x = 3 in
let y = 5 in
x / y

is captured by the OCaml expression

# Let ("x", Int 3,

# Let ("y", Int 5,
# Binop (Divide, Var "x", Var "y"))) ;;
- : expr =
Let ("x", Int 3, Let ("y", Int 5, Binop (Divide, Var "x", Var
"y")))

Exercise 145
Augment the type definitions to allow for other binary operations (subtraction and
multiplication, say) and for unary operations (negation).

13.4.1 Implementing substitution

With a representation of expressions in hand, we can proceed to im-

plement various useful functions over the expressions. Rather than
provide implementations, we leave them as exercises.

Exercise 146
Write a function subst : expr -> varspec -> expr -> expr that performs substi-
tution, that is, subst p x q returns the expression that is the result of substituting q for
the variable x in the expression p. For example,
 SEMANTICS: THE SUBSTITUTION MODEL 229

# subst (Binop (Plus, Var "x", Var "y")) "x" (Int 3) ;;

- : expr = Binop (Plus, Int 3, Var "y")
# subst (Binop (Plus, Var "x", Var "y")) "y" (Int 3) ;;
- : expr = Binop (Plus, Var "x", Int 3)
# subst (Binop (Plus, Var "x", Var "y")) "z" (Int 3) ;;
- : expr = Binop (Plus, Var "x", Var "y")

13.4.2 Implementing evaluation

Now the semantics of the language – the evaluation of expressions

to their values – can be implemented as a recursive function eval :
expr -> expr, which follows the evaluation rules just introduced. The
type of the function indicates that the header line should be
let rec eval (exp : expr) : expr = ...

The computation proceeds based on the structure of exp, which might

be any of the structures introducing the semantic rules. Consequently,
we match on these structures:
let rec eval (exp : expr) : expr =
match exp with
| Int n -> ...
| Var x -> ...
| Binop (Plus, e1, e2) -> ...
| Binop (Divide, e1, e2) -> ...
| Let (var, def, body) -> ...

The computation for each of the cases mimics the computations in the
evaluation rules exactly. Integers, for instance, are self-evaluating.
let rec eval (exp : expr) : expr =
match exp with
| Int n -> Int n
| Var x -> ...
| Binop (Plus, e1, e2) -> ...
| Binop (Divide, e1, e2) -> ...
| Let (var, def, body) -> ...

The second pattern concerns what should be done for evaluating free
variables in expressions. (Presumably, any bound variables were sub-
stituted away by virtue of the final pattern-match.) We have provided
no evaluation rule for free variables, and for good reason. Expressions
with free variables, called O P E N E X P R E S S I O N S don’t have a value in
and of themselves. Consequently, we can simply report an error upon
evaluation of a free variable. We introduce an exception for this pur-
pose.
let rec eval (exp : expr) : expr =
match exp with
| Int n -> Int n
| Var x -> raise (UnboundVariable x)
| Binop (Plus, e1, e2) -> ...
| Binop (Divide, e1, e2) -> ...
| Let (var, def, body) -> ...
230 PROGRAMMING WELL

The binary operator rules work by recursively evaluating the operands

and applying an appropriate computation to the results.

let rec eval (exp : expr) : expr =

match exp with
| Int n -> Int n
| Var x -> raise (UnboundVariable x)
| Binop (Plus, e1, e2) ->
let Int m = eval e1 in
let Int n = eval e2 in
Int (m + n)
| Binop (Divide, e1, e2) ->
let Int m = eval e1 in
let Int n = eval e2 in
Int (m / n)
| Let (var, def, body) -> ...

Finally, the naming rule R let performs substitution of the value of

the definition in the body, and evaluates the result. We appeal to the
function subst from Exercise 146.

# exception UnboundVariable of string ;;

exception UnboundVariable of string

# let rec eval (exp : expr) : expr =

# match exp with
# | Int n -> Int n (* R_int *)
# | Var x -> raise (UnboundVariable x)
# | Binop (Plus, e1, e2) -> (* R_+ *)
# let Int m = eval e1 in
# let Int n = eval e2 in
# Int (m + n)
# | Binop (Divide, e1, e2) -> (* R_/ *)
# let Int m = eval e1 in
# let Int n = eval e2 in
# Int (m / n)
# | Let (var, def, body) -> (* R_let *)
# let def' = eval def in
# eval (subst body var def') ;;
Lines 7-8, characters 0-11:
7 | let Int n = eval e2 in
8 | Int (m + n)
Warning 8: this pattern-matching is not exhaustive.
Here is an example of a case that is not matched:
(Var _|Binop (_, _, _)|Let (_, _, _))
Lines 6-8, characters 0-11:
6 | let Int m = eval e1 in
7 | let Int n = eval e2 in
8 | Int (m + n)
Warning 8: this pattern-matching is not exhaustive.
Here is an example of a case that is not matched:
(Var _|Binop (_, _, _)|Let (_, _, _))
Lines 11-12, characters 0-11:
11 | let Int n = eval e2 in
 SEMANTICS: THE SUBSTITUTION MODEL 231

12 | Int (m / n)
Warning 8: this pattern-matching is not exhaustive.
Here is an example of a case that is not matched:
(Var _|Binop (_, _, _)|Let (_, _, _))
Lines 10-12, characters 0-11:
10 | let Int m = eval e1 in
11 | let Int n = eval e2 in
12 | Int (m / n)
Warning 8: this pattern-matching is not exhaustive.
Here is an example of a case that is not matched:
(Var _|Binop (_, _, _)|Let (_, _, _))
val eval : expr -> expr = <fun>

Two problems jump out: First, violating the edict of intention,

we’ve not provided information about what to do in cases where the
arguments to an integer operator evaluate to something other than in-
tegers. These show up as “pattern-matching not exhaustive” warnings.
Second, violating the edict of irredundancy, the code for binary op-
erators is quite redundant. We’ll solve both problems simultaneously
by factoring out the redundancy into a function for evaluating binary
operator expressions. We’ll introduce another exception for reporting
ill-formed expressions.
# exception UnboundVariable of string ;;
exception UnboundVariable of string
# exception IllFormed of string ;;
exception IllFormed of string

# let binopeval (op : binop) (v1 : expr) (v2 : expr)

# : expr =
# match op, v1, v2 with
# | Plus, Int x1, Int x2 -> Int (x1 + x2)
# | Plus, _, _ ->
# raise (IllFormed "can't add non-integers")
# | Divide, Int x1, Int x2 -> Int (x1 / x2)
# | Divide, _, _ ->
# raise (IllFormed "can't divide non-integers") ;;
val binopeval : binop -> expr -> expr -> expr = <fun>

# let rec eval (e : expr) : expr =

# match e with
# | Int _ -> e
# | Var x -> raise (UnboundVariable x)
# | Binop (op, e1, e2) ->
# binopeval op (eval e1) (eval e2)
# | Let (x, def, body) ->
# eval (subst body x (eval def)) ;;
val eval : expr -> expr = <fun>

This function allows evaluating expressions in the language reflecting

the semantics of those expressions.
# eval (Binop (Plus, Int 5, Int 10)) ;;
- : expr = Int 15
232 PROGRAMMING WELL

# eval (Let ("x", Int 3,

# Let ("y", Int 5,
# Binop (Divide, Var "x", Var "y")))) ;;
- : expr = Int 0

13.5 Problem section: Semantics of booleans and condi-

tionals

Exercise 147
Augment the abstract syntax of the language to introduce boolean literals true and
false. Add substitution semantics rules for the new constructs. Adjust the definitions of
subst and eval to handle these new literals.

Exercise 148
Augment the abstract syntax of the language to add conditional expressions (if ⟨⟩
then ⟨⟩ else ⟨⟩ ). Add substitution semantics rules for the new construct. Adjust the
definitions of subst and eval to handle conditionals.

13.6 Semantics of function application

We can extend our language further, by introducing (anonymous)

functions and their application. We augment the language with two
rules for function expressions and function applications as follows:

⟨binop ⟩ ::= +|-|*|/

⟨var ⟩ ::= x|y|z|⋯
⟨expr ⟩ ::= ⟨integer ⟩
| ⟨var ⟩
| ⟨expr1 ⟩ ⟨binop ⟩ ⟨expr2 ⟩
| let ⟨var ⟩ = ⟨exprdef ⟩ in ⟨exprbody ⟩
| fun ⟨var ⟩ -> ⟨exprbody ⟩
| ⟨exprfun ⟩ ⟨exprarg ⟩

To complete the semantics for this language, we simply have to add

rules for the evaluation of functions and applications.
The case of functions is especially simple. Functions are pending
computations; they don’t take effect until they are applied. So we can
take functions to be values, that is, they self-evaluate.

fun x -> B ⇓ fun x -> B (R fun )

All the work happens upon application. To evaluate an application,

we must evaluate the function part to get the function to be applied
and evaluate the argument part to get the argument’s value, and then
evaluate the body of the function, after substituting in the argument
 SEMANTICS: THE SUBSTITUTION MODEL 233

for the variable bound by the function.

P Q⇓
RRR P ⇓ fun x -> B
RRR
RRR Q ⇓ v (R app )
RRR Q
RRR B [x ↦ v ] ⇓ v
R Q B

⇓ vB

Exercise 149

Give glosses for these two rules R fun and R app , as was done for the previous rules R + and
R let .
Let’s try an example:

(fun x -> x + x) (3 * 4)

Intuitively, this should evaluate to 24. The derivation proceeds as

follows:

(fun x -> x + x) (3 * 4)

⇓
RRR (fun x -> x + x) ⇓ (fun x -> x + x)
RRR
RRR 3
RRR * 4⇓
RRR 3⇓3
RRR ∣
RRR 4 ⇓4
RRR
RRR ⇓ 12
RRR
RRR 12 + 12 ⇓
RRR
RRR 12 ⇓ 12
RRR ∣
RRR 12 ⇓ 12
RRR
RRR ⇓ 24
⇓ 24

The combination of local naming and anonymous functions gives

us the ability to give names to functions:

let double = fun x -> 2 * x in

double (double 3)
234 PROGRAMMING WELL

The derivations start getting a bit complicated:

let double = fun x -> 2 * x in double (double 3)

⇓
RRR fun x -> 2
RRR * x ⇓ fun x -> 2 * x
RRR (fun x -> 2 * x) ((fun x -> 2 * x) 3)
RRR
RRR ⇓
RRR
RRR RRR fun x -> 2
RRR RRR * x ⇓ fun x -> 2 * x
RRR RRR (fun x -> 2 * x) 3
RRR RRR
RRR RRR ⇓
RRR RRR
RRR R RRR fun x -> 2
RRR RRR * x ⇓ fun x -> 2 * x
RRR RRR R
RRR RRR R
RRR 3 ⇓ 3
RRR RRR RRR
RRR RRR RRR 2 * 3 ⇓
RRR RRR RRR
RRR RRR RRR 2⇓2
RRR RRR RRR ∣
RRR RRR RRR 3 ⇓3
RRR RRR RRR
RRR RRR R ⇓6
RRR RRR
RRR RRR ⇓6
RRR RRR 2
RRR R * 6 ⇓ 12
RRR ⇓ 12
R
⇓ 12

Exercise 150

Carry out similar derivations for the following expressions:

1. (fun x -> x + 2) 3

2. let f = fun x -> x in

f (f 5)

3. let square = fun x -> x * x in

let y = 3 in
square y

4. let id = fun x -> x in

let square = fun x -> x * x in
let y = 3 in
id square y

13.6.1 More on capturing free variables

There is still a problem in our definition of substitution. Consider the

following expression: let f = fun z -> y in (fun y -> f 3)
1. Intuitively speaking, this expression seems ill-formed; it defines a
function f that makes use of an unbound variable y in its body. But
using the definitions that we have given so far, we would have the
 SEMANTICS: THE SUBSTITUTION MODEL 235

following derivation:

let f = fun z -> y in (fun y -> f 3) 1

⇓
RRR fun z -> y ⇓ fun z -> y
RRR
RRR (fun y -> (fun z -> y) 3) 1
RRR
RRR ⇓
RRR
RRR RRR (fun y -> (fun z > y) 3) ⇓ (fun y -> (fun z -> y) 3)
RRR RRR
RRR RRR 1 ⇓ 1
RRR RRR
RRR RRR (fun z -> 1) 3 ⇓
RRR RRR
RRR RRR fun z -> 1 ⇓ fun z -> 1
RRR RRR ∣
RRR RRR 1⇓1
RRR RRR
RRR RRR ⇓1
RRR
RRR ⇓1
⇓1
The problem happens in the highlighted expression. We’re sneaking
a y inside the scope of the variable y bound by the fun. That’s not
kosher. We need to change the definition of substitution to make sure
that such VA R I A B L E C A P T U R E doesn’t occur. The following rules for
substituting inside a function work by replacing the bound variable y
with a new freshly minted variable, say z, that doesn’t occur elsewhere,
renaming all occurrences of y accordingly.

(fun x -> P )[x ↦ Q ] = fun x -> P

(fun y -> P )[x ↦ Q ] = fun y -> P [x ↦ Q ]

where x ≡
/ y and y ∈/ F V (Q )

(fun y -> P )[x ↦ Q ] = fun z -> P [ y ↦ z ][x ↦ Q ]

where x ≡
/ y and y ∈ F V (Q ) and z is a fresh variable

Exercise 151
Carry out the derivation for
let f = fun z -> y in (fun y -> f 3) 1
as above but with this updated definition of substitution. What happens at the step
highlighted above?

Exercise 152
What should the corresponding rule or rules defining substitution on let ⋯ in ⋯
expressions be? That is, how should the following rule be completed? You’ll want to think
about how this construct reduces to function application in determining your answer.
(let y = Q in R )[x ↦ P ] = ⋯
Try to work out your answer before checking it with the full definition of substitution in
Figure 13.4.
236 PROGRAMMING WELL

Exercise 153

Use the definition of the substitution operation above to determine the results of the
following substitutions:

1. (fun x -> x + x)[x ↦ 3]

2. (fun x -> y + x)[x ↦ 3]

3. (let x = y * y in x + x)[x ↦ 3]

4. (let x = y * y in x + x)[y ↦ 3]

The implementation of substitution should be updated to han-

dle this issue of avoiding the capture of free variables. The next two
exercises do so.

Exercise 154

Write a function free_vars : expr -> varspec Set.t that returns a set of varspecs
corresponding to the free variables in the expression as per Figure 13.3. (Recall the
discussion of the OCaml library module Set in Section 12.8.)

Exercise 155

Revise the definition of subst to eliminate the problem of variable capture by imple-
menting the set of rules given in Figure 13.4.

F V (m ) = ∅ (integers) (13.1)
F V (x ) = {x } (variables) (13.2)
F V (P + Q ) = F V (P ) ∪ F V (Q ) (and similarly for other binary operators) (13.3)
F V (P Q ) = F V (P ) ∪ F V (Q ) (applications) (13.4)
F V (fun x -> P ) = F V (P ) − {x } (functions) (13.5)
F V (let x = P in Q ) = (F V (Q ) − {x }) ∪ F V (P ) (binding) (13.6)

Figure 13.3: Definition of F V , the set

of free variables in expressions for a
functional language with naming and
arithmetic.

13.7 Substitution semantics of recursion

You may observe that the rule for evaluating let ⟨⟩ in ⟨⟩ expressions
doesn’t allow for recursion. For instance, the Fibonacci example pro-
 SEMANTICS: THE SUBSTITUTION MODEL 237

m [x ↦ P ] = m (13.7)
x [x ↦ P ] = P (13.8)
y [x ↦ P ] = y where x ≡
/y (13.9)
(Q + R )[x ↦ P ] = Q [x ↦ P ] + R [x ↦ P ] (13.10)
and similarly for other binary operators
Q R [x ↦ P ] = Q [x ↦ P ] R [x ↦ P ] (13.11)
(fun x -> Q )[x ↦ P ] = fun x -> Q (13.12)
(fun y -> Q )[x ↦ P ] = fun y -> Q [x ↦ P ] (13.13)
where x ≡
/ y and y ∈/ F V (P )
(fun y -> Q )[x ↦ P ] = fun z -> Q [ y ↦ z ][x ↦ P ] (13.14)
where x ≡
/ y and y ∈ F V (P ) and z is a fresh variable
(let x = Q in R )[x ↦ P ] = let x = Q [x ↦ P ] in R (13.15)
(let y = Q in R )[x ↦ P ] = let y = Q [x ↦ P ] in R [x ↦ P ] (13.16)
where x ≡
/ y and y ∈/ F V (P )
(let y = Q in R )[x ↦ P ] = let z = Q [x ↦ P ] in R [ y ↦ z ][x ↦ P ] (13.17)
where x ≡
/ y and y ∈ F V (P ) and z is a fresh variable

Figure 13.4: Definition of substitution of

expressions for variables in expressions
for a functional language with naming
and arithmetic.
 238 PROGRAMMING WELL

ceeds as follows:

let f = fun n -> if n = 0 then 1 else n * f (n - 1) in f 2

⇓
RRR fun n -> if n = 0 then 1 else n
RRR * f (n - 1) ⇓ fun n -> if n = 0 then 1 else n * f (n - 1)
RRR (fun n -> if n = 0 then 1 else n
RRR * f (n - 1)) 2
RRR ⇓
RRR
RRR RRR fun n -> if n = 0 then 1 else n
RRR RRR * f (n - 1) ⇓ fun n -> if n = 0 then 1 else n * f (n - 1)
RRR RRR 2 ⇓ 2
RRR RRR
RRR RRR if 2 = 0 then 1 else 2
RRR R * f (2 - 1) ⇓ ???
RRR ⇓ ???
⇓ ???

The highlighted expression, if 2 = 0 then 1 else 2 * f (2 - 1),

eventually leads to an attempt to apply the unbound variable f to its
argument 1.
Occurrences of the name definiendum in the body are properly
replaced with the definiens, but occurrences in the definiens itself are
not. But what should those recursive occurrences of f be replaced by?
It doesn’t suffice simply to replace them with the definiens, as that
still has a free occurrence of the definiendum. Rather, we’ll replace
them with their own recursive let construction, thereby allowing
later occurrences to be handled as well. In the factorial example, we’ll
replace the free occurrence of f in the definiens by let rec f = fun
n -> if n = 0 then 1 else n * f (n - 1) in f, that is, an
expression that evaluates to whatever f evaluates to in the context of
the recursive definition itself.
Thus the substitution semantics rule for let rec, subtly different
from the let rule, will be as follows:

let rec x = D in B ⇓

D ⇓ vD
∣
B [x ↦ v D [x ↦ let rec x = v D in x ]] ⇓ v B

⇓ vB
(R letrec )

Continuing the factorial example above, we would substitute for f in

the third line the expression let rec f = fun n -> if n = 0 then
1 else n * f (n - 1) in f, forming

(fun n -> if n = 0 then 1

else n * (let rec f = fun n -> if n = 0 then 1
else n * f (n-1) in f) (n-1)) 2

Proceeding further, the final line becomes

 SEMANTICS: THE SUBSTITUTION MODEL 239

if 2 = 0 then 1
else 2 * (let rec f = fun n -> if n = 0 then 1
else n * f (n-1) in f) (2-1))

which will (eventually) evaluate to 2.

Exercise 156
Thanklessly continue this derivation until it converges on the final result for the factorial
of 2, viz., 2. Then thank your lucky stars that we have computers to do this kind of rote
repetitive task for us.
We’ll provide an alternative approach to semantics of recursion
when we introduce environment semantics in Chapter 19.

We defined a set of formal rules providing the meanings of OCaml

expressions via simplifying substitutions of equals for equals, resulting
finally in the values that most simply encapsulate the meanings of
complex expressions.
An interpreter for a programming language (the object language)
written in the same programming language (as metalanguage) – a
M E TA C I R C U L A R I N T E R P R E T E R – provides another way of getting at the
semantics of a language. In fact, the first semantics for the program-
ming language L I S P was given as a metacircular interpreter.
In both cases, we see the advantage of having a language with a
small core, sprinkled liberally with syntactic sugar, since only the core
need be given the formal treatment through rules or metacircular
interpretation. The syntactic sugar can be translated out. For instance,
although the metacircular interpreter that we started developing here
does not handle the more compact function definition notation seen
in

let f x = x + 1

this expression can be taken as syntactic sugar for (that is, a variant
concrete syntax for the abstract syntax of) the expression

let f = fun x -> x + 1

which we already have defined formal rules to handle. In the case of a

metacircular interpreter, we can imagine that the parser for the former
expression will simply provide the abstract syntax of the latter.
Our exploration of rigorous semantics for programs will continue
once the substitution approach starts to falter in the presence of state
change and imperative programming.
240 PROGRAMMING WELL

Figure 13.5: Substitution semantics

rules for evaluating expressions, for a
functional language with naming and
n⇓n (R int ) arithmetic.

fun x -> B ⇓ fun x -> B (R fun )

P + Q⇓

P ⇓m
∣ (R + )
Q ⇓n

⇓ m +n

P / Q⇓

P ⇓m
∣ (R / )
Q ⇓n

⇓ ⌊m /n ⌋

P Q⇓
RRR P ⇓ fun x -> B
RRR
RRR Q ⇓ v (R app )
RRR Q
RRR B [x ↦ v ] ⇓ v
R Q B

⇓ vB

let x = D in B ⇓

D ⇓ vD
∣ (R let )
B [x ↦ v D ] ⇓ v B

⇓ vB

let rec x = D in B ⇓

D ⇓ vD
∣
B [x ↦ v D [x ↦ let rec x = v D in x ]] ⇓ v B

⇓ vB
(R letrec )
14
Efficiency, complexity, and recurrences

We say that some agent is efficient if it makes the best use of a scarce
resource to generate a desired output. Furnaces turn the scarce re-
source of fuel into heating, so an efficient furnace is one that generates
the most heat using the least fuel. Similarly, an efficient shooter in
basketball generates the most points using the fewest field goal at-
tempts. Standard measurements of efficiency reflect these notions.
Furnaces are rated for Annual Fuel Utilization Efficiency, NBA players
for Effective Field Goal Percentage.
Computer programs use scarce resources to generate desired out-
puts as well. Most prominently, the resources expended are time and
“space” (the amount of memory required during the computation),
though power is increasingly becoming a resource of interest.
Up to this point, we haven’t worried about the efficiency of the pro-
grams we’ve written. And for good reason. Donald Knuth, Professor
Emeritus of the Art of Computer Programming at Stanford Univer-
sity and Turing-Award–winning algorithmist, warns of P R E M AT U R E
O P T I M I Z AT I O N :

Programmers waste enormous amounts of time thinking about, or

worrying about, the speed of noncritical parts of their programs, and
these attempts at efficiency actually have a strong negative impact when
debugging and maintenance are considered. We should forget about
small efficiencies, say about 97% of the time: premature optimization is
the root of all evil. (Knuth, 1974)

Knuth’s point is that programmers’ time is a scarce resource too, and

often the most important one.
Nonetheless, sometimes issues of code efficiency become impor-
Figure 14.1: Donald Knuth (1938–
tant – Knuth’s 3% – and in any case the special ways of thinking and
), Professor Emeritus of the Art of
tools for reasoning about efficiency of computation are important Computer Programming at Stanford
aspects of computational literacy, most centrally ideas of University. In this photo, he holds a
volume of his seminal work The Art of
Computer Programming.
• Complexity as the scaling of resource usage,
242 PROGRAMMING WELL

• Comparison of asymptotic scaling,

• Big-O notation for specifying asymptotic scaling, and

• Recurrences as the means to capture and solve for resource usage.

In this chapter, we describe these important computational notions

in the context of an extended example, the comparison of two sorting
functions.

14.1 The need for an abstract notion of efficiency

With furnaces and basketball players, we can express a notion of ef-

ficiency as a single number – Annual Fuel Utilization Efficiency or
Effective Field Goal Percentage. With computer programs, things are
not so simple. Consider, for example, one of the most fundamental of
all computations, S O RT I N G – ordering the elements of a list according
to a comparison function. Given a particular function to sort lists, we
can’t characterize its efficiency – how long it takes to sort lists – as a
single number. What number would we use? That is, how long does it
take to sort a list of integers using the function? The answer, of course,
is “it depends”; in particular, it depends on

• Which input? How many elements are in the list? What order are
they in? Are there a lot of duplicate items, or very few?

• How computed? Which computer are you using, and which soft-
ware environment? How long does it take to execute the primitive
computations out of which the function is built?

All of these issues affect the running time of a particular sorting func-
tion. To make any progress on comparing the efficiency of functions in
the face of such intricacy, it is clear that we will need to come up with a
more abstract way of characterizing the efficiency of computations.
We address these two issues separately. To handle the question of
“which input”, we might characterize the efficiency of the sorting pro-
gram not as a number (a particular running time), but as a function
from inputs to numbers. However, this doesn’t seem an appealing
option; we want to be able to draw some general conclusions for com-
paring sorting programs, not have to reassess for each possible input.
Nonetheless, the idea of characterizing efficiency in terms of some
function is a useful one. Broadly speaking, algorithms take longer on
bigger problems, so we might use a function that provides the time re-
quired as a function of the size of the input. In the case of sorting lists,
we might take the size of the input to be the number of elements in the
list to be sorted. Unfortunately, for any given input size, the program
 E F F I C I E N C Y, C O M P L E X I T Y, A N D R E C U R R E N C E S 243

might require quite different amounts of time. What should we take

to be the time required for problems of a given size. There are several
options: We might consider the time required on average for instance.
But we will use the time required in the worst case. When comparing
algorithms, we might well want to plan for the worst case behavior of
a program, just to play it safe. We will refer to the function from input
sizes to worst-case time needed as the W O R S T- C A S E C O M P L E X I T Y of
the algorithm.
We’ve made some progress. Rather than thinking of resource usage
as a single number (too coarse) or a function from problem inputs
to numbers (too fine), we use the programs worst-case complexity, a
function from sizes of inputs to worst-case resource usage on inputs
with those sizes. But even this is not really well defined, because of the
“How computed?” question. One and the same program, running on
different computers, say, may have wildly different running times.
To make further progress, let’s take a concrete example. We’ll exam-
ine two particular sorting algorithms.

14.2 Two sorting functions

A module signature for sorting can be given by

# module type SORT =
# sig
# (* sort lt xs -- Returns the list xs sorted in increasing
# order by the "less than" function lt. *)
# val sort : ('a -> 'a -> bool) -> 'a list -> 'a list
# end ;;
module type SORT =
sig val sort : ('a -> 'a -> bool) -> 'a list -> 'a list end

The sort function takes as its first argument a comparison function,

which specifies when one element should be sorted before another in
the desired ordering.
A simple implementation of the signature is the I N S E RT I O N S O RT
algorithm, which operates by inserting the elements of the unsorted
list one by one into an empty list, each in its appropriate place.1
# module InsertSort : SORT =
# struct
Figure 14.2: An example of the recursive
# let rec insert lt xs x = insertion sort algorithm, sorting the list
# match xs with [1, 3, 5, 7, 8, 6, 4, 2]. Each
# | [] -> [x] recursive call is marked with a rounded
# | hd :: tl -> if lt x hd then x :: xs box, in which the tail is sorted, and the
# else hd :: (insert lt tl x) head then inserted.
1
# Insertion sort could have been imple-
mented more elegantly using a single
# let rec sort (lt : 'a -> 'a -> bool)
fold_left, but we make the recursion
# (xs : 'a list)
explicit to facilitate the later complexity
# : 'a list =
analysis.
244 PROGRAMMING WELL

# match xs with
# | [] -> []
# | hd :: tl -> insert lt (sort lt tl) hd
# end ;;
module InsertSort : SORT

We can use insertion sort to sort some integers in increasing order:

# InsertSort.sort (<) [1; 3; 5; 7; 8; 6; 4; 2] ;;
- : int list = [1; 2; 3; 4; 5; 6; 7; 8]

or some floats in decreasing order:

# InsertSort.sort (>) [2.71828; 1.41421; 3.14159; 1.61803] ;;
- : float list = [3.14159; 2.71828; 1.61803; 1.41421]

An especially elegant implementation of sorting is the M E R G E S O RT

algorithm, first described by John von Neumann in 1945 (according
to Knuth (1970)). It works by dividing the list to be sorted into two
lists of (roughly) equal size. Each of the halves is then sorted, and the
resulting sorted halves are merged together to form the sorted full
list. This recursive process of dividing the list in half can’t continue
indefinitely; at some point the recursion must “bottom out”, or the
process will never terminate. In the implementation below, we bottom
out when the list to be sorted contains at most a single element. The
sort function can be defined then as
let rec sort lt xs =
match xs with
| []
| [_] -> xs
| _ -> let first, second = split xs in
merge lt (sort lt first) (sort lt second) ;;

The mergesort definition above makes use of functions

split : ’a list -> ’a list * ’a list

and Figure 14.3: An example of the recursive

mergesort algorithm, sorting the list
merge : (’a -> ’a -> bool) -> ’a list -> ’a list -> ’a
[1, 3, 5, 7, 8, 6, 4, 2]. Each
list . recursive call is marked with a rounded
box, in which the list is split, sorted, and
A call to split lst returns a pair of lists, each containing half of merged.
the elements of lst. (In case, lst has an odd number of elements,
the extra element can go in either list in the returned pair.) A call to
merge lt xs ys returns a list containing all of the elements of xs and
ys sorted according to lt; it assumes that xs and ys are themselves
already sorted.

Exercise 157
Provide implementations of the functions split and merge, and package them together
with the sort function just provided in a module MergeSort satisfying the SORT module
type. You should then have a module that allows for the following interactions:
 E F F I C I E N C Y, C O M P L E X I T Y, A N D R E C U R R E N C E S 245

# MergeSort.sort (<) [1; 3; 5; 7; 8; 6; 4; 2] ;;

- : int list = [1; 2; 3; 4; 5; 6; 7; 8]
# MergeSort.sort (>) [2.7183; 1.4142; 3.1416; 1.6180] ;;
- : float list = [3.1416; 2.7183; 1.618; 1.4142]

(Another elegant recursive sorting algorithm, quicksort, is explored

further in Section 16.4.)
2
We’re taking advantage of sev-
eral useful functions here. The
14.3 Empirical efficiency map function from the List library
module is familiar from Chapter 8.
The Absbook module, available at
How efficient are these algorithms? The time usage of the algorithms http://url.cs51.io/absbookml provides
can be compared by timing each of them on the same input. Here, some useful functions that we’ll use
throughout the book, for example,
we make use of a simple timing function call_timed : (’a -> ’b)
the range function and several timing
-> ’a -> (’b * float). Calling call_timed f x evaluates the functions.
application of the function f to x, returning the result paired with the
number of milliseconds required to perform the computation.
Now we can sort a list using the two sorting algorithms, reporting
the timings as well.2

# (* Generate lists of random integers *)

# let shortlst = List.map (fun _ -> Random.int 1000)
# (Absbook.range 1 10) ;;
val shortlst : int list = [344; 685; 182; 641; 439; 500; 104; 20;
921; 370]
# let longlst = List.map (fun _ -> Random.int 1000)
# (Absbook.range 1 1000) ;;
val longlst : int list =
[217; 885; 949; 678; 615; 412; 401; 606; 428; 869; ...]

# (* Sort the lists two ways *)

# Absbook.call_reporting_time ~count:100
# (InsertSort.sort (<)) shortlst ;;
time (msecs): 0.174999
- : int list = [20; 104; 182; 344; 370; 439; 500; 641; 685; 921]
# Absbook.call_reporting_time ~count:100
# (MergeSort.sort (<)) shortlst ;;
time (msecs): 0.233889
- : int list = [20; 104; 182; 344; 370; 439; 500; 641; 685; 921]
# Absbook.call_reporting_time ~count:100
# (InsertSort.sort (<)) longlst ;;
time (msecs): 1017.933130
- : int list = [0; 0; 0; 2; 3; 4; 4; 7; 11; 12; ...]
# Absbook.call_reporting_time ~count:100
# (MergeSort.sort (<)) longlst ;;
time (msecs): 85.466146
- : int list = [0; 0; 0; 2; 3; 4; 4; 7; 11; 12; ...]

Not surprisingly, it appears that sometimes the insertion sort algo-

rithm is faster (as on shortlst) and sometimes mergesort is faster (as
on longlst). It doesn’t seem possible to give a definitive answer as to
which is faster in general.
246 PROGRAMMING WELL

Figure 14.4: Run time in seconds for

sorting random lists of lengths varying
from 1,000 to 10,000 elements, gener-
ated by averaging run time over 100
trials. The two lines show performance
for insertion sort and merge sort, with
insertion sort times using the right scale
to allow for comparison.

If we examine the performance of the algorithm for a broader range

of cases, however, a pattern emerges. For short lists, insertion sort is
somewhat faster, but as the lists grow in length, the time needed to
sort them grows faster for insertion sort than for mergesort, so that
eventually mergesort shows a consistent performance advantage.
The pattern is quite clear from the graph in Figure 14.4. The key to
comparing the algorithms, then, is not their comparative efficiency on
any particular list, but rather the character of their efficiency as their
inputs grow in size. As we argued in Section 14.1, thinking about the
time required as a function of the size of the inputs looks like a good
idea.
However, as also noted above, a problem with analyzing algorithms,
as we have just done, by running them with particular implementa-
tions on particular computers on particular lists, is that the results may
apply only for those particulars. Instead, we’d like a way of character-
izing the algorithms’ relative performance whatever the particulars.
Measuring running times empirically is subject to idiosyncrasies of the
measurement exercise: the relative time required for different prim-
itive operations on the particular computer being used and with the
particular software tools, what other operations were happening on
the computer at the same time, imprecision in the computer’s clock,
whether the operating system is slowing down or speeding up the CPU
for energy-saving purposes, and on and on. The particularities also
may not be predictive of the future as computers change over time,
 E F F I C I E N C Y, C O M P L E X I T Y, A N D R E C U R R E N C E S 247

with processing and memory retrieval and disk accesses becoming

faster – and faster at varying rates. The empirical approach doesn’t get
at the intrinsic properties of the algorithms.
The approach we will take, then, is to analyze the algorithms in
terms of the intrinsic growth rate of their performance as the size of
their inputs grow, their worst-case complexity. Detailed measure-
ment and analysis can be saved for later, once the more fundamental
complexity issues are considered. We thus take an abstract view of
performance, rather than a concrete one. This emphasis on abstrac-
tion, as usual, comes from thinking like a computer scientist, and not a
computer programmer.
The time complexity of the two sorting algorithms can be thought
of as functions (!) from the size of the input to the amount of time
needed to sort inputs of that size. As it turns out – and as we will show
in Sections 14.5.5 and 14.5.9 – for insertion sort on a list of size n, the
time required to sort the list grows as the function

T i s (n ) = a ⋅ n 2 + b

whereas for mergesort, the time required to sort the list grows as the
function

Tms (n ) = c ⋅ n log n + d

where a, b, c, and d are some constants. For a given n, which is larger?

That depends on these constants of course. But regardless of the con-
stants, as n increases Ti s grows “faster” than Tms in a way that we will
make precise shortly.
In order to make good on this idea of comparing algorithms by
comparing their growth functions, then, we must pay on two promis-
sory notes:

1. How to figure out the growth function for a given algorithm, and

2. How to determine which growth functions are growing faster.

In the remainder of this chapter, we will address the first of these with
a technique of recurrence equations, and the second with the idea of
asymptotic complexity and “big-O” notation.

14.4 Big-O notation

Which is better, an algorithm (like insertion sort) with a complexity

that grows as a ⋅ n 2 + b or an algorithm (like mergesort) with a complex-
ity that grows as c ⋅ n log n + d ? The answer “it depends on the values
of the constants” seems unsatisfactory, since intuitively, a function
248 PROGRAMMING WELL

that grows quadratically (as the square of the size) like the former will
eventually outstrip a function that grows like the latter. Figure 14.5
shows this graphically. The gray lines all grow as c ⋅ n log n for increas-
ing values of c. But regardless of c, the red line, displaying quadratic
growth, eventually outpaces all of the gray lines. In a sense, then, we’d
eventually like to use the n log n algorithm regardless of the constants.
It is this A S Y M P T OT I C (that is, long term or eventual) sense that we’d
like to be able to characterize.
To address the question of how fast a function grows asymptotically,
independent of the annoying constants, we introduce a generic way of
expressing the growth rate of a function – B I G -O N OTAT I O N .
We’ll assume that problem sizes are non-negative integers and that
times are non-negative as well. Given a function f from non-negative
integers to non-negative numbers, O ( f ) is the set of functions that
grow no faster than f , in the following precise sense:3 We define O ( f )
to be the set of all functions g such that for all “large enough” n (that is, Figure 14.5: A graph of functions with
different growth rates. The highlighted
n larger than some value n 0 ), g (n ) ≤ c ⋅ f (n ). line grows as n 2 . The three gray lines
The roles of the two constants n 0 and c are exactly to move beyond grow as c ⋅ n log n, where c is, from
bottom to top, 1, 2, and 4.
the details of constants like the a, b, c, and d in the sorting algorithm 3
Since it takes a function as its argu-
growth functions. In deciding whether a function grows no faster than ment and returns sets of functions as
f , we don’t want to be misled by a few input values here and there its output, O is itself a higher-order
function!
where g (n ) may happen to be larger than f (n ), so we allow exempting
values smaller than some fixed value n 0 . The point is that as the inputs
grow in size, eventually we’ll get past the few input sizes n where g (n )
is larger than f (n ). Similarly, if the value of g (n ) is always, say, twice
the value of f (n ), the two aren’t growing at qualitatively different rates.
Perhaps that factor of 2 is based on just the kinds of idiosyncrasies that
can change as computers change. We want to ignore such constant
multiplicative factors. For that reason, we don’t require that g (n ) be
less than f (n ); instead we require that g (n ) be less than some constant
multiple c of f (n ).
As an example of big-O notation, consider two simple polynomial
functions. It will be convenient to use Church’s elegant lambda nota-
tion (see Section A.1.4) to specify these functions directly: λn.10n 2 + 3
and λn.n 2 .
Is the function λn.10n 2 + 3 an element of the set O (λn.n 2 )? To
demonstrate that it is, we need to find constants c and n 0 such that
for all n > n 0 , 10n 2 + 3 ≤ c ⋅ n 2 . It turns out that the values n 0 = 0 and
c = 13 do the trick, that is, for all n > 0, 10n 2 + 3 ≤ 13n 2 . We can prove
this as follows: Since n ≥ 1, it follows that n 2 ≥ 1 and thus 3 ≤ 3n 2 . Thus
10n 2 + 3 ≤ 10n 2 + 3n 2 = 13n 2 . We conclude, then, that

λn.10n 2 + 3 ∈ O (λn.n 2 ) .
 E F F I C I E N C Y, C O M P L E X I T Y, A N D R E C U R R E N C E S 249

Of course, the converse is also true:

λn.n 2 ∈ O (λn.10n 2 + 3) .

We can just take n 0 again to be 0 and c to be 1, since n 2 < 10n 2 + 3 for all
n.

14.4.1 Informal function notation

It is conventional, when using big-O notation, to stealthily move be-

tween talk of functions (like λn.n 2 ) to the corresponding body ex-
pression (like n 2 ), leaving silent the particular variable (in this case
n) that represents the input of the function. Typically, the variable is
clear from context (and indeed is frequently the variable n itself ). For
instance, we might say

10n 2 + 3 ∈ O (n 2 ) ,

rather than the more complex formulation above.

Continuing this abuse of notation, we sometimes write variables
for functions, like f or g , not only to stand for a function, but also the
corresponding body expression. This allows us to write things like k ⋅ f
(where k is a constant) to mean the function whose body expression
is the product of k and the body expression of f . (This turns out to be
equivalent to the rather more cumbersome λn.k ⋅ f (n ).) Suppose f is
the function λn.n 2 . Then we write k ⋅ f to mean, not k ⋅ λn.n 2 (which is
an incoherent formula), but rather k ⋅ n 2 , which, again by convention,
glosses λn.k ⋅ n 2 .
This convention of sliding between functions and their body expres-
sions may seem complicated, but it soon becomes quite natural. And
it allows us to formulate important properties of big-O notation very
simply, as we do in the next section.
Of course, there are problems with playing fast and loose with nota-
tion in this way. First, writing O (n 2 ) makes it look like O is a function
that takes integers as input, since n 2 looks like it specifies an integer.
But O is a function from functions, not from integers. Second, what are
we to make of something like O (m ⋅ n 2 )? Does this specify the set of
functions that grow no faster than the function from m to m ⋅ n 2 , that
is, O (λm.m ⋅ n 2 )? Or does it specify the set of functions that grow no
faster than the function from n to m ⋅ n 2 , that is, O (λn.m ⋅ n 2 )? The
notation doesn’t make clear which variable is the one representing the
input to the function – which variable the growth is relative to. In cases
such as this, computer scientists rely on context to make clear what the
notation is supposed to mean.
We’ll stick to this informal notation since it is universally used.
250 PROGRAMMING WELL

But you’ll want to always remember that O maps functions to sets of

functions.

14.4.2 Useful properties of O

In general, it’s tedious to prove particular cases of big-O membership

like the example in Section 14.4. Instead, you’ll want to acquire a
general understanding of these big-O sets of functions, and reason
on the basis of that understanding.
The big-O notation brings together whole classes of functions
whose growth rates are similar. These classes of functions have cer-
tain properties that make them especially useful.4 First of all, every 4
The mathematically inclined might
want to take a stab at proving these
function grows no faster than itself:
properties of big-O.

f ∈ O( f )

Adding a constant to a function doesn’t change its big-O classification:

If g ∈ O ( f ), then5 5
Here’s our first instance of the informal
function notation in the wild.
g + k ∈ O( f ) .

We can reason immediately, then, that 2n 2 + 3 ∈ O (2n 2 ) (or, more

pedantically, λn.2n 2 + 3 ∈ O (λn.2n 2 )), without going through a specific
proof.
Similarly, multiplying by a constant (k) doesn’t affect the class ei-
ther. If g ∈ O ( f ), then

k ⋅ g ∈ O( f ) .

Thus, 2n 2 ∈ O (n 2 ). Together with the results above, we can conclude

that 2n 2 + 3 ∈ O (n 2 ).
In fact, adding in any lower degree terms doesn’t matter. If f ∈
O (n k ) and g ∈ O (n c ), where k > c:

f + g ∈ O (n k )

The upshot of all this is that in determining the big-O growth rate
of a polynomial function, we can always just drop lower degree terms
and multiplicative constants. In thinking about the growth rate of a
complicated function like 4n 3 + 142n + 3, we can simply ignore all but
the largest degree term (4n 3 ) and even the multiplicative constant 4,
and conclude that

4n 3 + 142n + 3 ∈ O (n 3 )

Exercise 158
Which of these claims about the growth rates of various functions hold?
 E F F I C I E N C Y, C O M P L E X I T Y, A N D R E C U R R E N C E S 251

1. 3n + 5 ∈ O (n )
2. n ∈ O (3n + 5)
3. n + n 2 ∈ O (n )
4. n 3 + n 2 ∈ O (n 3 + 2n )
5. n 2 ∈ O (n 3 )
6. n 3 ∈ O (n 2 )
7. 32n 3 ∈ O (n 2 + n + k )

Finally, the sum or product of functions grows no faster than the

sum or product, respectively, of their respective growth rates. If f ′ ∈
O ( f ) and g ′ ∈ O (g ), then

f ′ + g ′ ∈ O( f + g )
f ′ ⋅ g ′ ∈ O( f ⋅ g )

We can thus conclude that

(5n 3 + n 2 ) ⋅ (3 log n + 7) ∈ O (n 3 ⋅ log n )

14.4.3 Big-O as the metric of relative growth

We are interested in the big-O classification of functions in particular

because we can use it to compare functions as to which asymptotically
grows faster. In particular, if f ∈ O (g ) but g ∈/ O ( f ), then g grows faster
than f , which we notate g ≫ f .
For example,

n 2 ∈ O (n 3 ) ,

but the converse doesn’t hold:

n 3 ∈/ O (n 2 ) .

We can conclude, then that

n3 ≫ n2 ,

that is, n 3 grows faster than n 2 .

More generally,

• Functions with bigger exponents grow faster:

nk ≫ nc when k > c

• Linear functions grow faster than logarithmic functions:

n ≫ log n
252 PROGRAMMING WELL

• Exponentials grow faster than polynomials:

2n ≫ n k

• The exponential base matters; exponentials with larger base grow

faster:

3 n ≫ 2n

We can think of big-O as defining classes of functions that grow

at similar rates, up to multiplicative constants. Thus, O (n ) is the set
of functions whose growth rate is (at most) linear, O (n 2 ) the set of
functions whose growth rate is (at most) quadratic, O (n 3 ) the set of
cubic functions, O (2n ) the set of base-two exponential functions. We
can then place these classes in an ordering (≫) as to which classes
grow faster inherently (and not just because of the values of some
contingent constants).
From the properties above, we can conclude that n 2 ≫ n log n, and
therefore, since Ti s ∈ O (n 2 ) and Tms ∈ O (n log n ), that Ti s ≫ Tms .6 6
Strictly speaking, we’d have to further
show that Ti s ∈/ O (n log n ), but we’ll
Mergesort has lower complexity – is asymptotically more efficient –
ignore this nicety in general here and in
than insertion sort. This conclusion is independent of which comput- the following discussion.
ers we time the algorithms on, or other particularities that affect the
constants.
Of course, this reasoning relies on knowing the functions for how
each algorithm’s performance scales. (In the discussion above, we
merely asserted the growth rate functions for the two sorting algo-
rithms.) Only then can we use big-O to determine which algorithm
scales better, which is more efficient in a deeper sense than just test-
ing a particular instance or two. We still need a way to determine for a
particular algorithm the particular resource-usage function. This is the
second promissory note, and the one that we now address.
Problem 159
Two friends who work at EuclidCo tell you that they’re looking for a fast algorithm
to solve a problem they’re working on. So far, they’ve each developed an algorithm:
algorithm A has time complexity O (n 3 ) and algorithm B is O (2n ). They prefer algorithm
A, and use three different arguments to convince you of their preference. For each
argument, evaluate the truth of the bolded statement, and justify your answer.
1. “We’re all about speed at EuclidCo, and A will always be faster than B.”
2. “In a high stakes industry like ours, we can’t afford to have more than a finite number
of inputs that run slower than polynomial time, and we can avoid this if we go with
A.”
3. “We work with big data at EuclidCo. For suitably large inputs, A will be faster on
average than B.”

14.5 Recurrence equations

Given an algorithm, how are we to determine how much time it needs

as a function of the size of its input? In this section, we introduce one
 E F F I C I E N C Y, C O M P L E X I T Y, A N D R E C U R R E N C E S 253

method, based on the solving of recurrence equations, to address this

question.
We start with a simple example, the append function to append two
lists, defined as

# let rec append xs ys =

# match xs with
# | [] -> ys
# | hd :: tl -> hd :: (append tl ys) ;;
val append : 'a list -> 'a list -> 'a list = <fun>

An appropriate measure for the size of the input to the function is the
sizes of the two lists it is to append. Let’s use Tappend (n, m ) for the time
required to run the append function on lists with n and m elements
respectively. What do we know about this Tappend ?
When the first argument, xs, is the empty list (so n = 0), the function
performs just a few simple actions, pattern-matching the input against
the empty list pattern, and then returning ys. If we say that the time for
the pattern match is some constant c match and the time for the return
is some constant c returnys , then we have that

Tappend (0, m ) = c match + c returnys .

Since the sum of the two constants is itself a constant, we can simplify
by treating the whole as a new constant c:

Tappend (0, m ) = c

When the first argument is nonempty, the computation performed

again has a few parts: the match against the first pattern (which fails),
the match against the second pattern (which succeeds), the recursive
call to append, the cons of h and the result of the recursive call. Each
of these (except for the recursive call) takes some constant time, so we
can characterize the amount of time as

Tappend (n + 1, m ) = c matchcons + Tappend (n, m ) .

Here, we use n + 1 as the length of the first list, as we know it is at least

one element long. The recursive call is appending the tail of xs, a list of
length n, to ys, a list of length m, and thus (by hypothesis) takes time
Tappend (n, m ).
Merging and renaming constants, we thus have the following two
R E C U R R E N C E E QUAT I O N S that characterize the running time of the
append function in terms of the size of its arguments:

Tappend (0, m ) = c
Tappend (n + 1, m ) = k + Tappend (n, m )
254 PROGRAMMING WELL

14.5.1 Solving recurrences by unfolding

Because the recurrence equations defining Tappend use Tappend itself,

recursively, in the definition (hence the term “recurrence”), they don’t
provide a C L O S E D - F O R M (nonrecursive) solution to the question of
characterizing the running time of the function. To get a solution in
closed form, we will use a method called U N F O L D I N G to solve the
recurrence equations.
Consider the general case of Tappend (n, m ) and assume that n > 0.
By the second recurrence equation,

Tappend (n, m ) = k + Tappend (n − 1, m ) .

Now Tappend (n − 1, m ) itself can be unfolded as per the second recur-

rence equation, so

Tappend (n, m ) = k + k + Tappend (n − 2, m ) .

Continuing in this vein, we can continue to unfold until the first argu-
ment to Tappend becomes 0:

Tappend (n, m ) = k + Tappend (n − 1, m )

= k + k + Tappend (n − 2, m )
= k + k + k + Tappend (n − 3, m )
=⋯
= k + k + k + ⋯ + k + Tappend (0, m )

How many unfoldings are required until the first argument reaches 0?
We’ll have had to unfold n times. There will therefore be n instances of
k being summed in the unfolded equation. Completing the derivation,
then, using the first recurrence equation,

Tappend (n, m ) = k ⋅ n + Tappend (0, m )

= k ⋅n +c

We now have the closed-form solution

Tappend (n, m ) = k ⋅ n + c

Notice that the time required is independent of m, the size of the

second argument. That makes sense because the code for append
never looks inside the structure of the second argument; the computa-
tion therefore doesn’t depend on its size.
Now, the function k ⋅ n + c ∈ O (n ). Thus the time complexity of
append is O (n ) or linear in the length of its first argument. This is
typical of algorithms that operate by recursively marching down a list
one element at a time.
 E F F I C I E N C Y, C O M P L E X I T Y, A N D R E C U R R E N C E S 255

In order to apply the same kinds of techniques to determine the

time complexity of the two sorting algorithms, we’ll work through a
series of examples.

14.5.2 Complexity of reversing a list

There are multiple ways of implementing list reversal. We show that

they can have quite different time complexities. We start with a naive
implementation, which works by reversing the tail of the list and ap-
pending the head on the end:

# let rec rev xs =

# match xs with
# | [] -> []
# | hd :: tl -> append (rev tl) [hd] ;;
val rev : 'a list -> 'a list = <fun>

We define recurrence equations for the time Tr ev (n ) to reverse a

list of length n using this implementation. If the list is empty, we have
(similarly to the case of append, and introducing constants as needed):

Trev (0) = c match + c return = q

For nonempty lists, we must perform the appropriate match, reverse

the tail, cons the head onto the empty list, and perform the append:

Trev (n + 1) = c match + c cons + Trev (n ) + Tappend (n, 1)

= r + Trev (n ) + Tappend (n, 1)
= r + Trev (n ) + k ⋅ n + c
= k ⋅ n + s + Trev (n )

The closed form solution for append from the previous section be-
comes useful here. And again, notice our free introduction of new
constants to simplify things. We take the sum of c match and c cons to be
r , then for r + c we introduce s. Summarizing, the reverse implemen-
tation above yields the recurrence equations

Trev (0) = q
Trev (n + 1) = k ⋅ n + s + Trev (n )

which we must now solve to find a closed form.

256 PROGRAMMING WELL

We again unfold Trev (n ):

7
We digress to present the proof of
Trev (n ) = k ⋅ (n − 1) + s + Trev (n − 1) this formula for ∑n i , the sum of
i =1
= k ⋅ (n − 1) + s + k ⋅ (n − 2) + s + Trev (n − 2) all the integers from 1 to n. Famously
(if apocryphally), the seven-year-old
= k ⋅ (n − 1 ) + s + k ⋅ (n − 2 ) + s mathematical prodigy Carl Friedrich
Gauss (1777–1855) solved this problem
+ k ⋅ (n − 3) + s + Trev (n − 3) in his head. Gauss was asked by his
teacher to sum all of the integers from
=⋯ 1 to 100. That’ll keep him quiet for a
n −1 bit, the teacher presumably thought.
= k ⋅ ∑ (n − i ) + s ⋅ n + Tr ev (0) But Gauss came up with the answer
i =1 immediately: 5050. How?
n −1 Define the sum in question to be S:
= k ⋅ ∑ (n − i ) + s ⋅ n + q n
i =1 S= ∑i
n −1 i =1

= k ⋅ ∑ i +s ⋅n +q We can think of adding all the values

i =1 from 1 to n, or conversely, all the
n numbers from n to 1, that is all the
= k ⋅∑i −k ⋅n +s ⋅n +q values of (n − i + 1):
i =1
n
n S = ∑ (n − i + 1 )
= k ⋅ ∑ i + (s − k ) ⋅ n + q i =1
i =1
Adding these two together,
7 n n
We use a simple identity
2S = ∑ i + ∑ (n − i + 1)
i =1 i =1
n
n ⋅ (n + 1)
∑i = but the two sums can be brought
i =1 2 together as a single sum and simplified:
n
to simplify the equation for Tr ev : 2S = ∑ (i + (n − i + 1))
i =1
n n
Tr ev (n ) = k ⋅ ∑ i + (s − k ) ⋅ n + q = ∑ (n + 1 )
i =1 i =1

n ⋅ (n + 1 ) Now we’re just summing up n instances

=k⋅ + (s − k ) ⋅ n + q of n + 1, that is, multiplying n and n + 1:
2
k k 2S = n ⋅ (n + 1)
= n 2 + n + (s − k ) ⋅ n + q
2 2 so that
k k
= n + (s − ) ⋅ n + q
2
S=
n ⋅ (n + 1 )
2 2 2
∈ O (n 2 ) For Gauss’s problem, where n is 100, he
presumably calculated 1002⋅101 = 5050.
concluding that the function has quadratic (O (n 2 )) complexity. The A graphical version of this proof is
shown in Figure 14.6.
last step really shows the power of big-O notation, allowing to strip
away all of the constants and lower order terms to get at the essence of
the growth rate.
Problem 160
Recall that the Stdlib.compare function compares two values, returning an int based
on their relative magnitude: compare x y returns 0 if x is equal to y, -1 if x is less than y, 8
For reference, this built-in length
and +1 if x is greater than y.
function is, unsurprisingly, linear in the
A function compare_lengths : ’a list -> ’b list -> int that compares
length of its argument.
the lengths of two lists can be implemented using compare by taking advantage of the
length function8 from the List module:
 E F F I C I E N C Y, C O M P L E X I T Y, A N D R E C U R R E N C E S 257

let compare_lengths xs ys =
compare (List.length xs) (List.length ys) ;;

For instance,
# compare_lengths [1] [2; 3; 4] ;;
- : int = -1
# compare_lengths [1; 2; 3] [4] ;;
- : int = 1
# compare_lengths [1; 2] [3; 4] ;;
- : int = 0

However, this implementation of compare_lengths does a little extra work than it needs
to. Its complexity is O (n ) where n is the length of the longer of the two lists.
Why does compare_lengths have this big-O complexity? In particular, why does
the length of the shorter list not play a part in the complexity? We’re looking for a brief
informal argument here, not a full derivation of its complexity.
Provide an alternative implementation of compare_lengths whose complexity is
O (n ) where n is the length of the shorter of the two lists, not the longer. Figure 14.6: A graphical proof that
n n ⋅ (n + 1 )
∑i = .
14.5.3 Complexity of reversing a list with accumulator i =1 2

Two triangles, each formed by piling up

An alternative method of reversing a list uses an accumulator. As squares with rows from 1 to n can be
each element in the list is processed, it is consed on the front of the combined to form a rectangle of area
n ⋅ (n + 1). Each triangle is half that area,
accumulating list. The process begins with the empty accumulator. n ⋅(n +1)
that is, 2
. A more algebraic proof
# let rec revappend xs accum = is given in footnote 7.
# match xs with
# | [] -> accum
# | hd :: tl -> revappend tl (hd :: accum) ;;
val revappend : 'a list -> 'a list -> 'a list = <fun>

# let rev xs = revappend xs [] ;;

val rev : 'a list -> 'a list = <fun>

As before, we can set up recurrence equations for this version of rev

and its auxiliary function revappend.

Trev (n ) = q + Trevapp (n, 0)

Trevapp (0, m ) = c
Trevapp (n + 1, m ) = k + Trevapp (n, m + 1)

By an unfolding argument similar to that for append, we can solve

these recurrence equations to closed form:

Trevapp (n, m ) = k ⋅ n + c
∈ O (n )

so that

Trev (n ) = q + Trevapp (n, 0)

= q +k ⋅n +c
∈ O (n )
258 PROGRAMMING WELL

Unlike the quadratic simple reverse, the revappend approach

is linear. The difference is born out empirically as well, as shown in
Figure 14.7.

Figure 14.7: Time in microseconds

to reverse lists of lengths 100 to 1000
using the naive (square) and revappend
(circle, highlighted) implementations.
The left graph places both lines on
the same (left) vertical scale. The right
graph places the revappend line on
the right vertical scale (equivalent to
multiplying all of the revappend times
by 50) to emphasize the difference in
growth rate of the functions. Despite the
change in constants, the naive reverse
still eventually overtakes the revappend.

14.5.4 Complexity of inserting in a sorted list

The insertion sort algorithm uses a function insert to insert an ele-

ment in its place in a sorted list:

# let rec insert xs x =

# match xs with
# | [] -> [x]
# | hd :: tl -> if x > hd then hd :: (insert tl x)
# else x :: xs ;;
val insert : 'a list -> 'a -> 'a list = <fun>

As usual, we construct appropriate recurrence equations for Tinsert (n )

where n is the length of the list being inserted into. (We ignore the ele-
ment argument, as its size is irrelevant to the time required.) Inserting
into the empty list takes constant time.

Tinsert (0) = c

Inserting into a nonempty list (of size n + 1) is more subtle. The time
required depends on whether the element should come at the start
of the list (the else clause of the conditional) or not (the then clause).
In the former case, the cons operation takes constant time, say k 2 ; in
the latter case, it involves a recursive call to insert (Tinsert (n )) plus
 E F F I C I E N C Y, C O M P L E X I T Y, A N D R E C U R R E N C E S 259

some further constant overhead (k 1 ). Since we don’t know which way

the computation will branch, we have to make the worst-case assump-
tion: whichever is bigger. Which of the two is bigger depends on the
constants, but we can be sure, in any case, that the time required is
certainly less than the sum of the two.

Tinsert (n + 1) = max(k 1 + Tinsert (n ), k 2 )

≤ k1 + Tinsert (n ) + k2
= k + Tinsert (n )

Unfolding these proceeds as usual:

Tinsert (n ) = k + Tinsert (n − 1)
= k + k + Tinsert (n − 2)
=⋯
= k ⋅ n + Tinsert (0)
= k ⋅n +c
∈ O (n )

Insertion is thus linear in the size of the list to be inserted into.

14.5.5 Complexity of insertion sort

Recall the implementation of insertion sort:

let rec sort (lt : 'a -> 'a -> bool)
(xs : 'a list)
: 'a list =
match xs with
| [] -> []
| hd :: tl -> insert lt (sort lt tl) hd ;;

Using similar arguments as above, the recurrence equations can be

determined to be

Tisort (0) = c
Tisort (n + 1) = k + Tisort (n ) + Tinsert (n )

Solving the recurrence equations:

Tisort (n ) = k + Tisort (n − 1) + O (n − 1)
= k + k + Tisort (n − 2) + O (n − 1) + O (n − 2)
= k ⋅ n + Tisort (0) + O (n − 1) + O (n − 2) + ⋯ + O (0)
n
= k ⋅ n + c + ∑ O (i )
i =1

∈ O (n )
2

We conclude that insertion sort is quadratic in its run time.

260 PROGRAMMING WELL

14.5.6 Complexity of merging lists

Continuing our exploration of the time complexity of sorting algo-

rithms, we turn to the components of mergesort. The merge function,
defined by

let rec merge lt xs ys =

match xs, ys with
| [], _ -> ys
| _, [] -> xs
| x :: xst, y :: yst ->
if lt x y
then x :: (merge lt xst ys)
else y :: (merge lt xs yst) ;;

takes two list arguments; their sizes will be two of the arguments of the
complexity function Tmerge . Each recursive call of merge reduces the
total number of items in the two lists. We will for that reason use the
sum of the sizes of the two lists as the argument to Tmerge .
If the total number of elements in the two lists is 1, then one of the
two lists must be empty, and we have

Tmerge (1) = c

In the worst case, neither element will become empty until the to-
tal number of elements in the lists is 2. Thus, for n ≥ 2, we have the
“normal” case, when the lists are nonempty, which involves (in ad-
dition to some constant overhead) a recursive call to merge with one
fewer element in the lists. In the worst case, both elements will still be
nonempty.

Tmerge (n + 1) = k + Tmerge (n )

Solving these recurrence equations:

Tmerge (n ) = k + Tmerge (n − 1)
= k + k + Tmerge (n − 2)
=⋯
= k ⋅ n + Tmerge (1)
= k ⋅n +c
∈ O (n )

14.5.7 Complexity of splitting lists

We leave as an exercise to show that the split function defined by

let rec split lst =

match lst with
 E F F I C I E N C Y, C O M P L E X I T Y, A N D R E C U R R E N C E S 261

| []
| [_] -> lst, []
| first :: second :: rest ->
let first', second' = split rest in
first :: first', second :: second' ;;

has linear time complexity.

Exercise 161
Show that split has time complexity linear in the size of its first list argument.

14.5.8 Complexity of divide and conquer algorithms

Before continuing to the analysis of mergesort, we look more generally

at algorithms that (like mergesort) attack problems by dividing them
into equal parts, recursively solving them, and putting the subsolutions
back together to solve the full problem – D I V I D E - A N D - C O N QU E R
algorithms.
The recurrences of such algorithms are typically structured with a
base case requiring constant time

T (1 ) = c

and a recursive case that involves two recursive calls on some prob-
lems each of half the size. At first, we’ll assume that the time to break
apart and put together the two parts takes constant time k.

T (n ) = k + 2 ⋅ T (n /2)

For simplicity in solving these recurrence equations, we assume that

n is a power of 2. Then unfolding a few times:

T (n ) = k + 2 ⋅ T (n /2)
= k + k + 4 ⋅ T (n /4 )
= k + k + k + 8 ⋅ T (n /8)
=⋯

How many times can we unfold? The denominator keeps doubling. We

can keep doubling, then, m times until 2m = n, that is, m = log n:
⎫
T (n ) = k + 2 ⋅ T (n /2) ⎪
⎪
⎪
⎪
⎪
⎪
= k + k + 4 ⋅ T (n /4) ⎪
⎪
⎪
⎪
⎪
⎪
= k + k + k + 8 ⋅ T (n /8) ⎬ log n times
⎪
⎪
⎪
=⋯ ⎪
⎪
⎪
⎪
⎪
⎪
⎪
= k ⋅ log n + n ⋅ T (n /n ) ⎪
⎪
⎭
= k ⋅ log n + c ⋅ n
∈ O (n )
262 PROGRAMMING WELL

More realistically, however, the time required to divide the problem

up and to merge the subsolutions together may take time linear in the
size of the problem. In that case, the recurrence would be something
like

T (n ) = k ⋅ n + 2 ⋅ T (n /2)

and the closed form is derived as

⎫
T (n ) = k ⋅ n + 2 ⋅ T (n /2) ⎪
⎪
⎪
⎪
⎪
⎪
= k ⋅ n + k ⋅ n + 4 ⋅ T (n /4) ⎪
⎪
⎪
⎪
⎪
⎪
= k ⋅ n + k ⋅ n + k ⋅ n + 8 ⋅ T (n /8) ⎬ log n times
⎪
⎪
⎪
=⋯ ⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
= k ⋅ n ⋅ log n + n ⋅ T (n /n ) ⎭
= k ⋅ n ⋅ log n + c ⋅ n
∈ O (n log n )

The O (n log n ) complexity is the hallmark of divide-and-conquer

algorithms. Since log n grows extremely slowly, such algorithms are
almost linear in their complexity, thus very efficient.

14.5.9 Complexity of mergesort

Having determined the time complexity for the components of merge-

sort, we put them together to determine the complexity of the merge-
sort function itself:

let rec msort xs =

match xs with
| [] -> xs
| [_] -> xs
| _ -> let fst, snd = split xs in
merge (msort fst) (msort snd) ;;

Tmsort (0) = Tmsort (1) = c 1

Tmsort (n ) = c 2 + Tsplit (n ) + 2 ⋅ Tmsort (n /2) + Tmerge (n )

Since both Tsplit and Tmerge are linear, we can write

Tmsort (n ) = k ⋅ n + c + 2 ⋅ Tmsort (n /2)

These recurrence equations are just of the divide-and-conquer sort, so

we know immediately that the complexity of mergesort is O (n log n ).
And since

n 2 ≫ n log n
 E F F I C I E N C Y, C O M P L E X I T Y, A N D R E C U R R E N C E S 263

mergesort is shown to be asymptotically more efficient than insertion

sort.
Consistent with this analysis of the sorting algorithms is their
empirical performance, as shown in Figure 14.4. The figure depicts
well the almost linear behavior of mergesort and the much steeper
quadratic growth of insertion sort.

14.5.10 Basic Recurrence patterns

Table 14.1 summarizes some of the basic types of recurrence equations

and their closed-form solution in terms of big-O.

Table 14.1: Some common recurrence

patterns and their closed-form solution
in terms of big-O.
T (n ) = c + T (n − 1 ) T (n ) ∈ O (n )
T (n ) = c + k ⋅ n + T (n − 1 ) T (n ) ∈ O (n 2 )
T (n ) = c + k ⋅ n d + T (n − 1 ) T (n ) ∈ O (n d +1 )
T (n ) = c + 2 ⋅ T (n /2) T (n ) ∈ O (n )
T (n ) = c + T (n /2) T (n ) ∈ O (log n )
T (n ) = c + k ⋅ n + 2 ⋅ T (n /2) T (n ) ∈ O (n ⋅ log n )

14.6 Problem section: Complexity of the Luhn check

Recall the Luhn check algorithm from Section 9.6, and its various
component functions: evens, odds, doublemod9, sum.
Problem 162
What is an appropriate recurrence equation for defining the time complexity of the odds
function from Problem 69 in terms of the length of its list argument?
Problem 163
What is the time complexity of the odds function from Problem 69 (in big-O notation)?
Problem 164
If the function f (n ) is the time complexity of odds on a list of n elements, which of the
following is true?
• f ∈ O (1 )
• f ∈ O (log n )
• f ∈ O (log n /c ) for all c > 0
• f ∈ O (c ⋅ log n ) for all c > 0
• f ∈ O (n )
• f ∈ O (n /c ) for all c > 0
• f ∈ O (c ⋅ n ) for all c > 0
• f ∈ O (n 2 )
264 PROGRAMMING WELL

• f ∈ O (n 2 /c ) for all c > 0

• f ∈ O (c ⋅ n 2 ) for all c > 0
• f ∈ O (2 n )
• f ∈ O (2n /c ) for all c > 0
• f ∈ O (c ⋅ 2n ) for all c > 0

Problem 165
What is the time complexity of the luhn function implemented in Problem 73 in terms of
the length n of its list argument? Use big-O notation. Explain why your implementation
has that complexity.

14.7 Problem set 6: The search for intelligent solutions

This section provides substantive background for one of the course prob-
lem sets. For common background on logistics and procedures for prob-
lem sets, including such topics as accessing the problem set code, compil-
ing and testing your code, reflecting on the process, and submitting your
solution, please review the course document “Problem set procedures”.

In this assignment, you will apply your knowledge of OCaml mod-

ules and functors to complete the implementation of a program for
solving search problems, a core problem in the field of artificial intelli-
gence. In the course of working on this assignment, you’ll implement
a more efficient queue module using two stacks; create a higher-order
functor that abstracts away details of search algorithms and puzzle im-
plementations; and compare, visualize, and analyze the performance
of various search algorithms on different puzzles.

14.7.1 Search problems

The field of A RT I F I C I A L I N T E L L I G E N C E pursues the computational

emulation of behaviors that in humans are indicative of intelligence.
A hallmark of intelligent behavior is the ability to figure out how to
achieve some desired goal. Let’s consider an idealized version of
this behavior – puzzle solving. A puzzle can be in any of a variety of
S TAT E S . The puzzle starts in a specially designated I N I T I A L S TAT E , and
we desire to reach a G O A L S TAT E by finding a sequence of M O V E S that,
when executed starting in the initial state, reach the goal state. Fig-
ure 14.8 provides some examples of this sort of puzzle – peg solitaire,
the 8-puzzle, and a maze puzzle.
A good example is the 8 puzzle, depicted in Figure 14.9. (You may
know it better as the 15 puzzle, its larger 4 by 4 version.) A 3 by 3 grid of
numbered tiles, with one tile missing, allows sliding of a tile adjacent to
the empty space. The goal state is to be reached by repeated moves of
this sort. But which moves should you make?
Solving goal-directed problems of this sort requires a S E A R C H
among all the possible move sequences for one that achieves the
 E F F I C I E N C Y, C O M P L E X I T Y, A N D R E C U R R E N C E S 265

Figure 14.8: Some puzzles based on

1 6 4 search for a goal state. (a) the peg
solitaire puzzle; (b) the sliding-tile 8

5 8 puzzle; (c) a maze puzzle.

3 2 7
(a) (b) (c)

1 6 4 1 2 3 Figure 14.9: The 8 puzzle: (a) an initial

state, (b) the goal state, (c-f) the states

5 8 4 5 6 resulting from moving up, down,

left, and right from the initial state,
3 2 7 7 8 respectively.

(a) (b)

1 6 4 1 4 1 6 4 1 6 4
5 2 8 5 6 8 5 8 5 8
3 7 3 2 7 3 2 7 3 2 7
(c) (d) (e) (f)

goal. You can think of this search process as a walk of a S E A R C H T R E E ,

where the nodes in the tree are the possible states of the puzzle and the
directed edges correspond to moves that change the state from one to
another. Figure 14.10 depicts a small piece of the tree corresponding to
the 8 puzzle.

Figure 14.10: A snippet from the search

1 6 4 tree for the 8 puzzle.

5 8
3 2 7
down left
up right

1 4 1 6 4 1 6 4 1 6 4
5 6 8 5 2 8 5 8 5 8
3 2 7 3 7 3 2 7 3 2 7

To solve a puzzle of this sort, you maintain a collection of states to

be searched, which we will call the pending collection. The pending
collection is initialized with just the initial state. You can then take a
266 PROGRAMMING WELL

state from the pending collection and test it to see if it is a goal state. If
so, the puzzle has been solved. But if not, this state’s N E I G H B O R states
– states that are reachable in one move from the current state – are
added to the pending collection (or at least those that have not been
visited before) and the search continues.
To avoid adding states that have already been visited before, you’ll
need to keep track of a set of states that have already been visited,
which we’ll call the visited set, so you don’t revisit one that has already
been visited. For instance, in the 8 puzzle, after a down move, you don’t
want to then perform an up move, which would just take you back to
where you started. (The standard OCaml Set library will be useful here
to keep track of the set of visited states.)
Of course, much of the effectiveness of this process depends on the
order in which states are taken from the collection of pending states as
the search proceeds. If the states taken from the collection are those
most recently added to the collection (last-in, first-out, that is, as a
stack), the tree is being explored in a D E P T H - F I R S T manner. If the
states taken from the collection are those least recently added (first-in,
first-out, as a queue), the exploration is B R E A D T H - F I R S T . Other orders
are possible, for instance, the states might be taken from the collection
in order of how closely they match the goal state (using some metric
of closeness). This regime corresponds to B E S T- F I R S T or G R E E DY
S E A R C H.

14.7.2 Structure of the provided code

We have provided code to start off the process of building a general

puzzle-solving system that allows for different search regimes applied
to different puzzle types. You’ll want to familiarize yourself with the
provided code.
You will work with the following signatures, modules, and functors.
Some of the important components are listed by the file they are found
in. Components that you will write are shown in italics.

• collections.ml

– COLLECTION – A signature for collections of elements allowing

adding and taking elements
– MakeStackList – A functor for generating a stack Collection
implemented using lists
– MakeQueueList – A functor for generating a queue Collection
implemented using lists
– MakeQueueStack – A functor for generating a queue Collection,
where the Queue is implemented using two stacks
 E F F I C I E N C Y, C O M P L E X I T Y, A N D R E C U R R E N C E S 267

• puzzledescription.ml

– PUZZLEDESCRIPTION – A signature for specifications of puzzles,

providing information about the states and moves, initial and
goal states, the neighbors structure of the puzzle, etc.

• mazes.ml – A specification of maze puzzles satisfying

PUZZLEDESCRIPTION

• tiles.ml – A specification of tile puzzles satisfying

PUZZLEDESCRIPTION

• puzzlesolve.ml

– PUZZLESOLVER – A signature for puzzle solvers

– MakePuzzleSolver – A higher-order functor for generating differ-
ent puzzle solvers

Collections The file collections.ml provides a signature for

COLLECTION modules. A COLLECTION module, which is very similar
to the OrderedCollection module in Problem Set 4, is a data struc-
ture with types elt and collection, and functions take and add. The
take function removes an elt from a collection, while add inserts an
elt into a collection. Collections are a generalization over a variety
of data structures including stacks, queues, and priority queues.
Collections will be used to store the states reachable from the initial
state that have yet to be explored, as the puzzle solver searches the
space of states in order to find a goal state. By using different collec-
tions that implement different orderings in how elements are taken
from the collection, different search regimes are manifested and differ-
ent efficiencies of the search process can result.
We have provided two functors for especially simple implementa-
tions of COLLECTION, one that implements a stack represented inter-
nally as a list, and one that implements a queue represented internally
with a list. You will implement a third functor MakeQueueStack, that
implements a queue represented internally with two stacks, similar
to the approach introduced in Section 15.5.2. (However, you should
be able to provide an implementation more elegant and more purely
functional than the examples there. There’s no reason to introduce
imperative programming techniques for MakeQueueStack.)
Problem 166
Implement the MakeQueueStack functor. You’ll want to test it fully as well.

In theory, the implementation in MakeQueueStack should be more

efficient than that of MakeQueueList. In Section 14.7.3, you will exper-
iment with that hypothesis and provide a report on your findings as
part of the problem set.
268 PROGRAMMING WELL

Puzzle description A module signature of a puzzle description, called

PUZZLEDESCRIPTION, specifies an interface that includes data types for
the states and moves; an initial state; a predicate for goal states; and
functions for generating neighbors, for executing move sequences, and
for visualizing the puzzle. You will want to read and understand it to
familiarize yourself with the functionality that a PUZZLEDESCRIPTION
provides.
To understand the notion of moves and states, consider a simple
maze. In a maze, the state would be represented as the current position
in the maze, and the moves would allow you to change your position
by walking up, down, left, or right. The neighbors of a state would be
the positions that you could move to in one step using those moves,
keeping in mind that not all four move types are possible from a given
state. Similarly, in the 8 puzzle, a state is a configuration of the tiles and
a move is the swapping of the empty tile with one of its four adjacent
tiles, the adjacent tile moving up, down, left, or right.
We’ve provided two implementations of the PUZZLE_DESCRIPTION
signature, one for tile puzzles like the 8 puzzle, and one for maze puz-
zles. Examining the files tiles.ml and mazes.ml should provide
further understanding of how these puzzles, and the search problems
they engender, are being represented.

Puzzle solving In file puzzlesolve.ml, we define a PUZZLESOLVER

signature, which provides a solve function to solve a puzzle described
as a PUZZLEDESCRIPTION. The type of the solve function is unit ->
move list * state list. When called (by applying it to unit), it
returns a pair. The first element of the pair is a solution to the puzzle,
a list of moves that when executed on the initial state reaches a goal
state. The second element is a list of all the states that were ever visited
in the search process in looking for a goal state, in any order. (These
are just the elements of the visited set; the Set.S.elements function
may come in handy for generating this from the visited set.)
The search process will thus need to maintain several data struc-
tures:

• A collection of states9 that are pending examination (initialized with 9

A subtlety about the pending collec-
tion. Instead of storing as its elements
just the initial state); and
just the states, it is helpful to store both
a state and the list of moves it took to get
• A set of all states that have been visited (that is, that have ever been to that state. Thus, the elements will be
removed from the pending collection for examination), initialized of type state * move list. That way,
with the empty set. when a goal state is found, you’ll know
the path to get there as well.

The search proceeds by taking a state from the pending collection (call
it the current state). If the current state has already been visited (that
is, it’s in the visited set), we can move on to the next pending state. But
 E F F I C I E N C Y, C O M P L E X I T Y, A N D R E C U R R E N C E S 269

if it has never been visited, we add it to the visited set and examine it
further. If it is a goal state, the search is over and appropriate infor-
mation can be returned from the search. If not, the neighbors of the
current state are generated and added to the pending collection, and
the search continues.
You will implement a functor MakePuzzleSolver that maps a col-
lection implementation (a functor that generates modules satisfy-
ing the signature COLLECTION) and a puzzle description (a module
satisfying the PUZZLEDESCRIPTION signature) to a module imple-
menting the PUZZLESOLVER signature. The solve function in the
generated PUZZLESOLVER module solves the puzzle described in the
PUZZLEDESCRIPTION using a search that uses the provided collection
regime.
The type signature of MakePuzzleSolver is

(functor (sig type t end -> COLLECTION))

-> PUZZLEDESCRIPTION
-> PUZZLESOLVER

Notice that MakePuzzleSolver is a functor that takes a functor as

its argument. It is a higher-order functor! (You may not have real-
ized that was even possible in OCaml.) The idea is that the functor
MakeCollection is used for generating the collection for storing
pending states that have yet to be searched. Using different collec-
tion regimes – stacks (MakeStackList), queues (MakeQueueList,
MakeQueueStack), etc. – leads to different search regimes – depth-
first, breadth-first, etc. The argument functor can be used to provide
a collection of elements of any type. We recommend that you use it
to store the collection of pending elements of type state * (move
list). (See footnote 9.) Each such element pairs a state remaining
to be searched with the list of moves that was used to reach it starting
from the initial state.
The PUZZLESOLVER signature provides for an exception
CantReachGoal for the solver to raise if it can’t find a solution to the
puzzle.
Problem 167
Implement the MakePuzzleSolver functor. Again, you’ll want to test it fully.

14.7.3 Testing, metering, and writeup

We’ve provided two sample puzzles, an 8-puzzle and a maze puzzle, in

files tiles.ml and mazes.ml, respectively. Once you’ve implemented
the solver (Section 14.7.2) you should be able to solve simple puzzles of
these sorts. The file tests.ml has code to generate some maze and tile
puzzles and solve them with various solvers. Once you’ve completed
270 PROGRAMMING WELL

the first two problems in the problem set, you should be able to build
tests.byte and see the puzzle solvers in action.
Running the tests uses OCaml’s graphics to display the puzzles and
even animates the search process. After each display, press any key to
move on to the next display. Figure 14.11 shows the display of the maze
solver as it animates the solution of a maze.
This code should provide some inspiration for your own systematic
testing of the solver on these kinds of puzzles. You can even implement
your own puzzles if you’re of a mind too. (Extra karma!)
You can try out both depth-first and breadth-first search for solving
the puzzles by using different collection functors in the solver. With
your two-stack implementation of queues, you can experiment with Figure 14.11: A frame from an ani-
mation of the maze solver. The initial
the relative efficiency of the two queue implementations. position of the maze is in the upper left,
Problem 168 and the goal in the lower right, marked
In order to assess the benefit of the additional implementation of collections, you should with a grey circle on purple background.
design, develop, test, and deploy a performance testing system for timing the perfor- The current position is marked with the
mance of the solver using different search regimes and implementations. Unlike in pre- purple circle slightly southeast of the
vious problem sets, we provide no skeleton code to carry out this part of the problem set center. The maze walls are brick red,
(though tests.ml may be useful to look at). You should add a new file experiments.ml and the visited positions are marked
that carries out your experiments. The design of this code is completely up to you. This with small squares.
part of the problem set allows you the freedom to experiment with ab initio design. The
deliverable for this part of the problem set, in addition to any code that you write, is a
report on the relative performance that you find.
You’ll want to write some OCaml code to time the solving process on appropriate
examples with several collection implementations. There are some timing functions that
may be useful in the Absbook module (absbook.ml), which we’ve used in tests.ml. You
may also find the time and gettimeofday functions in the Sys and Unix modules useful
for this purpose.
We recommend that to the extent possible any code that you write for performing
your testing not change the structure of code we provide, so that our unit tests will still
work. Ideally, all the code for your experimentation should be in new files, not the ones
we provided.
You should generate your writeup with the rest of your code as a raw text, Markdown,
or LATEX file named writeup.txt, writeup.md, or writeup.tex. (We recommend these
markup-based non-wysiwyg formats for reasons described here. For Markdown in
particular, there are many tools for writing and rendering Markdown files. Some of our
favorites are: ByWord, Marked, MultiMarkdown Composer, Sublime Text, and the Swiss
army knife of file format conversion tools, pandoc.) If you use Markdown or LATEX, you
should include the rendered version as a PDF file writeup.pdf in your submission as
well.

This portion of the problem set is purposefully underspecified.

There is no “right answer” that we are looking for. It is up to you to
determine what makes sense to evaluate the performance of the
code. It is up to you to decide how to present your results in clear,
well-structured, cogent prose and appropriate visualizations of the
experimental results.
15
Mutable state and imperative programming

The range of programming abstractions presented so far – first-order

and higher-order functions; strong, static typing; polymorphism;
algebraic data types; modules and functors – all fall squarely within a
view of functional programming that we might term P U R E , in which
computation is identified solely with the evaluation of expressions.
Pure programming has to do with what expressions are, not what they
do. Pure programs have values rather than effects. Indeed, the slightly
pejorative term S I D E E F F E C T is used in the functional programming
literature for effects that impure programs manifest while they are
being evaluated beyond their values themselves.
In a pure functional programming language, there are no side ef-
fects. Computation can be thought of as simplifying expressions to
their values by repeated substitution of equals for equals. Because this
notion of program meaning is so straightforward, functional programs
are easier to reason about. Hopefully, the preceding chapters have
shown that the functional paradigm is also more powerful than you
might have thought.
Strictly speaking, however, pure functional programming is point-
less. We write code to have an effect on the world. It might be pretty
to think that “side effects” aren’t the main point. But they’re the main
point.
Take this simple computation of the twentieth Fibonacci number:
# let rec fib n =
# if n <= 1 then 1
# else fib (n - 1) + fib (n - 2) ;;
val fib : int -> int = <fun>

# fib 20 ;;
- : int = 10946

The computation of fib 20 proceeds purely functionally – at least

until that very last step where the OCaml R E P L prints out the com-
puted value. Printing is the quintessential side effect. It’s a thing that
272 PROGRAMMING WELL

a program does, not a value that a program has. Without that one side
effect, the fib computation would be useless. We’d gain no informa-
tion from it.
So we need at least a little impurity in any programming system.
But there are some algorithms that actually require impurity – side
effects that change state. For instance, we’ve seen implementation of
a dictionary data type in Chapter 12. That implementation allowed
for linear insertion and linear lookup. More efficient implementations
allow for constant time insertion and linear lookup (or vice versa) or
for logarithmic insertion and lookup. But by taking advantage of side
effects that change state, we can implement mutable dictionaries,
which achieve constant time insertion and constant time lookup, for
instance, with hash tables. (In fact, we do so in Section 15.6.)
In this chapter and the next, we introduce I M P E R AT I V E P R O G R A M -
M I N G , a programming paradigm based on side effects and state
change. We start with mutable data structures, moving on to imper-
ative control structures in the next chapter.
In the pure part of OCaml, we don’t change the state of the compu-
tation, as encoded in the computer’s memory. In languages that have
mutable state, variables name blocks of memory whose contents can
change. Assigning a new value to such a variable mutates the memory,
changing its state by replacing the original value with the new one.
OCaml variables, by contrast, aren’t mutable. They name values, and
once having named a value, the value named doesn’t change.
You might think that OCaml does allow changing the value of a
variable. What about, for instance, a global renaming of a variable?

# let x = 42 ;;
val x : int = 42
# x ;; (* x is 42 *)
- : int = 42
# let x = 21 ;;
val x : int = 21
# x ;; (* ...but now it's 21 *)
- : int = 21

Hasn’t the value of x changed from 42 to 21?

No, it hasn’t. Rather, there are two separate variables that happen to
both have the same name, x. In the second expression, we are referring
to the first x variable. In the fourth expression, we are referring to the
second x variable, which shadows the first one. But the first x is still
there. We can tell by the following experiment:

# let x = 42 ;; (* establishing first x... *)

val x : int = 42
# x ;; (* ...whose value is 42 *)
- : int = 42
 M U TA B L E S TAT E A N D I M P E R AT I V E P R O G R A M M I N G 273

# let f () = x ;; (* f returns value in first x *)

val f : unit -> int = <fun>
# let x = 21 ;; (* establishing second x... *)
val x : int = 21
# x ;; (* ...with a different value *)
- : int = 21
# f () ;; (* but f still references first x *)
- : int = 42

The definition of the function f makes use of the first variable x, simply
by returning its value when called. Even if we add a new x naming a
different value, the application f () still returns 42, the value that the
first variable x names, thereby showing that the first x is still available.
The let naming constructs of OCaml thus don’t provide for mutable
state. If we want to make use of mutable state, for instance for the pur-
pose of building mutable data structures, we’ll need new constructs.
OCaml provides references for this purpose.

15.1 References

The OCaml language provides an abstract notion of R E F E R E N C E to a

block of mutable memory with its R E F E R E N C E T Y P E S . To maintain
the type discipline of the language, we want to keep track of the type 1
Yes, the same symbol, ref, is used at
of thing stored in the block; although the particular value stored there the type level for the type constructor
and at the value level for the value
may change, we don’t want its type to vary. Thus, we have separate
constructor. And to make matters
types for references to integers, references to strings, references to more confusing, the type constructor
functions from booleans to integers, and so forth. The postfix type is postfix while the value constructor is
prefix. Learning the concrete syntax of a
constructor ref is used to construct reference types: int ref, string new programming language sure can be
ref, (bool -> int) ref, and the like. frustrating.
To create a value of some reference type, OCaml provides the prefix
value constructor ref.1 The supplied expression must be of the type
appropriate for the reference type, and the value of that expression is
stored as the initial value in the block of memory that the reference ref-
erences. Here, for instance, we create a reference to a block of memory
storing the integer value 42:
# let r : int ref = ref 42 ;;
val r : int ref = {contents = 42}

As with all variables, r is an immutable name, but it is a name for

a block of memory that is itself mutable. (The value is printed as
{contents = 42} for reasons that we allude to in Section 15.2.1.)
The natural operations to perform on a reference value are two:
first, D E R E F E R E N C E , that is, retrieve the value stored in the referenced
block; and second, U P D AT E , modify the value stored in the referenced
block (with a value of the same type, of course). Dereferencing is done
with the prefix ! operator, and updating with the infix := operator.
274 PROGRAMMING WELL

# !r ;;
- : int = 42
# r := 21 ;;
- : unit = ()
# !r ;;
- : int = 21

Here, we’ve dereferenced the same variable r twice (in the two high-
lighted expressions), getting two different values – first 42, then 21.
This is quite different from the example with two x variables. Here,
there is only one variable r, and yet a single expression !r involving r
whose value has changed!2 2
But like all variables, r has not itself
changed its value. It still points to the
This example puts in sharp relief the difference between the pure
same block of memory.
language and the impure. In the pure language, an expression in a
given lexical context (that is, the set of variable names that are avail-
able) always evaluates to the same value. But in this example, two
instances of the expression !r evaluate to two different values, even
though the same r is used in both instances of the expression. The
assignment has the side effect of changing what value is stored in the
block that r references, so that reevaluating !r to retrieve the stored
value finds a different integer.
The expression causing the side effect here was easy to spot. But
in general, these side effects could happen as the result of a series of
function calls quite obscure from the code that manifests the side
effect. This property of side effects can make it difficult to reason about
what value an expression has.
In particular, the substitution semantics of Chapter 13 has Leibniz’s
law as a consequence. Substitution of equals for equals doesn’t change
the value of an expression. But here, we have a clear counterexample.
The first evaluation implies that !r and 42 are equal. Yet if we substi-
tute 42 for !r in the third expression, we get 42 instead of 21. Once we
add mutable state to the language, we need to extend the semantics
from one based purely on substitution. We do so in Chapter 19, where
we introduce environment semantics.

15.1.1 Reference operator types

The reference system is specifically designed so as to retain OCaml’s

strong typing regimen. Each of the operators, for instance, can be seen
as a well-typed function. The dereference operator !, for instance,
takes an argument of type ’a ref and returns the ’a referenced. It is
thus typed as (!) : ’a ref -> ’a. The reference value constructor
ref works in the opposite direction, taking an ’a and returning an ’a
ref, so it types as (ref) : ’a -> ’a ref.
Finally, the assignment operator := takes two arguments, a refer-
 M U TA B L E S TAT E A N D I M P E R AT I V E P R O G R A M M I N G 275

ence to update, of type ’a ref, and the new ’a value to store there.
But what should the assignment operator return? Assignment is per-
formed entirely for its side effect – the update in the state of memory
– rather than for its return value. Given that there is no information
in the return value, it makes sense to use a type that conveys no in-
formation. This is a natural use for the unit type (Section 4.3). Since
unit has only one value (namely, the value ()), that value conveys no
information. The hallmark of a function that is used only for its side
effects (which we might call a P R O C E D U R E ) is the unit return type.
The typing for assignment is appropriately then (:=) : ’a ref ->
’a -> unit.
These typings can be verified in OCaml itself:
# (!) ;;
- : 'a ref -> 'a = <fun>
# (ref) ;;
- : 'a -> 'a ref = <fun>
# (:=) ;;
- : 'a ref -> 'a -> unit = <fun>

15.1.2 Boxes and arrows

It can be helpful to visualize references using B O X A N D A R R OW D I A -

G R A M S. When establishing a reference,
# let r = ref 42 ;;
val r : int ref = {contents = 42}

we draw a box (standing for a block of memory) named r with an arrow

pointing to another box (block of memory) containing the integer 42
(Figure 15.1(a)). Adding another reference with
# let s = ref 42 ;;
val s : int ref = {contents = 42}

generates a new named box and its referent (Figure 15.1(b)), which
happens to store the same value. But we can tell that the referents are
distinct, since assigning to r changes !r but not !s (Figure 15.1(c)).
# r := 21 ;;
- : unit = ()
# !r, !s ;;
- : int * int = (21, 42)
Figure 15.1: Box and arrow diagrams
To have s refer to the value that r does, we need to assign to it as well for the state of memory as various
(Figure 15.1(d)). references are created and updated.

# s := !r ;;
- : unit = ()

We can have a reference s that points to the same block of memory

as r does (Figure 15.1(e)).
276 PROGRAMMING WELL

# let s = r ;;
val s : int ref = {contents = 21}

Now s and r have the same value (that is, refer to the same block of
memory). We say that s is an A L I A S of r. (The old s is shadowed by the
new one, as depicted by showing it in gray. Since we no longer have
access to it and whatever it references, the gray blocks of memory are
garbage. See the discussion in Section 15.1.3.)
Changing the value stored in a block of memory changes the value
of all its aliases as well. Here, updating the block referred to by r (Fig-
ure 15.1(f)) changes the value for s:

# r := 7 ;;
- : unit = ()
# !r, !s ;;
- : int * int = (7, 7)

In a language with references and aliases, we are confronted with

two different notions of equality. S T RU C T U R A L E QUA L I T Y holds when
two values have the same structure, regardless of where they are stored
in memory, such as r and s in Figure 15.1(d). P H Y S I C A L E QUA L I T Y
holds when two values are the identical “physical” block of memory, as
r and s in Figure 15.1(e). Values that are physically equal are of course
structurally equal as well but the converse needn’t hold.
In OCaml, structural equality and inequality are tested with (=) :
’a -> ’a -> bool and (<>) : ’a -> ’a -> bool, respectively,
whereas physical equality and inequality of mutable types are tested
with (==) : ’a -> ’a -> bool and (!=) : ’a -> ’a -> bool.3 3
The behavior of == and != tests on
immutable (pure) types is allowed to
Exercise 169 be implementation-dependent and
Construct an example defining values r and s that are structurally but not physically shouldn’t be relied on. These operators
equal. Construct an example defining values r and s that are both structurally and should only be used with values of
physically equal. Verify these conditions using the equality functions. mutable types.

15.1.3 References and pointers

You may have seen this kind of thing before. In programming lan-
guages like c, references to blocks of memory are manipulated through
POINTERS to memory, which are explicitly created (with malloc) and
freed (with free), dereferenced (with *), and updated (with =). Some
correspondences between OCaml and c syntax for these operations are
given in Table 15.1.
Notable differences between the OCaml and c approaches are:

• In OCaml, unlike in c, references can’t be created without initializing

them. Referencing uninitialized blocks of memory is a recipe for
difficult to diagnose bugs. OCaml’s type regime eliminates this
entire class of bugs, since a reference type like int ref specifies
 M U TA B L E S TAT E A N D I M P E R AT I V E P R O G R A M M I N G 277

Operation OCaml c

Create, initialize ref 42

Create, name int *r = malloc(sizeof int);
Create, initialize, name let r = ref 42 int *r = malloc(sizeof int);
*r = 42;
Dereference !r *r
Update r := 21 *r = 21
Free free(r)

that the block must at all times store an int and the operators
maintain this invariant. Table 15.1: Approximate equivalencies
between OCaml references and c
• In c, nothing conspires to make sure that the size of the block al- pointers.

located is appropriate for the value being stored, leading to the

possibility of B U F F E R O V E R F L OW S – assignments that overflow one
block of memory to overwrite others. Buffer overflows allow for
widely exploited security holes in code. In OCaml, the strong typing
again eliminates this class of bug. Similarly, B U F F E R O V E R - R E A D S
occur when a program reading from a block of memory continues
to read past the end of the block into adjacent memory, potentially
compromising the security of information in the adjacent block. An
infamous example is the H E A RT B L E E D bug in OpenSSL, so notori-
ous that it even acquired its own logo (Figure 15.2).

• In c, programs must free memory explicitly in order to reclaim the

previously allocated memory for future use. When blocks are freed
while still being used, the memory can be overwritten, leading to
M E M O RY C O R RU P T I O N and once again to insidious bugs. Con- Figure 15.2: The logo for H E A RT B L E E D ,
a buffer over-read bug in the widely
versely, not freeing blocks even when they are no longer needed,
used OpenSSL library (written in c) for
called a M E M O RY L E A K , leads to programs running out of memory securing web interactions. The bug
needlessly. was revealed in 2014 after two years
undiscovered in the field.
OCaml has no function for explicitly freeing memory. Instead,
blocks of memory that are no longer needed, as determined by the
OCaml run-time system itself, are referred to as G A R B A G E . The
run-time system reclaims garbage automatically, in a process called
G A R B A G E C O L L E C T I O N. Since computers can typically analyze
the status of memory blocks better than people, the use of garbage
collection eliminates memory corruption and memory leaks.
However, the garbage collection approach takes the timing of mem-
ory reclamation out of the hands of the programmer. The run-time
system may decide to perform computation-intensive garbage col-
lection at inopportune times. For applications where careful control
of such timing issues is necessary, the garbage collection approach
278 PROGRAMMING WELL

may be undesirable; use of a language, like c, that allows explicit

allocation and deallocation of memory may be necessary.4 4
A new class of functional programming
languages is exploring the design space
Problem 170 of languages with high-level abstraction
For each of the following expressions, give its type and value, if any. mechanisms as in OCaml, including
strongly typed safe references, while
1. let a = ref 3 in
let b = ref 5 in providing finer control of memory
let a = ref b in deallocation, in order to obtain the best
!(!a) ;; of both the explicit approach and the
2. let rec a, b = ref b, ref a in garbage collection approach. The prime
!a ;; example is Mozilla’s Rust language.
3. let a = ref 1 in
let b = ref a in
let a = ref 2 in
!(!b) ;;

4. let a = 2 in
let f = (fun b -> a * b) in
let a = 3 in
f (f a) ;;

5. let a = Cons(2, ref (Cons(3, ref Nil))) ;;

15.2 Other primitive mutable data types

In addition to references, OCaml provides two other primitive data

types that allow for mutability: mutable record fields and arrays. We
mention them briefly for completeness; full details are available in the
OCaml documentation.

15.2.1 Mutable record fields

Records (Section 7.4) are compound data structures with named fields,
each of which stores a value of a particular type. As introduced, each
field of a record, and hence records themselves, are immutable. How-
ever, when a record type is defined with the type construct, and the
individual fields are specified and typed, its individual fields can also
be marked as allowing mutability by adding the keyword mutable.
For instance, we can define a person record type with immutable
name fields but a mutable address field.

# type person = {lastname : string;

# firstname : string;
# mutable address : string} ;;
type person = {
lastname : string;
firstname : string;
mutable address : string;
}

Once constructed, the address of a person can be updated.

# let sms = {lastname = "Shieber";

# firstname = "Stuart";
# address = "123 Main"} ;;
 M U TA B L E S TAT E A N D I M P E R AT I V E P R O G R A M M I N G 279

val sms : person =

{lastname = "Shieber"; firstname = "Stuart"; address = "123
Main"}
# sms.address <- "124 Main" ;; (* I moved next door *)
- : unit = ()

To update a mutable record, the operator <- is used, rather than := as

for references.
In fact, reference types and their operators can be thought of as
being implemented using mutable records by the following type and
operator definitions:

type 'a ref_ = {mutable contents : 'a} ;;

let ref_ (v : 'a) : 'a ref_ = {contents = v} ;;

let (:=) (r : 'a ref_) (v : 'a) : unit = r.contents <- v ;;
let (!) (r : 'a ref_) : 'a = r.contents ;;

This should explain the otherwise cryptic references to contents when

the R E P L prints values of reference type.

15.2.2 Arrays

Arrays are a kind of cross between lists and tuples with added muta-
bility. Like lists, they can have an arbitrary number of elements all of
the same type. Unlike lists (but like tuples), they cannot be extended
in size; there is no cons equivalent for arrays. Finally, each element of
an array can be individually indexed and updated. An example may
indicate the use of arrays:

# let a = Array.init 5 (fun n -> n * n) ;;

val a : int array = [|0; 1; 4; 9; 16|]
# a ;;
- : int array = [|0; 1; 4; 9; 16|]
# a.(3) <- 0 ;;
- : unit = ()
# a ;;
- : int array = [|0; 1; 4; 0; 16|]

Here, we’ve created an array of five elements, each the square of its
index. We update the third element to be 0, and examine the result,
which now has a 0 in the appropriate location.

15.3 References and mutation

To provide an example of the use of mutating references, we consider

the task of counting the occurrences of an event. We start by establish-
ing a location to store the current count as an int ref named gctr
(for “global counter”).
280 PROGRAMMING WELL

# let gctr = ref 0 ;;

val gctr : int ref = {contents = 0}

Now we define a function that “bumps” the counter (adding 1) and

then returns the current value of the counter.
# let bump () =
# gctr := !gctr + 1;
# !gctr ;;
val bump : unit -> int = <fun>

We’ve used a new operator here, the binary sequencing operator (;),
which is a bit like the pair operator (,) in that it evaluates its left and
right arguments, except that the sequencing operator returns the
value only of the second.5 But then what could possibly be the point 5
You can think of P ; Q as being
syntactic sugar for let () = P in Q.
of evaluating the first argument? Since the argument isn’t used for
its value, it must be of interest for its side effects. That is the case in
this example; the expression gctr := !gctr + 1 has the side effect
of updating the counter to a new value, its old value (retrieved with
!gctr) plus one.6 Since the sequencing operator ignores the value 6
This part of the bump function that
does the actual incrementing of an
returned by its first argument, it requires that argument to be of type
int ref is a common enough activity
unit, the type for expressions with no useful value.7 that OCaml provides a function incr
We can test it out. : int ref -> unit in the Stdlib
library for just this purpose. It works as
# bump () ;; if implemented by
- : int = 1
let incr (r : int ref) : unit =
# bump () ;; r := !r + 1 ;;
- : int = 2
We could therefore have substituted
# bump () ;; incr gctr as the second line of the
- : int = 3 bump function.
7
Sometimes, you may want to sequence
Again, you see the hallmark of impure code – the same expression in an expression that returns a value other
the same context evaluates to different values. The change between than (). The ignore function of type ’a
-> unit in Stdlib comes in handy in
invocations happens because of the side effects of the earlier calls to
such cases.
bump. We can see evidence of the side effects also in the value of the
counter, which is globally visible.
# !gctr ;;
- : int = 3

In the case of the bump function, it is the intention to provide these

side effects. They are what generates the counting functionality. How-
ever, it is not necessarily the intention to make the current counter
visible to users of the bump function. Doing so enables unintended side
effects, like manipulating the value stored in the counter outside of the
manipulation by the bump function itself, enabling misuses such as the
following:
# gctr := -17 ;;
- : unit = ()
# bump () ;;
- : int = -16
 M U TA B L E S TAT E A N D I M P E R AT I V E P R O G R A M M I N G 281

To eliminate this abuse we’d like to avoid a global variable for the
counter. We’ve seen this kind of information hiding before – in the use
of local variables within functions, and in the use of signatures to hide
auxiliary values and functions from users of modules, all instances of
the edict of compartmentalization. But in the context of assignment,
making gctr a local variable (we’ll call it ctr) requires some thought. A
naive approach doesn’t work:
# let bump () =
# let ctr = ref 0 in
# ctr := !ctr + 1;
# !ctr ;;
val bump : unit -> int = <fun>

Exercise 171
What goes wrong with this definition? Try using it a few times and see what happens.
The problem: This code establishes the counter variable ctr upon
application of bump, and establishes a new such variable at each such
application. Instead, we want to define ctr just once, upon the defini-
tion of bump, and not its applications.
In this case, the compact notation for function definition, which
conflates the defining of the function and its naming, is doing us a
disservice. Fortunately, we aren’t obligated to use that syntactic sugar.
We can use the desugared version:
let bump =
fun () ->
ctr := !ctr + 1;
!ctr ;;

Now the naming (first line) and the function definition (second line
and following) are separate. We want the definition of ctr to outscope
the function definition but fall within the local scope of its naming:
# let bump =
# let ctr = ref 0 in
# fun () ->
# ctr := !ctr + 1;
# !ctr ;;
val bump : unit -> int = <fun>

The function is defined within the scope of – and therefore can access
and modify – a local variable ctr whose scope is only that function.
This definition operates as before to deliver incremented integers:
# bump () ;;
- : int = 1
# bump () ;;
- : int = 2
# bump () ;;
- : int = 3
 282 PROGRAMMING WELL

but access to the counter variable is available only within the function,
as it should be, and not outside of it:

# !ctr ;;
Line 1, characters 1-4:
1 | !ctr ;;
^^^
Error: Unbound value ctr
Hint: Did you mean gctr?

This example – the counter with local, otherwise inaccessible, per-

sistent, mutable state – is one of the most central to understand. We’ll
see a dramatic application of this simple pattern in Chapter 18, where
it underlies the idea of instance variables in object-oriented program-
ming.
Problem 172
Suppose you typed the following OCaml expressions into the OCaml R E P L sequentially.
1 let p = ref 11 ;;
2 let r = ref p ;;
3 let s = ref !r ;;
4 let t =
5 !s := 14;
6 !p + !(!r) + !(!s) ;;
7 let t =
8 s := ref 17;
9 !p + !(!r) + !(!s) ;;

Try to answer the questions below about the status of the various variables being defined
before typing them into the R E P L yourself.
1. After line 1, what is the type of p?
2. After line 2, what is the type of r?
3. After line 3, which of the following statements are true?
(a) p and s have the same type
(b) r and s have the same type
(c) p and s have the same value (in the sense that p = s would be true)
(d) r and s have the same value (in the sense that r = s would be true)
4. After line 6, what is the value of t?
5. After line 9, what is the value of t?

15.4 Mutable lists

To demonstrate the power of imperative programming, we use

OCaml’s imperative aspects to provide implementations of two mu-
table data structures: mutable lists and mutable queues.
As noted in Section 11.1, the OCaml list type operates as if defined
by

type 'a list =

| Nil
| Cons of 'a * 'a list ;;

A mutable list allows the tail of the list to be updated.

 M U TA B L E S TAT E A N D I M P E R AT I V E P R O G R A M M I N G 283

# type 'a mlist =

# | Nil
# | Cons of 'a * ('a mlist ref) ;;
type 'a mlist = Nil | Cons of 'a * 'a mlist ref

We can compute the length of such a list using the usual recursive
definition. We try

# let rec length (l : 'a mlist) : int =

# match l with
# | Nil -> 0
# | Cons(_hd, tl) -> 1 + length tl ;;
Line 4, characters 30-32:
4 | | Cons(_hd, tl) -> 1 + length tl ;;
^^
Error: This expression has type 'a mlist ref
but an expression was expected of type 'a mlist

but this goes south in trying to calculate the recursive length of the tail
tl. Of course, tl isn’t an ’a mlist; it’s a reference to one. The fix is
easy:

# let rec length (l : 'a mlist) : int =

# match l with
# | Nil -> 0
# | Cons(_hd, tl) -> 1 + length !tl ;;
val length : 'a mlist -> int = <fun>

We can build some mutable lists and experiment a bit.

# let r = ref Nil ;;

val r : '_weak1 mlist ref = {contents = Nil}
# let s = Cons(1, r) ;;
Nil
val s : int mlist = Cons (1, {contents = Nil})
# let t = Cons(2, ref s);;
Cons(1, r )
val t : int mlist = Cons (2, {contents = Cons (1, {contents = s
Nil})})
# length !r ;;
Cons(2, )
- : int = 0 t
# length s ;;
- : int = 1
# length t ;; Nil
- : int = 2

Cons(1, r )
Box and arrow diagrams (Figure 15.3) help in figuring out what’s going s
on here.
Cons(2, )
Exercise 173 t
Write functions mhead and mtail that extract the head and the (dereferenced) tail from a Figure 15.3: Pictorial representation of
mutable list. For example, (top) the state of memory after building
# mhead t ;; some mutable list structures, and
- : int = 2 (bottom) updating with r := t. The
# mtail t ;;
nil has become garbage and the lists s
- : int mlist = Cons (1, {contents = Nil})
and t now have cycles in them.
284 PROGRAMMING WELL

Problem 174
For each of the following expressions, give its type and value, if any.
1. let a = Cons(2, ref (Cons(3, ref Nil))) ;;
let Cons(_h, t) = a in
let b = Cons(1, ref a) in
t := b;
mhead (mtail (mtail b)) ;;

Because the lists are mutable, we can modify the tail of s (equiva-
lently, r) to point to t.

# r := t ;;
- : unit = ()

Since the tail of s points to t and the tail of t to s, we’ve constructed a

CYCLIC data structure. Doing so uncovers a bug in the length func-
tion,

# length t ;;
Stack overflow during evaluation (looping recursion?).

demonstrating once again how adding impure features to a language

introduces new and quite subtle complexities.
Problem 175
Provide an implementation of the length function that handles cyclic lists, so that
# length t ;;
- : int = 2

You’ll notice that the requirement to handle cyclic lists dramatically increases the
complexity of implementing length. (Hint: Keep a list of sublists you’ve already visited
and check to see if you’ve already visited each sublist. What is a reasonable value to
return in that case?)
Problem 176
Define a function first that returns a list (immutable) of the first n elements of a
mutable list mlst:
Problem 177
Write code to define a mutable integer list alternating such that for all integers n, the
expression first n alternating returns a list of alternating 1s and 2s, for example,
# first 5 alternating ;;
- : int list = [1; 2; 1; 2; 1]
# first 8 alternating ;;
- : int list = [1; 2; 1; 2; 1; 2; 1; 2]

15.5 Imperative queues

By way of review, the pure functional queue data structure in Sec-

tion 12.4 implemented the following signature:

# module type QUEUE = sig

# type 'a queue
# val empty_queue : 'a queue
# val enqueue : 'a -> 'a queue -> 'a queue
# val dequeue : 'a queue -> 'a * 'a queue
# end ;;
module type QUEUE =
 M U TA B L E S TAT E A N D I M P E R AT I V E P R O G R A M M I N G 285

sig
type 'a queue
val empty_queue : 'a queue
val enqueue : 'a -> 'a queue -> 'a queue
val dequeue : 'a queue -> 'a * 'a queue
end

Each call to enqueue and dequeue returns a new queue, differing from
its argument queue in having an element added or removed.
In an imperative implementation of queues, the enqueuing and
dequeuing operations can and do mutate the data structure, so that
the operations don’t need to return an updated queue. The types for
the operations thus change accordingly. We’ll use the following IMP_-
QUEUE signature for imperative queues:

# module type IMP_QUEUE = sig

# type 'a queue
# val empty_queue : unit -> 'a queue
# val enqueue : 'a -> 'a queue -> unit
# val dequeue : 'a queue -> 'a option
# end ;;
module type IMP_QUEUE =
sig
type 'a queue
val empty_queue : unit -> 'a queue
val enqueue : 'a -> 'a queue -> unit
val dequeue : 'a queue -> 'a option
end

Here again, you see the sign of a side-effecting operation: the enqueue
operation returns a unit. Dually, to convert a procedure that modifies
its argument and returns a unit into a pure function, the standard
technique is to have the function return instead a modified copy of its
argument, leaving the original untouched. Indeed, when we generalize
the substitution semantics of Chapter 13 to handle state and state
change in Chapter 19, we will use just this technique of passing a
representation of the computation state as an argument and returning
a representation of the updated state as the return value.
Another subtlety introduced by the addition of mutability is the
type of the empty_queue value. In the functional signature, we had
empty_queue : ’a queue; the empty_queue value was an empty
queue. In the mutable signature, we have empty_queue : unit ->
’a queue; the empty_queue value is a function that returns a (new,
physically distinct) empty queue. Without this change, the empty_-
queue value would be “poisoned” as soon as something was inserted
in it, so that further references to empty_queue would see the modified
(non-empty) value. Instead, the empty_queue function can generate a
new empty queue each time it is called.
286 PROGRAMMING WELL

15.5.1 Method 1: List references

Perhaps the simplest method to implement an imperative queue is as a

(mutable) reference to an (immutable) list of the queue’s elements.
# module SimpleImpQueue : IMP_QUEUE =
# struct
# type 'a queue = ('a list) ref
# let empty_queue () = ref []
# let enqueue elt q =
# q := (!q @ [elt])
# let dequeue q =
# match !q with
# | first :: rest -> (q := rest; Some first)
# | [] -> None
# end ;;
module SimpleImpQueue : IMP_QUEUE

This is basically the same as the list implementation from Section 12.4,
but with the imperative signature. Nonetheless, internally the opera-
tions are still functional, and enqueuing an element requires time lin-
ear in the number of elements in the queue. (Recall from Section 14.5
that the functional append function (here invoked as Stdlib.(@)) is
linear.)
We’ll examine two methods for generating constant time implemen-
tations of an imperative queue.

15.5.2 Method 2: Two stacks

Figure 15.4: Pictorial representation of
An old trick is to use two stacks to implement a queue. The two stacks implementing a queue with two stacks.
hold the front of the queue (the first elements in, and hence the first
out) and the reversal of the rear of the queue. For example, a queue
containing the elements 1 through 4 in order might be represented by
the two stacks (implemented as int lists) [1; 2] and [4; 3], or
pictorially as in Figure 15.4 (upper left).
Enqueuing works by adding an element (5 in upper right) to the rev
rear stack. Dequeuing works by popping the top element in the front
stack, if there is one (middle right and left and lower right). If there are
no elements to dequeue in the front stack (middle left), the rev rear
stack is reversed onto the front stack first (lower left).
The stacks can be implemented with type ’a list ref and the two
stacks packaged together in a record.
module TwoStackImpQueue : IMP_QUEUE =
struct
type 'a queue = {front : 'a list ref;
revrear : 'a list ref}
...

The empty queue has two empty lists.

 M U TA B L E S TAT E A N D I M P E R AT I V E P R O G R A M M I N G 287

module TwoStackImpQueue : IMP_QUEUE =

struct
type 'a queue = {front : 'a list ref;
revrear : 'a list ref}
let empty_queue () =
{front = ref []; revrear = ref []}
...

Enqueuing simply places the element on the top of the rear stack.
module TwoStackImpQueue : IMP_QUEUE =
struct
type 'a queue = {front : 'a list ref;
revrear : 'a list ref}
let empty_queue () =
{front = ref []; revrear = ref []}
let enqueue elt q =
q.revrear := elt :: !(q.revrear)
...

Dequeuing is the more complicated operation.

# module TwoStackImpQueue : IMP_QUEUE =
# struct
# type 'a queue = {front : 'a list ref;
# revrear : 'a list ref}
# let empty_queue () =
# {front = ref []; revrear = ref []}
# let enqueue elt q =
# q.revrear := elt :: !(q.revrear)
# let rec dequeue q =
# match !(q.front) with
# | h :: t -> (q.front := t; Some h)
# | [] -> if !(q.revrear) = [] then None
# else ((* reverse revrear onto front *)
# q.front := List.rev (!(q.revrear));
# (* clear revrear *)
# q.revrear := [];
# (* try the dequeue again *)
# dequeue q)
# end ;;
module TwoStackImpQueue : IMP_QUEUE

As in method 1, the enqueue operation takes constant time. But de-

queuing usually takes constant time too, unless we have to perform the
reversal of the rear stack. Since the stack reversal takes time linear in
the number of enqueues, the time to enqueue and dequeue elements
is, on average, constant time per element.

Exercise 178
An alternative is to use mutable record fields, so that the queue type would be
type 'a queue = {mutable front : 'a list;
mutable revrear : 'a list}

Reimplement the TwoStackImpQueue module using this type for the queue implementa-
tion.
288 PROGRAMMING WELL

15.5.3 Method 3: Mutable lists

To allow for manipulation of both the head of the queue (where en-
queuing happens) and the tail (where dequeuing happens), a final
implementation uses mutable lists. The queue type

module MutableListQueue : IMP_QUEUE =

struct
type 'a queue = {front : 'a mlist ref;
rear : 'a mlist ref}
...

provides a reference to the front of the queue as well as a reference to

the last element in the queue. When the queue is empty, both of these
lists will be Nil.

module MutableListQueue : IMP_QUEUE =

struct
type 'a queue = {front : 'a mlist ref;
rear : 'a mlist ref}
let empty_queue () = {front = ref Nil;
rear = ref Nil}
...

Enqueuing a new element differs depending on whether the queue is

empty. If it already contains at least one element, the rear will have a
head and a Nil tail (because the rear always points to the last element.

module MutableListQueue : IMP_QUEUE =

struct
type 'a queue = {front : 'a mlist ref;
rear : 'a mlist ref}
let empty_queue () = {front = ref Nil;
rear = ref Nil}
let enqueue elt q =
match !(q.rear) with
| Cons (hd, tl) -> (assert (!tl = Nil);
tl := Cons(elt, ref Nil);
q.rear := !tl)
| Nil -> ...

If the queue is empty, we establish a single element mutable list with

front and rear pointers to its single element.

module MutableListQueue : IMP_QUEUE =

struct
type 'a queue = {front : 'a mlist ref;
rear : 'a mlist ref}
let empty_queue () = {front = ref Nil;
rear = ref Nil}
let enqueue elt q =
match !(q.rear) with
| Cons (hd, tl) -> (assert (!tl = Nil);
tl := Cons(elt, ref Nil);
 M U TA B L E S TAT E A N D I M P E R AT I V E P R O G R A M M I N G 289

q.rear := !tl)
| Nil -> (assert (!(q.front) = Nil);
q.front := Cons(elt, ref Nil);
q.rear := !(q.front))
...

Finally, dequeuing involves moving the front pointer to the next ele-
ment in the list, and updating the rear to Nil if the last element was
dequeued and the queue is now empty.

# module MutableListQueue : IMP_QUEUE =

# struct
# type 'a queue = {front : 'a mlist ref;
# rear : 'a mlist ref}
#
# let empty_queue () = {front = ref Nil;
# rear = ref Nil}
# let enqueue elt q =
# match !(q.rear) with
# | Cons (_hd, tl) -> (assert (!tl = Nil);
# tl := Cons(elt, ref Nil);
# q.rear := !tl)
# | Nil -> (assert (!(q.front) = Nil);
# q.front := Cons(elt, ref Nil);
# q.rear := !(q.front))
# let dequeue q =
# match !(q.front) with
# | Cons (hd, tl) ->
# (q.front := !tl;
# (match !tl with
# | Nil -> q.rear := Nil
# | Cons(_, _) -> ());
# Some hd)
# | Nil -> None
# end ;;
module MutableListQueue : IMP_QUEUE

Figure ?? depicts the queue data structure as it performs the following

operations:

# let open MutableListQueue in

# let q = empty_queue () in
# enqueue 1 q;
# enqueue 2 q;
# dequeue q ;;
- : int option = Some 1

15.6 Hash tables

A hash table is a data structure implementing a mutable dictionary.

We’ve seen functional key-value dictionaries already in Section 12.6,
which implement a signature like the following:
290 PROGRAMMING WELL

module type DICT =

sig
type key
type value
type dict

(* An empty dictionary *)
val empty : dict
(* Returns as an option the value associated with the
provided key. If the key is not in the dictionary,
returns None. *)
val lookup : dict -> key -> value option
(* Returns true if and only if the key is in the
dictionary. *)
val member : dict -> key -> bool
(* Inserts a key-value pair into the dictionary. If the
key is already present, updates the key to have the
new value. *)
val insert : dict -> key -> value -> dict
(* Removes the key from the dictionary. If the key is
not present, returns the original dictionary. *)
val remove : dict -> key -> dict
end ;;

In a mutable dictionary, the data structure state is actually modified

by side effect when inserting or removing key-value pairs. Conse-
quently, those functions need not (and should not) return an updated
dictionary. (As with mutable lists, because dictionaries can be mod-
ified by side effect, care must also be taken with specifying an empty
dictionary. Instead of a single empty dictionary value, we provide a
function from unit that returns a new empty dictionary.) An appropri-
ate signature for a mutable dictionary, then, is
# module type MDICT =
# sig
# type key
# type value
# type dict
#
# (* Returns an empty dictionary. *)
# val empty : unit -> dict
# (* Returns as an option the value associated with the
# provided key. If the key is not in the dictionary,
# returns None. *)
# val lookup : dict -> key -> value option
# (* Returns true if and only if the key is in the
# dictionary. *)
# val member : dict -> key -> bool
# (* Inserts a key-value pair into the dictionary. If the
# key is already present, updates the key to have the
# new value. *)
# val insert : dict -> key -> value -> unit
# (* Removes the key from the dictionary. If the key is
 M U TA B L E S TAT E A N D I M P E R AT I V E P R O G R A M M I N G 291

# not present, leaves the original dictionary unchanged. *)

# val remove : dict -> key -> unit
# end ;;
module type MDICT =
sig
type key
type value
type dict
val empty : unit -> dict
val lookup : dict -> key -> value option
val member : dict -> key -> bool
val insert : dict -> key -> value -> unit
val remove : dict -> key -> unit
end

In a H A S H TA B L E implementation of this signature, the key-value

pairs are stored in a mutable array of a given size at an index speci-
fied by a H A S H F U N C T I O N , a function from keys to integers within
the range provided. The idea is that the hash function should assign
well distributed locations to keys, so that inserting or looking up a
particular key-value pair involves just computing the hash function
to generate the location where it can be found. Thus, insertion and
lookup are constant-time operations.
An important problem to resolve is what to do in case of a H A S H
C O L L I S I O N , when two different keys hash to the same value. We as-
sume that only a single key-value pair can be stored at a given location
in the hash table – called C L O S E D H A S H I N G – so in case of a collision
when inserting a key-value pair, we keep searching in the table at the
sequentially following array indices until an empty slot in the table is
found. Similarly, when looking up a key, if the key-value pair stored
at the hash location does not match the key being looked up, we se-
quentially search for a pair that does match. This process of trying
sequential locations is known as L I N E A R P R O B I N G . Frankly, linear
probing is not a particularly good method for handling hash collisions
(see Exercises 180 and 181), but it will do for our purposes here.
To define a new kind of hash table, we need to provide types for the
keys and values, a size for the array, and an appropriate hash function.
We package all of this up in a module that can serve as the argument to
a functor.
# module type MDICT_ARG =
# sig
# (* Types to be used for the dictionary keys and values *)
# type key
# type value
# (* size -- Number of elements that can be stored in the
# dictionary *)
# val size : int
# (* hash key -- Returns the hash value for a key. *)
292 PROGRAMMING WELL

# val hash_fn : key -> int

# end ;;
module type MDICT_ARG =
sig type key type value val size : int val hash_fn : key -> int
end

Here is the beginning of an implementation of such a functor:

module MakeHashtableDict (D : MDICT_ARG)

: (MDICT with type key = D.key
and type value = D.value) =
struct
type key = D.key
type value = D.value

(* A hash record is a key value pair *)

type hashrecord = { key : key;
value : value }
(* An element of the hash table array is a hash record
(or empty) *)
type hashelement =
| Empty
| Element of hashrecord
(* The hash table itself is a (mutable) array of hash
elements *)
type dict = hashelement array

let empty () = Array.make D.size Empty

...
end ;;

With a full implementation of the MakeHashtableDict functor

(Exercise 179), we can build an IntStringHashtbl hash table module
for hash tables that map integers to strings as follows:8 8
The hash function we use here is an
especially poor choice; we use it to
# module IntStringHashtbl : (MDICT with type key = int make it easy to experiment with hash
# and type value = string) = collisions.
# MakeHashtableDict (struct
# type key = int
# type value = string
# let size = 100
# let hash_fn k = (k / 3) mod size
# end) ;;
module IntStringHashtbl :
sig
type key = int
type value = string
type dict
val empty : unit -> dict
val lookup : dict -> key -> value option
val member : dict -> key -> bool
val insert : dict -> key -> value -> unit
val remove : dict -> key -> unit
end
 M U TA B L E S TAT E A N D I M P E R AT I V E P R O G R A M M I N G 293

Let’s experiment:

# open IntStringHashtbl ;;
# let d = empty () ;;
val d : IntStringHashtbl.dict = <abstr>
# insert d 10 "ten" ;;
- : unit = ()
# insert d 9 "nine" ;;
- : unit = ()
# insert d 34 "34" ;;
- : unit = ()
# insert d 1000 "a thousand" ;;
- : unit = ()
# lookup d 10 ;;
- : IntStringHashtbl.value option = Some "ten"
# lookup d 9 ;;
- : IntStringHashtbl.value option = Some "nine"
# lookup d 34 ;;
- : IntStringHashtbl.value option = Some "34"
# lookup d 8 ;;
- : IntStringHashtbl.value option = None
# remove d 9 ;;
- : unit = ()
# lookup d 10 ;;
- : IntStringHashtbl.value option = Some "ten"
# lookup d 9 ;;
- : IntStringHashtbl.value option = None
# lookup d 34 ;;
- : IntStringHashtbl.value option = Some "34"
# lookup d 8 ;;
- : IntStringHashtbl.value option = None

Exercise 179
Complete the implementation by providing implementations of the remaining func-
tions lookup, member, insert, and remove.

Exercise 180
Improve the collision handling in the implementation by allowing the linear probing to
“wrap around” so that if it reaches the end of the array it keeps looking at the beginning
of the array.

Exercise 181
A problem with linear probing is that as collisions happen, contiguous blocks of the
array get filled up, so that further collisions tend to yield long searches to get past
these blocks for an empty location. Better is to use a method of rehashing that leaves
some gaps. A simple method to do so is QUA D R AT I C P R O B I N G : each probe increases
quadratically, adding 1, then 2, then 4, then 8, and so forth. Modify the implementation
so that it uses quadratic probing instead of linear probing.

15.7 Conclusion

With the introduction of references, we move from thinking about

what expressions mean to what they do. The ability to mutate state
294 PROGRAMMING WELL

means that data structures can now undergo change. By modifying

existing data structures, we may be able to avoid building new copies,
thereby saving some space. More importantly, performing small up-
dates may be much faster than constructing large copies, leading to
improvements in both space and time complexity.
But making good on these benefits requires much more subtle
reasoning about what programs are up to. The elegant substitution
model – which says that expressions are invariant under substitution
of one subexpression by another with the same value – doesn’t hold
when side effects can change those values out from under us. Aliasing
means that changes in one part of the code can have ramifications far
afield. Modifying data structures means that the hierarchical structures
can be modified to form cycles, with the potential to fall into infinite
loops. (We explore the changes needed to the substitution semantics
of Chapter 13 to allow for mutable state in Chapter 19.)
Nonetheless, the underlying structure of modern computer hard-
ware is based on stateful memory to store program and data, so that
at some point imperative programming is a necessity. Imperative pro-
gramming can be a powerful way of thinking about implementing
functionality.

We’ve now introduced essentially all of the basic language con-

structs that we need. In the following chapters, we deploy them in new
combinations that interact to provide additional useful programming
abstractions – providing looping constructs to enable the repetition of
side effects (Chapter 16); the ability to perform computation “lazily”,
delaying it until its result is needed (Chapter 17); and the encapsula-
tion of computations within data objects that they act on (Chapter 18).
16
Loops and procedural programming

Back in Section 7.3.1, we implemented a function to compute the

length of a list, by capturing how the length is defined: the length of
the empty list is 0; the length of a non-empty list is one more than the
length of its tail. This definition can be immediately cashed out as

# let rec length (lst : 'a list) : int =

# match lst with
# | [] -> 0
# | _hd :: tl -> 1 + length tl ;;
val length : 'a list -> int = <fun>

An alternative approach, in the spirit of imperative programming,

is to think not about what the length is but about what one does
when calculating the length: For each element of the list, add one to
a counter until the end of the list is reached.
This approach – which we might term P R O C E D U R A L P R O G R A M -
MING because it emphasizes the steps in the procedure to be carried
out – is typical of how introductory programming is taught, with an
emphasis on commands with side effects that are executed repeatedly
through loops.
In this chapter, we’ll provide examples of procedural programming,
emphasizing one of the main benefits of the paradigm, S PA C E E F F I -
C I E N C Y. Procedural programming can be more space efficient in a
couple of ways. First, it can reduce the need for storing suspended
computations in so-called “stack frames”, though as we’ll see, the func-
tional language technique of tail-recursion optimization can provide
this benefit as well. Second, it can reduce the need for copying data
structures as they are manipulated.
Although OCaml is at its core a functional programming language,
it supports procedural programming as well. There are, for instance,
while loops:
296 PROGRAMMING WELL

while ⟨exprcondition ⟩ do
⟨exprbody ⟩
done

which specify that the body expression be executed repeatedly so long

as the condition expression is true.
In addition, the for loop, familiar from other procedural languages,
is expressed as follows to count up from a start value to an end value:
for ⟨variable ⟩ = ⟨exprstart ⟩ to ⟨exprend ⟩ do
⟨exprbody ⟩
done

or, counting down,

for ⟨variable ⟩ = ⟨exprstart ⟩ downto ⟨exprend ⟩ do

⟨exprbody ⟩
done

16.1 Loops require impurity

In a pure language, an expression in a given context always has the

same value. Thus, in a while loop of the form
while ⟨exprcondition ⟩ do
⟨exprbody ⟩
done

if the condition expression ⟨exprcondition ⟩ is true the first time it’s eval-
uated, it will remain so perpetually and the loop will never terminate.
Conversely, if the condition expression is false the first time it’s eval-
uated, it will remain so perpetually and the loop body will never be
evaluated. Similarly, the body expression ⟨exprbody ⟩ will always evalu-
ate to the same value, so what could possibly be the point of evaluating
it more than once?
In summary, procedural programming only makes sense in a lan-
guage with side effects, the kind of impure constructs (like variable
assignment) that we introduced in the previous chapter. You can see
this need in attempting to implement the length function in this pro-
cedural paradigm. Here is a sketch of a procedure for calculating the
length of a list:

let length (lst : 'a list) : int =

(* initialize the counter *)
while (* the list is not empty *) do
(* increment the counter *)
(* drop an element from the list *)
done;
(* return the counter *) ;;
 LOOPS AND PROCEDURAL PROGRAMMING 297

We’ll need to establish the counter in such a way that its value can
change. Similarly, we’ll need to update the list each time the loop
body is executed. We’ll thus need both the counter and the list being
manipulated to be references, so that they can change. Putting all this
together, we get the following procedure for computing the length of a
list:
# let length_iter (lst : 'a list) : int =
# let counter = ref 0 in (* initialize the counter *)
# let lst_ref = ref lst in (* initialize the list *)
# while !lst_ref <> [] do (* while list not empty... *)
# incr counter; (* increment the counter *)
# lst_ref := List.tl !lst_ref (* drop element from list *)
# done;
# !counter ;; (* return the counter value *)
val length_iter : 'a list -> int = <fun>

# length_iter [1; 2; 3; 4; 5] ;;
- : int = 5

16.2 Recursion versus iteration

Is this impure, iterative, procedural method better than the pure,

recursive, functional approach? It certainly seems more complex,
and gaining an understanding that it provides the correct values as
specified in the definition of list length is certainly more difficult.

16.2.1 Saving stack space

There is one way, however, in which this approach might be supe-

rior. Think of the calculation of the length of a list, say [1; 2; 3],
using the functional definition. Since the list is non-empty, we need
to add one to the result of evaluating length [2; 3], and we’ll need
to suspend the addition until that evaluation completes. Likewise, to
evaluate length [2; 3] we’ll need to add one to the result of evaluat-
ing length [3], again suspending the addition until that evaluation
completes. Continuing on in this way, at run time we’ll eventually have length [1; 2; 3]
⇒ 1 + length [2; 3]
a nested stack of suspended calls. Each element of this stack, carry- ⇒ 1 + (1 + length [3])
ing information about the suspended computation, is referred to as a ⇒ 1 + (1 + (1 + length []))
⇒ 1 + (1 + (1 + 0))
S TA C K F R A M E . Only once we reach length [] can we start unwind-
⇒ 1 + (1 + 1)
ing this stack, performing all of the suspended additions specified in ⇒ 1 + 2
the stack frames, to calculate the final answer. Figure 16.1 depicts this ⇒ 3

linearly growing stack of suspended calls. Figure 16.1: The nested stack of sus-
The iterative approach, on the other hand, needs no stack of sus- pended calls in evaluating a non-tail-
recursive length function.
pended computations. The single call to length_iter invokes the
while loop to iteratively increment the counter and drop elements
from the list. The computation is “flat”.
298 PROGRAMMING WELL

The difference can be seen forcefully when computing the length of

a very long list. Here, we’ve defined very_long_list to be a list with
one million elements.

# let very_long_list = List.init 1_000_000 (fun x -> x) ;;

val very_long_list : int list = [0; 1; 2; 3; 4; 5; 6; 7; ...]

The iterative procedure for computing its length works well.

# length_iter very_long_list ;;
- : int = 1000000

But the functional recursive version overflows the stack dedicated to

storing the suspended computations. Apparently, one million stack
frames is more than the computer has space for.

# length very_long_list ;;
Stack overflow during evaluation (looping recursion?).

16.2.2 Tail recursion

The profligate use of space for stack frames is not inherent in all purely
functional recursive computations however. Consider the following
purely functional method length_tr for implementing the length
calculation.

# let length_tr lst =

# let rec length_plus lst acc =
# match lst with
# | [] -> acc
# | _hd :: tl -> length_plus tl (1 + acc) in
# length_plus lst 0 ;;
val length_tr : 'a list -> int = <fun>

Here, a local auxiliary function length_plus takes two arguments,

the list and an integer accumulator of the count of elements counted
so far. It returns the length of its list argument plus the value of its
accumulator. Thus, the call to length_plus lst 0 calculates the the
length of lst plus 0, which is just the length desired.
This length_tr version of calculating list length still operates re- length_tr [1; 2; 3]
⇒ length_plus [1; 2; 3] 0
cursively; length_plus is the locus of the recursion as indicated by ⇒ length_plus [2; 3] 1
the rec keyword. The nesting of recursive calls proceeds as shown in ⇒ length_plus [3] 2
⇒ length_plus [] 3
Figure 16.2.
⇒ 3
As with the previous recursive version, the number of such recursive
Figure 16.2: The call structure in
computations is linear in the length of the list. One might think, then,
evaluating a tail-recursive length
that the same problem of stack overflow will haunt the length_tr function. Note the lack of nesting of
implementation as well. Let’s try it. suspended calls.

# length_tr very_long_list ;;
- : int = 1000000
 LOOPS AND PROCEDURAL PROGRAMMING 299

This version doesn’t have the same problem. It’s easy to see why. For
the recursive length, the result of each call is a computation using the
result of the embedded call to length; that computation must there-
fore be suspended, and a stack frame must be allocated to store infor-
mation about that pending computation. But the result of each call to
the recursive length_plus is not just a computation using the result of
the embedded call to length_plus; it is the result of that nested call.
We don’t need to store any information about a suspended computa-
tion – no need to allocate a stack frame – because the embedded call
result is all that is needed.
Recursive programs written in this way, in which the recursive invo-
cation is the result of the invoking call, are deemed TA I L R E C U R S I V E
(hence the _tr in the function’s name). Tail-recursive functions need
not use a stack to keep track of suspended computations. Program-
ming language implementations that take advantage of this possibility
by not allocating a stack frame to tail-recursive applications are said to
perform TA I L - R E C U R S I O N O P T I M I Z AT I O N , effectively turning the re-
cursion into a corresponding iteration, and yielding the benefits of the
procedural iterative solution. The OCaml interpreter is such a language
implementation.
Thus, this putative advantage of loop-based procedures over recur-
sive functions – the ability to perform computations space-efficiently –
can often be replicated in functional style through careful tail-recursive
implementation where needed.
You’ll see discussion of this issue, for instance, in the description
of functions in the List library, which calls out those functions that
are not tail-recursive.1 For instance, the library function fold_left is 1
From the List library documentation:
“Some functions are flagged as not
implemented in a tail-recursive manner, so it can fold over very long
tail-recursive. A tail-recursive function
lists without running out of stack space. By contrast, the fold_right uses constant stack space, while a
implementation is not tail-recursive, so may not be appropriate when non-tail-recursive function uses stack
space proportional to the length of its
processing extremely long lists. list argument, which can be a problem
with very long lists. . . . The above
considerations can usually be ignored
16.3 Saving data structure space if your lists are not longer than about
10000 elements.”

Another advantage of procedural programming is the ability to avoid

building of new data structures. Think of the map function over lists,
which can be implemented as follows:

# let rec map (fn : 'a -> 'b) (lst : 'a list) : 'b list =
# match lst with
# | [] -> []
# | hd :: tl -> fn hd :: map fn tl ;;
val map : ('a -> 'b) -> 'a list -> 'b list = <fun>

We can use map to increment the values in a list:

300 PROGRAMMING WELL

# let original = [1; 2; 3] ;;

val original : int list = [1; 2; 3]
# map succ original ;;
- : int list = [2; 3; 4]

The result is a list with different values. Most notably, the result is a new
list. The original is unchanged.

# original ;;
- : int list = [1; 2; 3]

The new list is created by virtue of the repeated construction of

conses with the :: operator highlighted in the map definition above.
Every time map is called to operate over a list, more conses will be
needed. There’s no free lunch here. Under the hood, every cons takes
up space; storage must be allocated for each one. If we start with a list
of length n, we’ll end up allocating n more conses to compute the map.

16.3.1 Problem section: Metering allocations

We can determine how many allocations are going on by metering

them. Imagine there were a module Metered satisfying the following
signature:

# module type METERED =

# sig
# (* reset () -- Resets the count of allocations *)
# val reset : unit -> unit
# (* count () -- Returns the number of allocations
# since the last reset *)
# val count : unit -> int
# (* cons hd tl -- Returns the list cons of `hd` and
# `tl`, increasing the allocation count accordingly *)
# val cons : 'a -> 'a list -> 'a list
# (* pair first second -- Returns the pair of `first`
# and `second`, increasing the allocation count
# accordingly *)
# val pair : 'a -> 'b -> 'a * 'b
# end ;;
module type METERED =
sig
val reset : unit -> unit
val count : unit -> int
val cons : 'a -> 'a list -> 'a list
val pair : 'a -> 'b -> 'a * 'b
end

The functions cons and pair could be used to replace their built-in
counterparts for consing (::) and pairing (,) to track the number of
allocations required.
Problem 182
Implement the module Metered.
 LOOPS AND PROCEDURAL PROGRAMMING 301

Problem 183
Reimplement the zip function of Section 10.3.2 using metered conses and pairs.

Having metered the zip function, we can observe the count of

allocations.

# Metered.reset () ;;
- : unit = ()
# zip [1; 2; 3; 4; 5] [5; 4; 3; 2; 1] ;;
- : (int * int) list = [(1, 5); (2, 4); (3, ...); ...]
# Metered.count () ;;
- : int = 10

16.3.2 Reusing space through mutable data structures

Now consider, by contrast to the functional map over lists above, a

function (call it map_array) to map a function over a mutable data
structure, an array. Instead of returning a new data structure, we’ll
mutate the values in the original data structure. For that reason, map_-
array doesn’t itself need to return an array. 2 2
Notice that the function being applied
must be of type ’a -> ’a since the
# let map_array (fn : 'a -> 'a) (arr : 'a array) : unit = output of the function is being stored
# for i = 0 to Array.length arr - 1 do
in the same location as the input, and
must thus be of the same type. For
# arr.(i) <- fn arr.(i)
that reason, map_array can’t be as
# done ;;
polymorphic as map.
val map_array : ('a -> 'a) -> 'a array -> unit = <fun>

We can perform a similar computation, mapping the successor func-

tion over the elements of an array.

# let original = [|1; 2; 3|] ;;

val original : int array = [|1; 2; 3|]
# map_array succ original ;;
- : unit = ()

We see the effect of the map this time not in the return value but in the
modified original array.

# original ;;
- : int array = [|2; 3; 4|]

By using imperative techniques, we gain access to the incremented

values, and without incurring the cost of allocating further storage.
There is a cost, however. We no longer have access to the original
unincremented values. They’ve been destroyed, replaced by the new
values. Under what conditions the tradeoff – reduced storage versus
loss of access to prior results – is a judgement call. But as an issue
of efficiency, we’d want to heed Knuth’s warning against premature
optimization.
302 PROGRAMMING WELL

16.4 In-place sorting

As another example of the use of procedural programming to reduce

storage requirements, we consider one of the most elegant sorting
algorithms, QU I C K S O RT . Quicksort works by selecting a pivot value
– the first element of the list, say – and partitioning the list into those
elements less than the pivot and those that are greater. The two sub-
lists are recursively sorted, and then concatenated to form the final
sorted list. A recursive implementation of quicksort, following the SORT
signature of Section 14.2, is as follows:
# module QuickSort : SORT =
# struct
# (* partition lt pivot xs -- Returns two lists
# constituting all elements in `xs` less than (according
# to `lt`) than the `pivot` value and greater than the
# pivot `value`, respectively *)
# let rec partition lt pivot xs =
# match xs with
# | [] -> [], []
# | hd :: tl ->
# let first, second = partition lt pivot tl in
# if lt hd pivot then hd :: first, second
# else first, hd :: second
#
# (* sort lt xs -- Quicksorts `xs` according to the comparison
# function `lt` *)
# let rec sort (lt : 'a -> 'a -> bool)
# (xs : 'a list)
# : 'a list =
# match xs with
# | [] -> []
# | pivot :: rest ->
# let first, second = partition lt pivot rest in
# sort lt first @ [pivot] @ sort lt second
# end ;;
module QuickSort : SORT

Problem 184
Implement a metered version of quicksort, and experiment with how many allocations
are needed to sort lists of different lengths.

Just as we built a version of map that mutated an array to map over

its elements, we can build a version of quicksort that mutates an array
to sort its elements. This approach, I N - P L A C E sorting, is much more
space-efficient. As we’ll see, though, there is a cost in transparency of
the implementation.
The type for an in-place sort differs from its pure alternative, which
allocates extra space. A signature for an in-place sorting module makes
clear the differences.
# module type SORT_IN_PLACE =
# sig
 LOOPS AND PROCEDURAL PROGRAMMING 303

# (* sort lt xs -- Sorts the array `xs` in place in increasing

# order by the "less than" function `lt`. *)
# val sort : ('a -> 'a -> bool) -> 'a array -> unit
# end ;;
module type SORT_IN_PLACE =
sig val sort : ('a -> 'a -> bool) -> 'a array -> unit end

First, we’re sorting a mutable data structure, an array, rather than a

list. Second, the sort function returns a unit as it works by side effect
rather than by returning a sorted version of the unchanged argument
list. The sorting function, then, begins with a header line

let sort (lt : 'a -> 'a -> bool) (arr : 'a array) : unit =

The primitive operation of in-place sorting is the swapping of two

elements in the array, specified by their indices. We’ll make use of a
function swap to perform this operation.

let swap (i : int) (j : int) : unit =

let temp = arr.(i) in
arr.(i) <- arr.(j);
arr.(j) <- temp

We’ll need to partition a region of the array, by which we mean a

contiguous subportion of the array between two indices. For that pur-
pose, we’ll have a function partition that takes two indices (left and
right) demarcating the region to partition (the elements between the
indices inclusive). The partition function returns the index of the
split point in the region, the position that marks the border between
the left partition and the right partition where the pivot element re-
sides. We note that for our purposes, there should and will always be
at least two elements in the region; otherwise, no recursive sorting is
necessary, hence no need to partition.
To partition the region, we select the leftmost element as the pivot.
We keep a “current” index that moves from left to right as we process
each element in the region. At the same time, we maintain a moving
“border” index, again moving from left to right. At any point, all of the
elements to the left of the border will be guaranteed to be less than
the pivot value. Those between the border and the current index are
greater than or equal to the pivot. Those to the right of the current
index are yet to be processed. Eventually, when we’ve processed all
elements, we swap the pivot element itself into the correct position at
the border. Here’s the implementation of this quite subtle process:

let partition (left : int) (right : int) : int =

(* region has at least two elements *)

assert (left < right);
304 PROGRAMMING WELL

(* select the pivot element to be the first element in

the region *)
let pivot_val = arr.(left) in
(* all elements to the left of `border` are guaranteed
to be strictly less than pivot value *)
let border = ref (left + 1) in
(* current element being partitioned, starting just
after pivot *)
let current = ref (left + 1) in

(* process each element, moving those less than the

pivot to before the border *)
while !current <= right do
if lt arr.(!current) pivot_val then
begin
(* current should be left of pivot *)
swap !current !border; (* swap into place at border *)
incr border (* move border to the right to make room *)
end;
incr current
done

(* the split point is just to left of the border *)

let split = !border - 1 in
(* move pivot into place at the split point *)
swap left split;
(* return the split index *)
split

With the availability of the partition function, we can implement

a function sort_region to sort a region, again picked out by two
indices.
let rec sort_region (left : int) (right : int) : unit =
if left >= right then ()
else
let split = partition left right in
(* recursively sort left and right regions *)
sort_region left (split - 1);
sort_region (split + 1) right

Finally, to sort the entire array, we can sort the region between the
leftmost and rightmost indices
sort_region 0 ((Array.length arr) - 1)

Putting this all together leads to the implementation shown in Fig-

ure 16.3. (We’ve placed the swap and partition functions within
the sort function so that they are within the scope of (and can thus
access) the lt and arr arguments of sort.)
You’ll note that the in-place quicksort is considerably longer than
the pure version. In part that is because of the much more detailed
work that must be done in partitioning a region, maintaining complex
 LOOPS AND PROCEDURAL PROGRAMMING 305

module QuickSort : SORT_IN_PLACE = Figure 16.3: Implementation of an

struct in-place quicksort.
let sort (lt : 'a -> 'a -> bool) (arr : 'a array) : unit =

(* swap i j -- Update the `arr` array by swapping the

elements at indices `i` and `j` *)
let swap (i : int) (j : int) : unit =
let temp = arr.(i) in
arr.(i) <- arr.(j);
arr.(j) <- temp in

(* partition left right -- Partition the region of the

`arr` array between indices `left` and `right`
*inclusive*, returning the split point, that is, the
index of the pivot element. Assumes the region
contains at least two elements. At the end,
everything to left of the split is less than the
pivot; everything to the right is greater. *)
let partition (left : int) (right : int) : int =

(* region has at least two elements *)

assert (left < right);

(* select the pivot element to be the first element in

the region *)
let pivot_val = arr.(left) in
(* all elements to the left of `border` are guaranteed
to be strictly less than pivot value *)
let border = ref (left + 1) in
(* current element being partitioned, starting just
after pivot *)
let current = ref (left + 1) in
306 PROGRAMMING WELL

(* process each element, moving those less than the Figure 16.3: (continued) Implementa-
pivot to before the border *) tion of an in-place quicksort.
while !current <= right do
if lt arr.(!current) pivot_val then
begin
(* current should be left of pivot *)
swap !current !border; (* swap into place *)
incr border (* move border right to make room *)
end;
incr current
done;

(* the split point is just to left of the border *)

let split = !border - 1 in
(* move pivot into place at the split point *)
swap left split;
(* return the split index *)
split in

(* sort_region left right -- quicksort the subarray of

the `arr` array between indices `left` and `right`
*inclusive* *)
let rec sort_region (left : int) (right : int) : unit =
if left >= right then ()
else
let split = partition left right in
(* recursively sort left and right regions *)
sort_region left (split - 1);
sort_region (split + 1) right
in

(* sort the whole `arr` array *)

sort_region 0 ((Array.length arr) - 1)
end
 LOOPS AND PROCEDURAL PROGRAMMING 307

invariants concerning the left, right, current, and border indices and
the elements in the various subregions. In part the length is a result
of considerably more documentation in the implementation, but that
is not a coincidence. The implementation requires this additional
documentation to be remotely as understandable as the pure version.
(Even still, an understanding of the in-place version is arguably more
complex. It’s hard to imagine understanding how the partition func-
tion works without manually “playing computer” on some examples to
verify the procedure.)
The payoff is that the in-place version needs to allocate only a tiny
amount of space beyond the storage in the various stack frames for the
function applications – just the storage for the current and border
elements. Is the cost in code complexity and opaqueness worth it?
That depends on the application. If sorting huge amounts of data is
necessary, the reduction in space may be needed.
Problem 185
A completely in-place version of mergesort that uses only a fixed amount of extra space
turns out to be quite tricky to implement. However, a version that uses only a single
extra array is possible, and still more space-efficient than the pure version described in
Section 14.2. Implement a version of mergesort that uses a single extra array as “scratch
space” for use while merging. To sort a region, we sort the left and right subregions
recursively, then merge the two into the scratch array, and finally copy the merged region
back into the main array.
17
Infinite data structures and lazy programming

Combining functions as first-class values, algebraic data types, and

references enables programming with infinite data structures, the
surprising topic of this chapter. We’ll build infinite lists (streams) and
infinite trees. The primary technique we use, lazy evaluation, has many
other applications.

17.1 Delaying computation

OCaml is an E A G E R language. Recall the semantic rule for function

application from Chapter 13:

P Q⇓
RRR P ⇓ fun x -> B
RRR
RRR Q ⇓ v (R app )
RRR Q
RRR B [x ↦ v ] ⇓ v
R Q B

⇓ vB

According to this rule, before generating the result of the application

(by substituting into the body expression B ), we first evaluate the
argument Q. Similarly, in a local let expression,

let x = D in B ⇓

D ⇓ vD
∣ (R let )
B [x ↦ v D ] ⇓ v B

⇓ vB

before substituting the definition D into the body expression B , we first

evaluate D to a value.
There are disadvantages of this eager evaluation approach. For
instance, if the argument value is not used in the body of the function,
the computation to generate the value will still be carried out, an
310 PROGRAMMING WELL

entirely wasted effort. An extreme case occurs when the computation

of the argument value doesn’t even terminate:
# let rec forever n = 1 + forever n ;;
val forever : 'a -> int = <fun>
# (fun x -> "this value ignores x") (forever 42) ;;
Line 1, characters 5-6:
1 | (fun x -> "this value ignores x") (forever 42) ;;
^
Warning 27: unused variable x.
Stack overflow during evaluation (looping recursion?).

If we had delayed the computation of forever 42 until after it had

been substituted in as the argument of the function, we would never
have had to evaluate it at all, and the evaluation of the full expression
would have terminated with "this value ignores x".
Examples like this indicate the potential utility of L A Z Y E VA L U -
AT I O N – being able to D E L AY computation until such time as it is
needed, at which time the computation can be F O R C E D to occur.
There are, in fact, constructs of OCaml that work lazily. The condi-
tional expression if ⟨exprtest ⟩ then ⟨exprtrue ⟩ else ⟨exprfalse ⟩ delays
evaluation of ⟨exprtrue ⟩ and ⟨exprfalse ⟩ until after evaluating ⟨exprtest ⟩,
and in fact will refrain from evaluating the unchosen branch of the
conditional entirely. Thus the following computation terminates, even
though the else branch, if it were evaluated, would not.
# if true then 3 else forever 42 ;;
- : int = 3

Another construct that delays computation is the function itself.

The body of a function is not evaluated until the function is applied.
If application is postponed indefinitely, the body is never evaluated.
Thus the following “computation” terminates.
# fun () -> forever 42 ;;
- : unit -> int = <fun>

This latter approach provides a universal method for delaying and

forcing computations: wrapping the computation in a function (delay),
applying the function (forcing) if and when we need the value. What
should the argument to the function be? Its only role is to postpone
evaluation, so there needn’t be a real datum as argument – just a unit.
As noted above, we refer to this wrapping a computation in a function
from unit as delay of the computation. Conversely, we force the com-
putation when the delayed expression is applied to unit so as to carry
out the computation and get the value.
Though OCaml is eager in its evaluation strategy (with the few ex-
ceptions noted), some languages have embraced lazy evaluation as
the default, starting with Rod Burstall’s Hope language and finding
 I N F I N I T E D ATA S T RU C T U R E S A N D L A Z Y P R O G R A M M I N G 311

its widest use in the Haskell language named after Haskell Curry (Fig-
ure 6.2).
We’ll make use of lazy evaluation in perhaps the most counter-
intuitive application, the creation and manipulation of infinite data
structures. We start with the stream, a kind of infinite list.

17.2 Streams

Here’s a new algebraic data type definition for the S T R E A M .

# type 'a stream = Cons of 'a * 'a stream ;;
type 'a stream = Cons of 'a * 'a stream

It may look familiar; it shares much in common with the algebraic

type definition of the polymorphic list, from Section 11.1, except that it
dispenses with the Nil marking the end of the list.
We can define some operations on streams, like taking the head or
tail of a stream.
# let head (Cons (hd, _tl) : 'a stream) : 'a = hd ;;
val head : 'a stream -> 'a = <fun>

# let tail (Cons (_hd, tl) : 'a stream) : 'a stream = tl ;;

val tail : 'a stream -> 'a stream = <fun>

It’s all well and good to have streams and functions over them,
but how are we to build one? It looks like we have a chicken and egg
problem, requiring a stream in order to create one. Nonetheless, we
press on, building a stream whose head is the integer 1. We start with
let ones = Cons (1, ...) ;;

We need to fill in the ... with an int stream, but where are we to find
one? How about the int stream named ones itself?
# let ones = Cons (1, ones) ;;
Line 1, characters 20-24:
1 | let ones = Cons (1, ones) ;;
^^^^
Error: Unbound value ones

Of course, that doesn’t work, because the name ones isn’t itself avail-
able in the definition. That requires a let rec.
# let rec ones = Cons (1, ones) ;;
val ones : int stream = Cons (1, <cycle>)

It works! And the operations on this stream work as well:

# head ones ;;
- : int = 1
# head (tail ones) ;;
- : int = 1
# head (tail (tail ones)) ;;
- : int = 1
312 PROGRAMMING WELL

Its head is one, as is the head of its tail, and the head of the tail of the
tail. It seems to be an infinite sequence of ones!
What is going on here? How does the implementation make this
possible? Under the hood, the components of an algebraic data type
have implicit pointers to their values. When we define ones as above,
OCaml allocates space for the cons without initializing it (yet) and
connects the name ones to it. It then initializes the contents of the
cons, the head and tail, a pair of hidden pointers. The head pointer
points to the value 1, and the tail points to the cons itself. This explains
where the notation <cycle> comes from in the R E P L printing out the
value. In any case, the details of how this behavior is implemented isn’t
necessary to make good use of it.
Not all such cyclic definitions are well defined however. Consider
this definition of an integer x:

# let rec x = 1 + x ;;
Line 1, characters 12-17:
1 | let rec x = 1 + x ;;
^^^^^
Error: This kind of expression is not allowed as right-hand side of
`let rec'

We can allocate space for the integer and name it x, but when it comes
to initializing it, we need more than just a pointer to x; we need its
value. But that isn’t yet defined, so the whole process fails and we get
an error message.

17.2.1 Operations on streams

We can look to lists for inspiration for operations on streams, opera-

tions like map, fold, and filter. Here is a definition for map on streams,
which we call smap:

# let rec smap (f : 'a -> 'b) (s : 'a stream) : ('b stream) =
# match s with
# | Cons (hd, tl) -> Cons (f hd, smap f tl) ;;
val smap : ('a -> 'b) -> 'a stream -> 'b stream = <fun>

or, alternatively, using our recent definitions of head and tail,

# let rec smap (f : 'a -> 'b) (s : 'a stream) : ('b stream) =
# Cons (f (head s), smap f (tail s)) ;;
val smap : ('a -> 'b) -> 'a stream -> 'b stream = <fun>

Now, we map the successor function over the stream of ones to form a
stream of twos.

# let twos = smap succ ones ;;

Stack overflow during evaluation (looping recursion?).
 I N F I N I T E D ATA S T RU C T U R E S A N D L A Z Y P R O G R A M M I N G 313

Of course, that doesn’t work at all. We’re asking OCaml to add one to
each element in an infinite sequence of ones. Luckily, smap isn’t tail
recursive, so we blow the stack, instead of just hanging in an infinite
loop. This behavior makes streams as currently implemented less
than useful since there’s little we can do to them without getting into
trouble. If only the system were less eager about doing all those infinite
number of operations, doing them only if it “needed to”.
The problem is that when calculating the result of the map, we need
to generate (and cons together) both the head of the list (f (head s))
and the tail of the list (smap f (tail s)). But the tail already involves
calling smap.
Why isn’t this a problem in calling regular recursive functions, like
List.map? In that case, there’s a base case that is eventually called.
Why isn’t this a problem in defining regular recursive functions?
Why is there no problem in defining, say,
let rec fact n =
if n = 0 then 1
else n * fact (pred n) ;;

Recall that this definition is syntactic sugar for

let rec fact =
fun n ->
if n = 0 then 1
else n * fact (pred n) ;;

The name fact can be associated with a function that uses it because
functions are values. The parts inside are not further evaluated, at least
not until the function is called. In essence, a function delays the latent
computation in its body until it is applied to its argument.
We can take advantage of that in our definition of streams by using
functions to perform computations lazily. We achieve laziness by
wrapping the computation in a function delaying the computation
until such time as we need the value. We can then force the value by
applying the function.
To achieve the delay of computation, we’ll take a stream not to
be a cons as before, but a delayed cons, a function from unit to the
cons. Other functions that need access to the components of the de-
layed cons can force it as needed. We need a new type definition for
streams, which will make use of a simultaneously defined auxiliary
type stream_internal:1 1
The and connective allows mutually re-
cursive type definitions. Unfortunately,
# type 'a stream_internal = Cons of 'a * 'a stream OCaml doesn’t allow direct definition of
# and 'a stream = unit -> 'a stream_internal ;; nested types, like
type 'a stream_internal = Cons of 'a * 'a stream type 'a stream = unit -> (Cons of 'a * 'a stream)
and 'a stream = unit -> 'a stream_internal

An infinite stream of ones is now defined as so:

314 PROGRAMMING WELL

# let rec ones : int stream =

# fun () -> Cons (1, ones) ;;
val ones : int stream = <fun>

Notice that it returns a delayed cons, that is, a function which, when
applied to a unit, returns the cons.
We need to redefine the functions accordingly to take and return
these new lazy streams. In particular, head and tail now force their
argument by applying it to unit.
# let head (s : 'a stream) : 'a =
# match s () with
# | Cons (hd, _tl) -> hd ;;
val head : 'a stream -> 'a = <fun>

# let tail (s : 'a stream) : 'a stream =

# match s () with
# | Cons (_hd, tl) -> tl ;;
val tail : 'a stream -> 'a stream = <fun>

# let rec smap (f : 'a -> 'b) (s : 'a stream) : ('b stream) =
# fun () -> Cons (f (head s), smap f (tail s)) ;;
val smap : ('a -> 'b) -> 'a stream -> 'b stream = <fun>

The smap function now returns a lazy stream, a function, so that the
recursive call to smap isn’t immediately evaluated (as it was in the
old definition). Only when the cons is needed (as in the head or tail
functions) is the function applied and the cons constructed. That cons
itself has a stream as its tail, but that stream is also delayed.
Now, finally, we can map the successor function over the infinite
stream of ones to form an infinite stream of twos.
# let twos = smap succ ones ;;
val twos : int stream = <fun>
# head twos ;;
- : int = 2
# head (tail twos) ;;
- : int = 2
# head (tail (tail twos)) ;;
- : int = 2

We can convert a stream – or at least the first n of its infinity of

elements – into a corresponding list,
# let rec first (n : int) (s : 'a stream) : 'a list =
# if n = 0 then []
# else head s :: first (n - 1) (tail s) ;;
val first : int -> 'a stream -> 'a list = <fun>

allowing us to examine the first few elements of the streams we have

constructed:
# first 10 ones ;;
- : int list = [1; 1; 1; 1; 1; 1; 1; 1; 1; 1]
 I N F I N I T E D ATA S T RU C T U R E S A N D L A Z Y P R O G R A M M I N G 315

# first 10 twos ;;
- : int list = [2; 2; 2; 2; 2; 2; 2; 2; 2; 2]

So far, we’ve constructed a few infinite streams, but none of much

interest. But the tools are in hand to do much more. Think of the natu-
ral numbers: 0, 1, 2, 3, 4, 5, . . .. What is this sequence? We can think of it Start with the natural numbers
0 1 2 3 4 5 6 7 ...
as the sequence formed by taking the natural numbers, incrementing Increment them
them all to form the sequence 1, 2, 3, 4, 5, 6, . . ., and then prepending a 1 2 3 4 5 6 7 8 ...
Prepend a zero
zero to the front, as depicted in Figure 17.1. 0 1 2 3 4 5 6 7 8 ...
We’ll define a stream called nats in just this way.
Figure 17.1: Creating an infinite stream
# let rec nats = of natural numbers by taking the natural
# fun () -> Cons (0, smap succ nats) ;;
numbers, incrementing them, and
prepending a zero.
val nats : int stream = <fun>
# first 10 nats ;;
- : int list = [0; 1; 2; 3; 4; 5; 6; 7; 8; 9]

Let’s just pause for a moment to let that sink in.

A function to map over two streams simultaneously, like the
List.map2 function, allows even more powerful ways of building Start with the Fibonacci sequence
streams. 0 1 1 2 3 5 8 ...
Take its tail
# let rec smap2 f s1 s2 = 1 1 2 3 5 8 13 ...
Sum them
# fun () -> Cons (f (head s1) (head s2),
1 2 3 5 8 13 21 ...
# smap2 f (tail s1) (tail s2)) ;;
Prepend a zero and one
val smap2 : ('a -> 'b -> 'c) -> 'a stream -> 'b stream -> 'c stream
0 1 1 2 3 5 8 13 21 ...
= <fun>
Figure 17.2: Creating an infinite stream
We can, for instance, generate the Fibonacci sequence (see Exercise 32) of the Fibonacci numbers.

in this way. Figure 17.2 gives the recipe.

# let rec fibs =

# fun () -> Cons (0,
# fun () -> Cons (1,
# (smap2 (+) fibs (tail fibs)))) ;;
val fibs : int stream = <fun>

Here, we’ve timed generating the first 10 elements of the sequence.

It’s slow, but it works.

# Absbook.call_reporting_time (first 10) fibs ;;

time (msecs): 2.663136
- : int list = [0; 1; 1; 2; 3; 5; 8; 13; 21; 34]

17.3 Lazy recomputation and thunks

Recall the definition of streams:

type 'a stream_internal = Cons of 'a * 'a stream

and 'a stream = unit -> 'a stream_internal ;;
316 PROGRAMMING WELL

Every time we want access to the head or tail of the stream, we need
to rerun the function. In a computation like the Fibonacci defini-
tion above, that means that every time we ask for the n-th Fibonacci
number, we recalculate all the previous ones – more than once. But
if the value being forced is pure, without side effects, there’s no rea-
son to recompute it. We should be able to avoid the recomputation
by remembering its value the first time it’s computed, and using the
remembered value from then on. The term of art for this technique is
2
M E M O I Z AT I O N . 2
Not “memorization”. For unknown
reasons, computer scientists have
We’ll encapsulate this idea in a new abstraction called a T H U N K ,
settled on this bastardized form of the
essentially a delayed computation that stores its value upon being word.
forced. We implement a thunk as a mutable value (a reference) that
can be in one of two states: not yet evaluated or previously evaluated.
The type definition is thus structured with two alternatives.

# type 'a thunk = 'a thunk_internal ref

# and 'a thunk_internal =
# | Unevaluated of (unit -> 'a)
# | Evaluated of 'a ;;
type 'a thunk = 'a thunk_internal ref
and 'a thunk_internal = Unevaluated of (unit -> 'a) | Evaluated of
'a

Notice that in the unevaluated state, the thunk stores a delayed value
of type ’a. Once evaluated, it stores an immediate value of type ’a.
When we need to access the actual value encapsulated in a thunk,
we’ll use the force function. If the thunk has been forced before and
thus evaluated, we simply retrieve the value. Otherwise, we compute
the value, remember it by changing the state of the thunk to be evalu-
ated, and return the value.

# let rec force (t : 'a thunk) : 'a =

# match !t with
# | Evaluated v -> v
# | Unevaluated f ->
# t := Evaluated (f ());
# force t ;;
val force : 'a thunk -> 'a = <fun>

Here’s a thunk for a computation of, say, factorial of 15. To make the
timing clearer, we’ll give it a side effect of printing a short message.

# let fact15 =
# ref (Unevaluated (fun () ->
# print_endline "evaluating 15!";
# fact 15)) ;;
val fact15 : int thunk_internal ref = {contents = Unevaluated
<fun>}

which can be forced to carry out the calculation:

 I N F I N I T E D ATA S T RU C T U R E S A N D L A Z Y P R O G R A M M I N G 317

# Absbook.call_reporting_time force fact15 ;;

evaluating 15!
time (msecs): 0.015974
- : int = 1307674368000

Now that the value has been forced, it is remembered in the thunk
and can be returned without recomputation. You can tell that no
recomputation occurs because the printing side effect doesn’t happen,
and the computation takes orders of magnitude less time.
# fact15 ;;
- : int thunk_internal ref = {contents = Evaluated 1307674368000}
# Absbook.call_reporting_time force fact15 ;;
time (msecs): 0.000954
- : int = 1307674368000

17.3.1 The Lazy Module

Thunks give us the ability to delay computation, force a delayed com-

putation, and memoize the result. But the notation is horribly cum-
bersome. Fortunately, OCaml provides a module and some appropri-
ate syntactic sugar for working with lazy computation implemented
through thunks – the Lazy module.
In the built-in Lazy module, the type of a delayed computation
of an ’a value is given not by ’a thunk but by ’a Lazy.t. A de-
layed computation is specified not by wrapping the expression in
ref (Unevaluated (fun () -> ...)) but by preceding it with the
new keyword lazy. Finally, forcing a delayed value uses the function
Lazy.force.
Availing ourselves of the Lazy module, we can perform the same
experiment more simply:
# let fact15 =
# lazy (print_endline "evaluating 15!";
# fact 15) ;;
val fact15 : int lazy_t = <lazy>
# Lazy.force fact15 ;;
evaluating 15!
- : int = 1307674368000
# Lazy.force fact15 ;;
- : int = 1307674368000

Now we can reconstruct infinite streams using the Lazy module.

First, the stream type:
# type 'a stream_internal = Cons of 'a * 'a stream
# and 'a stream = 'a stream_internal Lazy.t ;;
type 'a stream_internal = Cons of 'a * 'a stream
and 'a stream = 'a stream_internal Lazy.t

Functions on streams will need to force the stream values. Here, for
instance, is the head function:
318 PROGRAMMING WELL

let head (s : 'a stream) : 'a =

match Lazy.force s with
| Cons (hd, _tl) -> hd ;;

Exercise 186
Rewrite tail, smap, smap2, and first to use the Lazy module.
The Fibonacci sequence can now be reconstructed. It runs hun-
dreds of times faster than the non-memoized version in Section 17.2.1:
# let rec fibs =
# lazy (Cons (0,
# lazy (Cons (1,
# smap2 (+) fibs (tail fibs))))) ;;
val fibs : int stream = <lazy>
# Absbook.call_reporting_time (first 10) fibs ;;
time (msecs): 0.006199
- : int list = [0; 1; 1; 2; 3; 5; 8; 13; 21; 34]

17.4 Application: Approximating π

A nice application of infinite streams is in the numerical approxima-

tion of the value of π. In 1715, the English mathematician Brook Taylor
showed how to approximate functions as an infinite sum of terms, a
technique we now call T AY L O R S E R I E S . For instance, the trigonomet-
ric arctangent function can be approximated by the following infinite
sum: Figure 17.3: English mathematician
Brook Taylor (1685–1731), inventor
x3 x5 x7 of the Taylor series approximation of
arctan x = x − + − +⋯ functions.
3 5 7
π
As a special case, the arctangent of 1 is 4
(Figure 17.4). So

π 1 1 1
= 1− + − +⋯
4 3 5 7
and
4 4 4
π = 4− + − +⋯ . Figure 17.4: The arctangent of 1, that
3 5 7 is, the angle whose ratio of opposite to
We can thus approximate π by adding up the terms in this infinite adjacent side is 1, is a 45 degree angle,
or π 4
in radians.
stream of numbers.
We start with a function to convert a stream of integers to a stream
of floats.
# let to_float = smap float_of_int ;;
val to_float : int stream -> float stream = <fun>

Next, we build a stream of odd integers to serve as the denominators in

all the terms in the Taylor series:
# let odds = smap (fun x -> x * 2 + 1) nats ;;
val odds : int stream = <lazy>
 I N F I N I T E D ATA S T RU C T U R E S A N D L A Z Y P R O G R A M M I N G 319

and a stream of alternating positive and negative ones to represent the

alternate adding and subtracting:

# let alt_signs =
# smap (fun x -> if x mod 2 = 0 then 1 else -1) nats ;;
val alt_signs : int stream = <lazy>

Finally, the stream of terms in the π sequence is

# let pi_stream = smap2 ( /. )

# (to_float (smap (( * ) 4) alt_signs))
# (to_float odds) ;;
val pi_stream : float stream = <lazy>

A check of the first few elements in these streams verifies them:

# first 5 odds ;;
- : int list = [1; 3; 5; 7; 9]
# first 5 alt_signs ;;
- : int list = [1; -1; 1; -1; 1]
# first 5 pi_stream ;;
- : float list =
[4.; -1.33333333333333326; 0.8; -0.571428571428571397;
0.44444444444444442]

Now that we have an infinite stream of terms, we can approximate

π by taking the sum of the first few elements of the stream, a PA RT I A L
S U M. The function pi_approx extracts the first n elements of the
stream and sums them up using a fold.

# let pi_approx n =
# List.fold_left ( +. ) 0.0 (first n pi_stream) ;;
val pi_approx : int -> float = <fun> The given sequence
# pi_approx 10 ;; 1 2 3 4 5 6 7
- : float = 3.04183961892940324 ...
# pi_approx 100 ;; . . . and its partial sums
- : float = 3.13159290355855369 1 3 6 10 15 21 28
# pi_approx 1000 ;; ...
Prepend a zero to the partial sums
- : float = 3.14059265383979413
0 1 3 6 10 15 21 28
# pi_approx 10000 ;;
...
- : float = 3.14149265359003449 . . . plus the original sequence
# pi_approx 100000 ;; 1 2 3 4 5 6 7 8
- : float = 3.14158265358971978 ...
. . . yields the partial sums
After 100,000 terms, we have a pretty good approximation of π, good to 1 3 6 10 15 21 28 36
about four decimal places. ...
Of course, this technique of partial sums isn’t in the spirit of infinite Figure 17.5: Creating an infinite stream
of partial sums of a given stream, in this
streams. Better would be to generate an infinite stream of all of the
case, the stream of positive integers.
partial sums. Figure 17.5 gives a recipe for generating a stream of We prepend a zero to the sequence’s
partial sums from a given stream. Starting with the stream, we take its partial sums and add in the original
sequence to generate the sequence
partial sums (!) and prepend a zero. Adding the original stream and the of partial sums. Only by virtue of lazy
prepended partial sums stream yields. . . the partial sums stream. This computation can this possibly work.
technique, implemented as a function over streams, is:
320 PROGRAMMING WELL

# let rec sums s =

# smap2 ( +. ) s (lazy (Cons (0.0, sums s))) ;;
val sums : float stream -> float stream = <fun>

Now the first few approximations of π are easily accessed:

# let pi_approximations = sums pi_stream ;;

val pi_approximations : float stream = <lazy>
# first 5 pi_approximations ;;
- : float list =
[4.; 2.66666666666666696; 3.46666666666666679; 2.89523809523809561;
3.33968253968254025]

If we want to find an approximation within a certain tolerance, say

², we can search for two terms in the stream of approximations whose
difference is less than ².

# let rec within epsilon s =

# let hd, tl = head s, tail s in
# if abs_float (hd -. (head tl)) < epsilon then hd
# else within epsilon tl ;;
val within : float -> float stream -> float = <fun>

We can now search for a value accurate to within any number of digits
we desire:

# within 0.01 pi_approximations ;;

- : float = 3.13659268483881615
# within 0.001 pi_approximations ;;
- : float = 3.14109265362104129

Continuing on in this vein, we might explore methods for S E R I E S

A C C E L E R AT I O N – techniques to cause series to converge more quickly
– or apply infinite streams to other applications. (In fact, series ac-
celeration is the subject of Section 17.8.2.) But for now, this should
be sufficient to give a sense of the power of computing with infinite
streams.

Exercise 187
As mentioned in Exercise 32, the ratios of successive numbers in the Fibonacci sequence
approach the golden ratio (1.61803 . . .). Show this by generating a stream of ratios of
successive Fibonacci numbers and use it to calculate the golden ratio within 0.000001.

17.5 Problem section: Circuits and boolean streams

A boolean circuit is a device with one or more inputs and a single

output that receives over time a sequence of boolean values on its
inputs and converts them to a corresponding sequence of boolean
values on its output. The building blocks of circuits are called gates.
For instance, the and gate is a boolean device with two inputs; its
output is true when its two inputs are both true, and false if either
 I N F I N I T E D ATA S T RU C T U R E S A N D L A Z Y P R O G R A M M I N G 321

output is false. The not gate is a boolean device with a single input; its
output is true when its input is false and vice versa.
In this problem, you’ll generate code for modeling boolean circuits.
The inputs and outputs will be modeled as lazy boolean streams.
Let’s start with an infinite stream of false values.

Exercise 188
Define a value falses to be an infinite stream of the boolean value false.

Exercise 189
What is the type of falses?

Exercise 190
A useful function is the trueat function. The expression trueat n generates a stream of
values that are all false except for a single true at index n:
# first 5 (trueat 1) ;;
- : bool list = [false; true; false; false; false]

Define the function trueat.

Exercise 191
Define a function circnot : bool stream -> bool stream to represent the not gate.
It should have the following behavior:
# first 5 (circnot (trueat 1)) ;;
- : bool list = [true; false; true; true; true]

Exercise 192
Define a function circand to represent the and gate. It should have the following
behavior:
# first 5 (circand (circnot (trueat 1)) (circnot (trueat 3))) ;;
- : bool list = [true; false; true; false; true]

A nand gate is a gate that computes the negation of an and gate.

That is, it negates the and of its two inputs, so that its output is false
only if both of its inputs are true.

Exercise 193
Succinctly define a function circnand using the functions above to represent the nand
gate. It should have the following behavior:
# first 5 (circnand falses (trueat 3)) ;;
- : bool list = [true; true; true; true; true]
# first 5 (circnand (trueat 3) (trueat 3)) ;;
- : bool list = [true; true; true; false; true]

17.6 A unit testing framework

With the additional tools of algebraic data types and lazy evaluation,
we can put together a more elegant framework for unit testing. Lazy
evaluation in particular is useful here, since a unit test is nothing other
than an expression to be evaluated for its truth at some later time
when the tests are run. Algebraic data types are useful in a couple of
ways, first to package together the components of a test and second to
express the alternative ways that a test can come out.
322 PROGRAMMING WELL

Of course, tests can pass or fail, which we represent by an expres-

sion that returns either true or false respectively. But tests can have
other outcomes as well; there are other forms of failing than returning
false. In particular, a test might raise an exception, or it might not
terminate at all. In order to deal with tests that might not terminate,
we’ll need a way of safely running these tests in a context in which we
cut off computation after a specified amount of time. The computation
will be said to have T I M E D O U T . To record the outcome of a test, we’ll
define a variant type:

# type status =
# | Passed
# | Failed
# | Raised_exn of string
(* string describing exn *)
# | Timed_out of int (* timeout in seconds *) ;;
type status = Passed | Failed | Raised_exn of string | Timed_out of
int

A unit test type will package together such a delayed expression,

the test condition itself, a mnemonic label for the test, and a timeout
period in seconds.

# type test =
# { label : string;
# condition : bool Lazy.t;
# time : int } ;;
type test = { label : string; condition : bool Lazy.t; time : int;
}

Notice that the condition of the test is a lazy boolean, so that the con-
dition will not be evaluated until the test is run.
To construct a test, we provide a function that packages together the
components.3 3
We make use of an optional argument
for the time, which defaults to five
# (* test ?time label condition -- Returns a test with the seconds if not provided. For the inter-
# given label and condition, with optional timeout time ested, details of optional arguments are
# in seconds. *) discussed here.
# let test ?(time=5) label condition =
# {label; condition; time} ;;
val test : ?time:int -> string -> bool Lazy.t -> test = <fun>

The crux of the matter is the running of a test. Doing so generates

a value of type status. The run_test function will be provided a
function continue to be applied to the label of the test and its status.
For instance, an appropriate such function might print out a line in a
report describing the outcome, like this:

# (* present labels status -- Prints a line describing the

# outcome of a test. Appropriate for use as the continue
# function in run_test. *)
# let present (label : string) (status : status) : unit =
 I N F I N I T E D ATA S T RU C T U R E S A N D L A Z Y P R O G R A M M I N G 323

# let open Printf in

# match status with
# | Passed ->
# printf "%s: passed\ n" label
# | Failed ->
# printf "%s: failed\ n" label
# | Timed_out secs ->
# printf "%s: timed out after %d seconds\ n" label secs
# | Raised_exn msg ->
# printf "%s: raised %s\ n" label msg ;;
val present : string -> status -> unit = <fun>

The run_test function needs to evaluate the test by forcing evaluation

of the delayed condition. As a first cut, we’ll look only to the normal
case, where a test returns true or false.

# (* run_test test continue -- Runs the test, applying the

# continue function to the test label and status. *)
# let run_test ({label; condition; _} : test)
# (continue : string -> status -> unit)
# : unit =
# let result = Lazy.force condition in
# if result then continue label Passed
# else continue label Failed ;;
val run_test : test -> (string -> status -> unit) -> unit = <fun>

But what if the test raises an exception? We’ll evaluate the test condi-
tion in a try ⟨⟩ with ⟨⟩ to deal with this case.

# (* run_test test continue -- Runs the test, applying the

# continue function to the test label and status. *)
# let run_test ({label; condition; _} : test)
# (continue : string -> status -> unit)
# : unit =
# try
# let result = Lazy.force condition in
# if result then continue label Passed
# else continue label Failed
# with
# | exn -> continue label
# (Raised_exn (Printexc.to_string exn)) ;;
val run_test : test -> (string -> status -> unit) -> unit = <fun>

Finally, we need to deal with timeouts. We appeal to a function

timeout that forces a lazy computation, but raises a special Timeout
exception if the computation goes on too long. The workings of this
function are well beyond the scope of this text, but we provide the code
in Figure 17.6.
Using the timeout function to force the condition and checking for
the Timeout exception handles the final possible status of a unit test.

# (* run_test test continue -- Runs the test, applying the

# continue function to the test label and status. *)
324 PROGRAMMING WELL

Figure 17.6: The function timeout used

in the evaluation of unit tests, based on
the timeout function of Chailloux et al.
# (* timeout time f -- Forces delayed computation f, returning
(2000)
# what f returns, except that after time seconds it raises

# a Timeout exception. *)

# exception Timeout ;;

exception Timeout

# let sigalrm_handler =

# Sys.Signal_handle (fun _ -> raise Timeout) ;;

val sigalrm_handler : Sys.signal_behavior = Sys.Signal_handle <fun>

# let timeout (time : int) (f : 'a Lazy.t) : 'a =

# let old_behavior =

# Sys.signal Sys.sigalrm sigalrm_handler in

# let reset_sigalrm () =

# ignore (Unix.alarm 0);

# Sys.set_signal Sys.sigalrm old_behavior in

# ignore (Unix.alarm time) ;

# let res = Lazy.force f in

# reset_sigalrm () ; res ;;

val timeout : int -> 'a Lazy.t -> 'a = <fun>

 I N F I N I T E D ATA S T RU C T U R E S A N D L A Z Y P R O G R A M M I N G 325

# let run_test ({label; time; condition} : test)

# (continue : string -> status -> unit)
# : unit =
# try
# if timeout time condition
# then continue label Passed
# else continue label Failed
# with
# | Timeout -> continue label (Timed_out time)
# | exn -> continue label
# (Raised_exn (Printexc.to_string exn)) ;;
val run_test : test -> (string -> status -> unit) -> unit = <fun>

By iterating over a list of unit tests, we can generate a nice report of

all the tests.
# (* report tests -- Generates a report based on the
# provided tests. *)
# let report (tests : test list) : unit =
# List.iter (fun test -> run_test test present) tests ;;
val report : test list -> unit = <fun>

With this infrastructure in place, we can define a test suite that

demonstrates all of the functionality of the unit testing framework.
# let tests =
# [ test "should fail" (lazy (3 > 4));
# test "should pass" (lazy (4 > 3));
# test "should time out" (lazy (let rec f x = f x in f 1));
# test "should raise exception" (lazy ((List.nth [0; 1] 3) = 3))
# ] ;;
val tests : test list =
[{label = "should fail"; condition = <lazy>; time = 5};
{label = "should pass"; condition = <lazy>; time = 5};
{label = "should time out"; condition = <lazy>; time = 5};
{label = "should raise exception"; condition = <lazy>; time =
5}]

# report tests ;;
should fail: failed
should pass: passed
should time out: timed out after 5 seconds
should raise exception: raised Failure("nth")
- : unit = () Figure 17.7: Peter Landin (1930–2009),
developer of many innovative ideas
in programming languages, including
17.7 A brief history of laziness the roots of lazy programming. His
influence transcended his role as a
The idea of lazy computation probably starts with Peter Landin (Fig- computer scientist, especially in his
active support of gay rights.
ure 17.7). He observed “a relationship between lists and functions”:
In this relationship a nonnull list L is mirrored by a none-adic function S
that produces a 2-list consisting of (1) the head of L, and (2) the function
mirroring the tail of L. . . . This correspondence serves two related pur-
poses. It enables us to perform operations on lists (such as generating
326 PROGRAMMING WELL

them, mapping them, concatenating them) without using an “exten-

sive,” item-by-item representation of the intermediately resulting lists;
and it enables us to postpone the evaluation of the expressions specify-
ing the items of a list until they arc actually needed. The second of these
is what interests us here. (1965)

The idea of a “function mirroring the tail of” a list is exactly the delay-
ing of the tail computation that we’ve seen in the stream data type.
Landin is notable for many other ideas of great currency. For in-
stance, he invented the term “syntactic sugar” for the addition of
extra concrete syntax to abbreviate some useful but otherwise com-
plicated abstract syntax. His 1966 paper “The next 700 programming
languages” (Landin, 1966) introduced several innovative ideas in-
cluding the “offside rule” of concrete syntax, allowing the indentation
pattern of a program to indicate its structure. Python is typically noted
for making use of this Landin innovation. Indeed, the ISWIM language
that Landin described in this paper is arguably the most influential
programming language that no one ever programmed in.
Following Landin’s observation, Wadsworth proposed the lazy
lambda calculus in 1971, and Friedman and Wise published an article
proposing that “Cons should not evaluate its arguments” in 1976. The
first programming language to specify lazy evaluation as the evaluation
regime was Burstall’s Hope language (which also introduced the idea,
found in nascent form in ISWIM, of algebraic data types). A series of
lazy languages followed, most notably Miranda, but the lazy program-
ming community came together to converge on the now canonical lazy
language Haskell, named after Haskell Curry.

17.8 Problem set 7: Refs, streams, and music

This section provides substantive background for one of the course prob-
lem sets. For common background on logistics and procedures for prob-
lem sets, including such topics as accessing the problem set code, compil-
ing and testing your code, reflecting on the process, and submitting your
solution, please review the course document “Problem set procedures”.

You are allowed (and encouraged) to work with a partner on this problem
set. You are also allowed to work alone, if you prefer. See the course
document “Problem set procedures” for further information on working
with partners on problem sets.

In this problem set you will work with two new ideas: First, we pro-
vide a bit of practice with imperative programming, emphasizing mu-
table data structures and the interaction between assignment and lex-
ical scoping. Since this style of programming is probably most familar
to you, this portion of the problem set is brief. Second, we introduce
 I N F I N I T E D ATA S T RU C T U R E S A N D L A Z Y P R O G R A M M I N G 327

lazy programming and its use in modeling infinite data structures. This
part of the problem set is more extensive, and culminates in a project
to generate infinite streams of music.

17.8.1 Mutable lists and cycles

We’ve provided some problems involving mutable lists in refs.ml.

Remember that you must test each function in this part in refs_-
test.ml. Your solutions will not be graded for time or space efficiency.
Recall from lecture that a major difficulty with mutable lists is that it
is possible to introduce cycles, links to elements that appeared earlier
in the list, which cause a naive traversal of the list to loop forever.
Figure 17.8(a) shows an example of a mutable list that doesn’t have
a cycle, even though the same element is repeated twice, whereas
Figure 17.8(b) is an example of a mutable list with a cycle.

1 2 3 Figure 17.8: Example mutable lists: (a)

no cycle; (b) with cycle.

1 2 3 2 2

(a) (b)
Problem 194
Write a function has_cycle that returns true if and only if a mutable list has a cycle.
Your function must not alter the original mutable list, and it must terminate eventually.
Think about how you will know if a node that you are visiting has been seen before.
Testing whether the reference a_ref points to b can be done with (!a_ref) == b. The
(==) function tests equality at the level of memory location rather than value.
Problem 195
Write a function flatten that takes a mutable list and removes any cycle in it destruc-
tively by removing backward links. This means that the data structure should be changed
in-place, such that the list passed as an argument itself is altered if necessary. Note that
this is very different from the functional programming approach that we have been using
up to this point, where functions might return an altered copy of a data structure. Sup-
pose you pass in the mutable list from Figure 17.8(b); flatten should alter it such that
it looks like Figure 17.8(a), and then return unit. If you are unsure how to destructively
alter a mutable list, take a look at reflist in refs_test.ml.
Problem 196
Write mlength, which finds the number of elements in a mutable list. This should always
terminate, even if the list has a cycle. For example, both Figures 17.8(a) and (b) have
length 4. The mlength function must be nondestructive, that is, the original mutable list
should not change.
Problem 197
Challenge problem: It’s possible to complete Problem 194 in such a way that it doesn’t
use any additional space other than that taken up by the list passed as an argument.
Attempt this only if you’ve finished the rest of the assignment and are up for a challenge.

17.8.2 Lazy evaluation

In this section you’ll gain practice with lazy evaluation using the
OCaml Lazy module through problems with infinite streams and
infinite trees.
328 PROGRAMMING WELL

Series acceleration with infinite streams In Section 17.4, we

showed how to use Taylor series to approximate π. The code needed
to do so is provided in a module SampleStreams, which uses
NativeLazyStreams as its implementation of lazy streams. The
method relies on generating the series of terms of a Taylor series, com-
puting the partial sums that approximate π, and then finding a pair of
consecutive approximations that are within the desired tolerance ², as
shown here:
# let pi_sums = sums pi_stream ;;
val pi_sums : float NativeLazyStreams.stream = <lazy>

# first 5 pi_sums ;;
- : float list =
[4.; 2.66666666666666696; 3.46666666666666679; 2.89523809523809561;
3.33968253968254025]

# within 0.1 pi_sums ;;

- : int * float = (19, 3.09162380666784)

# within 0.01 pi_sums ;;

- : int * float = (199, 3.13659268483881615)

# within 0.001 pi_sums ;;

- : int * float = (1999, 3.14109265362104129)

The method works, but converges quite slowly. It takes some 200 terms
in the expansion to get within 0.01 of π. In this section of the problem
set, you will use a technique called series acceleration to speed up
the process of converging on a value. A simple method is to average
adjacent elements in the approximation stream. The necessary code
for you to use and modify can be found in the file streamstrees.ml.
Problem 198
Write a function average that takes a float stream and returns a stream of floats each
of which is the average of adjacent values in the input stream. For example:
# first 5 (average (to_float nats)) ;;
- : float list = [0.5; 1.5; 2.5; 3.5; 4.5]

You should then be able to define a stream pi_avgs of the averages of the partial sums
in pi_sums. How many steps does it take to get within 0.01 of π using pi_avgs instead of
pi_sums?

An even better accelerator is Aitken’s method. Given a sequence s,

Aitken’s method generates the sequence s ′ given by

(s n − s n −1 )2
s n′ = s n −
s n − 2s n −1 + s n −2
Problem 199
Implement a function aitken that applies Aitken’s method to a float stream returning
the resulting float stream.
Problem 200
Try the various methods to compute approximations of π and fill out the table in the
streamstrees.ml file with what you find.
 I N F I N I T E D ATA S T RU C T U R E S A N D L A Z Y P R O G R A M M I N G 329

Infinite trees Just as streams are a lazy form of list, we can have a lazy
form of trees. In the definition below, each node in a lazy tree of type
’a tree holds a value of some type ’a, and a (conventional, finite) list
of one or more (lazy) child trees.

type 'a treeval = Node of 'a * 'a tree list

and 'a tree = 'a treeval Lazy.t ;;

Problem 201
Complete the implementation by writing functions node, children, print_depth, tmap,
tmap2, bfenumerate, onest, levels, and tree_nats as described in streamstrees.ml.
We recommend implementing them in that order.

17.8.3 The song that never ends

In this part, you will explore a fun application of streams: music. Be-
fore you begin, if you’re not already familiar with basic music notation
and theory, you should learn some of these concepts. Numerous on-
line tutorials exist, such as this one, though if you’re short on time, the
following introduction should be sufficient for this assignment.

A brief introduction to music theory The major musical objects we

use in this assignment are notes and rests. Notes indicate when a sound
is heard, and rests indicate a certain period of silence. Notes have a
pitch and a duration, rests have only a duration. A pitch has one of
twelve names: C, D♭ (pronounced D-flat, also called C♯, pronounced
C-sharp), D, E♭ (also called D♯), E, F, G♭ (F♯), G, A♭ (G♯), A, B♭ (A♯), B.
The distance between two consecutive pitches in this list is called a
half-step. Each of these pitches can be played in many octaves. For
example, a piano keyboard has many Cs. The one in the middle of the
keyboard is called “middle C”, but the key 12 half-steps (that is, twelve
piano keys counting white and black keys) above and below it are both
Cs as well, in different octaves. Thus, a pitch can be thought of as a
name and an octave.
A basic unit of time in music is called the measure, and notes are
named based on what fraction of a measure they last. The most com-
mon notes are whole notes, half notes, quarter notes and eighth notes.
Rests can be similarly named for their duration, and so there are whole
rests, half rests and so on.

Data types for musical objects The top of music.ml provides simple
definitions for musical data types. The type pitch is a tuple of a p,
which has one constructor for each of the 12 pitch names, and an
integer representing the octave. (Note that our definition includes only
flats and not sharps. We apologize if this offends any music theorists.)
There is also a data type representing musical objects, obj. An obj can
330 PROGRAMMING WELL

either be a Note, which contains a pitch, the duration as a float, and

the volume (how loud the note is played) as an integer from 0 to 128; or
a Rest, which contains the duration as a float. Durations are in units of
measures, so 1.0 is a whole note or rest, 0.5 is a half note or rest, and so
on. You may also indicate other float values, like 0.75 (called a dotted
half note or rest) or 1.5, 2.0, 3.0, and so forth, for multi-measure notes
and rests.
This data type is useful for representing the kind of music notation
you’d see in sheet music, but what does this have to do with streams?
For the purpose of representing or playing music on a computer (for
example, in a MIDI sequencer), a different representation is more
convenient. In this representation, a piece of music is a stream of
events, where an event is the start of a certain tone or the end of a
certain tone. Both kinds of events carry two pieces of information: how
much time should elapse between the previous event and this one,
and the pitch of the event. Note that stops, in addition to starts, must
have pitch. This is so that we can have multiple notes played at a time.
The sequencer has to know which of the notes currently playing we
want to stop. Starts also have a volume. (It so happens that in the MIDI
specification a stop can be represented like a start with a volume of
0.) A data type for musical events is also given at the top of music.ml.
An event can be a Tone of a time, a pitch, and a volume, or a Stop
of a time and a pitch. Remember that the times stored with events are
relative to the previous event and not absolute.
A natural way to represent music is as a stream of events. For this
purpose, you’ll use the NativeLazyStreams module. The sequencer
doesn’t care about the rest of the piece, it just takes events as they
come and processes them. This also allows us to lazily perform actions
on an entire piece of music. The downside of this is that, because our
streams are always infinite, songs represented in this way can’t end.
With the exception of certain songs commonly performed in the back
seats of cars on family trips, pieces of music are of finite length, but
we’ll get back to this later.
Problem 202
Write a function list_to_stream that builds a music stream (event stream) from a
finite list of musical objects. The stream should repeat this music forever. (You may want
to look out for invalid lists, which don’t represent any meaningful music, though we will
not test for that.)
Hint: Use a recursive helper function as defined, which will change the list but keep
the original list around as lst. Both need to be recursive, since you will call both the
inner and outer functions at some point. If you’d rather solve the problem a different
way, you may remove the definition of the recursive helper as long as you keep the type
signature of list_to_stream the same.
Problem 203
Write a function pair that merges two event streams into one. Events that happen
earlier in time should appear earlier in the merged stream. Events in the merged stream
should happen at the same absolute time they happened in the individual streams.
 I N F I N I T E D ATA S T RU C T U R E S A N D L A Z Y P R O G R A M M I N G 331

For example, if one stream has a start at time 0, and a stop at time 0.5 after that, and
the other stream has a start at time 0.25 and a stop at time 0.5 after that, the combined
stream should have a start at time 0, the second start at 0.25 after that, the first stop
at 0.25 after that, and the second stop at 0.25 after that. This will require some careful
thought to update the time differences of the event that is not chosen.
Problem 204
Write a function transpose that takes an event stream and transposes it (moves each
pitch up by a given number of half_steps.) For example, a C transposed up 7 half-steps
is a G. A C transposed up 12 half-steps is the C in the next octave up. Use the helper
function transpose_pitch, which transposes a single pitch value by the correct number
of half-steps.

What fun would an assignment about music be if there was no way

to hear music? We’ve given you a function output_midi that takes
a string filename, integer n, and an event stream str and outputs
the first n events of str as a MIDI file with the given filename. Note
that the integer argument is necessary because streams are infinite
and we can’t have infinite MIDI files. To hear some music and test the
functions you’ve written so far, uncomment the lines of code under
the comment “Start off with some scales.” When you compile and
run the file at the command line or execute the file’s definitions in
the OCaml toplevel, this will output a file scale.mid, which plays a C
major scale together with a G major scale. You should be able to play
the generated MIDI file with an application such as GarageBand on
Mac OS or Windows Media Player on Windows.
Please watch this informative YouTube video that relates to the next
problem.
As any fourth-grader in school band can tell you, scales are boring.
This example also doesn’t show us why we should care about the power
music streams give us. For a more realistic example, we need a piece
of music that is repetitive, and has several parts played together which
resemble each other with small modifications. To be a good example,
it should also be instantly recognizable and in the public domain so we
don’t get sued. Such a piece is Johann Pachelbel’s Canon in D major.
(We guarantee you’ve heard it.)
We’ll be building a music stream of a simplified version of eight
measures of Pachelbel’s canon. This segment contains a basso con-
tinuo, defined as the event stream bass. This is a two-measure pattern
that repeats over and over. (Ah, so this is where infinite streams are
useful.) It also contains a melody, of which six measures are contained
in the stream melody. The bass plays for two measures, and then the
melody starts and plays over it. Two measures later, the melody starts
again, playing over the bass and the first melody. Finally, two measures
after this (six measures from the start), the melody starts once more.
For the final two measures, there are four streams playing at once: the
bass and three copies of the melody, at different points.
Problem 205
332 PROGRAMMING WELL

Define a stream canon that represents this piece. Use the functions shift_start and
pair, and the streams bass and melody. When you’re done, uncomment the line below
the definition and compile and run the code. It will produce a file canon.mid which,
if you’ve done the problem correctly, will contain the first eight measures of the piece.
(We export 176 events since these eight measures should contain 88 notes; you could
increase this number if you want to hear more music, though it won’t be true to the
original.) If it sounds wrong, it probably is. If it sounds right, either it is or you’re a
brilliant composer. (Note: As this is not a music class, brilliant composers will not earn
extra points on this question.)
Problem 206
Challenge problem: There’s lots of opportunity for extending your system. If you want,
try implementing other functions for manipulating event streams. Maybe one that
increases or decreases the timescale, or one that doubles every note, or increases or
decreases the volume. See what kind of music you can create. We’ve given you some
more interesting streams of music to play around with if you’d like, named part1
through part4. They’re all the same length, 2.25 measures.
18
Extension and object-oriented programming

Think of your favorite graphical user interface (GUI). It probably has

various W I D G E T S – buttons, checkboxes, textboxes, radio buttons,
menus, icons, and so forth. These widgets might undergo various op-
erations – we might want to draw them in a window, click on them,
change their location, remove them, highlight them, select from them.
Each of these operations seems like a function. We’d organize func-
tions like this:

Figure 18.1: Function-oriented organi-

how to draw: how to click on: how to highlight: zation of widget software
a button a button a button
a checkbox a checkbox a checkbox
a textbox a textbox a textbox
... … ...

But new widgets are being invented all the time. Every time a new
widget type is added, we’d have to change every one of these functions.
Instead, we might want to organize the code a different way:

Figure 18.2: Object-oriented organiza-

buttons: checkboxes: textboxes: tion of widget software
how to draw how to draw how to draw
how to click how to click how to click
how to highlight how to highlight how to highlight
... ... ...

This way, adding a new widget doesn’t affect any of the existing
ones. The changes are localized, and therefore likely to be much more
reliably added. We are carving the software at its joints, following the
edict of decomposition.
This latter approach to code organization, organizing by “object”
334 PROGRAMMING WELL

rather than by function, is referred to as O B J E C T- O R I E N T E D . It’s prob-

ably no surprise that the rise in popularity of object-oriented pro-
gramming tracks the development of graphical user interfaces; as seen
above, it’s a natural fit. In particular, the idea of object-oriented pro-
gramming was popularized by the Smalltalk programming language
and system, which pioneered many of the fundamental ideas of graph-
ical user interfaces that we are now accustomed to – windows, icons,
menus, buttons. Smalltalk with its graphical user interface was devel-
oped in the early 1970’s at Xerox PARC by Alan Kay, Adele Goldberg,
Dan Ingalls, and others (Figure 18.3). Steve Jobs, seeing the Smalltalk
environment in a 1979 visit to Xerox PARC, immediately imported the
ideas into Apple’s Lisa and Macintosh computers, thereby disseminat-
ing and indeed universalizing the ideas.
In this chapter, we introduce object-oriented programming, a pro-
gramming paradigm based on organizing functionalities (in the form
of methods) together with the data that they operate on, as opposed to
the functional paradigm, which organizes functionalities (in the form
of functions) separate from the corresponding data.

18.1 Drawing graphical elements

To motivate such a reorganization, consider a program to draw graph-

ical elements on a window. We’ll start by organizing the code in a
function-oriented, not object-oriented, style.
Positions in the window can be captured with a point data type:
# type point = {x : int; y : int} ;;
type point = { x : int; y : int; }

We might want data types for the individual kinds of graphical ele-
ments – rectangles, circles, squares – each with its own parameters
specifying pertinent positions, sizes, and the like:
# type rect = {rect_pos : point;
# rect_width : int; rect_height : int} ;;
type rect = { rect_pos : point; rect_width : int; rect_height :
int; }

# type circle = {circle_pos : point; circle_radius : int} ;; Figure 18.3: Alan Kay, Adele Goldberg,
type circle = { circle_pos : point; circle_radius : int; } and Dan Ingalls, developers of the influ-
ential Smalltalk language, a pioneering
# type square = {square_pos : point; square_width : int} ;; object-oriented language, with an inno-
type square = { square_pos : point; square_width : int; } vative user interface based on graphical
widgets and direct manipulation.
We can think of a scene as being composed of a set of these display
elements:
# type display_elt =
# | Rect of rect
 E X T E N S I O N A N D O B J E C T- O R I E N T E D P R O G R A M M I N G 335

# | Circle of circle
# | Square of square ;;
type display_elt = Rect of rect | Circle of circle | Square of
square

# type scene = display_elt list ;;

type scene = display_elt list

In order to make use of these elements to actually draw on a screen,

we’ll make use of the OCaml Graphics module, which you may want
to familiarize yourself with before proceeding. (We rename the module
G for brevity.)

# module G = Graphics ;;
module G = Graphics

We can write a function to draw a display element of whatever vari-

ety by dispatching (matching) based on the variant of the display_elt
type:1

# let draw (d : display_elt) : unit =

# match d with
# | Rect r ->
# G.set_color G.black;
# G.fill_rect (r.rect_pos.x - r.rect_width / 2)
1
# (r.rect_pos.y - r.rect_height / 2) All of the subtractions of half the
widths and heights is because the
# r.rect_width r.rect_height
Graphics module often draws graphics
# | Circle c ->
based on the lower left hand corner
# G.set_color G.black; position, instead of the center of the
# G.fill_circle c.circle_pos.x c.circle_pos.y graphic that we’re using.
# c.circle_radius
# | Square s ->
# G.set_color G.black;
# G.fill_rect (s.square_pos.x - s.square_width / 2)
# (s.square_pos.y - s.square_width / 2)
# s.square_width s.square_width ;;
val draw : display_elt -> unit = <fun>

and use it to draw an entire scene on a fresh canvas:

# let draw_scene (s : scene) : unit =

# try
# G.open_graph ""; (* open the canvas *)
# G.resize_window 200 300; (* erase and resize *)
# List.iter draw s; (* draw the elements *)
# ignore (G.read_key ()) (* wait for a keystroke *)
# with
# exn -> (G.close_graph () ; raise exn) ;;
val draw_scene : scene -> unit = <fun>

Let’s test it on a simple scene of a few rectangles and circles:

# let test_scene =
# [ Rect {rect_pos = {x = 0; y = 20};
336 PROGRAMMING WELL

# rect_width = 15; rect_height = 80};

# Circle {circle_pos = {x = 40; y = 100};
# circle_radius = 40};
# Circle {circle_pos = {x = 40; y = 140};
# circle_radius = 20};
# Square {square_pos = {x = 65; y = 160};
# square_width = 50} ] ;;
val test_scene : display_elt list =
[Rect {rect_pos = {x = 0; y = 20}; rect_width = 15; rect_height =
80};
Circle {circle_pos = {x = 40; y = 100}; circle_radius = 40};
Circle {circle_pos = {x = 40; y = 140}; circle_radius = 20};
Square {square_pos = {x = 65; y = 160}; square_width = 50}] (a)

# draw_scene test_scene ;;
- : unit = ()

A window pops up with the scene (Figure 18.4(a)).

Sadly, the scene is not centered very well in the canvas. Fortunately,
it’s easy to add functionality in the functional programming paradigm:
just add functions. We can easily add functions to translate a display
element or a scene by a given amount in the x and y directions.
# let translate (p : point) (d : display_elt) : display_elt =
# let vec_sum {x = x1; y = y1} {x = x2; y = y2} =
# {x = x1 + x2; y = y1 + y2} in
# match d with
# | Rect r ->
# Rect {r with rect_pos = vec_sum p r.rect_pos} (b)
# | Circle c -> Figure 18.4: (a) A test scene. (b) The test
# Circle {c with circle_pos = vec_sum p c.circle_pos} scene translated.
# | Square s ->
# Square {s with square_pos = vec_sum p s.square_pos} ;;
val translate : point -> display_elt -> display_elt = <fun>

# let translate_scene (p : point) : scene -> scene =

# List.map (translate p) ;;
val translate_scene : point -> scene -> scene = <fun>

Using these, we can translate the scene to center it before drawing:

# draw_scene (translate_scene {x = 42; y = 50} test_scene) ;;
- : unit = ()

to get the depiction in Figure 18.4(b).

So adding functionality is easy. What about adding new types of
data, new display elements? Suppose we want to add a textual display
element to place some text in the scene.
# type text = {text_pos : point;
# text_title : string} ;;
type text = { text_pos : point; text_title : string; }

We’ll have to modify the display_elt data type to incorporate text

elements:
 E X T E N S I O N A N D O B J E C T- O R I E N T E D P R O G R A M M I N G 337

# type display_elt =
# | Rect of rect
# | Circle of circle
# | Square of square
# | Text of text ;;
type display_elt =
Rect of rect
| Circle of circle
| Square of square
| Text of text

Now the draw function complains (unsurprisingly) of an inexhaustive

match:

# let draw (d : display_elt) : unit =

# match d with
# | Rect r ->
# G.set_color G.black;
# G.fill_rect (r.rect_pos.x - r.rect_width / 2)
# (r.rect_pos.y - r.rect_height / 2)
# r.rect_width r.rect_height
# | Circle c ->
# G.set_color G.black;
# G.fill_circle c.circle_pos.x c.circle_pos.y
# c.circle_radius
# | Square s ->
# G.set_color G.black;
# G.fill_rect (s.square_pos.x - s.square_width / 2)
# (s.square_pos.y - s.square_width / 2)
# s.square_width s.square_width ;;
Lines 2-16, characters 0-29:
2 | match d with
3 | | Rect r ->
4 | G.set_color G.black;
5 | G.fill_rect (r.rect_pos.x - r.rect_width / 2)
6 | (r.rect_pos.y - r.rect_height / 2)
...
13 | G.set_color G.black;
14 | G.fill_rect (s.square_pos.x - s.square_width / 2)
15 | (s.square_pos.y - s.square_width / 2)
16 | s.square_width s.square_width...
Warning 8: this pattern-matching is not exhaustive.
Here is an example of a case that is not matched:
Text _
val draw : display_elt -> unit = <fun>

We’ll have to augment it to handle drawing text. Ditto for the

translate function. In fact, every function that manipulates display
elements will have to be changed. If we’re going to be adding new types
of elements to display, translate, and the like, this will get unwieldy
quickly. But there’s a better way – objects.
338 PROGRAMMING WELL

18.2 Objects introduced

What do we care about about display elements? That they can be

drawn. That’s it. We want to abstract away from all else.
We’ll define a data type, an abstraction, display_elt, that is a
record with a single field called draw that stores a drawing function.
# type display_elt = {draw : unit -> unit} ;;
type display_elt = { draw : unit -> unit; }

Then rectangles, circles, squares, and texts are just ways of building
display elements with that drawing functionality.
Take rectangles for example. A rectangle is a display_elt whose
draw function displays a rectangle. We can establish a rect function
that builds such a display element given its initial parameters – posi-
tion, width, and height:
# let rect (p : point) (w : int) (h : int) : display_elt =
# { draw = fun () ->
# G.set_color G.black ;
# G.fill_rect (p.x - w/2) (p.y - h/2) w h } ;;
val rect : point -> int -> int -> display_elt = <fun>

Similarly with circles and squares:

# let circle (p : point) (r : int) : display_elt =
# { draw = fun () ->
# G.set_color G.black;
# G.fill_circle p.x p.y r } ;;
val circle : point -> int -> display_elt = <fun>

# let square (p : point) (w : int) : display_elt =

# { draw = fun () ->
# G.set_color G.black ;
# G.fill_rect (p.x - w/2) (p.y - w/2) w w } ;;
val square : point -> int -> display_elt = <fun>

Now to draw a display element, we just extract the draw function and
call it. The display element data object knows how to draw itself.
# let draw (d : display_elt) = d.draw () ;;
val draw : display_elt -> unit = <fun>

If we want to add a new display element, a text, say, we just have to

provide a way to draw such a thing. No other code (draw, draw_scene)
needs to change.
# let text (p : point) (s : string) : display_elt =
# { draw = (fun () ->
# let (w, h) = G.text_size s in
# G.set_color G.black;
# G.moveto (p.x - w/2) (p.y - h/2);
# G.draw_string s) } ;;
val text : point -> string -> display_elt = <fun>
 E X T E N S I O N A N D O B J E C T- O R I E N T E D P R O G R A M M I N G 339

Of course, we’d probably want display elements to have more func-

tionality than just drawing themselves – for instance, moving them to a
new position, querying and changing their color, and much more. Let’s
start with these.

# type display_elt =
# { draw : unit -> unit;
# set_pos : point -> unit;
# get_pos : unit -> point;
# set_color : G.color -> unit;
# get_color : unit -> G.color } ;;
type display_elt = {
draw : unit -> unit;
set_pos : point -> unit;
get_pos : unit -> point;
set_color : G.color -> unit;
get_color : unit -> G.color;
}

Notice that display elements now (apparently) must have mutable

state. Their position and color can be modified over time. We’ll im-
plement this state by creating appropriate references, called pos and
color, respectively, that are generated upon creation of an object and
are specific to it. Here, for instance, is the circle function to create a
circular display element object:

# let circle (p : point) (r : int) : display_elt =

# let pos = ref p in
# let color = ref G.black in
# { draw = (fun () -> G.set_color (!color);
# G.fill_circle (!pos).x (!pos).y r);
# _
set pos = (fun p -> pos := p);
# get_pos = (fun () -> !pos);
# set_color = (fun c -> color := c);
# get_color = (fun () -> !color) } ;;
val circle : point -> int -> display_elt = <fun>

The scoping is crucial. The definitions of pos and color are within
the scope of the circle function. Thus, new references are generated
each time circle is invoked and are accessible only to the record
structure (the object) created by that invocation.2 Similarly, we’ll want 2
Recall the similar idea of local, other-
wise inaccessible, persistent, mutable
a function to create rectangles and text boxes, each with its own state
state first introduced in the bump func-
and functionality as specified by the display_elt type. tion from Section 15.3, and reproduced
here:
# let rect (p : point) (w : int) (h : int) : display_elt =
# let bump =
# let pos = ref p in
# let ctr = ref 0 in
# let color = ref G.black in
# fun () ->
# { draw = (fun () -> # ctr := !ctr + 1;
# G.set_color (!color); # !ctr ;;
# G.fill_rect ((!pos).x - w/2) ((!pos).y - h/2) val bump : unit -> int = <fun>
# w h);
# set_pos = (fun p -> pos := p);
340 PROGRAMMING WELL

# get_pos = (fun () -> !pos);

# set_color = (fun c -> color := c);
# get_color = (fun () -> !color) };;
val rect : point -> int -> int -> display_elt = <fun>

# let text (p : point) (s : string) : display_elt =

# let pos = ref p in
# let color = ref G.black in
# { draw = (fun () ->
# let (w, h) = G.text_size s in
# G.set_color (!color);
# G.moveto ((!pos).x - w/2) ((!pos).y - h/2);
# G.draw_string s);
# set_pos = (fun p -> pos := p);
# get_pos = (fun () -> !pos);
# set_color = (fun c -> color := c);
# get_color = (fun () -> !color) } ;;
val text : point -> string -> display_elt = <fun>

What we’ve done is to generate a wholesale reorganization of the

display element code, organizing it not by functionality (with a draw
function, a set_pos function, and so forth), but instead by variety of
“object” bearing that functionality. We’ve organized the code in an
O B J E C T- O R I E N T E D manner.
Think of a table (as in Table 18.1) that describes for each function-
ality (draw, move, getting and setting color) and each class of object
(rectangle, circle, text) the code necessary to carry out that function-
ality for that class of object. We can organize the code by functional-
ity, packaging the rows into functions; this is the function-oriented
paradigm. Alternatively, we can organize the code by class of ob-
ject, packaging the columns into objects; this is the object-oriented
paradigm.

rectangle circle text

G.set_color (!color);
G.set_color (!color); G.set_color (!color)
G.moveto (!pos).x
G.fill_rect (!pos).x G.fill_circle (!pos).x
draw (!pos).y;
(!pos).y w h (!pos).y r
G.draw_string s

move pos := p pos := p pos := p

set color color := c color := c color := c

get color !color !color !color

Which is the better approach? The edict of decomposition appeals Table 18.1: The matrix of functionality
(rows) and object classes (columns)
to cutting up software at its joints. Which of row or column constitutes for the display elements example.
the natural joints will vary from case to case. It is thus a fundamental The code can be organized by row –
function-oriented – or by column –
object-oriented.
 E X T E N S I O N A N D O B J E C T- O R I E N T E D P R O G R A M M I N G 341

design decision as to whether to use a function- or object-oriented

structuring of code. If you expect a need to add additional columns
with regularity, whereas adding rows will be rare, the object-oriented
approach will fare better. Conversely, if new rows, new functional-
ity, will be needed over a relatively static set of classes of data, the
function-oriented approach is preferable.

18.3 Object-oriented terminology and syntax

The object-oriented programming paradigm that we’ve reconstructed

here comes with its own set of terminology. First, the data structure
that encapsulates the various bits of functionality – here implemented
as a simple record structure – is an O B J E C T . The various components
providing the functionality are its M E T H O D S , and the state variables
(like color and pos) its I N S TA N C E VA R I A B L E S . The specification of
what methods are provided by an object (like display_elt) is its
C L A S S I N T E R F A C E , and the creation of an object is specified by its
CLASS (like circle or text).
We create an object by I N S TA N T I AT I N G the class, in this example,
the circle class,
# let circle1 = circle {x = 100; y = 100} 50 ;;
val circle1 : display_elt =
{draw = <fun>; set_pos = <fun>; get_pos = <fun>; set_color =
<fun>;
get_color = <fun>}

which satisfies the display_elt class interface.

When we make use of a method, for instance, the set_pos method,
# circle1.set_pos {x = 125; y = 125} ;;
- : unit = ()

we are said to I N V O K E the method

It should be clear that the object-oriented programming paradigm
can be carried out in any programming language with the abstractions
that we’ve relied on here, basically, first-class functions, lexical scoping,
and mutable state. But, as with other programming paradigms we’ve
looked at, providing some syntactic sugar in support of the paradigm
can be quite useful. OCaml does just that. Indeed, the “O” in “OCaml”
indicates that the language was developed as an extension to the Caml
language that added syntactic support for object-oriented program-
ming.
The object-oriented syntax extensions in OCaml are summarized in
Table 18.2.
The display element example can thus be stated in colloquial
OCaml as follows. We start with the display_elt class interface:
342 PROGRAMMING WELL

Table 18.2: Syntactic extensions in

Concept Syntax
OCaml supporting object-oriented
Class interfaces class type ⟨interfacename ⟩ = ... programming.

Class definition class ⟨classname ⟩ ⟨args ⟩ = ...

Object definition object ... end
Instance variables val (mutable) ⟨varname ⟩ = ...
Methods method ⟨methodname ⟩ ⟨args ⟩ = ...
Instance variable update ... <- ...
Instantiating classes new ⟨classname ⟩ ⟨args ⟩
Invoking methods ⟨object ⟩#⟨methodname ⟩ ⟨args ⟩

# class type display_elt =

# object
# method draw : unit
# method set_pos : point -> unit
# method get_pos : point
# method set_color : G.color -> unit
# method get_color : G.color
# end ;;
class type display_elt =
object
method draw : unit
method get_color : G.color
method get_pos : point
method set_color : G.color -> unit
method set_pos : point -> unit
end

and define some classes that satisfy the interface:

# class circle (p : point) (r : int) : display_elt =

# object
# val mutable pos = p
# val mutable color = G.black
# method draw = G.set_color color;
# G.fill_circle pos.x pos.y r
# method set_pos p = pos <- p
# method get_pos = pos
# method set_color c = color <- c
# method get_color = color
# end ;;
class circle : point -> int -> display_elt

# class rect (p : point) (w : int) (h : int) : display_elt =

# object
# val mutable pos = p
# val mutable color = G.black
# method draw = G.set_color color;
# G.fill_rect (pos.x - w/2) (pos.y - h/2)
# w h
# method set_pos p = pos <- p
 E X T E N S I O N A N D O B J E C T- O R I E N T E D P R O G R A M M I N G 343

# method get_pos = pos

# method set_color c = color <- c
# method get_color = color
# end ;;
class rect : point -> int -> int -> display_elt

Now we can use these to create and draw some display elements. We
create a new circle,
(a)
# let _ = G.open_graph "";
# G.clear_graph ;;
- : unit -> unit = <fun>
# let b = new circle {x = 100; y = 100} 40 ;;
val b : circle = <obj>

but nothing appears yet until we draw the element.

# let _ = b#draw ;;
- : unit = ()

(Notice that invoking the method doesn’t require the application to

a unit. In the object-oriented syntax, method invocation with no
arguments can be implicit in this way.) The circle now appears, as in
Figure 18.5(a).
We can erase the object by setting its color to white and redrawing it (b)

(Figure 18.5(b)).

# let _ = b#set_color G.white;

# b#draw ;;
- : unit = ()

We move it to a new position and change its color (Figure 18.5(c)).

# let _ = b#set_pos {x = 150; y = 150};

# b#set_color G.red;
# b#draw ;;
- : unit = ()

18.4 Inheritance
(c)
The code we’ve developed so far violates the edict of irredundancy.
The implementations of the circle and rect classes, for instance, are
almost identical, differing only in the arguments of the class and the
details of the draw method.
To capture the commonality, the object-oriented paradigm allows
for definition of a class expressing the common aspects, from which
both of the classes can I N H E R I T their behaviors. We refer to the class
Figure 18.5: A circle appears (a) and
(or class type) that is being inherited from as the S U P E R C L A S S and the disappears (b). It moves and reappears
inheriting class as the S U B C L A S S . with a changed color (c).
We’ll define a shape superclass that can handle the position and
color aspects of the more specific classes. Its class type is given by
344 PROGRAMMING WELL

# class type shape_elt =

# object
# method set_pos : point -> unit
# method get_pos : point
# method set_color : G.color -> unit
# method get_color : G.color
# end ;;
class type shape_elt =
object
method get_color : G.color
method get_pos : point
method set_color : G.color -> unit
method set_pos : point -> unit
end

and a simple implementation of the class is

# class shape (p : point) : shape_elt =

# object
# val mutable pos = p
# val mutable color = G.black
# method set_pos p = pos <- p
# method get_pos = pos
# method set_color c = color <- c
# method get_color = color
# end ;;
class shape : point -> shape_elt

Notice that the new shape_elt signature provides access to the four
methods, but not directly to the instance variables used to implement
those methods. The only access to those instance variables will be
through the methods, an instance of the edict of compartmentaliza-
tion that seems appropriate.
The display_elt class type can inherit the methods from shape_-
elt, adding just the additional draw method.

# class type display_elt =

# object
# inherit shape_elt
# method draw : unit
# end ;;
class type display_elt =
object
method draw : unit
method get_color : G.color
method get_pos : point
method set_color : G.color -> unit
method set_pos : point -> unit
end

The inherit specification works as if the contents of the inherited su-

perclass type were simply copied into the subclass type at that location
in the code.
 E X T E N S I O N A N D O B J E C T- O R I E N T E D P R O G R A M M I N G 345

The rect and circle subclasses can inherit much of their behavior
from the shape superclass, just adding their own draw methods. How-
ever, without the ability to refer directly to the instance variables, the
draw method will need to call its own methods for getting and setting
the position and color. We can add a variable to name the object itself,
by adding a parenthesized name after the object keyword. Although
any name can be used, by convention, we use this or self. We can
then invoke the methods from the shape superclass with, for instance,
this#get_color.

# class rect (p : point) (w : int) (h : int) : display_elt =

# object (this)
# inherit shape p
# method draw =
# G.set_color this#get_color ;
# G.fill_rect (this#get_pos.x - w/2)
# (this#get_pos.y - h/2)
# w h
# end ;;
class rect : point -> int -> int -> display_elt

# class circle (p : point) (r : int) : display_elt =

# object (this)
# inherit shape p
# method draw =
# G.set_color this#get_color;
# G.fill_circle this#get_pos.x this#get_pos.y r
# end ;;
class circle : point -> int -> display_elt

Notice how the inherited shape class is provided the position argu-
ment p so its instance variables and methods can be set up properly.
Using inheritance, a square class can be implemented with a single
inheritance from the rect class, merely by specifying that the width
and height of the inherited rectangle are the same:
# class square (p : point) (w : int) : display_elt =
# object
# inherit rect p w w
# end ;;
class square : point -> int -> display_elt

Exercise 207
Define a class text : point -> string -> display_elt for placing a string of text at
a given point position on the canvas. (You’ll need the Graphics.draw_string function
for this.)

18.4.1 Overriding

Inheritance in OCaml allows for subclasses to override the methods

in superclasses. For instance, we can implement a class of bordered
346 PROGRAMMING WELL

rectangles (rather than the filled rectangles of the rect class) simply by
overriding the draw method:

# class border_rect (p : point)

# (w : int) (h : int)
# : display_elt =
# object (this)
# inherit rect p w h as super

# method! draw = G.set_color this#get_color;

# G.fill_rect (this#get_pos.x - w/2 - 2)
# (this#get_pos.y - h/2 - 2)
# (w+4) (h+4) ;
# let c = this#get_color in
# this#set_color G.white ;
# super#draw ;
# this#set_color c
# end ;;
class border_rect : point -> int -> int -> display_elt

Here, we’ve introduced the overriding draw method with method!,

where the exclamation mark diacritic explicitly marks the method as
overriding the superclass’s draw method. Without that, OCaml will
provide a helpful warning to the programmer in case the overriding
was unintentional.
When a subclass overrides the method of a superclass, the subclass
may still want access to the superclass’s version of the method. That’s
the case here, where the subclass’s draw method needs to call the su-
perclass’s. In the presence of overriding, then, it becomes important to
have a name for the superclass object so as to be able to call its meth-
ods. The inherited superclass can be given a name for this purpose by
the as construct used above in the inherit specification. The variable
following the as – conventionally super though any variable can be
used – then names the superclass providing access to its version of any
overridden methods.

18.5 Subtyping 3
The type of the scene is displayed not,
as one might expect, as display_elt
Back in Section 18.1, we defined a scene as a set of drawable ele- list but as border_rect list. OCaml
uses class names, not class type names,
ments, so as to be able to iterate over a scene to draw each element. to serve the purpose of reporting typing
We can obtain that ability by defining a new function that draw a list of information for objects. The elements
of scene are instances of various
display_elt objects:
classes (all consistent with class type
display_elt). OCaml selects the first
# let draw_list (d : display_elt list) : unit =
element of the list, which happens to be
# List.iter (fun x -> x#draw) d ;;
a border_rect instance, to serve as the
val draw_list : display_elt list -> unit = <fun> printable name of the type. This quirk
of OCaml reveals that the grafting of
We’ve put together a small scene (Figure 18.6), evocatively called the “O” part of the language isn’t always
seamless.
scene, to test the process.3
 E X T E N S I O N A N D O B J E C T- O R I E N T E D P R O G R A M M I N G 347

Figure 18.6: A test scene.

let scene =
(* generate some graphical objects *)
let box = new border_rect {x = 100; y = 110} 180 210 in
let l1 = new rect {x = 70; y = 60} 20 80 in
let l2 = new rect {x = 135; y = 100} 20 160 in
let b = new circle {x = 100; y = 100} 40 in
let bu = new circle {x = 100; y = 140} 20 in
let h = new rect {x = 150; y = 170} 50 20 in
let t = new text {x = 100; y = 200} "The CS51 camel" in
(* bundle them together *)
let scene = [box; l1; l2; b; bu; h; t] in
(* change their color and translate them *)
List.iter (fun x -> x#set_color 0x994c00) scene;
List.iter (fun o -> let {x; y} = o#get_pos in
o#set_pos {x = x + 50; y = y + 40})
scene;
(* update the surround color *)
box#set_color G.blue;
scene ;;

# scene ;;
- : border_rect list = [<obj>; <obj>; <obj>; <obj>; <obj>; <obj>;
<obj>]

We can draw this scene in a fresh window using draw_list.

# let test scene =

# try
# G.open_graph "";
# G.resize_window 300 300;
# G.clear_graph ();
# draw_list scene;
# ignore (G.read_key ())
# with
# exn -> (G.close_graph (); raise exn) ;;
val test : display_elt list -> unit = <fun>

# test scene ;;
- : unit = ()

We defined draw_list to operate on display_elt lists. But there’s

no reason to be so specific. It ought to be the case that any object with
a draw method should be able to participate in a scene. We can define
a new class type of drawable elements

# class type drawable =

# object
# method draw : unit
348 PROGRAMMING WELL

# end ;;
class type drawable = object method draw : unit end

and redefine draw_list accordingly:

# let draw_list (d : drawable list) : unit =

# List.iter (fun x -> x#draw) d ;;
val draw_list : drawable list -> unit = <fun>

We’ve defined drawable as a S U P E RT Y P E of display_elt. It’s a super-

type because anything that can be done with a drawable can be done
with a display_elt, but also potentially with other classes as well
(namely, any that have a draw method). The idea is that an object with
a “wider” interface (a subtype, like display_elt) can be used where
an object with a “narrower” interface (a supertype, like drawable) is
needed.
There is a family resemblance in this idea to polymorphism. Any
4
function with a more polymorphic type (like ’a -> ’a list, say) can There would seem to be a correlation
between subclasses and subtypes. Of
be used where an object with a less polymorphic type (like int -> int
course, not all subtypes are subclasses;
list) is needed. they may not be related by inheritance.
But in a subclass, you have all the
Exercise 208 functionality of the superclass, plus you
Test out this polymorphism subtyping behavior in OCaml by defining two functions can add some more. So are subclasses
mono : int -> int list and poly : ’a -> ’a list, along with a function need : always subtypes?
(int -> int list) -> int list. Then apply need to both mono and poly, thereby No. For instance, in the subclass,
showing that need works with an argument of its required type (int -> int list) and you could redefine a method to have a
also a subtype thereof (’a -> ’a list). more restrictive signature. In that case,
the subclass would not be a subtype;
Anything that’s a display_elt or inherits from display_elt or it would have a narrower interface (at
satisfies the display_elt interface will have at least the functionality least for that method), not a wider one.
of a drawable. So they are subtypes of drawable.4
The advantage of subtyping – allowing functions with a wider in-
terface to be used where one with a narrower interface is called for –
is just the advantage of polymorphism. It allows reuse of functionality,
which redounds to the benefit of the edict of irredundancy. Rather
than reimplement functions for the different interface “widths”, we
reuse them instead. We’ll see that OCaml allows this kind of reuse,
though with a little less automaticity than the reuse from polymor-
phism.
It ought to be the case, for instance, that, as display_elt is a sub-
type of drawable, our revision of draw_scene to apply to drawable
objects ought to allow scenes composed of display_elt objects. Let’s
try it.

# let test scene =

# try
# G.open_graph "";
# G.resize_window 300 300;
# G.clear_graph ();
# draw_list scene;
 E X T E N S I O N A N D O B J E C T- O R I E N T E D P R O G R A M M I N G 349

# ignore (G.read_key ())

# with
# exn -> (G.close_graph (); raise exn) ;;
val test : drawable list -> unit = <fun>

The type of test shows that it now takes a drawable list argument.
We apply it to our scene.

# test scene ;;
Line 1, characters 5-10:
1 | test scene ;;
^^^^^
Error: This expression has type border_rect list
but an expression was expected of type drawable list
Type
border_rect =
< draw : unit; get_color : G.color; get_pos : point;
set_color : G.color -> unit; set_pos : point -> unit >
is not compatible with type drawable = < draw : unit >
The second object type has no method get_color

But the draw_list call no longer works. We’ve tripped over a limita-
tion in OCaml’s type inference. A subtype ought to be allowed where
a supertype is needed, as it is in the case of polymorphic subtypes of
less polymorphic supertypes. But in the case of class subtyping, OCaml
is not able to perform the necessary type inference to view the sub-
type as the supertype and use it accordingly. We have to give the type
inference system a hint.
We want the call to draw_list to view scene not as its display_elt
list subtype but rather as the drawable list supertype. We use the
:> operator to specify that view. The expression scene :> drawable
list specifies scene viewed as a drawable list.

# let test scene =

# try
# G.open_graph "";
# G.resize_window 300 300;
# G.clear_graph ();
# draw_list (scene :> drawable list) ;
# ignore (G.read_key ())
# with
# exn -> (G.close_graph (); raise exn) ;;
val test : #drawable list -> unit = <fun>

# test scene ;;
- : unit = ()

Figure 18.7: The rendered test scene.

Voila! The scene (Figure 18.7) appears. A little advice to the type infer-
ence mechanism has resolved the problem.
350 PROGRAMMING WELL

18.6 Problem section: Object-oriented counters

Here is a class type and class definition for “counter” objects. Each
object maintains an integer state that can be “bumped” by adding
an integer. The interface guarantees that only the two methods are
revealed.

class type counter_interface =

object
method bump : int -> unit
method get_state : int
end ;;

class counter : counter_interface =

object
val mutable state = 0
method bump n = state <- state + n
method get_state = state
end ;;

Problem 209
Write a class definition for a class loud_counter obeying the same interface that works
identically, except that it also prints the resulting state of the counter each time the
counter is bumped.
Problem 210
Write a class type definition for an interface reset_counter_interface, which is
just like counter_interface except that it has an additional method of no arguments
intended to reset the state back to zero.
Problem 211
Write a class definition for a class loud_reset_counter satisfying the reset_counter_-
interface that implements a counter that both allows for resetting and is “loud”
(printing the state whenever a bump or reset occurs).

18.7 Problem set 8: Force-directed graph drawing

This section provides substantive background for one of the course prob-
lem sets. For common background on logistics and procedures for prob-
lem sets, including such topics as accessing the problem set code, compil-
ing and testing your code, reflecting on the process, and submitting your
solution, please review the course document “Problem set procedures”.

You’ll be familiar with graph drawings, those renderings of nodes

and edges between them that depict all kinds of networks – both phys-
ical and virtual. These drawings are ubiquitous, in large part because
of their fabulous utility. Examples date from as early as the Middle Ages
(see Figure 18.8(a)), when they were used to depict family trees and
categorizations of vices and virtues. These days, they are used to depict
everything from molecular interactions to social networks.
To gain the best benefit from visualizing graphs through a graph
drawing, the nodes and edges must be laid out well. In this problem
 E X T E N S I O N A N D O B J E C T- O R I E N T E D P R O G R A M M I N G 351

Figure 18.8: Two sample graph draw-

ings several hundred years apart.
(a) A graph drawing from the 14th
century with nodes depicting logi-
cal propositions in an argument and
edges depicting relations among
them. From Kruja et al. (2001). (b)
Snapshot of a dynamic interactive force-
directed graph drawing built using D3
(https://mbostock.github.io/d3/talk/
20111116/force-collapsible.html), from
the D3 gallery.

(a) (b)

set, you’ll complete the implementation of a system for force-directed

graph layout. A modern example of what can be done with force-
directed graph drawing is provided in Figure 18.8(b). If you’d like to get
a sense of what can be done with force-directed graph drawing, you
can play around with the graph visualization from which this snapshot
came. In carrying out this project, you’ll be making use of the object-
oriented programming paradigm supported by OCaml.
A note of assuagement: Although this problem set document uses
a lot of physics terminology, you really don’t need to know any physics
whatsoever to do the problem set. All of the physics-related code is in
portions of the code-base (graphdraw.ml and controls.ml) that we
have provided for you and that you won’t need to modify.

18.7.1 Background

A G R A P H is a mathematical object defined as a set of N O D E S and

EDGES connecting the nodes. As an example, consider a set of four
nodes (numbered 0 to 3) connected with edges cyclically, 0 to 1, 1 to
2, 2 to 3, and 3 to 0, plus an extra edge from 0 to 2. A G R A P H D R AW -
ING is a depiction of a graph in two (or sometimes three) dimensions
indicating the nodes in the graph by graphical symbols of various
sorts (circles, squares, and the like) and edges by lines drawn between
the nodes. Other aspects of the graph are also typically manifested in
graphical properties. For instance, groups of nodes might be aligned
horizontally or vertically, or grouped with a zone box surrounding
them, or laid out symmetrically or in a hub-and-spoke motif.
For the example four-node graph just presented, if we depict the
nodes as small circles, placed more or less randomly on a drawing
“canvas”, we might get a graph drawing like Figure 18.9(a). It’s not
352 PROGRAMMING WELL

Figure 18.9: Four different drawings of

the same graph. (a) Nodes randomly
placed. (b) With fixed length spring
constraints between nodes connected
by edges. (c) With fixed length spring
constraints between nodes connected
by outside edges, plus a horizontal
alignment constraint on nodes 0 and 1
and a vertical alignment constraint on
nodes 0 and 3. (d) An overconstrained
(a) (b) (c) (d)
layout with the constraints from (c) but
with all of the edge constraints from (b),
including the fixed length constraint
between 0 and 2.

particularly visually pleasing.

Much more attractive layouts can be generated by thinking of the
positions at which the nodes are to be placed as physical M A S S E S sub-
ject to various kinds of F O R C E S . The forces encourage the satisfying
of graphical constraints, such as nodes being a particular distance
from each other, or far away from each other, or horizontally or verti-
cally aligned. For instance, if we imagine a spring with a certain R E S T
LENGTH connecting two masses, those masses will have forces push-
ing them towards each other if they are farther apart than the rest
length or away from each other if they are closer together than the rest
length. (See Figure 18.10 for a visual depiction.) According to Hooke’s
law, the force applied is directly proportional to the difference between
the current distance and the rest length.
We can use this kind of mass-spring physical system to help with
graph layout. We imagine that there is a mass for each node initially
placed at the locations shown in Figure 18.9(a), and for each edge
in the graph there is a Hooke’s law spring of a given rest length, 80
pixels, say, connecting the masses representing the nodes at the end of
the edge. We refer to a force-generating element like the Hooke’s law
spring as a C O N T R O L . If we physically simulate how the forces on the
masses generated by the controls would work, eventually the masses
will come to rest at locations different from where they started, and
indeed, if we place the graph nodes at those locations, we get exactly
the layout in Figure 18.9(b). Notice how all of the edge-connected
nodes are the same length apart from each other – as it turns out, 80
pixels apart.
This methodology for graph layout is called F O R C E - D I R E C T E D
G R A P H L AY O U T based on its use of simulated forces to move the nodes
and edges around. The method can be generalized to much more ex-
pressive graphical constraints than just establishing fixed distances
between nodes with Hooke’s-law springs. For instance, we can have
force-generating controls that push masses to be in horizontal align-
ment, or vertical alignment. Using these controls, we can generate
 E X T E N S I O N A N D O B J E C T- O R I E N T E D P R O G R A M M I N G 353

(a) (b) (c)

Figure 18.10: A Hooke’s law spring
layouts like the one in Figure 18.9(c). Care must be taken however. If connecting two masses (labeled 0
we add too many controls in ways that overconstrain the physical sys- and 1) and its generated forces. The
pale red bar indicates the spring’s rest
tem, the result of finding the resting positions may not fully satisfy any length. (a) The spring at rest. No forces
of the constraints, leading to unattractive layouts as in Figure 18.9(d). on the masses. (b) When the spring is
stretched (the masses are farther apart
than the spring’s rest length), forces
18.7.2 Building a force-directed graph layout system (red arrows) are applied to the two
masses pushing them towards each
We’ve provided you with most of the components of a force-directed other. (c) Conversely, when the spring
is compressed (the masses are closer
graph layout system, written in an object-oriented design that is par-
together than the spring’s rest length),
ticularly appropriate for this task. We start with two-dimensional forces are applied to the two masses
P O I N T S. A point has x and y coordinates, and can thus represent a pushing them away from each other.

position on the canvas. The signature of a point class is specified in

the file points.mli. The interface file documents the functionality of
objects in the class, and we’ve provided a partial implementation of the
class in the file points.ml. You’ll notice that in addition to retrieving
the position of a point, a point can be moved directly to a new position.
Further, operations can be performed on points interpreted as
vectors, for instance, adding two points, or multiplying a point by a
scaling factor. We describe the various point operations by example.
The sum of points at (1, 3) and (7, 6) would be (8, 9). Scaling the result
by 2 would yield (16, 18). Subtracting (10, 10) yields (6, 8). The norm
of that point (its distance from the origin) is
√ √ √
62 + 82 = 36 + 64 = 100 = 10 .

The unit vector that corresponds to the vector (6, 8) is the vector in
the same direction but whose norm is 1, which we can generate by just
dividing the vector by its norm: (0.6, 0.8). The distance between two
points is the norm of their difference, so the distance between (5, 5)
and (5.6, 5.8) is 1.
M A S S E S are like points, except that they have a physical mass and
forces can act upon them. Look at the file masses.mli for information
about the mass class, a subclass of point. Again, the interface file
documents the functionality of objects in the class, and we’ve provided
a partial implementation of the class.
C O N T R O L S are force-generating objects – like springs or alignment
constraints. We’ve provided the code for those in controls.ml. There
is a control class along with subclasses for different types of controls:
springs, alignment constraints, repulsive forces, and the like. Note
how the controls affect masses, so control objects typically have one
354 PROGRAMMING WELL

or more masses as elements. For instance, a spring control (see the

bispring class) will have two masses as elements, the masses that the
spring connects.
G R A P H I C A L O B J E C T S are the kinds of things that get rendered on
the graphics canvas – small circles, squares, or other shapes repre-
senting nodes, edges drawn as connecting lines, boxes representing
a zone that includes several other graphical objects. These graphical
objects have various properties governing how they appear: what color
they are drawn in, what line thickness should be used, and so forth.
You’ll notice that some of these are common to all graphical objects
and appear in the drawable class. Others are particular to a subclass of
drawable, like the radius property of the circle class.
These classes make heavy use of OCaml’s ability to have named
arguments that are optional, taking a default value if the argument is
not provided. You may want to look at the discussion of labeled and
optional arguments in Real World OCaml to learn about the syntax
used.
Finally, the file graphdraw.ml implements functionality to “solve” a
graph layout problem. The solve function is provided a list of masses;
a list of constraints generating forces on those masses; and a scene,
a list of drawable graphical items located relative to those masses. It
uses a renderer (which we also provide) to display the scene in OCaml’s
graphics window (using the X11 windowing system that you installed
at the start of term), and animates the scene, moving the graphical
objects around the canvas as the forces apply to them, until the scene
settles into a final state. To get a sense of what the system looks like as
it performs the physical simulation of masses and forces, you can view
the demo movies at http://url.cs51.io/graphdraw.
Figure 18.11 depicts the interrelationships among the various
classes in the problem set (as a graph drawing!).

18.7.3 Completing the graph drawing system

The following files make up the system:

• points.ml and points.mli: The point class.

• masses.ml and masses.mli: The mass class.

• controls.ml: The various controls classes, for instance, bispring,

align, logisticrepel.

• graphobj.ml: Graphical object classes, including drawable and its

subclasses. We provide the circle subclass. You’ll add classes for
rectangle, square, edge, and zone.
 E X T E N S I O N A N D O B J E C T- O R I E N T E D P R O G R A M M I N G 355

drawable

control

point rectangle square circle edge zone

bispring align repel

mass

Legend

class we class you inherits references component

subclass superclass class
provide implement class

Figure 18.11: A map of the various

classes involved in this problem set.
• graphdraw.ml: Code for carrying out the physical simulation and
for rendering graphs in the graphics window. All of the physics
happens here, so you won’t need to deal with that part.

• testXXX.ml: Files that provide tests that you can run to see the
system in action. You may want to add your own tests.

The parts that you’ll need to do are as follows:

1. Implement the point class in points.ml consistent with the inter-

face in points.mli. Test it thoroughly.

2. Complete the implementation of the mass class in masses.ml

consistent with the interface in masses.mli. Test it thoroughly.

3. You should already be able to test the system on examples that only
use the circle graphical objects that we’ve provided. For instance,
the example in testuniformcentered.ml should already work.
Try building and running it and verify that the graphics window
launches and you see the animated layout process.

4. Add the additional graphical objects to graphobj.ml – rectangle

and square nodes, edges, and zone boxes – and try the other tests
we provided.

5. Construct an example graph drawing of your own, placing the

code in a file example.ml. The file should culminate in a function
example : unit -> unit, which when called uses the system to
lay out the graph nicely and display the result. We’ve provided a file
example.mli that it should be consistent with. If you’d like, post a
screenshot or video to Piazza. We’ll award a prize for the best graph
posted!
356 PROGRAMMING WELL

18.8 Problem set 9: Simulating an infectious process

This section provides substantive background for one of the course prob-
lem sets. For common background on logistics and procedures for prob-
lem sets, including such topics as accessing the problem set code, compil-
ing and testing your code, reflecting on the process, and submitting your
solution, please review the course document “Problem set procedures”.

Imagine an infection among a population of people where the agent

is transmitted from infected people to susceptible people nearby.
The time course of such a process depends on many factors: How
infectious is the agent? How much mixing is there of the population?
How nearby must people get to be subject to infection? How long does
recovery take? Is immunity conferred?
To get a sense of how such factors affect the overall course of the
infection, we can simulate the process, with configurable parameters
to control these and other aspects of the simulation.

18.8.1 The simulation

In this simulation, a population of people can be in one of several

states:

• Susceptible – The person has not been infected or has been infected
but is no longer immune.

• Infected – The person is infected and is therefore infectious and can

pass the infection on to susceptibles nearby.

• Recovered – The person was infected but recovered and has immu-
nity from further infection for a period of time.

• Deceased – The person was infected but did not recover.

(In the field of epidemiology, this kind of simulation is known as an

SIRD model for obvious reasons.)
The simulation proceeds through a series of time steps. At each
time step members of the population move on a two-dimensional grid
to nearby squares. (How far they move – how many squares in each
direction – is a configurable parameter.) Each person’s status updates
after they’ve moved. A susceptible person in the vicinity of infecteds
may become infected. (This depends on how large a vicinity is con-
sidered to be “nearby” and how infectious each of the people in that
vicinity are.) An infected person after a certain number of time steps
may recover or die. (The relative proportion depends on a mortality
parameter.) A recovered person after a certain number of time steps
may lose immunity, becoming susceptible again.
 E X T E N S I O N A N D O B J E C T- O R I E N T E D P R O G R A M M I N G 357

18.8.2 The simulator

We’ve provided you the basics of such a simulator, which you will
augment to be able to experiment with a wide range of scenarios.
The simulation can be visualized by showing the locations of the
people, color-coded as to their status. Figure 18.12 shows a visualiza-
tion of a simulation after some simulated time has elapsed. Suscep-
tibles are shown as small blue circles, infecteds as red, recovereds as
gray, and deceaseds as light gray “x” shapes. The radius around in-
fecteds where they can infect others is marked with a thin red circle as
well.
The visualization also provides a summary chart that shows the
proportion of the population in the different statuses over time. Fig-
ure 18.12 shows the full visualization containing the map and sum-
mary chart, and Figure 18.13 shows the visualization at the conclusion
of the simulated run. In this particular scenario, by the end of the run,
the infection had been eradicated.

Figure 18.12: A visualization of an infec-

tion spreading through a population.
On the left is a map of the population,
color-coded by infection status. Suscep-
tibles are shown as small blue circles,
infecteds as red, recovereds as gray, and
deceaseds as light gray “x” shapes. The
radius around infecteds where they can
infect others is marked with a thin red
circle as well. On the right is a stacked
bar chart showing the proportion of
the population in different statuses
over time. This snapshot was about 40
percent through the simulation.

The simulator is made up of several files, most of which you will not
need to modify. Those that you will be augmenting are given in italics.

• utilities.ml – Some generic utilities, such as functions to sample

from distributions, to flip weighted coins, to clip values, and the
like.

• counter.ml – Object-oriented counters that keep track of a running

total that can be incremented or decremented.

• statistics.ml – A set of counters to keep track of the number of

people in the population in various states.

• registry.ml – Defines a class type thing_type for things in the

world that participate in the simulation, and a module Registry for
358 PROGRAMMING WELL

Figure 18.13: The visualization after the

simulation has concluded.

storing a population of objects of this type to allow for easy access

to the objects.

• people.ml – Defines a person object, and provides the beginnings

of a set of subclasses for people of different statuses. In particular, it
provides a full implementation of susceptible people and a partial
implementation of infected people, but no implementation of
recovered or deceased people. This is the main file that you will be
augmenting.

• visualization.ml – Provides all of the code for generating visual-

izations of the time course of the infection.

• simulation.ml – Runs a simulation of an infectious process for a

population for a fixed number of time steps, generating the visual-
ization as the simulation proceeds.

• config.ml – Provides configurable parameters that govern all of the

details of the simulation and its visualization. You can experiment
with different scenarios by adjusting these parameters.

• run.ml – Runs the simulation.

18.8.3 Implementing the simulation

You’ll want to start by familiarizing yourself with the code by reading

through it. Figure 18.14 depicts a graph of the primary dependencies
among the files. The graph shows, for instance, that the people file pri-
marily makes use of registry and visualization. It makes sense to
start at the top of this graph and work your way down to get a sense of
the overall structure of the code, even though you’ll only be modifying
a few files. You’ll want to look through the file config.ml to get a sense
 E X T E N S I O N A N D O B J E C T- O R I E N T E D P R O G R A M M I N G 359

of the configurable parameters that you may want to use in completing

the simulation.

run simulation people Figure 18.14: The files implementing

the simulation, with arrows indicating
which files make substantial use of
which others.

statistics visualization registry utilities

counter config

Problem 212
By way of a warm-up exercise, you should implement the counter class in counter.ml.
This simple class is used to build some counters for maintaining statistics of how
many people are in each of the possible statuses and for tracking the time-steps in
the simulation. You can see how these counters are established in statistics.ml.

Once the counter class is working, the system should already be

able to be compiled and run. You can
% ocamlbuild run.byte

and then
% ./run.byte

to see the simulation in action. However, since only the two statuses of
susceptible and infected are implemented, and the latter only partially,
the simulation will not be complete and its visualization won’t conform
to the desired one.
Problem 213
Complete the implementation of the draw method for the infected class so that
infecteds show up as red circles with a radius marker as in Figure 18.12.

Next, you’ll complete the implementation of the infected class

by allowing for infecteds to recover after a certain number of time
steps. The number of time steps required before recovery should be
determined by sampling from a Gaussian distribution with mean
and standard deviation given by the parameter cRECOVERY_PERIOD
from the file config.ml. (This sounds more difficult than it is. We’ve
provided a Gaussian sampler in the utilities.ml file. You can call it
to get the sample, and store it somewhere appropriate. Then at each
update, you’ll decrement it until it reaches zero, at which time the
infected has recovered.)
Problem 214
Implement the update method for the infected class. It should check to see if the
infected has recovered, and if so, it should replace the infected object in the registry
with a recovered object. (The process is similar to how the susceptible object is
replaced with an infected object in the susceptible class we’ve provided.)
360 PROGRAMMING WELL

In order to complete this change, you’ll need a recovered class.

Problem 215
Add a recovered class, in addition to the susceptible and infected classes. Objects in
this class should have an immunity period, again sampled from an appropriate Gaussian
distribution (using the parameter cIMMUNITY_PERIOD). After the immunity period is
over, recovereds become susceptible again.

You now have a full simulation of the infection process, with people
cycling through from susceptible to infected to recovered to suscepti-
ble again. Use the opportunity to start experimenting with the various
configuration parameters in config.ml. What happens if you decrease
the neighbor radius (cNEIGHBOR_RADIUS), which is akin to “social
distancing”? What happens if you increase the step size (cSTEP_-
SIZE_SUSCEPTIBLE), which might be thought of as modeling increased
traveling. What happens if you greatly increase the immunity period?
Try out different scenarios and see what happens.
Finally, and perhaps most dramatically, the infection might have an
additional outcome, by virtue of its mortality.
Problem 216
Add a deceased class. In the infected class update method, after the infection period
is over, a proportion of people (governed by cMORTALITY) will become deceased rather
than recovered.

18.8.4 Exploration

Once you’ve got this all working, try out different scenarios. See how
parameters affect the results.
Feel free to augment the implementation. Here are some possibili-
ties, but you can certainly come up with your own.

• You might add a notion of “central quarantining”. After a certain

number of time steps have elapsed, at each time step thereafter
a small proportion of infecteds might change their properties (if
not their object class) to become quarantined. They move to a
central location (say, the middle of the grid) and their movement is
restricted by resetting their step size to zero and their infectiousness
to zero. When they recover, they move back to their location before
the quarantine.

• You might add a small number of locations on the map – “grocery

stores” we’ll call them – where people tend to congregate. Every
time step, a small proportion of people are relocated to the grocery
stores for a few time steps before returning back to where they came
from.

• You might establish a capacity for treating infecteds such that when
there are more infecteds than the capacity, mortality increases. It
 E X T E N S I O N A N D O B J E C T- O R I E N T E D P R O G R A M M I N G 361

would then become more important to “flatten the curve”. Can you
adjust parameters to do so? If so, which parameters work best?

Problem 217
To present your experiments, we ask that you make a short video of perhaps three to five
minutes presenting some scenarios that you’ve looked at. If they involve extensions of
the sort above, all the better.
To make the recording, you can use whatever video or screen-recording system you
prefer, but a simple one-person Zoom session using Zoom’s built-in local recording
feature should be sufficient.
19
Semantics: The environment model

The addition of mutability – which enables impure programming

paradigms like imperative and procedural programming, with its
potential for efficiencies in both time and space – comes at a cost.
Leibniz’s law no longer applies. One and the same expression in the
same context can evaluate to different values, making reasoning about
programs more difficult.
That complexity ramifies in providing explicit semantics for the
language as well. The simple substitution semantics of Chapter 13 is
no longer sufficient. For that reason, and looking forward to the im-
plementation of an interpreter for a larger fragment of OCaml (Chap-
ter 21), we revisit the formal substitution semantics from Chapter 13,
modifying and augmenting it to provide a rigorous semantics for ref-
erences and assignment, showing where the additional complexity
arises and clarifying issues such as scope, side effects, and order of
evaluation.

19.1 Review of substitution semantics

Recall from Section 13.6 the abstract syntax of a simple functional

language with arithmetic:

⟨binop ⟩ ::= +|-|*|/

⟨var ⟩ ::= x|y|z|⋯
⟨expr ⟩ ::= ⟨integer ⟩
| ⟨var ⟩
| ⟨expr1 ⟩ ⟨binop ⟩ ⟨expr2 ⟩
| let ⟨var ⟩ = ⟨exprdef ⟩ in ⟨exprbody ⟩
| fun ⟨var ⟩ -> ⟨exprbody ⟩
| ⟨exprfun ⟩ ⟨exprarg ⟩
364 PROGRAMMING WELL

The semantics for this language was provided through the apparatus of
evaluation rules, which defined derivations for judgements of the form

P ⇓v

where P is an expression and v is its value (a simplified expression that

means the same and that cannot be further evaluated).
The substitution semantics is sufficient for this simple language
because it is a pure functional programming language. But binding
constructs like let, let rec, and fun are awkward to implement,
and extending the language to handle references, mutability, and
imperative programming is quite challenging if not impossible. For
that reason, we start by modifying the substitution semantics to make
use of an E N V I R O N M E N T that stores a mapping from variables to
their values. In the next two sections, we develop the environment
semantics for the language of Chapter 13 in two variants: a dynamic
environment semantics and a lexical environment semantics. We then
augment the environment semantics with a model of a mutable store
to allow for reference values and their assignment.

19.2 Environment semantics

In an environment semantics, instead of substituting for variables

the value that they specify, we directly model a mapping between
variables and their values, which we call an E N V I R O N M E N T . We use
the following notation for mappings in the semantics. A mapping from
elements, say, x, y, z, to elements a, b, c, respectively, will be notated
as {x ↦ a; y ↦ b; z ↦ c }. The notation purposefully evokes the OCaml
record notation, since a record also provides a kind of mapping from
a finite set of elements (labels) to associated values. It also evokes,
through the use of the ↦ symbol, the idea of substitution, as these
mappings will replace substitutions in the environment semantics.
Indeed, the environments that give their name to environment
semantics are just such mappings – from variables to their values. We’ll
conventionally use the symbol E and its primed versions (E ′ , E ′′ , . . . )
as variables standing for environments. The empty environment will
be notated {}, and the environment E augmented so as to add the
mapping of the variable x to the value v will be notated E {x ↦ v }.
To look up what value an environment E maps a variable x to, we use
Euler’s function application notation: E (x ).
Having introduced the necessary notation, we turn to modifying the
substitution semantics to use environments instead.
 SEMANTICS: THE ENVIRONMENT MODEL 365

19.2.1 Dynamic environment semantics

Recall that the substitution semantics is given through a series of rules

defining judgements of how expressions evaluate to values. (Reviewing
Figure 13.5 may refresh your memory.)
In an environment semantics, expressions aren’t evaluated in isola-
tion. Rather, they are evaluated in the context of an environment that
specifies which variables have which values. Instead of defining rules
for P evaluating to v (written as the judgement P ⇓ v), we define rules
for P evaluating to v in an environment E (written as the judgement
E ⊢ P ⇓ v). The rule for evaluating numbers, for instance, becomes

E ⊢n⇓n (R int )

stating that “in environment E a numeral n evaluates to itself (inde-

pendent of the environment)”, and the rule for addition provides the
environment as context for evaluating the subexpressions:

E ⊢P + Q ⇓

E ⊢P ⇓m
∣ (R + )
E ⊢Q ⇓n

⇓ m +n

Glossing again, the rule says “to evaluate an expression of the form P
+ Q in an environment E , first evaluate P in the environment E to an
integer value m and Q in the environment E to an integer value n. The
value of the full expression is then the integer literal representing the
sum of m and n.”
To construct a derivation for a whole expression using these rules,
we start in the empty environment {}. For instance, a derivation for
the expression 3 + 5 would be

{} ⊢ 3 + 5 ⇓
{} ⊢ 3 ⇓ 3
∣
{} ⊢ 5 ⇓ 5
⇓8

So far, not much is different from the substitution semantics. The

differences show up in the handling of binding constructs like let.
Recall the R let rule for let binding in the substitution semantics.

let x = D in B ⇓

D ⇓ vD
∣ (R let )
B [x ↦ v D ] ⇓ v B

⇓ vB
366 PROGRAMMING WELL

This rule specifies that an expression of the form let x = D in

B evaluates to the value v B , whenever the definition expression D
evaluates to v D and the body expression B after substituting v B for the
variable x evaluates to v B .
The corresponding environment semantics rule doesn’t substi-
tute into B . It evaluates B directly, but it does so in an environment
augmented with a new binding of x to its value v D :

E ⊢ let x = D in B ⇓

E ⊢ D ⇓ vD
∣ (R let )
E {x ↦ v D } ⊢ B ⇓ v B

⇓ vB
According to this rule, “to evaluate an expression of the form let x
= D in B in an environment E , first evaluate D in E resulting in a
value v D and then evaluate the body B in an environment that is like E
except that the variable x is mapped to the value v D . The result of this
latter evaluation, v B , is the value of the let expression as a whole.”
In the substitution semantics, we will have substituted away all of
the bound variables in a closed expression, so no rule is needed for
evaluating variables themselves. But in the environment semantics,
since no substitution occurs, we’ll need to be able to evaluate expres-
sions that are just variables. Presumably, those variables will have
values in the prevailing environment; we’ll just look them up.

E ⊢ x ⇓ E (x ) (R var )

A gloss for this rule is “evaluating a variable x in an environment E

yields the value of x in E .”
Putting all these rules together, we can derive a value for the expres-
sion let x = 3 in x + x:
{} ⊢ let x = 3 in x + x ⇓
RRR
RRR {} ⊢ 3 ⇓ 3
RRR {x ↦ 3} ⊢ x + x ⇓
RRR
RRR {x ↦ 3} ⊢ x ⇓ 3
RRR ∣
RRR {x ↦ 3} ⊢ x ⇓ 3
RRR
RRR ⇓ 6
R
⇓6
The derivation makes clear how the environment semantics differs
from the substitution semantics. Rather than replacing a bound vari-
able with its value, we add the bound variable with its value to the
environment; when an occurrence of the variable is reached, we sim-
ply look up its value in the environment.
 SEMANTICS: THE ENVIRONMENT MODEL 367

Exercise 218
Construct the derivation for the expression
let x = 3 in
let y = 5 in
x + y ;;

Exercise 219
Construct the derivation for the expression
let x = 3 in
let x = 5 in
x + x ;;

Continuing the translation of the substitution semantics directly

into an environment semantics, we turn to functions and their appli-
cation. Maintaining functions as values is reflected in this simple rule:

E ⊢ fun x -> P ⇓ fun x -> P (R fun )

and the application of a function to its argument again adds the ar-
gument’s value to the environment used in evaluating the body of the
function:

E ⊢P Q ⇓
RRR E ⊢ P ⇓ fun x -> B
RRR
RRR E ⊢ Q ⇓ v (R app )
RRR Q
RRR E {x ↦ v } ⊢ B ⇓ v
R Q B

⇓ vB

Exercise 220
Provide glosses for these two rules.
We can try the example from Section 13.6:
(fun x -> x + x) (3 * 4)

whose evaluation to 24 is captured by the following derivation:

{} ⊢ (fun x -> x + x) (3 * 4)
⇓
RRR {} ⊢ (fun x -> x + x) ⇓ (fun x -> x + x)
RRR
RRR {} ⊢ 3
RRR * 4⇓
RRR {} ⊢ 3 ⇓ 3
RRR ∣
RRR {} ⊢ 4 ⇓ 4
RRR
RRR ⇓ 12
RRR
RRR {x ↦ 12} ⊢ x + x ⇓
RRR
RRR {x ↦ 12} ⊢ x ⇓ 12
RRR ∣
RRR {x ↦ 12} ⊢ x ⇓ 12
RRR
RRR ⇓ 24
⇓ 24
368 PROGRAMMING WELL

The full set of dynamic environment semantics rules so far is pre-

sented in Figure 19.1.

Figure 19.1: Dynamic environment

semantics rules for evaluating expres-
sions, for a functional language with
E ⊢n⇓n (R int ) naming and arithmetic.

E ⊢ x ⇓ E (x ) (R var )

E ⊢ fun x -> P ⇓ fun x -> P (R fun )

E ⊢P + Q ⇓

E ⊢P ⇓m
∣ (R + )
E ⊢Q ⇓n

⇓ m +n

(and similarly for other binary operators)

E ⊢ let x = D in B ⇓

E ⊢ D ⇓ vD
∣ (R let )
E {x ↦ v D } ⊢ B ⇓ v B

⇓ vB

E ⊢P Q ⇓
RRR E ⊢ P ⇓ fun x -> B
RRR
RRR E ⊢ Q ⇓ v (R app )
RRR Q
RRR E {x ↦ v } ⊢ B ⇓ v
R Q B

⇓ vB

Problems with the dynamic semantics The environment semantics

captured in these rules (Figure 19.1) seems like it should generate the
same evaluations as the substitution semantics (Figure 13.5). After
all, the only difference would seem to be that instead of the binding
constructs (let and fun) substituting a value for the variables they
bind, they place the value in the environment, to be retrieved when the
variables they bind need them. But there are subtle differences, hidden
in the decision as to which variable occurrences see which values.
Recall (Section 5.4) that in OCaml the connection between occur-
rences of variables and the binding constructs they are bound by is
 SEMANTICS: THE ENVIRONMENT MODEL 369

determined by the lexical structure of the code. For instance, in the

expression

# let x = 1 in
# let f = fun y -> x + y in
# let x = 2 in
# f 3 ;;
Line 3, characters 4-5:
3 | let x = 2 in
^
Warning 26: unused variable x.
- : int = 4

the highlighted occurrence of the variable x is bound by the outer let

x, not the inner. For that reason, the result of evaluating the expression
is 4, and not 5. The substitution semantics reflects this fact, as seen in
the derivation

let x = 1 in let f = fun y -> x + y in let x = 2 in f 3

⇓
RRR
RRR 1 ⇓ 1
RRR let f = fun y -> 1 + y in let x = 2 in f 3
RRR
RRR ⇓
RRR
RRR RRR
RRR RRR fun y -> 1 + y ⇓ fun y -> 1 + y
RRR RRR let x = 2 in (fun y -> 1 + y) 3
RRR RRR
RRR RRR ⇓
RRR RRR
RRR R
RRR RRR
RRR RRR RRR 2 ⇓ 2
RRR RRR RRR (fun y -> 1 + y) 3
RRR RRR RRR
RRR RRR RRR ⇓
RRR RRR RRR
RRR R
RRR RRR fun y -> 1 + y ⇓ fun y -> 1 + y
RRR RRR
RRR RRR RRR RRR 3 ⇓ 3
RRR RRR RRR RRR
RRR RRR R
RRR RRR 1 + 3 ⇓
RRR RRR RRR RRR
RRR RRR RRR RRR
RRR RRR RRR 1⇓1
RRR RRR ∣
RRR RRR R
R ⇓3
RRR RRR RRR RRR 3
RRR RRR RRR RRR ⇓4
RRR RRR RRR R
RRR RRR RRR ⇓4
RRR RRR R
RRR RRR ⇓ 4
RRR R
RRR ⇓ 4
R
⇓4

But the environment semantics evaluates this expression to 5.

Exercise 221
Before proceeding, see if you can construct the derivation for this expression according
to the environment semantics rules. Do you see where the difference lies?
According to the environment semantics developed so far, a deriva-
 370 PROGRAMMING WELL

tion for this expression proceeds as

{} ⊢ let x = 1 in let f = fun y -> x + y in let x = 2 in f 3

⇓
RRR {} ⊢ 1 ⇓ 1
RRR
RRR {x ↦ 1} ⊢ let f = fun y -> x + y in let x = 2 in f 3
RRR
RRR ⇓
RRR
RRR RRR
RRR RRR {x ↦ 1} ⊢ fun y -> x + y ⇓ fun y -> x + y
RRR RRR
RRR RRR {x ↦ 1; f ↦ fun y -> x + y} ⊢ let x = 2 in f 3
RRR RRR
RRR RRR ⇓
RRR RRR RRR {x ↦ 1; f ↦ fun y -> x + y} ⊢ 2 ⇓ 2
RRR RRR RRR
RRR RRR RRR {f ↦ fun y -> x + y; x ↦ 2} ⊢ f 3
RRR RRR RRR
RRR RRR RRR
RRR RRR RRR ⇓
RRR R
RRR RRR RRR {f ↦ fun y -> x + y; x ↦ 2} ⊢ f ⇓ fun y -> x + y
RRR RRR RRR RRR
RRR RRR R
RRR RRR {f ↦ fun y -> x + y; x ↦ 2} ⊢ 3 ⇓ 3
RRR RRR RRR RRR
RRR RRR RRR RRR {f ↦ fun y -> x + y; x ↦ 2; y ↦ 3} ⊢ x + y
RRR RRR RRR RRR
RRR RRR RRR RRR ⇓
RRR RRR RRR RRR
RRR RRR RRR RRR {f ↦ fun y -> x + y; x ↦ 2; y ↦ 3} ⊢ x ⇓ 2
RRR RRR RRR RRR ∣
RRR RRR RRR R
RRR {f ↦ fun y -> x + y; x ↦ 2; y ↦ 3} ⊢ y ⇓ 3
RRR RRR RRR RRR
RRR RRR RRR RR ⇓5
RRR RRR RRR
RRR RRR RR ⇓5
RRR RRR
RRR RR ⇓5
RRR
RR ⇓ 5

⇓5
The crucial difference comes when augmenting the environment
during application of the function fun y -> x + y to its argument.
Examine closely the two highlighted environments in the derivation
above. The first is the environment in force when the function is de-
fined, the L E X I C A L E N V I R O N M E N T of the function. The second is the
environment in force when the function is applied, its DY N A M I C E N -
V I R O N M E N T. The environment semantics presented so far augments
the dynamic environment with the new binding induced by the appli-
cation. It manifests a DY N A M I C E N V I R O N M E N T S E M A N T I C S . But for
consistency with the substitution semantics (which substitutes occur-
rences of a bound variable when the binding construct is defined, not
applied), we should use the lexical environment, thereby manifesting a
L E X I C A L E N V I R O N M E N T S E M A N T I C S.
In Section 19.2.2, We’ll develop a lexical environment semantics
that cleaves more faithfully to the lexical scope of the substitution
semantics, but first, we note some other divergences between dynamic
and lexical semantics.
Consider this simple application of a curried function:
 SEMANTICS: THE ENVIRONMENT MODEL 371

(fun x -> fun y -> x + y) 1 2

The substitution semantics rules specify that this expression evaluates

to 3. But the dynamic semantics misbehaves:

{} ⊢ (fun x -> fun y -> x + y) 1 2

⇓
RRR {} ⊢ (fun x -> fun y -> x + y) 1
RRR
RRR ⇓
RRR
RRR RRR {} ⊢ fun x -> fun y -> x + y ⇓ fun x -> fun y -> x + y
RRR RRR
RRR RRR {} ⊢ 1 ⇓ 1
RRR RRR
RRR RRR {x ↦ 1} ⊢ fun y -> x + y ⇓ fun y -> x + y
RRR R
RRR ⇓ fun y -> x + y
RRR
RRR {} ⊢ 2 ⇓ 2
RRR
RRR {y ↦ 2} ⊢ x + y
RRR
RRR ⇓
RRR
RRR {y ↦ 2} ⊢ x ⇓ ???
RRR ∣
RRR ⋯
RRR
RR ⇓ ???

⇓ ???

We can start the derivation, but the dynamic environment available

when we come to evaluate the x in the function body contains no
binding for x! (If only we had been evaluating the body in its lexical
environment.) In a dynamic semantics, currying – so central to many
functional idioms – becomes impossible.
On the other hand, under a dynamic semantics, recursion needs no
special treatment. By using the dynamic environment in evaluating the
definiendum of a let, the definition of the bound variable is already
available. We revisit the derivation for factorial from Section 13.7, but
 372 PROGRAMMING WELL

this time using the dynamic environment semantics:

{} ⊢ let f = fun n -> if n = 0 then 1 else n * f (n - 1) in f 2

⇓
RRR {} ⊢ fun n -> if n = 0 then 1 else n
RRR * f (n - 1)
RRR ⇓ fun n -> if n = 0 then 1 else n * f (n - 1)
RRR
RRR {f ↦ fun n -> if n = 0 then 1 else n * f (n - 1)} ⊢ f 2
RRR
RRR ⇓
RRR
RRR RRR {f ↦ fun n -> if n = 0 then 1 else n
RRR * f (n - 1)} ⊢ f
RRR
RRR R
RRR ⇓ fun n -> if n = 0 then 1 else n * f (n - 1)
RRR RRR
RRR RRR {f ↦ fun n -> if n = 0 then 1 else n * f (n - 1)} ⊢ 2
RRR RRR
RRR RRR ⇓2
RRR RRR
RRR RRR { f ↦ if n = 0 then 1 else n * f (n - 1); n ↦ 2} ⊢ if n = 0 then 1 else n * f (n - 1)
RRR RRR
RRR RRR ⇓
RRR RRR ∣ ⋯
RRR RRR
RRR RRR ⇓2
RRR R
RRR ⇓2
R
⇓2

Notice how the body of the function, with its free occurrence of the
variable f, is evaluated in an environment in which f is bound to the
function itself. By using the dynamic environment semantics rules, we
get recursion “for free”. Consequently, the dynamic semantics rule for
the let rec construction can simply mimic the let construction:

E ⊢ let rec x = D in B ⇓

E ⊢ D ⇓ vD
∣ (R letrec )
E {x ↦ v D } ⊢ B ⇓ v B

⇓ vB

To truly reflect the intended semantics of expressions in an envi-

ronment semantics, we need to find a way of using the lexical envi-
ronment for functions instead of the dynamic environment; we need a
lexical environment semantics.

19.2.2 Lexical environment semantics

1
The term comes from the terminology
To modify the rules to provide a lexical rather than dynamic environ- of open versus closed expressions.
ment semantics, we must provide some way of capturing the lexical Open expressions have free variables
in them; closed expressions have none.
environment when functions are defined. The technique is to have
By capturing the defining environment,
functions evaluate not to themselves (awaiting the dynamic environ- we essentially use it to close the free
ment for interpretation of the variables within them), but rather to variables in the function. The closure
thus turns what would otherwise
have them evaluate to a “package” containing the function and its be an open expression into a closed
lexical (defining) environment. This package is called a C L O S U R E .1 expression.
 SEMANTICS: THE ENVIRONMENT MODEL 373

We’ll notate the closure that packages together a function P and its
environment E as [E ⊢ P ]. In evaluating a function, then, we merely
construct such a closure, capturing the function’s defining environ-
ment.

E ⊢ fun x -> P ⇓ [E ⊢ fun x -> P ] (R fun )

We make use of closures constructed in this way when the function is

applied:

Ed ⊢ P Q ⇓
RRR E ⊢ P ⇓ [E ⊢ fun x -> B ]
RRR d l
RRR E ⊢ Q ⇓ v (R app )
RRR d Q
RRR E {x ↦ v } ⊢ B ⇓ v
R l Q B

⇓ vB

Rather than augmenting the dynamic environment E d in evaluating

the body, we augment the lexical environment E l extracted from the
closure.
The lexical environment semantics properly reflects the intended
semantics for several of the problematic examples in Section 19.2.1,
as demonstrated in the following exercises. However, the handling
of recursion still requires some further work, which we’ll return to in
Section 19.4.

Exercise 222
Carry out the derivation using the lexical environment semantics for the expression
let x = 1 in
let f = fun y -> x + y in
let x = 2 in
f 3 ;;

What value does it evaluate to under the lexical environment semantics?

Exercise 223
Carry out the derivation using the lexical environment semantics for the expression
(fun x -> fun y -> x + y) 1 2 ;;

Problem 224
In problem 170, you evaluated several expressions as OCaml would, with lexical scoping.
Which of those expressions would evaluate to a different value using dynamic scoping?

19.3 Conditionals and booleans

In Section 13.5, exercises asked you to develop abstract syntax and

substitution semantics rules for booleans and conditionals. In this
section, we call for similar rules for environment semantics (applicable
to dynamic or lexical variants).
374 PROGRAMMING WELL

Exercise 225
Adjust the substitution semantics rules for booleans from Exercise 147 to construct
environment semantics rules for the constructs.

Exercise 226
Adjust the substitution semantics rules for conditional expressions (if ⟨⟩ then ⟨⟩ else ⟨⟩
) from Exercise 148 to construct environment semantics rules for the construct.

19.4 Recursion

The dynamic environment semantics already allows for recursion – in

fact, too much recursion – because of its dynamic nature. Think about
an ill-formed “almost-recursive” function, like

let f = fun x -> if x = 0 then 1 else f (x - 1) in f 1 ;;

It’s ill-formed because the lack of a rec keyword means that the f in
the definition part ought to be unbound. But it works just fine in the
dynamic environment semantics. When f 1 is evaluated in the dy-
namic environment in which f is bound to fun x -> if x = 0 then
1 else f (x - 1), all of the subexpressions of the definiens, includ-
ing the occurrence of f itself, will be evaluated in an augmentation
of that environment, so the “recursive” occurrence of f will obtain a
value. (It is perhaps for this reason that the earliest implementations of
functional programming languages, the original versions of LISP, used
a dynamic semantics.)
The lexical semantics, of course, does not benefit from this fortu-
itous accident of definition. The lexical environment in force when f
is defined is empty, and thus, when the body of f is evaluated, it is the
empty environment that is augmented with the argument x bound to
1. There is no binding for the recursively invoked f, and the deriva-
tion cannot be completed – consistent, by the way, with how OCaml
behaves:

# let f = fun x -> if x = 0 then 1 else f (x - 1) in f 1 ;;

Line 1, characters 38-39:
1 | let f = fun x -> if x = 0 then 1 else f (x - 1) in f 1 ;;
^
Error: Unbound value f
Hint: If this is a recursive definition,
you should add the 'rec' keyword on line 1

To allow for recursion in the lexical environment semantics, we

should add a special rule for let rec then. A let rec expression is
built from three parts: a variable name (x), a definition expression (D),
and a body (B ). To evaluate it, we ought to first evaluate the definition
part D, but using what environment? Any functions inside the defi-
nition part will see this environment as their lexical environment, to
 SEMANTICS: THE ENVIRONMENT MODEL 375

be captured in a closure. We’ll thus want to make a value for x avail-

able in that environment. But what will we use for the value of x in the
environment? We can’t merely map x to the definition D, with its free
occurence of x; that just postpones the problem.
In one sense, it doesn’t matter what value we use for x in evaluating
the definition D, because in evaluating D, we won’t (or at least better
not) make use of x directly, as for instance in

# let rec x = x + 1 in x ;;
Line 1, characters 12-17:
1 | let rec x = x + 1 in x ;;
^^^^^
Error: This kind of expression is not allowed as right-hand side of
`let rec'

That wouldn’t be a well-founded recursion. Instead, the occurrences

of x in D will have to be in contexts where they are not evaluated.
Canonically, that would be within an unapplied function, like the
factorial example

# let rec f = fun n -> if n = 0 then 1 else n * f (n - 1) in f 2 ;;

- : int = 2

Because of this requirement for well-founding of the recursion, what-

ever value we use for x, we’ll be able to evaluate the definition to some
value, call it v D . That value, however, may involve closures that capture
the binding for x, and we’ll need to look up the value for x later in eval-
uating the body. Thus, the environment used in evaluating the body
best have a binding for x to v D .
These considerations call for the following approach to handling
the semantics of let rec in an environment E . We start by forming an
environment E ′ that extends E with a binding for x, but a binding that
is mutable, so it can be changed later. Initially, x can be bound to some
recognizable and otherwise ungenerable value, say, Unassigned. We
evaluate the definition D in environment E ′ to a value v D , which may
capture E ′ (or extensions of it) in closures. We then change the value
stored for x in E ′ to v D , and evaluate the body B in E ′ (thus modified).
By mutating the value bound to x, any closures that have captured E ′
will see this new value for x as well, so that (recursive) lookups of x in
the body will see the evaluated v D as well.
Because this approach relies on mutation, our notation for environ-
ment semantics isn’t up to the task of formalizing this idea, and doing
so is beyond the scope of this discussion, so we’ll leave it at that for
now. But once mutability is incorporated into the semantics – that was
the whole point in moving to an environment semantics, remember –
we’ll revisit the issue and give appropriate rules for let rec.
Even without formal rules for let rec, you’ll see in Chapter 21 how
376 PROGRAMMING WELL

this approach can be implemented in an interpreter for a language

with a let rec construction.

Figure 19.2: Lexical environment

semantics rules for evaluating expres-
sions, for a functional language with
E ⊢n⇓n (R int ) naming and arithmetic.

E ⊢ x ⇓ E (x ) (R var )

E ⊢ fun x -> P ⇓ [E ⊢ fun x -> P ] (R fun )

E ⊢P + Q ⇓

E ⊢P ⇓m
∣ (R + )
E ⊢Q ⇓n

⇓ m +n

(and similarly for other binary operators)

E ⊢ let x = D in B ⇓

E ⊢ D ⇓ vD
∣ (R let )
E {x ↦ v D } ⊢ B ⇓ v B

⇓ vB

Ed ⊢ P Q ⇓
RRR E ⊢ P ⇓ [E ⊢ fun x -> B ]
RRR d l
RRR E ⊢ Q ⇓ v (R app )
RRR d Q
RRR E {x ↦ v } ⊢ B ⇓ v
R l Q B

⇓ vB

19.5 Implementing environment semantics

In Section 13.4.2, we presented an implementation of the substitution

semantics in the form of a function eval : expr -> expr. Modifying
it to follow the environment semantics requires just a few simple
changes. First, evaluation is relative to an environment, so the eval
function should take an additional argument, of type, say env. Second,
under the lexical environment semantics, expressions evaluate to
values that include more than just the pertinent subset of expressions.
In particular, expressions may evaluate to closures, so that an extended
 SEMANTICS: THE ENVIRONMENT MODEL 377

notion of value, codified in a type value is needed. In summary, the

type of eval should be expr -> env -> value.
The new env type, a simple mapping from variables to the values
they are bound to, can be implemented as an association list

type env = (varid * value ref) list

and the value type can include expression values and closures in a
simple variant type

type value =
| Val of expr
| Closure of (expr * env)

(The env data structure maps variables to mutable value refs to

allow for the mutation required in implementing let rec as described
in Section 19.4.) The carrying out of this exercise is the subject of
Chapter 21.

19.6 Semantics of mutable storage

In this section, we further extend the lexical environment semantics to

allow for imperative programming with references and assignment. (As
a byproduct, we’ll have the infrastructure to probide a formal seman-
tics rule for let rec.) To do so, we’ll start by augmenting the syntax of
the language, and then adjust the environment semantic rules so that
the context of evaluation includes not only an environment, but also a
model for the mutable storage that references require.
We’ll start with adding to the syntax a unit value () and operators
(ref, !, and :=) to manipulate reference values:

⟨binop ⟩ ::= +|-|*|/

⟨var ⟩ ::= x|y|z|⋯
⟨expr ⟩ ::= ⟨integer ⟩
| ⟨var ⟩
| ⟨expr1 ⟩ ⟨binop ⟩ ⟨expr2 ⟩
| let ⟨var ⟩ = ⟨exprdef ⟩ in ⟨exprbody ⟩
| fun ⟨var ⟩ -> ⟨exprbody ⟩
| ⟨exprfun ⟩ ⟨exprarg ⟩
| ()
| ref ⟨expr ⟩
| ! ⟨expr ⟩
| ⟨var ⟩ := ⟨expr ⟩

The plan for handling references is to add a new kind of value, a

L O C AT I O N , which is an index or pointer into an abstract model of
memory that we will call the S T O R E . A store S will be a finite mapping
378 PROGRAMMING WELL

(like the environment) from locations to values. So a reference to a

value v will be a location l such that the store S maps l to v. Evaluation
will need to be relative to a store in addition to an environment, so
evaluation judgements will look like E , S ⊢ P ⇓ ⋯.
E , S ⊢ P ⇓ vP , S′
Because the store can change as a side effect of evaluation (that’s ° ® b
²
the whole point of mutability), the result of evaluation can’t simply be a
a c
value. We’ll need access to the modified store as well. So the right-hand ´¹¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¸¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¶
side of the evaluation arrow ⇓ will provide both a value and a store. Our d
final form for evaluation judgements is thus ´¹¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¸¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¶
e
Figure 19.3: Anatomy of an evaluation
E , S ⊢ P ⇓ vP , S′ . judgement. (a) The context of evalua-
tion, including an environment E and a
store S. (b) The expression to be evalu-
ated. (c) The result of the evaluation, a
(See Figure 19.3 for a breakdown of such a judgement.)
value and a store mutated by side effect.
A semantic rule for references reflects these ideas: (d) The evaluation of P to its result.
(e) The judgement as a whole. “In the
environment E and store S, expression
E , S ⊢ ref P ⇓ P evaluates to value v P with modified
store S ′ .”
∣ E , S ⊢ P ⇓ vP , S′ (R ref )

⇓ l , S ′ {l ↦ v P } (where l is a new location)

According to this rule, “to evaluate an expression of the form ref P in

an environment E and store S, we evaluate P in that environment and
store, yielding a value v P for P and a new store S ′ (as there may have
been side effects to S in the evaluation). The value for the reference is
a new location l , and as side effect, a new store that is S ′ augmented so
that l maps to the value v P .”
To dereference such a reference, as in an expression of the form
! P , P will need to be evaluated to a location, and the value at that
location retrieved.

Exercise 227

Write a semantic rule for dereferencing references.

Finally, and most centrally to the idea of mutable storage, is assign-
ment to a reference. Evaluating an assignment of the form P := Q
involves evaluating P to a location l and evaluating Q to a value vQ ,
and updating the store so that l maps to vQ . Along the way, the various
subevaluations may themselves have side effects leading to updated
stores, which must be dealt with. For instance, starting with an envi-
ronment E and store S, evaluating P may result in an updated store
S ′ . That updated store would then be the store with respect to which
Q would be evaluated, leading to a possibly updated store S ′′ . It is this
final store that would be augmented with the new assignment. A rule
 SEMANTICS: THE ENVIRONMENT MODEL 379

specifying this semantics is

E , S ⊢ P := Q ⇓

E , S ⊢ P ⇓ l , S′
∣ (R assign )
E , S ′ ⊢ Q ⇓ vQ , S ′′

⇓ (), S ′′ {l ↦ vQ }

The important point of the rule is the update to the store. But like
all evaluation rules, a value must be returned for the expression as a
whole. Here, we’ve simply returned the unit value ().

Exercise 228
In the presence of side effects, sequencing (with ;) becomes important. Write an evalua-
tion rule for sequencing.
To complete the semantics of mutable state, the remaining rules
must be modified to use and update stores appropriately. Figure 19.4
provides a full set of rules.
As an example of the deployment of these semantic rules, we con-
sider the expression

let x = ref 3 in
x := 42;
!x

Here is the derivation in full.

{}, {} ⊢ let x = ref 3 in x := 42; !x

⇓
RRR {}, {} ⊢ ref 3 ⇓
RRR
RRR ∣ {}, {} ⊢ 3 ⇓ 3, {}
RRR
RRR ⇓ l 1 , {l 1 ↦ 3}
RRR
RRR {x ↦ l 1 }, {l 1 ↦ 3} ⊢ x := 42; !x
RRR
RRR ⇓
RRR
RRR RRR {x ↦ l }, {l ↦ 3} ⊢ x := 42
RRR RRR 1 1
RRR RRR ⇓
RRR RRR
RRR RRR {x ↦ l 1 }, {l 1 ↦ 3} ⊢ x ⇓ l 1 , {l 1 ↦ 3}
RRR RRR ∣
RRR R
RRR {x ↦ l 1 }, {l 1 ↦ 3} ⊢ 42 ⇓ 42, {l 1 ↦ 3}
RRR RRR
RRR RRR ⇓ () , {l 1 ↦ 42}
RRR RRR
RRR RRR {x ↦ l 1 }, {l 1 ↦ 42} ⊢ !x
RRR RRR
RRR RRR ⇓
RRR RRR ∣ {x ↦ l 1 }, {l 1 ↦ 42} ⊢ x ⇓ l 1 , {l 1 ↦ 42}
RRR RRR
RRR RRR ⇓ 42, {l 1 ↦ 42}
RRR R
RRR ⇓ 42 , { l 1 ↦ 42}
R
⇓ 42, {l 1 ↦ 42}
380 PROGRAMMING WELL

Figure 19.4: Lexical environment

semantics rules for evaluating ex-
pressions, for a functional language
E , S ⊢ n ⇓ n, S (R int ) with naming, arithmetic, and mutable
storage.
E , S ⊢ x ⇓ E ( x ), S (R var )

E , S ⊢ fun x -> P ⇓ [E ⊢ fun x -> P ], S (R fun )

E,S ⊢ P + Q ⇓

E , S ⊢ P ⇓ m, S ′
∣ (R + )
E , S ′ ⊢ Q ⇓ n, S ′′

⇓ m + n, S ′′

(and similarly for other binary operators)

E , S ⊢ let x = D in B ⇓

E , S ⊢ D ⇓ vD , S′
∣ (R let )
E {x ↦ v D }, S ′ ⊢ B ⇓ v B , S ′′

⇓ v B , S ′′

Ed , S ⊢ P Q ⇓
RRR E , S ⊢ P ⇓ [E ⊢ fun x -> B ], S ′
RRR d l
RRR E , S ′ ⊢ Q ⇓ v , S ′′ (R app )
RRR d Q
RRR E {x ↦ v }, S ′′ ⊢ B ⇓ v , S ′′′
R l Q B

⇓ v B , S ′′′
 SEMANTICS: THE ENVIRONMENT MODEL 381

Figure 19.4: (continued) Lexical envi-

ronment semantics rules for evaluating
expressions, for a functional language
E , S ⊢ ref P ⇓
with naming, arithmetic, and mutable
∣ E , S ⊢ P ⇓ vP , S′ (R ref ) storage.

⇓ l , S {l ↦ v P }
′
(where l is a new location)

E,S ⊢ ! P ⇓
∣ E , S ⊢ P ⇓ l , S′ (R deref )

⇓ S ′ (l ), S ′

E , S ⊢ P := Q ⇓

E , S ⊢ P ⇓ l , S′
∣ (R assign )
E , S ′ ⊢ Q ⇓ vQ , S ′′

⇓ (), S ′′ {l ↦ vQ }

E,S ⊢ P ; Q ⇓

E , S ⊢ P ⇓ (), S ′
∣ (R seq )
E , S ′ ⊢ Q ⇓ vQ , S ′′

⇓ vQ , S ′′

19.6.1 Lexical environment semantics of recursion

The extended language with references and assignment is sufficient to

provide a semantics for the recursive let rec construct. A simple way
to observe this is to reconstruct a let rec expression of the form

let rec x = D in B

as syntactic sugar for an expression that caches the recursion out using
just the trick described in Section 19.4: first assigning to x a mutable
reference to a special unassigned value, then evaluating the definition
D, replacing the value of x with the evaluated D, and finally, evaluating
B in that environment. We can carry out that recipe with the following
expression, which we can think of as the desugared let rec:
let x = ref unassigned in
x := D [x ↦ !x ];
B [x ↦ !x ]

(Since we’ve changed x to a reference type, we need to replace occur-

rences of x in D and B with !x to retrieve the referenced value.)
One way to verify that this approach works is to test it out in OCaml
itself. Take this application of the factorial function, for instance:
382 PROGRAMMING WELL

# let rec f = fun n -> if n = 0 then 1

# else n * f (n - 1) in
# f 4 ;;
- : int = 24

Desugaring it as above, we get

# let unassigned = fun _ -> failwith "unassigned" ;;

val unassigned : 'a -> 'b = <fun>

# let f = ref unassigned in

# (f := fun n -> if n = 0 then 1
# else n * !f (n - 1));
# !f 4 ;;
- : int = 24

(To serve as the “unassigned” value, we define unassigned to simply

raise an exception.)
This expression, note, makes use of only the language constructs
provided in the semantics in the previous section. That semantics,
with its lexical environment and mutable store, thus has enough ex-
pressivity for capturing the approach to recursion described informally
in Section 19.4. In fact, we could even provide a semantic rule for let
rec by carrying through the semantics for the desugared expression.
This leads to the following let rec semantic rule for the let rec
construction:

E , S ⊢ let rec x = D in B ⇓

E {x ↦ l }, S {l ↦ unassigned} ⊢ D [x ↦ !x ] ⇓ v D , S ′
∣
E {x ↦ l }, S ′ {l ↦ v D } ⊢ B [x ↦ !x ] ⇓ v B , S ′′

⇓ v B , S ′′
(R letrec )

Problem 229
For the formally inclined, prove that the semantic rule for let rec above is equivalent to
the syntactic sugar approach.
20
Concurrency

In 1965, Gordon Moore, one of the founders of the pioneering electron-

ics company Fairchild Semiconductor, noted the exponential growth
in the number of components that were being placed on integrated
circuit chips, the building blocks of all kinds of electronics but espe-
Figure 20.1: Gordon Moore’s chart on
cially of computers. Extrapolating from just four points on a chart the basis of which his 1965 eponymous
(Figure 20.1), Moore saw that the number of integrated circuit compo- “law” was extrapolated.
nents had been growing at “a rate of roughly a factor of two per year”,
and he expected that rate to continue for the foreseeable future. Ten
years later, he revised his estimate to a doubling per two years. This
prediction became “Moore’s law”, and has been generalized to many
other aspects of computer technology and performance.
The generalized form of Moore’s law would have it that computer
performance, measured, say, in total number of instructions executed
per second, should grow exponentially as well, as indeed it has. Em-
pirical data on the number of standardized operations performed
per second, charted as squares in Figure 20.2, shows that Moore’s law
as applied to the performance of microprocessor chips has held up
remarkably well for many decades. Data on clock speed, the rate at
which individual instructions can be executed, charted as circles,
shows a different story. The clock speed of the processors showed the
same exponential growth through the mid to late 2000’s, but flattened
after that. What could account for this differential? If the processors
weren’t executing instructions faster, how could they be executing
more instructions in the same amount of time. The answer is given
by the third series, shown as triangles in Figure 20.2, which graphs the
number of processors per chip. Over the last decade or so, we’ve seen
a regular rise in the number of processors per chip, making up the
difference in Moore’s law by having multiple instructions executed in
parallel.
These days, specialized architectures like graphics processing units
(GPUs) and AI accelerators take advantage of even larger scale paral-
384 PROGRAMMING WELL

Figure 20.2: Chart showing growth in

clock speed (in megaHertz (MHZ),
as circles), throughput (in Dhrystone
millions of instructions per second
(DMIPS), as squares), and number
of cores per chip for Intel and recent
AMD microcomputer chips. Note the
logarithmic vertical scale.

lelism to speed up complex highly structured computations for graph-

ics or machine learning. In fact, parallel computing is responsible for
the recent breakthroughs in machine learning performance, and is
in large part the future of maintaining the tremendous performance
improvements that Moore’s law has captured.
There’s no free lunch. Programming computations that happen
concurrently introduces new challenges, requiring new programming
abstractions to manage them. In this chapter, we’ll explore some of the
promise, difficulty, and tools of concurrent programming. As usual,
in the effort to simplify the management of the daunting problems of
concurrency, new abstractions will be crucial. 1
Exactly what constitutes an atomic
step depends on the particularities
of the hardware; we needn’t concern
ourselves with the details here. We’ll just
20.1 Sequential, concurrent, and parallel computation assume that operations like reading a
value from memory (as, for instance,
It will be helpful, in thinking about these issues, to imagine compu- accessing a variable’s value), modifying
a value in memory (instantiating
tation as proceeding sequentially in a series of small atomic steps.
a variable or updating a mutable
Indeed, computation does proceed that way, down at the level of ab- variable, for instance), performing a
straction at which the hardware processors operate. The role of a simple operation on retrieved values
(arithmetic operations, for instance),
compiler is to translate programs written using higher-level abstrac- and the like are atomic. In the examples
tions down to a sequence of atomic instructions directly runnable on we’ll use, we’ll write the atomic steps on
separate lines, so that any line of code
(possibly virtual) hardware.1
will be assumed to execute atomically.
Suppose we have two tasks (A and B) to complete, each requiring
 CONCURRENCY 385

(a) (b) (c) (d)

Figure 20.3: Two tasks running in vari-
execution of a sequence of atomic steps. One way of completing the ous forms of sequential and concurrent
tasks is to execute the two tasks S E QU E N T I A L LY , all of the steps of Task computation. Each task is depicted
as atomic steps (the individual boxes)
A before any of the steps of Task B, as depicted in Figure 20.3(a). Alter- executing through time (running from
natively, we might execute some of the steps in Task A, then some from top to bottom). (a) Task A runs sequen-
tially to completion before task B. (b)
Task B, then the remainder of Task A and the remainder of Task B (Fig-
Coarsely concurrent execution of the
ure 20.3(b)). The tasks are said to be running C O N C U R R E N T LY . Even two tasks, with some steps of task B first
more fine-grained concurrency is possible of course (Figure 20.3(c)). running after four steps of task A. (c)
Finer concurrent execution, interleaving
Why might such concurrency be useful? Through concurrent exe- at each atomic step. (d) Parallel compu-
cution, Task B might be able to generate some useful behavior earlier tation of the tasks, with task B beginning
execution after the fourth step of task A
than having to wait for Task A to complete. Perhaps Task A part way
and running simultaneously. The arrows
through its execution computes some value that is needed by Task denote a dependency requiring task
B, or vice versa. Waiting for Task A to complete may postpone Task B.1 to run after task A.4. Note that that
dependency is violated in (c).
B for a long time. Indeed, some computations are intended never to
complete. Think of the process that runs a bank ATM, which is always
running a single program to handle requests from users as they walk
up to and interact with it. Although the ATM process never completes,
other processes may want to interact with it and intersperse their
computations on the same processor, perhaps to report changes to a
central database. In sum, concurrency allows multiple separate pro-
cesses to interact and communicate without requiring one of them to
complete before the other begins.
Where such concurrency is possible, a further benefit can accrue
– carrying out the steps of the tasks I N PA R A L L E L (Figure 20.3(d)) by
making use of separate hardware for processing the sequences of
instructions. Parallelism allows both tasks to complete in fewer time
steps, effectively trading time for “space” (hardware).

20.2 Dependencies

In taking advantage of concurrency or parallelism, delicate issues

quickly arise when there are dependencies between the two sequences
386 PROGRAMMING WELL

of instructions. For instance, Task B might read a value at one of its

steps (its first step, say) that Task A computes at its, say, fourth step.
We’ve indicated such a dependency with the arrows in Figure 20.3.
If Task A and B run sequentially in that order, then of course the
value generated by Task A will be available to Task B at the proper time.
But concurrent computation is also possible, say if Task A completes
its first four steps before Task B begins. But other interleavings can
be problematic, for instance, if Tasks A and B interleave after each
step. Task B will then attempt to make use of the value that Task A will
calculate before it has actually been calculated. This kind of depen-
dency, where one task must read a value only after another task writes
it, is sometimes referred to as a R E A D - A F T E R - W R I T E D E P E N D E N C Y .
What happens when concurrent execution violates the read-after-write
dependency may not be well defined, but it certainly is not a good
situation.
In addition to read-after-write dependencies, other kinds of de-
pendencies ( W R I T E - A F T E R - R E A D , W R I T E - A F T E R - W R I T E ) are also
important. The details are beyond the scope of this discussion. At this
point, we’re merely concerned with how to allow concurrency while
avoiding violations of ordering dependencies whatever they might be.
In summary, if we just allow tasks to interleave however they hap-
pen to, with no control over which parts of which task run when, de-
pendencies introduce a kind of race between the tasks. Will Task A’s
write step run faster and execute, as it should, before Task B’s read? Or
will Task B win the race, performing its read before task A has a chance
to write? This kind of R A C E C O N D I T I O N leads to the possibility of a
new kind of error. Gaining the benefits of concurrency and parallelism,
while avoiding race conditions and other new classes of errors, is the
challenge of concurrent and parallel programming.

20.3 Threads

In order to demonstrate these issues and experiment with abstractions

that can help avoid these new classes of errors, we need a way to im-
plement concurrency. OCaml provides a programming abstraction
that allows us to experiment with concurrency, the T H R E A D . A thread
can be thought of as providing a separate virtual processor.2 2
OCaml thread’s provide concurrency,
not true parallelism, but the issues
Suppose we need to do two tasks – call them Task A and B as before
they raise apply equally well to parallel
– implemented as OCaml functions named accordingly. Perhaps we processing, so they’re all we’ll need
want to sum the results returned by these two tasks. We can easily to demonstrate the problems. Other
OCaml modules, and aspects of many
execute them sequentially, task A before B: other programming languages, provide
concurrency and parallelism constructs
let resultA = taskA () in that introduce just the same issues.
let resultB = taskB () in
 CONCURRENCY 387

Figure 20.4: For reference, some lo-

gistical code used in the concurrency
(* log thread_name msg -- Prints a message recording that a demonstrations.
thread with the given `thread_name` has generated a log
message `msg`. Also prints an indication of time in seconds
since an initialization time. Used for tracking concurrent
executions. *)
let log =
(* store a fixed marker time for comparison *)
let init_time = Unix.gettimeofday () in
fun thread_name msg ->
Printf.printf "[%3.4f %s: %s]\ n%!"
((Unix.gettimeofday ()) -. init_time)
thread_name
msg ;;

(* task_delayed name delay value -- Prints a message

recording that a thread with the given `thread_name` has
generated a log message `msg`. Also prints an indication of
time in seconds since an initialization time. Used for
tracking concurrent executions. *)
let task_delayed (name : string)
(delay : float)
(value : 'a)
: 'a =
log name "starts";
Thread.delay delay;
log name "ends";
value ;;

(* Two sample tasks taking differing lengths of time and

returning different values. *)
let task_short () = task_delayed "task_short" 0.1 1 ;;
let task_long () = task_delayed "task_long " 0.2 2 ;;
388 PROGRAMMING WELL

resultA + resultB ;;

We can think of the two tasks (along with the computation of their
sum) as being executed in a single thread of computation. The se-
mantics of the let construct ensures that taskA () will be evaluated,
generating resultA, before taskB () begins its evaluation.
In order to demonstrate the idea, and prepare for the significantly
more subtle examples to come, we define a test function that takes two
functions as its argument, which play the roles of tasks A and B.
# let test_sequential taskA taskB =
# let resultA = taskA () in
# let resultB = taskB () in
# resultA + resultB ;;
val test_sequential : (unit -> int) -> (unit -> int) -> int = <fun>

We can test this sequential computation using some simulated tasks.

The unit function task_short simulates a task that engages in a
shorter computation (0.1 seconds) returning the value 1. The corre-
sponding function task_long takes longer (0.2 seconds) and returns
the value 2. (The details of how they’re implemented aren’t important,
but for completeness, they’re provided in Figure 20.4.) Let’s test it out.
# test_sequential task_short task_long ;;
[1.1189 task_short: starts]
[1.2191 task_short: ends]
[1.2191 task_long : starts]
[1.4193 task_long : ends]
- : int = 3

The test returns the summed results 3. Along the way, various key
events are logged. We see the start of the short task and its ending,
followed by the start and end of the long task, indicating their sequen-
tiality.
If we’d like them to execute concurrently, we can establish a separate
thread (that is, a separate virtual processor) corresponding to taskA.
We refer to this process as F O R K I N G a new thread. We use OCaml’s
Thread.create function,3 which takes a function and its argument 3
The Thread module is part of OCaml’s
threads library. To make use of it in the
and returns a separate new thread of computation (a value of type
R E P L , you’ll need to make it available
Thread.t) in which the function is applied to its argument. Its type with
is thus (’a -> ’b) -> ’a -> Thread.t. So we can evaluate tasks A #use "topfind" ;;
#thread ;;
and B in separate threads, concurrently, as follows:
let threadA = Thread.create taskA () in
let resultB = taskB () in
...

The evaluation of the Thread.create expression returns immediately,

without waiting for the result of the application of taskA to () to finish
in the new thread. Thus when taskB () is evaluated, it doesn’t wait
until taskA completes.
 CONCURRENCY 389

20.4 Interthread communication

We’ve enabled two tasks to operate concurrently using threads. But

we have no way as of yet for threads to communicate with each other.
For instance, in the example above, how can taskA, isolated in its
own thread, inform the thread running taskB about its return value?
Similarly, how can taskB communicate information to taskA if it
needs to?
A simple mechanism for this interthread communication is for the
threads to share mutable values, which serve as channels of commu-
nication between the threads. Let’s start with how the created thread
executing taskA can communicate its return value to the main thread
that needs to calculate the sum.
We’ll define another test function called test_communication
to test the communication between two tasks executed in separate
threads as above.

let test_communication taskA taskB =

...

We’ll use a shared mutable value called shared_result of type int

option, initially None since no result is yet available.

...
let shared_result = ref None in
...

Now we can create a new thread for executing task A, saving its return
value in the shared result.

...
let _thread = Thread.create
(fun () -> shared_result := Some (taskA ())) () in
...

In the original thread, we perform task B, saving its result.

...
let resultB = taskB () in
...

Finally, we can extract the result from task A from the shared value, and
compute with the two results, by adding them as before.

...
match !shared_result with
| Some resultA ->
(* compute with result1 and result2 *)
resultA + resultB
| None ->
(* Oops, taskA hasn't completed! *)
failwith "shouldn't happen!" ;;
390 PROGRAMMING WELL

Putting it all together, we have

# let test_communication taskA taskB =
# let shared_result = ref None in
# let _thread = Thread.create
# (fun () -> shared_result := Some (taskA ())) () in
# let resultB = taskB () in
# match !shared_result with
# | Some resultA ->
# (* compute with result1 and result2 *)
# resultA + resultB
# | None ->
# (* Oops, taskA hasn't completed! *)
# failwith "shouldn't happen!" ;;
val test_communication : (unit -> int) -> (unit -> int) -> int =
<fun>

Again, we can test using the simulated tasks. To start, we fork the
new thread running the shorter task, with the longer task in the main
thread.
# test_communication task_short task_long ;;
[2.0848 task_long : starts]
[2.0849 task_short: starts]
[2.1850 task_short: ends]
[2.2850 task_long : ends]
- : int = 3

the communication works as expected. The short task returns 1 –

passed through and retrievable from the shared variable – and the long
task returns 2. The test as a whole computes their sum, 3 as expected.
The logged events show the starting of the long task in the main
thread, followed by the short task starting in the newly created thread.
The latter short thread completes quickly (it’s shorter, after all), ending
before the long task does. The main thread can extract the completed
value for the short task and add it to the result from the long task.
But what if the task in the forked thread takes longer than that in the
main thread?
# test_communication task_long task_short ;;
[2.3367 task_short: starts]
[2.3367 task_long : starts]
[2.4368 task_short: ends]
Exception: Failure "shouldn't happen!".

In this version of the test, the short task in the main thread completes
before the forked thread has time to complete the long task and update
the shared variable, leading to a run-time exception. The code has a
race condition with respect to a read-after-write dependency. These
two executions of the test demonstrate that, depending on which task
“wins the race”, the value to be read may or may not be written in time
as it needs to be.
 CONCURRENCY 391

In general, one doesn’t have the kind of detailed information about

run times of various tasks as we have for task_short and task_long.
This kind of concurrent computation, without careful controls, thus
leads to indeterminacy at runtime. And debugging these intermittent
bugs that can come and go, perhaps appearing only rarely, can be
especially confounding. More tools are needed.
The lesson here is that the main thread shouldn’t attempt to use the
shared variable until the forked thread has completed. We thus need
a way of guaranteeing that a thread has completed. One solution you
may have thought of is to have the main thread test if the shared value
has not been properly set, and if not, to just “try again later”. We can
implement this with a simple loop,
while !shared_result == None do
Thread.delay 0.01
done;

which continually waits for a fraction of a second so long as the shared

result has not been properly set, a technique called B U S Y WA I T I N G .
# let test_communication taskA taskB =
# let shared_result = ref None in
# let _thread = Thread.create
#
# (fun () -> shared_result := Some (taskA ())) () in
# let resultB = taskB () in
# while !shared_result == None do
# Thread.delay 0.01
# done;
# match !shared_result with
# | Some resultA ->
# (* compute with result1 and result2 *)
# resultA + resultB
# | None ->
# (* Oops, taskA hasn't completed! *)
# failwith "shouldn't happen!" ;;
val test_communication : (unit -> int) -> (unit -> int) -> int =
<fun>

The errant race condition is now handled properly.

# test_communication task_long task_short ;;
[3.5003 task_short: starts]
[3.5004 task_long : starts]
[3.6006 task_short: ends]
[3.7006 task_long : ends]
- : int = 3

But this kind of brute force trick of repeatedly polling the shared vari-
able until it is ready is profligate and inelegant. It can waste compu-
tation that would be better allocated to other threads, and can waste
time if the delay is longer than needed.
392 PROGRAMMING WELL

Instead, we’d like to be able to directly specify to wait until

the forked thread completes. The companion to the fork function
Thread.create is the join function Thread.join. Thread.join
takes a thread as its argument and returns only once that thread has
completed. By requiring the join before accessing the shared variable,
we are guaranteed that the variable will have been updated at the time
that we need it.
# let test_communication taskA taskB =
# let shared_result = ref None in
# let thread = Thread.create
# (fun () -> shared_result := Some (taskA ())) () in
# let resultB = taskB () in
# Thread.join thread;
# match !shared_result with
# | Some resultA ->
# (* compute with result1 and result2 *)
# resultA + resultB
# | None ->
# (* Oops, taskA hasn't completed! *)
# failwith "shouldn't happen!" ;;
val test_communication : (unit -> int) -> (unit -> int) -> int =
<fun>

Using this version of the test, the race condition is avoided, and the
calculation completes properly.
# test_communication task_long task_short ;;
[4.4136 task_short: starts]
[4.4137 task_long : starts]
[4.5139 task_short: ends]
[4.6139 task_long : ends]
- : int = 3

20.5 Futures

The structure of this small example, in which a thread is forked to

allow it to compute a return value that is needed in the future, is so
common that it deserves its own abstraction, a kind of value dubbed
a F U T U R E . This abstraction is implemented via two functions: The
future function takes a task to be carried out for its return value, and
returns a “future value”. We can later use the force function to force
the future value to be extracted when available. A module signature
can help clarify the needed functionality:
# module type FUTURE =
# sig
# type 'result future
#
# (* future fn x -- Forks a new thread within which `fn`
# is applied to `x`. Immediately returns a `future`
 CONCURRENCY 393

# which can be used to synchronize with the thread

# and extract the result. *)
# val future : ('arg -> 'result) -> 'arg -> 'result future
#
# (* force fut -- Causes the calling thread to wait until the
# thread computing the future value `fut` is done and then
# returns its value. *)
# val force : 'result future -> 'result
# end ;;
module type FUTURE =
sig
type 'result future
val future : ('arg -> 'result) -> 'arg -> 'result future
val force : 'result future -> 'result
end

There are multiple ways to implement this functionality, but we’ll

use the shared value method from the previous example. In this imple-
mentation, a future value (an element of the future type) is a record
that contains the thread identifier in which the future task is being car-
ried out and the mutable variable for communicating the result back to
the calling thread.
# module Future : FUTURE =
# struct
# type 'result future = {tid : Thread.t;
# value : 'result option ref}
#
# let future (f : 'arg -> 'result) (x : 'arg) : 'result future =
# let r : 'result option ref = ref None in
# let t = Thread.create (fun () -> r := Some (f x)) ()
# in {tid = t; value = r}
#
# let force (f : 'result future) : 'result =
# Thread.join f.tid;
# match !(f.value) with
# | Some v -> v
# | None -> failwith "impossible!"
# end ;;
module Future : FUTURE

With the future abstraction in hand, the test_communication

example above can be greatly simplified.
# let test_future taskA taskB =
# let futureA = Future.future taskA () in
# let resultB = taskB () in
# Future.force futureA + resultB ;;
val test_future : (unit -> int) -> (unit -> int) -> int = <fun>

# test_future task_long task_short ;;

[6.2352 task_short: starts]
[6.2353 task_long : starts]
[6.3355 task_short: ends]
394 PROGRAMMING WELL

[6.4355 task_long : ends]

- : int = 3

This is hardly more complicated than the sequential version that we

started with, reproduced from above, displaying the effectiveness of
the abstraction.
let test_sequential taskA taskB =
let resultA = taskA () in
let resultB = taskB () in
resultA + resultB ;;

Exercise 230
Exercise 104 concerned implementing a fold operation over binary trees defined by
# type 'a bintree =
# | Empty
# | Node of 'a * 'a bintree * 'a bintree ;;
type 'a bintree = Empty | Node of 'a * 'a bintree * 'a bintree

Define a version of the fold operation, foldbt_conc, that performs the recursive folds of
the left and right subtrees concurrently, making use of futures to ensure that results are
available when needed.

Exercise 231
In

20.6 Futures are not enough

The sharing of mutable data across two concurrent threads is a valu-

able ability. It implements a kind of communication channel between
the threads. But managing this communication is complex. We’ve al-
ready seen this in the context of a thread’s “return value”. The calling
thread mustn’t read the shared variable that will be storing the called
thread’s return value until the latter has completed its computation
and updated the return value. Managing this ordering is the whole
point of the future/force abstraction.
Sharing mutable data across threads is a useful technique well
beyond just allowing for return values to be communicated.

1. Threads may have need for coordinating data manipulation beyond

the mere passing of a return value. For instance, think of multiple
threads manipulating a shared database.

2. In the case of threads that are not intended to terminate, the whole
notion of a return value is inapplicable. Importantly, not all con-
current computations are intended to terminate. Indeed, one of the
benefits of concurrency as a programming concept is that it allows
multiple threads of nonterminating computation to interact. We
still need to manage the interaction so that the concurrent com-
putations satisfy the various dependencies among them without
dangerous race conditions.
 CONCURRENCY 395

A standard example of this kind of concurrent nonterminating

computation is the ATM. ATMs are computers that run a program
that interacts with bank patrons to allow them to manipulate their
bank accounts in various ways. The bank accounts constitute a shared
database of mutable data. And because banks have multiple geograph-
ically distributed ATMs, multiple instances of the program are running
concurrently, and potentially transforming the same shared data, the
balances of the various accounts.
To demonstrate the problem, let’s think of a bank as having multiple
accounts each of which is an instance of an account class defined as
follows:

# class account (initial_balance : int) =

# object
# val mutable balance = initial_balance
#
# method balance = balance
#
# method deposit (amt : int) : unit =
# balance <- balance + amt
#
# method withdraw (amt : int) : int =
# if balance >= amt then begin
# balance <- balance - amt;
# amt
# end else 0
# end ;;
class account :
int ->
object
val mutable balance : int
method balance : int
method deposit : int -> unit
method withdraw : int -> int
end

The deposit and withdraw methods both potentially affect the value
of the mutable balance variable. The withdraw function, in particular,
verifies that the balance is sufficient to cover the withdrawal amount,
updates the balance accordingly, and returns the amount to be dis-
pensed (0 if the balance is insufficient).
Now what happens when we try multiple concurrent withdrawals
from the same account? To simulate such an occurrence, the following
test_wds function carries out withdrawals of $75 and $50 in separate
threads (call them “thread A” and “thread B” for ease of reference) from
a single account with initial balance of $100, using a future for the
larger withdrawal. To track what goes on, the test function returns
the amount dispensed in thread A and thread B, along with the final
balance in the account.
396 PROGRAMMING WELL

# let test_wds () =
# let acct = new account 100 in
# let threadA_ftr = Future.future acct#withdraw 75 in
# let threadB = acct#withdraw 50 in
# let threadA = (Future.force threadA_ftr) in
# threadA, threadB, acct#balance ;;
val test_wds : unit -> int * int * int = <fun>

What behavior would we like to see in this case? One or the other
of the two withdrawals, whichever comes first, should see a sufficient
balance, dispense the requested amount, and update the balance
accordingly. The other attempted withdrawal should see a reduced and
insufficient balance and dispense no funds. Let’s try it.
# test_wds () ;;
- : int * int * int = (0, 50, 50)

Hmm.
In order to experiment with the possibility of interleavings of the
various components of the withdrawals, we make two changes to the
withdrawal simulation. First, we divide the balance update (balance
<- balance - amt)into two parts: the computation of the updated
balance and the update of the balance variable itself (let diff =
balance - amt in balance <- diff). Doing so separates the read-
ing of the shared balance from its writing, allowing interposition of
other threads in between. Then, we introduce some random delays
at various points in the computation: before the withdrawal first exe-
cutes, immediately after the balance check, and after computing the
updated balance just before carrying out the update. For this pur-
pose, we use a function random_delay, which pauses a thread for a
randomly selected time interval.
# let random_delay (max_delay : float) : unit =
# Thread.delay (Random.float max_delay) ;; valid? first second balance count
val random_delay : float -> unit = <fun> 75 50 −25 29
75 50 50 26
Updating the withdrawal function to insert these delays, we have 75 50 25 21
✓ 0 50 50 12
method withdraw (amt : int) : int =
✓ 75 0 25 12
random_delay 0.004;
if balance >= amt then begin Figure 20.5: Table of outcomes from
random_delay 0.001; multiple runs of simultaneous with-
let diff = balance - amt in drawals. Each row represents a possible
random_delay 0.001; outcome, with columns showing the
balance <- diff; amount dispensed for the first with-
drawal, the amount dispensed for the
amt
second withdrawal, the final balance,
end else 0 ;;
and the number of times this outcome
occurred in 100 trials. Only the check-
Here is a typical outcome from this simulation. marked trials are valid in respecting
dependencies.
# test_wds () ;;
- : int * int * int = (75, 50, -25)
 CONCURRENCY 397

thread A ($75 withdrawal) thread B ($50 withdrawal) Figure 20.6: An unproblematic (essen-
tially sequential) interleaving of the
1. if balance >= amt then begin
2. let diff = balance - amt in threads.
3. balance <- diff;
4. amt
5. ⋯ if balance >= amt then begin
⋯
6. end else 0

If we run the simulation many times, we see (Figure 20.5) that the
result is quite variable. Certainly, there are many occurrences (about
half) showing the desired behavior, with either $75 or $50 dispensed
and a final balance of $25 or $50, respectively. But we also see plenty
of instances where both withdrawals go through, dispensing both $75
and $50, leaving a final balance of $−25. Or $25. Or $50. The use of
future ensures that the return value dependency is properly obeyed,
but the various dependencies having to do with the updates to and
uses of the account’s balance are uncontrolled. Different interleavings
of these operations can yield different results. Let’s examine a few of
the many possible interleavings.
First, thread A (the $75 withdrawal) may execute fully before thread
B (the $50 withdrawal) begins. That is, they may execute sequentially.
This interleaving is depicted in Figure 20.6. In this representation of
the two threads executing, the executed lines of thread A are on the
left, thread B on the right. We assume that each line of code executes
atomically, with the order of the numbered lines indicating the order
in which they are executed in the concurrent computation. The el-
lipses (⋯) indicate code lines that were not executed since they fell in
the non-chosen branch of a conditional. In line 1, the balance test in
thread A is evaluated. Since the balance is initially 100, and the with-
drawal amount is 75, the condition holds and lines 2-4 in the then
branch are executed. Line 3 in particular updates the shared balance
to 25, so that in line 5 when thread B tests the balance, the test fails
and the second withdrawal does not complete (line 6). In summary,
the $75 withdrawal attempt succeeds, dispensing the $75, and the $50
withdrawal attempt fails, leaving a balance of $25.
Of course, if thread B had executed fully before thread A, the cor-
responding result would have occurred, dispensing only the $50 and
leaving a balance of $50.
But other results are also possible. For instance, consider the in-
terleaving in Figure 20.7. Each thread verifies the balance as being
adequate and computes its updated value before the other performs
the balance update. Both threads go on to update the balance (lines
5 and 7); since thread B updates the balance later, its balance value,
$50, overwrites thread A’s $25 balance, so the final balance is $50. In
398 PROGRAMMING WELL

thread A ($75 withdrawal) thread B ($50 withdrawal) Figure 20.7: A problematic interleaving
of the threads.
1. if balance >= amt then begin
2. let diff = balance - amt in
3. if balance >= amt then begin
4. let diff = balance - amt in
5. balance <- diff;
6. amt
7. ⋯ balance <- diff;
8. amt
⋯

summary, both attempted withdrawals succeed, dispensing both $75

and $50, leaving a surprising $50 balance. Sure enough, Figure 20.5
indicates that such outcomes were actually attested in the simulations.

Exercise 232
Construct an interleaving in which both withdrawals succeed, leaving a balance of $25.

Exercise 233
Construct an interleaving in which both withdrawals succeed, leaving a balance of $ − 25.
As Figure 20.5 shows, and these possible interleavings explain,
there are important dependencies that are not being respected in the
concurrent implementation of the account operations. A solution to
this problem of controlling data dependencies requires further tools.

20.7 Locks

To gain better control over the interleavings, we introduce a new ab-

straction, the L O C K . The underlying idea is that while a thread is ex-
ecuting the withdrawal method, it ought to be the only thread with
access to the balance it is manipulating. Just as you might lock your
door to prevent others from using your car, you might want to lock
some data to prevent others from manipulating it. OCaml provides a
simple interface to a locking mechanism called M U T E X L O C K S in its
Mutex library. The name comes from the idea of mutual exclusion;
other threads should be excluded from certain regions of code when a
lock is in force.
To create a mutex lock for a given datum, the mutable balance, say,
in the ATM example, we use Mutex.create.

# let balance_lock = Mutex.create () ;;

val balance_lock : Mutex.t = <abstr>

4
As shown, this creates a lock of type Mutex.t. We can then lock Crucially, the testing for unlocked
status and subsequent locking occur
and unlock the lock as needed with the functions Mutex.lock and atomically, so that other threads can’t
Mutex.unlock. interleave between them. How this is
accomplished, the subject of fundamen-
The mutex locks work as follows. When Mutex.lock is called on a
tal research in concurrent computation,
lock, the lock is first verified to be in its unlocked state. If so, the lock is well beyond the scope of this text.
switches to the locked state and computation proceeds.4 But if not,
 CONCURRENCY 399

thread A ($75 withdrawal) thread B ($50 withdrawal) Figure 20.8: The problematic interleav-
ing, corrected by the use of locks.
1. Mutex.lock balance_lock;
2. if balance >= amt then begin
3. let diff = balance - amt in
Mutex.lock balance_lock;
4. balance <- diff;
5. amt

ÔÔ⇒
⋯ thread B suspended
6. Mutex.unlock balance_lock;
7. Mutex.lock balance_lock;
8. if balance >= amt then begin
9. let diff = balance - amt in
10. balance <- diff;
11. amt
⋯
12. Mutex.unlock balance_lock;

the thread in which the call was made is suspended until such time as
the lock becomes unlocked, presumably by a call to Mutex.unlock in
another thread.
Inserting the locks in the ATM example, we would have a withdraw
method like this:
method withdraw (amt : int) : int =
Mutex.lock balance_lock;
if balance >= amt then begin
balance <- balance - amt;
amt
end else 0;
Mutex.unlock balance_lock ;;

The code between the locking and unlocking is the C R I T I C A L R E G I O N ,

a computation that must be carried out atomically from the point of
view of the resource that is being locked. In this case, the entire body of
the withdraw method is a critical region.
Now consider the previous problematic case of Figure 20.7 – in
which thread B’s withdrawal code begins executing partway through
thread A’s withdrawal code – except now with the locking implementa-
tion above. As seen in Figure 20.8, thread A now begins by establishing
the balance lock in step 1. When the first step of thread B executes at
the intermediate point within thread A’s execution (after step 3) and
attempts to itself acquire the balance lock, the lock causes thread B
to suspend until such time as it becomes available, which is not until
thread A releases the lock at step 6. The delay in thread B changes the
interleaving to a safe one, like that of Figure 20.6.

20.7.1 Abstracting lock usage

This idiom – wrapping a critical region with a lock at the beginning and
an unlock at the end – captures the stereotypical use of locks.
In this idiom, the lock is explicitly unlocked after the need for the
lock is over. The unlocking is crucial; without it, other threads would
400 PROGRAMMING WELL

be permanently prevented from carrying out their own computations

requiring the lock. We can codify the importance of matching the locks
and unlocks by way of an abstracted function that wraps a computa-
tion with the lock and its corresponding unlock. We call the function
with_lock:

# (* with_lock l f -- Run thunk `f` in context of acquired lock `l`,

# unlocking on return *)
# let with_lock (l : Mutex.t) (f : unit -> 'a) : 'a =
# Mutex.lock l;
# let result = f () in
# Mutex.unlock l;
# result ;;
val with_lock : Mutex.t -> (unit -> 'a) -> 'a = <fun>

If we stick with using with_lock, we never need to worry that we will

perform a lock without the matching unlock, in keeping with the edict
of prevention.
Or will we? What would happen if the computation of f () raised
an exception of some sort? The body of the let will never be per-
formed, and the lock will not be unlocked! (Of course, that possibility
also held for the withdraw method just above.) We’ll want to fix that by
adjusting with_lock to make sure to handle exceptions properly, fur-
ther manifesting the edict of prevention. We leave that for an exercise.

Exercise 234
Define a version of with_lock that handles exceptions by making sure to unlock the
lock.
Using with_lock, the withdraw method becomes

method withdraw (amt : int) : int =

with_lock balance_lock (fun () ->
if balance >= amt then begin
balance <- balance - amt;
amt
end else 0) ;;
valid? first second balance count
With this modified implementation of accounts, the simulation of ✓ 0 50 50 51
many trials of simultaneous deposits performs much better, with only ✓ 75 0 25 49

valid results, as depicted in Figure 20.9. Figure 20.9: Rerunning the test of
simultaneous withdrawals, with locking
in place, all trials now respect the
20.8 Deadlock dependencies, though the results can
still vary depending on which of the two
withdrawals in each trial happens to
occur first.
21
Final project: Implementing MiniML

The culminative final project for CS51 is the implementation of a

small subset of an OCaml-like language. Unlike the problem sets, the
final project is more open-ended, and we expect you to work more
independently, using the skills of design, abstraction, testing, and
debugging that you’ve learned during the course.

21.1 Overview

Unlike OCaml and the ML programming language it was derived from,

the language you will be implementing includes only a subset of con-
structs, has only limited support for types (including no user-defined
types), and does no type inference (enforcing type constraints only at
run-time). On the other hand, the language is “Turing-complete”, as
expressive as any other programming language in the sense specified
by the Church-Turing thesis. Because the language is so small, we refer
to it as MiniML (pronounced “minimal”).
The implementation of this OCaml subset MiniML is in the form of
an interpreter for expressions of the language written in OCaml itself,
a M E TA C I R C U L A R I N T E R P R E T E R . Actually, you will implement a series
of interpreters that vary in the semantics they manifest. The first is
based on the substitution model (Chapter 13); the second a dynami-
cally scoped environment model (Chapter 19); and the third, a version
of the second implementing one or more extensions of your choosing,
with lexical scoping being a simple and highly recommended option.
This chapter builds on the idea of specifying the semantics of a pro-
gramming language and implementing that specification begun in
Chapters 13 and 19. The exercises herein are to test your understand-
ing. We recommend that you do the exercises, but you won’t be turning
them in and we won’t be supplying answers. The S TA G E S provide a
sequence of nine stages to implement the MiniML interpreter. It’s the
result of working on these stages that you will be turning in and on
402 PROGRAMMING WELL

which the project grade will be based.

This project specification is divided into three sections (correspond-
ing to the section numbers marked below):

Substitution model (Section 21.2) Implementation of a MiniML inter-

preter using the substitution semantics for the language.

Dynamic scoped environment model (Section 21.3) Implementation of

a MiniML interpreter using the environment model and manifesting
dynamic scoping.

Extensions (Section 21.4) Implementation of one or more extensions

to the basic MiniML language of your choosing. Special attention is
paid below to an extension to the environment model manifesting
lexical scoping (Section 21.4.2).

21.1.1 Grading and collaboration

As with all the individual problem sets in the course, your project is to
be done individually, under the course’s standard rules of collabora-
tion. (The sole exception is described in Section 21.6.) You should not
share code with others, nor should you post public questions about
your code on Piazza. If you have clarificatory questions about the
project assignment, you can post those on Piazza and if appropriate
we will answer them publicly so the full class can benefit from the
clarification.
The final project will be graded based on correctness of the imple-
mentation of the first two stages; design and style of the submitted
code; and scope of the project as a whole (including the extensions) as
demonstrated by a short paper describing your extensions, which is
assessed for both content and presentation.
It may be that you are unable to complete all the code stages of the
final project. You should make sure to keep versions at appropriate
milestones so that you can always roll back to a working partial project
to submit. Using git will be especially important for this version
tracking if used properly.
Some students or groups might prefer to do a different final project
on a topic of their own devising. For students who have been doing
exceptionally well in the course to date, this may be possible. See
Section 21.6 for further information.

21.1.2 A digression: How is this project different from a problem

set?

We frequently get questions about the final project of the following

sort: Do I need to implement X? Am I supposed to handle Y? Is it a
 F I N A L P R O J E C T: I M P L E M E N T I N G M I N I M L 403

sufficient extension to do Z? Should I provide tests for W? Is U the right

way to handle V? Do I have to discuss P in the writeup?
The final project description doesn’t specify answers to many ques-
tions of this sort. This is not an oversight; it is a pedagogical choice.
In the world of software design and development, there are an infi-
nite number of choices to make, and there are often no right answers,
merely tradeoffs. Part of the point of the course is that there are many
ways to implement software for a particular purpose, and they are not
all equally good. (See Section 1.2.) The final project is the place in the
course where you are most clearly on your own to deploy the ideas
from the course to make these choices and demonstrate your best un-
derstanding of the tradeoffs involved. By implementing X, you may not
have time to test Y. By implementing only Z, you may be able to do so
with a more elegant or generalizable approach. By adding tests for W,
you may not have time to fully discuss P in the writeup. So it goes.
Perhaps the most important of the major tradeoffs is that between
spending time to make improvements to the CS51 final project soft-
ware and writeup and spending time on other non-CS51 efforts. Be-
cause choices made in negotiating this tradeoff don’t fall solely within
the environment of CS51, it is inherently impossible for course staff
to give you advice on what to do. You’ll have to decide whether your
time is better spent, say, systematizing your unit tests for the project, or
working on the final paper in your Gen Ed course; further augmenting
your implementation of int arithmetic to handle bignums, or studying
for the math midterm that the instructor fatuously scheduled during
reading period; generating further demonstrations of the mutable ar-
ray extension you added by implementing a suite of in-place sorting
algorithms, or wrangling members of the student organization you find
yourself running because the president is awol.
With the final project, you are on your own. Not for issues of clari-
fication of this project description, where the course staff stand ready
to help on Piazza and in office hours. But on deontic issues, issues of
what’s better or worse, what you “should” do or mustn’t, what is re-
quired or forbidden. This is a kind of freedom, and like all freedoms,
it is not without consequences, but they are consequences you must
inevitably reconcile on your own.

21.2 Implementing a substitution semantics for MiniML

You’ll start your implementation with a substitution semantics for

MiniML. The abstract syntax of the language is given by the following
type definition:
404 PROGRAMMING WELL

type unop =
| Negate ;;

type binop =
| Plus
| Minus
| Times
| Equals
| LessThan ;;

type varid = string ;;

type expr =
| Var of varid (* variables *)
| Num of int (* integers *)
| Bool of bool (* booleans *)
| Unop of unop * expr (* unary operators *)
| Binop of binop * expr * expr (* binary operators *)
| Conditional of expr * expr * expr (* if then else *)
| Fun of varid * expr (* function def'ns *)
| Let of varid * expr * expr (* local naming *)
| Letrec of varid * expr * expr (* rec. local naming *)
| Raise (* exceptions *)
| Unassigned (* (temp) unassigned *)
| App of expr * expr ;; (* function app'ns *)

These type definitions can be found in the partially implemented

Expr module in the files expr.ml and expr.mli. You’ll notice that
the module signature requires additional functionality that hasn’t
been implemented, including functions to find the free variables in
an expression, to generate a fresh variable name, and to substitute
expressions for free variables, as well as to generate various string
representations of expressions.

Exercise 235
Write a function exp_to_concrete_string : expr -> string that converts an
abstract syntax tree of type expr to a concrete syntax string. The particularities of what
concrete syntax you use is not crucial so long as you do something sensible along the
lines we’ve exemplified. (This function will actually be quite helpful in later stages.)
To get things started, we also provide a parser for the MiniML lan-
guage, which takes a string in a concrete syntax and returns a value of
this type expr; you may want to extend the parser in a later part of the
project (Section 21.4.3).1 The compiled parser and a read-eval-print 1
The parser that we provide makes use
of the OCaml package menhir, which
loop for the language are available in the following files:
is a parser generator for OCaml. You
should have installed it as per the setup
evaluation.ml The future home of anything needed to evaluate ex- instructions provided at the start of the
pressions to values. Currently, it provides a trivial “evaluator” course by running the following opam
command:
eval_t that merely returns the expression unchanged.
% opam install -y menhir
miniml.ml Runs a read-eval-print loop for MiniML, using the The menhir parser generator will be
Evaluation module that you will complete. discussed further in Section 21.4.3.
 F I N A L P R O J E C T: I M P L E M E N T I N G M I N I M L 405

miniml_lex.mll A lexical analyzer for MiniML. (You should never need

to look at this unless you want to extend the parser.)

miniml_parse.mly A parser for MiniML. (Ditto.)

What’s left to implement is the Evaluation module in

evaluation.ml.
Start by familiarizing yourself with the code. You should be able to
compile miniml.ml and get the following behavior.2 2
In building the project, you may find
that you get a warning of the form:
# ocamlbuild -use-ocamlfind miniml.byte + menhir -ocamlc ’ocamlfind ocamlc
-thread -strict-sequence
Finished, 13 targets (12 cached) in 00:00:00. -package graphics
-package CS51Utils -w
# ./miniml.byte A-4-33-40-41-42-43-34-44’
-infer miniml_parse.mly
Entering miniml.byte... Warning: 15 states have
shift/reduce conflicts.
<== 3 ;; Warning: one state has
Fatal error: exception Failure("exp_to_abstract_string reduce/reduce conflicts.
Warning: 198 shift/reduce
not implemented") conflicts were arbitrarily
resolved.
Warning: 18 reduce/reduce
conflicts were arbitrarily
Stage 236 resolved.

Implement the function exp_to_abstract_string : expr -> You can safely ignore this message
from the parser generator, which is
string to convert abstract syntax trees to strings representing their reporting on some ambiguities in the
structure and test it thoroughly. If you did Exercise 235, the experience MiniML grammar that it has resolved
automatically.
may be helpful here, and you’ll want to also implement exp_to_-
concrete_string : expr -> string for use in later stages as well.
The particularities of what concrete syntax you use to depict the ab-
stract syntax is not crucial – we won’t be checking it – so long as you do
something sensible along the lines we’ve exemplified.
After this (and each) stage, it would be a good idea to commit the
changes and push to your remote repository as a checkpoint and
backup.

Once you write the function exp_to_abstract_string, you should

have a functioning read-eval-print loop, except that the evaluation
part doesn’t do anything. (The R E P L calls the trivial evaluator eval_t,
which essentially just returns the expression unchanged.) Conse-
quently, it just prints out the abstract syntax tree of the input concrete
syntax:

# ./miniml.byte
Entering miniml.byte...
<== 3 ;;
==> Num(3)
<== 3 4 ;;
==> App(Num(3), Num(4))
<== (((3) ;;
xx> parse error
406 PROGRAMMING WELL

<== let f = fun x -> x in f f 3 ;;

==> Let(f, Fun(x, Var(x)), App(App(Var(f), Var(f)), Num(3)))
<== let rec f = fun x -> if x = 0 then 1 else x * f (x - 1) in f 4 ;;
==> Letrec(f, Fun(x, Conditional(Binop(Equals, Var(x), Num(0)), Num(1),
Binop(Times, Var(x), App(Var(f), Binop(Minus, Var(x), Num(1)))))),
App(Var(f), Num(4)))
<== Goodbye.

Exercise 237
Familiarize yourself with how this “almost” R E P L works. How does eval_t get called?
What does eval_t do and why? What’s the point of the Env.Val in the definition? Why
does eval_t take an argument _env : Env.env, which it just ignores? (These last two
questions are answered a few paragraphs below. Feel free to read ahead.)
To actually get evaluation going, you’ll need to implement a substi-
tution semantics, which requires completing the functions in the Expr
module.

Stage 238
Start by writing the function free_vars in expr.ml, which takes an
expression (expr) and returns a representation of the free variables
in the expression, according to the definition in Figure 13.3. Test this
function completely.

Stage 239
Next, write the function subst that implements substitution as defined
in Figure 13.4. In some cases, you’ll need the ability to define new fresh
variables in the process of performing substitutions. You’ll see we call
for a function new_varname to play that role. Looking at the gensym
function that you wrote in lab might be useful for that. Once you’ve
written subst make sure to test it completely.

You’re actually quite close to having your first working interpreter

for MiniML. All that is left is writing a function eval_s (the ‘s’ is for
substitution semantics) that evaluates an expression using the substitu-
tion semantics rules. (Those rules are, conveniently, described in detail
in Chapter 13, and summarized in Figure 13.5.) The eval_s func-
tion walks an abstract syntax tree of type expr, evaluating subparts
recursively where necessary and performing substitutions when ap-
propriate. The recursive traversal bottoms out when it gets to primitive
values like numbers or booleans or in applying primitive functions like
the unary or binary operators to values. It is at this point that the eval-
uator can see if the operators are being applied to values of the right
type, integers for the arithmetic operators, for instance, or integers or
booleans for the comparison operators.
For consistency with the environment semantics that you will im-
plement later as the function eval_d, both eval_t and eval_s take
 F I N A L P R O J E C T: I M P L E M E N T I N G M I N I M L 407

a second argument, an environment, even though neither evaluator

needs an environment. Thus your implementation of eval_s can just
ignore the environment.
We’d also like the various evaluation functions eval_t, eval_s,
eval_d, and (if implemented) eval_l to all have the same return type
as well. Looking ahead, the lexically-scoped environment semantics
implemented in eval_l must allow for the result of evaluation to go
beyond the simple expression values we’ve used so far. In particular,
for the lexical environment semantics, we’ll want to add closures as
a new sort of value, as described in Section 21.4.2. We’ve provided a
variant type Env.value that allows for both the simple expression
values of the sort that eval_s and eval_d generate and for closures,
which only the environment-based lexical-scoped evaluator needs to
generate. For consistency, then, you should make sure that eval_s,
as well as the later evaluation functions, are of type Expr.expr ->
Env.env -> Env.value. This will ensure that your code is consistent
with our unit tests as well. You’ll note that the eval_t evaluator that
we provide already does this. In order to be type-consistent, it takes an
extra env argument that it doesn’t need or use, and it converts its expr
argument to the value type by adding the Env.Val value constructor
for that type. (This may help with Exercise 237.)

Stage 240
Implement the eval_s : Expr.expr -> Env.env -> Env.value
function in evaluation.ml. (You can hold off on completing the
implementation of the Env module for the time being. That comes into
play in later sections.) We recommend that you implement it in stages,
from the simplest bits of the language to the most complex. You’ll want
to test each stage thoroughly using unit tests as you complete it. Keep
these unit tests around so that you can easily unit test the later versions
of the evaluator that you’ll develop in future sections.

Using the substitution semantics, you should be able to handle

evaluation of all of the MiniML language. If you want to postpone
handling of some parts while implementing the evaluator, you can
always just raise the EvalError exception, which is intended just
for this kind of thing, when a MiniML runtime error occurs. Another
place EvalError will be useful is when a runtime type error occurs, for
instance, for the expressions 3 + true or 3 4 or let x = true in y.
Now that you have implemented a function to evaluate expressions,
you can make the R E P L loop worthy of its name. Notice at the bottom
of evaluation.ml the definition of evaluate, which is the function
that the R E P L loop in miniml.ml calls. Replace the definition with the
one calling eval_s and the R E P L loop will evaluate the read expres-
408 PROGRAMMING WELL

sion before printing the result. It’s more pleasant to read the output
expression in concrete rather than abstract syntax, so you can replace
the exp_to_abstract_string call with a call to exp_to_concrete_-
string. You should end up with behavior like this:

# miniml_soln.byte
Entering miniml_soln.byte...
<== 3 ;;
==> 3
<== 3 + 4 ;;
==> 7
<== 3 4 ;;
xx> evaluation error: (3 4) bad redex
<== (((3) ;;
xx> parse error
<== let f = fun x -> x in f f 3 ;;
==> 3
<== let rec f = fun x -> if x = 0 then 1 else x * f (x - 1) in f 4 ;;
xx> evaluation error: not yet implemented: let rec
<== Goodbye.

Some things to note about this example:

• The parser that we provide will raise an exception

Parsing.Parse_error if the input doesn’t parse as well-formed
MiniML. The R E P L handles the exception by printing an appropri-
ate error message.

• The evaluator can raise an exception Evaluation.EvalError at

runtime if a (well-formed) MiniML expression runs into problems
when being evaluated.

• You might also raise Evaluation.EvalError for parts of the eval-

uator that you haven’t (yet) implemented, like the tricky let rec
construction in the example above.

Stage 241
After you’ve changed evaluate to call eval_s, you’ll have a complete
working implementation of MiniML. As usual, you should save a snap-
shot of this using a git commit and push so that if you have trouble
down the line you can always roll back to this version to submit it.

21.3 Implementing an environment semantics for MiniML

The substitution semantics is sufficient for all of MiniML because it is

a pure functional programming language. But binding constructs like
 F I N A L P R O J E C T: I M P L E M E N T I N G M I N I M L 409

let and let rec are awkward to implement, and extending the lan-
guage to handle references, mutability, and imperative programming
is impossible. For that, you’ll extend the language semantics to make
use of an environment that stores a mapping from variables to their
values, as described in Chapter 19. We’ve provided a type signature for
environments. It stipulates types for environments and values, and
functions to create an empty environment (which we’ve already imple-
mented for you), to extend an environment with a new B I N D I N G , that
is, a mapping of a variable to its (mutable) value, and to look up the
value associated with a variable.
The implementation of environments for the purpose of this project
follows that described in Section 19.5. We make use of an environment
that allows the values to be mutable:

type env = (varid * value ref) list

This will be helpful in the implementation of recursion.

Stage 242
Implement the various functions involved in the Env module and test
them thoroughly.

How will these environments be used? Atomic literals – like numer-

als and truth values – evaluate to themselves as usual, independently
of the environment. But to evaluate a variable in an environment, we
look up the value that the environment assigns to it and return that
value.
A slightly more complex case involves function application, as in
this example:

(fun x -> x + x) 5

The abstract syntax for this expression is an application of one expres-

sion to another. Recall the environment semantics rule for applications
from Figure 19.1:

E ⊢P Q ⇓
RRR E ⊢ P ⇓ fun x -> B
RRR
RRR E ⊢ Q ⇓ v (R app )
RRR Q
RRR E {x ↦ v } ⊢ B ⇓ v
R Q B

⇓ vB

According to this rule, to evaluate an application P Q in an environ-

ment E ,

1. Evaluate P in E to a value v P , which should be a function of the

form fun x -> B . If v P is not a function, raise an evaluation error.
410 PROGRAMMING WELL

2. Evaluate Q in the environment E to a value vQ .

3. Evaluate B in the environment obtained by extending E with a

binding of x to vQ .

The formal semantics rule translates to what is essentially pseudocode

for the interpreter.
In the example: (1) fun x -> x + x is already a function, so evalu-
ates to itself. (2) The argument 5 also evaluates to itself. (3) The body x
+ x is thus evaluated in an environment that maps x to 5.
For let expressions, a similar evaluation process is used. Recall the
semantics rule:

E ⊢ let x = D in B ⇓

E ⊢ D ⇓ vD
∣ (R let )
E {x ↦ v D } ⊢ B ⇓ v B

⇓ vB

We’ll apply this rule in evaluating an expression like

let x = 3 * 4 in x + 1 ;;

To evaluate this expression in, say, the empty environment, we first

evaluate (recursively) the definition part in the same empty envi-
ronment, presumably getting the value 12 back. We then extend the
environment to associate that value with the variable x to form a new
environment, and then evaluate the body x + 1 in the new environ-
ment. In turn, evaluating x + 1 involves recursively evaluating x and
1 in the new environment. The latter is straightforward. The former
involves just looking up the variable in the environment, retrieving
the previously stored value 12. The sum can then be computed and
returned as the value of the entire let expression.
Don’t be surprised that this dynamically scoped evaluator exhibits
all of the divergences from the substitution-based evaluator that were
discussed in Section 19.2.1. For instance, the evaluator will return
different values for certain expressions; it will allow let-bound vari-
ables to be used recursively; and it will fail on simple curried functions.
That’s fine. Indeed, it’s a sign you’ve implemented the dynamic scope
regime correctly. But it does motivate implementation of a lexical-
scoped version of the evaluator described below.

Stage 243
Implement another evaluation function eval_d : Expr.expr ->
Env.env -> Env.value (the ‘d’ is for dynamically scoped environment
semantics), which works along the lines just discussed. Make sure to
test it on a range of tests exercising all the parts of the language.
 F I N A L P R O J E C T: I M P L E M E N T I N G M I N I M L 411

21.4 Extending the language

In this final part of the project, you will extend MiniML in one or more
ways of your choosing.

21.4.1 Extension ideas

Here are a few ideas for extending the language, very roughly in or-
der from least to most ambitious. Especially difficult extensions are
marked with ¢ symbols.

1. Add additional atomic types (floats, strings, unit, etc.) and corre-
sponding literals and operators.

2. Modify the environment semantics to manifest lexical scope in-

stead of dynamic scope (Section 21.4.2).

3. Augment the syntax by allowing for one or more bits of syntactic

sugar, such as the curried function definition notation seen in let
f x y z = x + y * z in f 2 3 4.

4. Add lists to the language.

5. Add records to the language.

6. Add references to the language, by adding operators ref, !, and :=.

Since the environment is already mutable, you can even implement
this extension without implementing stores and modifying the type
of the eval function, though you may want to anyway.

7. Add laziness to the language (by adding refs and syntactic sugar for
the lazy keyword). If you’ve also added lists, you’ll be able to build
infinite streams.

8. Add better support for exceptions, for instance, multiple different

exception types, exceptions with arguments, exception handling
with try...with....

9. ¢ Add simple compile-time type checking to the language. For this

extension, the language would be extended so that every intro-
duction of a bound variable (in a let, let rec, or fun construct)
is accompanied by its (monomorphic) type. The abstract syntax
would need to be extended to store those types, and you would
write a function to walk the tree to verify that every expression in
the program is well typed. This is a quite ambitious project.

10. ¢¢ Add type inference to the language, so that (as in OCaml) types
are inferred even when not given explicitly. This is extremely ambi-
tious, not for the faint of heart. Do not attempt to do this.
412 PROGRAMMING WELL

Most of the extensions (in fact, all except for (2)) require extensions
to the concrete syntax of the language. We provide information about
extending the concrete syntax in Section 21.4.3. Many other extensions
are possible. Don’t feel beholden to this list. Be creative!
In the process of extending the language, you may find the need to
expand the definition of what an expression is, as codified in the file
expr.mli. Other modifications may be necessary as well. That is, of
course, expected, but you should make sure that you do so in a manner
compatible with the existing codebase so that unit tests based on the
provided definitions continue to function. The ability to submit your
code for testing should help with this process. In particular, if you have
to make changes to mli files, you’ll want to do so in a way that extends
the signature, rather than restricting it.
Most importantly: It is better to do a great job (clean, elegant de-
sign; beautiful style; well thought-out implementation; evocative
demonstrations of the extended language; literate writeup) on a
smaller extension, than a mediocre job on an ambitious extension.
That is, the scope aspect of the project will be weighted substantially
less than the design and style aspects. Caveat scriptor.

21.4.2 A lexically scoped environment semantics

One possible extension is to implement a lexically scoped environ-

ment semantics, perhaps with some further extensions. Consider the
following OCaml expression, reproduced from Section 19.2.2:

let x = 1 in
let f = fun y -> x + y in
let x = 2 in
f 3 ;;

Exercise 244
What should this expression evaluate to? Test it in the OCaml interpreter. Try this
expression using your eval_s and eval_d evaluators. Which ones accord with OCaml’s
evaluation?
The eval_d evaluator that you’ve implemented so far is dynamically
scoped. The values of variables are governed by the dynamic ordering
in which they are evaluated. But OCaml is lexically scoped. The values
of variables are governed by the lexical structure of the program. (See
Section 19.2.2 for further discussion.) In the case above, when the
function f is applied to 3, the most recent assignment to x is of the
value 2, but the assignment to the x that lexically outscopes f is of the
value 1. Thus a dynamically scoped language calculates the body of f,
x + y, as 2 + 3 (that is, 5) but a lexically scoped language calculates
the value as 1 + 3 (that is, 4).
 F I N A L P R O J E C T: I M P L E M E N T I N G M I N I M L 413

The substitution semantics manifests lexical scope, as it should,

but the dynamic semantics does not. To fix the dynamic semantics, we
need to handle function values differently. When a function value is
computed (say the value of f, fun y -> x + y), we need to keep track
of the lexical environment in which the function occurred so that when
the function is eventually applied to an argument, we can evaluate
the application in that lexical environment – the environment when
the function was defined – rather than the dynamic environment – the
environment when the function was called.
The technique to enable this is to package up the function being
defined with a snapshot of the environment at the time of its defini-
tion into a closure. There is already provision for closures in the env
module. You’ll notice that the value type has two constructors, one
for normal values (like numbers, booleans, and the like) and one for
closures. The Closure constructor just packages together a function
with its lexical environment.

Stage 245
(if you decide to do a lexically scoped evaluator in service of your ex-
tension) Make a copy of your eval_d evaluation function and call it
eval_l (the ‘l’ for lexically scoped environment semantics). Modify the
code so that the evaluation of a function returns a closure containing
the function itself and the current environment. Modify the function
application part so that it evaluates the body of the function in the
lexical environment from the corresponding closure (appropriately
updated). As usual, test it thoroughly. If you’ve carefully accumulated
good unit tests for the previous evaluators, you should be able to fully
test this new one with just a single function call.
Do not just modify eval_d to exhibit lexical scope, as this will cause
our unit tests for eval_d (which assume that it is dynamically scoped)
to fail. That’s why we ask you to define the lexically scoped evaluator
as eval_l. The copy-paste recommendation for building eval_l from
eval_d makes for simplicity in the process, but will undoubtedly leave
you with redundant code. Once you’ve got this all working, you may
want to think about merging the two implementations so that they
share as much code as possible. Various of the abstraction techniques
you’ve learned in the course could be useful here.

Implementing recursion in the lexically-scoped evaluator By far the

trickiest bit of implementing lexical scope is the treatment of recur-
sion, so we address it separately. Consider this expression, which
makes use of an (uninteresting) recursive function:

let rec f = fun x -> if x = 0 then x else f (x - 1) in f 2 ;;

414 PROGRAMMING WELL

The let rec expression has three parts: a variable name, a definition
expression, and a body. To evaluate it, we ought to first evaluate the
definition part, but using what environment? If we use the incoming
(empty) environment, then what will we use for a value of f when we
reach it? Ideally, we should use the value of the definition, but we don’t
have it yet.
Following the approach described in Section 19.6.1, in the interim,
we’ll extend the environment with a special value, Unassigned, as the
value of the variable being recursively defined. You may have noticed
this special value in the expr type; uniquely, it is never generated by
the parser. We evaluate the definition in this extended environment,
hopefully generating a value. (The definition part better not ever eval-
uate the variable name though, as it is unassigned; doing so should
raise an EvalError. An example of this run-time error might be let
rec x = x in x.) The value returned for the definition can then re-
place the value for the variable name (thus the need for environments
to map variables to mutable values) and the environment can then be
used in evaluating the body.
In the example above, we augment the empty environment with a
binding for f to Unassigned and evaluate fun x -> if x = 0 then
x else f (x - 1) in that environment. Since this is a function, it
is already a value, so evaluates to itself. (Notice how we never had to
evaluate f in generating this value.)
Now the environment can be updated to have f have this function
as a value – not extended (using the extend function) but *actually
modified* by replacing the value stored in the value ref associated
with f in the environment. Finally, the body f 2 is evaluated in this
environment. The body, an application, evaluates f by looking it up in
this environment yielding fun x -> if x = 0 then x else f (x -
1) and evaluates 2 to itself, then evaluates the body of the function in
the prevailing environment (in which f has its value) augmented with a
binding of x to 2.
In summary, a let rec expression like let rec x = D in B is
evaluated via the following five-step process:

1. Extend the incoming environment with a binding of x to

Unassigned; call this extended environment env_x.

2. Evaluate the definition subexpression D in that environment to get a

value v_D.

3. Mutate env_x so that x now maps to v_D.

4. Evaluate the body subexpression B to get a value v_B.

5. Return v_B.
 F I N A L P R O J E C T: I M P L E M E N T I N G M I N I M L 415

21.4.3 The MiniML parser

We provided you with a MiniML parser that converts the concrete

syntax of MiniML to an abstract syntax representation using the expr
type. But to extend the implemented language, you’ll typically need
to extend the parser. Feel free to do so, but make sure that you extend
the language by adding new constructs to the expr type, without
changing the ones that are already given. For instance, if you want to
add support for multiple exceptions, you’ll want to leave the Raise
construct as is (so we can test it with our unit tests) and add your own
new construct, say RaiseExn for the extension.
The parser we provided was implemented using ocamllex and
menhir, programs designed to build lexical analyzers and parser
for programming languages. Documentation for them can be
found at http://caml.inria.fr/pub/docs/manual-ocaml/lexyacc.
html, http://cambium.inria.fr/~fpottier/menhir/manual.html,
and tutorial material is available at https://ohama.github.io/
ocaml/ocamllex-tutorial/ and https://dev.realworldocaml.org/
parsing-with-ocamllex-and-menhir.html.
In summary, ocamllex takes a specification of the tokens of a
programming language in a file, in our case miniml_lex.mll. The
ocamlbuild system knows how to use ocamllex to turn such files into
OCaml code for a lexical analyzer in the file miniml_lex.ml. Simi-
larly, a menhir specification of a parser in a file miniml_parse.mly
will be transformed by menhir (automatically with ocamlbuild) to
a parser in miniml_parse.ml. By modifying miniml_lex.mll and
miniml_parse.mly, you can modify the concrete syntax of the MiniML
language, which may be useful for many of the extensions you might
be interested in.

21.5 Submitting the project

Stage 246
Write up your extensions in a short but formal paper describing and
demonstrating any extensions and how you implemented them.
Use Markdown or LATEX format, and name the file writeup.md or
writeup.tex. You’ll submit both the source file and a rendered PDF
file.

In addition to submitting the code implementing MiniML to the

course grading server through the normal process, you should sub-
mit the writeup.md or writeup.tex file and the rendered PDF file
writeup.pdf as well.
Make sure to use git add to track any new files you create for
416 PROGRAMMING WELL

the final project (such as your writeup or any code files for testing)
before submitting. You can run git status to see if there are any
untracked files in your repository. Finally, remember that you can look
on Gradescope to check that your submissions contains the files you
expect. Unfortunately, we can’t accept any files that are not submitted
on time.

21.6 Alternative final projects

Students who have been doing exceptionally well in the course to date
can petition to do alternative final projects of their own devising, under
the following stipulations:

1. Alternative final projects can be undertaken individually or in

groups of up to four.

2. The implementation language for the project must be OCaml.

3. You will want to talk to course staff about your ideas early to get
initial feedback.

4. You will need to submit a proposal for the project by April 16, 2021.
The proposal should describe what the project goals are, how you
will go about implementing the project, and how the work will be
distributed among the members of the group (if applicable).

5. You will receive notification around April 19, 2021 as to whether

your request has been approved. Approval will be based on perfor-
mance in the course to date and the appropriateness of the project.

6. You will submit a progress report by April 26, 2021, including a

statement of progress, any code developed to date, and any changes
to the expected scope of the project.

7. You will submit the project results, including all code, a demon-
stration of the project system in action, and a paper describing the
project and any results, by May 5, 2021.

8. You will be scheduled to perform a presentation and demonstration

of your project for course staff during reading period.

9. The group as a whole may drop out of the process at any time.
Individual members of the group would then submit instead the
standard final project described here.
A
Mathematical background and notations

In this book, we make free use of a wide variety of mathematical con-

cepts and associated notations, some of which may be unfamiliar to
readers. Facility with learning and using notation is an important skill
to develop. In this chapter, we describe some of the notations we use,
both for reference and to help build this facility.

A.1 Functions

Mathematics is full of functions, and of notations for defining them. In

this section we present a menagerie of function-related notations.

A.1.1 Defining functions with equations

A standard technique is to define functions using a set of equations.

Each of the equations provides a part of the definition based on a
particular subset of the possible argument values of the function. For
instance, consider the factorial function, which we’ll denote with the
symbol “fact”. It is defined by these two equations:

fact (0) = 1
fact (n ) = n ⋅ fact (n − 1) for n > 0

Sometimes the cases are depicted overtly using a large brace:

1 for n = 0
fact (n ) = {
n ⋅ fact (n − 1) for n > 0

A ‘for’ or ‘where’ clause after an equation provides further con-

straint on the applicability of that equation. In the case at hand, the
second equation applies only when the argument n is greater than 0.
In equational definitions, each equation must apply disjointly. If there
were two equations that applied to a particular input, it would be un-
clear which of the two to use. These further constraints can guarantee
disjointness and remove ambiguity.
 Functions
Objective To define a function by using equations.
418 P R O G R A M M I N G W E L L
Tickets to the senior class play cost $5. Production expenses are $500. The
class's profit, p will depend on 11, the number of tickets sold.
profit = $5 . (number of tickets) - $500 or p = 511 - 500

A.1.2 Notating
The equation function
p = 511 application
500 describes a correspondence between the number of
tickets sold and the profit. This correspondence is a function whose domain is
Inthe
thesetfactorial
of ticketsexample,
that couldwe
possibly
used bethesold.
familiar mathematical notation
domain D {O, I, 2, ... }.
for applying a function to an argument – naming the function followed
The range is the set of profits that are possible, including "negative profits,"
by its argument in parentheses: fact (n ).
or losses, if too few tickets are sold.
range R = {-500, -495, -490, ...}.
If we call this profit function p. we can use arrow notation and write Figure A.1: A snippet from a typical
middle school algebra textbook (Brown
the rule P: 11 -7 511 - 500,
et al., 2000, page 379), introducing
which is read "the function P that assigns 511 - 500 to II" or "the function P standard mathematical function
that pairs 11 with 511 - 500." We could also use functional notation: application notation.
P(I1) = 511 500
which is read "P of 11 equals 511 - 500" or "the value of P at 11 is 511 - 500."
To specify a function completely, you must describe the domain of the
function as well as give the rule. The numbers assigned by the rule then form
the range of the function.
Some time in your primary education, perhaps in middle school,
you were taught this standard mathematical notation for applying a
Example 1 List the range of
function to one or g: more
x -7 arguments.
4 + 3x - x 2 In Figure A.1, a snapshot from
x 4 + 3x -./
a middle schoolifalgebra textbook shows where
the domain D = {-I, 0, I, 2}. this notation is first
-I 4 + 3(-1) (-1)2 = 0
taught: “We could also use functional notation: P (n ) = 5n − 500, which 2
Solution In 4 + 3x - x 2 replace x with each 0 4 + 3(0) - 0 = 4
is read ‘P of n equals 5n − 500.’” In this notation, functions can take one2
member of D to find the members
I 4 + 3(1) - 1 = 6
or more arguments,
of the notated
range R. by placing the arguments in parentheses
2
:. R 6} Answerthe function name. This
= {O, 4, following
2 4 + 3(2) - 2 = 6
and separated by commas notation
is so familiar that it’s hard to imagine that someone had to invent it.
Note that did.
But someone In fact,gitinwas
the function Example I assigns
the 18th the number
century 6 to both I
Swiss mathematician
and 2. In listing the range of g, however, you name 6 only once.
Leonhard Eulerofwho
Members in 1734
the range of a first used
function arethis notation
called (Figure
values of A.3). Since
the function. In
then, it hasI, become
Example the valuesuniversal.
of the function g are
At this 0, 4, the
point, and notation
6. To indicate
is sothat the
familiar
function g assigns to 2 the value 6, you write
that it is impossible to see f (1, 2, 3) without immediately interpreting it
g(2) = 6,
as the application of the function f to arguments 1, 2, and 3.
which is read "g of 2 equals 6" or "the value of g at 2 is 6." Note that g(2)
isIt110t
is thus perhaps
the product of gsurprising that the
and 2. It names OCaml doesn’t
number that g use this
assigns to notation
2.
for function application. Instead, it follows the notational convention
Introduction to Functions 379
proposed by the Princeton mathematician and logician Alonzo Church
in his so-called lambda calculus (Section A.1.4), a logic of functions.
In the lambda calculus, functions and their application are so central
(indeed, there’s basically nothing else in the logic) that the addition of Figure A.2: Leonhard Euler (1707–1783)
the parentheses in the function application notation is too onerous. invented the familiar parenthesized
notation for function application.
Instead, Church proposed merely prefixing the function to its argu-
ment. Instead of f (1), Church’s notation would have f 1. Instead of
f (g (1)), f (g 1).

A.1.3 Alternative mathematical notations for functions and

their application

Despite the ubiquity of Euler’s notation, mathematicians use a variety

of different notations for functions and their application.
Certainly, mathematics uses different conventions for denoting
 M AT H E M AT I C A L B A C KG R O U N D A N D N OTAT I O N S 419

Figure A.3: The first known instance of

the now standard function application
notation, in a 1734 paper by Leonhard
Euler. Note the f ( ax + c ). The function
is even named f !

operations than any given programming language. In the second fact

equation, for instance, a center dot ⋅ is used for multiplication instead
of the * more common in programming languages. In other cases,
simple juxtaposition is used for multiplication, as in 3x 2 where the jux-
taposition of the 3 and the x 2 indicates that they are to be multiplied.
The details of these notations are often left unspecified in mathemat-
ical writing, reflecting the reality that mathematics is written to be
read by people, people with sufficient common knowledge with the
author to know the background assumptions or to figure them out
from context. We don’t have such a privilege with computers, so nota-
tions are typically more carefully explicated in programming language
documentation.
The kind of thing that the argument must be (what computer sci-
entists would call its “type”) is often left implicit in mathematical
notation. In the factorial example, we didn’t state explicitly that the
argument of factorial must be a nonnegative integer, yet the definition
is only appropriate for that case. Negative integers are not provided
a well-founded definition for instance, nor are noninteger numbers.
Again, the omission of these requirements is based on an assumption
of shared context with the reader. So as not to have to make that as-
sumption, computer programs that implement function definitions
make use of type constraints (whether explicit or inferred) or invariant
assertions or (as a last resort) documentation to capture these assump-
tions.
The entire set of equations defines a single function, so that in
converting definitions of this sort to code, they will typically end up in
a single function definition. The individual equations correspond to
different cases, which will likely be manifest by conditionals or case
statements (such as OCaml match expressions).
Of course, the more standard notation for the factorial function is a
postfix exclamation mark (!):

0! = 1
n ! = n ⋅ (n − 1)! for n > 0
420 PROGRAMMING WELL

Figure A.4: Rules for taking deriva-

tives for a variety of expression types.
(Reproduced from Figure 11.8.)
( f (x ) + g (x ))′ = f ′ (x ) + g ′ (x )
( f (x ) − g (x ))′ = f ′ (x ) − g ′ (x )
( f (x ) ⋅ g (x ))′ = f ′ (x ) ⋅ g (x ) + f (x ) ⋅ g ′ (x )
f (x ) ( f ′ (x ) ⋅ g (x ) − f (x ) ⋅ g ′ (x ))
′
( ) =
g (x ) g (x )2
(sin f (x ))′ = f ′ (x ) ⋅ cos f (x )
(cos f (x ))′ = f ′ (x ) ⋅ ~ sin f (x )
f ′ (x )
(ln f (x ))′ =
f (x )
( f (x ) ) = h ⋅ f ′ (x ) ⋅ f (x )h −1
h ′

where h contains no variables

f ′ (x ) ⋅ g (x )
( f (x )g (x ) )′ = f (x )g (x ) ⋅ (g ′ (x ) ⋅ ln f (x ) + )
f (x )
(n )′ = 0 where n is any constant
(x )′ = 1

The point is that the Euler notation is not the only one that can be or is
used for function application. Here are some more examples:

• Frequently, superscripts are used to denote function application, for

instance, as in Figure 11.8 (reproduced here as Figure A.4), where a
superscript prime symbol specifies the derivative function.
d 3
• Newton’s notation for derivatives, for example, dx
x , provides yet
another example of a nonstandard notation for a function appli-
cation. Here, the function being applied is again the derivative
d
function, this time as depicted by the compound notation dx
, its
argument the expression x 3 .1 1
For the notation cognoscenti, what’s
really going on in this notation is that
• In Chapter 13, a specific notation is used to express the substitution the ddx is both a binding construct,
binding the x as the argument to an
function, a function over a variable (x) and two expressions (P
anonymous function that is (in Church’s
and Q) that returns the expression P with all free occurrences of x lambda calculus notation) λx.x 3 and a
replaced by Q. That function is not notated by the Euler notation function application of the derivative
function.
(say, subst (x, P ,Q )) but rather with a special notation employing
brackets and arrows P [x ↦ Q ]. Nonetheless, it’s still just a function
applied to some arguments.

The notational profligacy of mathematics – especially having many

different notations for functions – hides a lot of commonality shared
 M AT H E M AT I C A L B A C KG R O U N D A N D N OTAT I O N S 421

among mathematical processes. Don’t be confused; despite all the

notations, they’re all just functions.

A.1.4 The lambda notation for functions

Part of the notation for defining functions equationally involves giving

them a name. For instance, the A B S O L U T E VA L U E function can be
defined equationally as
√
abs (n ) = n2

One of the contributions of Church’s lambda calculus is a notation

for defining functions directly, without bestowing a name. In fact, the
√
expression on the right hand side of the equation, n 2 , almost serves
√
this purpose already, by specifying the function from n to n 2 . There
√
are two problems in using bare expressions like n 2 to specify func-
tions. First, how is the reader to know that the expression is intended
to specify a function rather than a number ? That is, how are we to real-
ize that the use of n is meant generically, and not as standing for some
particular number? Second, if the expression makes use of multiple
variables, how is the reader supposed to determine which variable
√
represents the input to the function? In the case of n 2 , there is only
one option, since the expression makes use of only one variable. But
for other expressions, like m ⋅ n 2 , it is unclear if the input is intended to
be m or n.
Church introduced his lambda notation to solve these problems.
He prefixes the expression with a Greek lambda (λ), followed by the
variable that is serving as the input to the function, followed by a
period. Table A.1 provides some examples.

√ √
λn. n 2 The function from n to n 2 , that is, the absolute Table A.1: A few functions in lambda
notation, with their English glosses and
value function, or, in OCaml:
their approximate OCaml equivalents.
fun n -> sqrt(n *. n)
λn.(m ⋅ n ) 2
the function from n to m ⋅ n 2 , so that m is implicitly
being viewed as a constant:
fun n -> m *. (n *. n)
λm.(m ⋅ n 2 ) the function from m to m ⋅ n 2 , so that n is implicitly
being viewed as a constant:
fun m -> m *. (n *. n)
λm.λn.(m ⋅ n ) 2
the function from m to a function from n to m ⋅ n 2 :
fun m -> fun n -> m *. (n *. n)

The lambda notation for specifying anonymous functions will be

familiar to OCaml programmers; it appears in OCaml as well, though
422 PROGRAMMING WELL

under a different concrete syntax. The keyword fun plays the role of λ
and the operator -> plays the role of the period. In fact, the ability to
p q p and q p or q not p
define anonymous functions, so central to functional programming
true true true true false
languages, is inherited directly from the lambda notation that gives its
true false false true false
name to Church’s calculus. false true false true true
As shown in Table A.1, each of the examples above could be false false false false true

rephrased in OCaml. You may recognize the last of these as an example Figure A.5: The three boolean operators
defined.
of a curried function (Section 6.1).
When there’s a need for specifying mathematical functions directly,
unnamed, we will take advantage of Church’s lambda notation, espe-
cially in Chapter 14.

A.2 Logic
hypotenuse
B
The logic of propositions, boolean logic, underlies the bool type. In-
c
formally, propositions are conceptual objects that can be either true or a
false. Propositions can be combined or transformed with various oper- right angle
ations. The C O N J U N C T I O N of two propositions p and q is true just in C A
b
case both p and q are true, and false otherwise. The D I S J U N C T I O N is
true just in case either p or q (or both) are true. The N E G AT I O N of p is Figure A.6: A right triangle. Angle C is a
true just in case p is not true (that is, p is false). Conjunction, disjunc- right angle. The opposite side, of length
c, is the hypotenuse. By Pythagorus’s
tion, and negation thus correspond roughly to the English words “and”,
theorem, a 2 + b 2 = c 2 .
“or”, and “not”, respectively, and for that reason, we sometimes speak
of the “and” of two boolean values, or their “or”. (See Figure A.5.)
There are other operations on boolean values considered in logic –
for instance, the conditional, glossed by “if . . . then . . . ”; or the exclusive
“or” – but these three are sufficient for our purposes. For more back-
ground on propositional logic, see Chapter 9 of the text by Lewis and
Zax (2019).
(x2 , y2 )
(x2 x1 )2 + (y2 y1 )2
A.3 Geometry y2 y1

The S L O P E of a line between two points x 1 , y 1 and x 2 , y 2 is the ratio x2 x1

y2 −y1 (x1 , y1 )
of their vertical difference and their horizontal difference, x 2 −x 1
. (See
Figure A.7.)
Figure A.7: Two points, given by a pair
A R I G H T T R I A N G L E is a triangle one of whose edges is a right (90○ ) of their x (horizontal) and y (vertical)
angle. The side opposite the right angle is called the H Y P OT E N U S E . coordinates. The slope of the line
y −y
between them is x2 −x1 . The distance
Pythagorus’s theorem holds that the sum of the squares of the adjacent 2 1
between them,
√ as per the Pythagorean
sides’ lengths is the square of the length of the hypotenuse. theorem, is (x 2 − x 1 )2 + ( y 2 − y 1 )2 .
Pythagorus’s theorem can be used to determine the D I S TA N C E
between two points specified with Cartesian (x-y) coordinates. As
depicted in Figure A.7, by Pythagorus’s theorem, we can square the
differences in each dimension, sum the squares, and take the square
 M AT H E M AT I C A L B A C KG R O U N D A N D N OTAT I O N S 423

root.
The ratio of the circumference of a circle and its diameter is (non- c
trivially, and perhaps surprisingly) a constant, conventionally called π r
(read, “pi”), and approximately 3.1416. This constant is also the ratio o
A d
of the area of a circle to the area of a square whose side is the circle’s
radius. Thus, using the nomenclature of Figure A.8, c = πd = 2πr and
A = πr 2 . Figure A.8: Geometry of the circle at
origin o of radius r , diameter d = 2r ,
circumference c, and area A.

A.4 Sets

A set is a collection of distinct (physical or mathematical) objects.

An E X T E N S I O N A L set definition (given by an explicit list of its mem-
bers) is notated by listing the elements in braces separated by commas,
as, for instance, {1, 2, 3, 4}. Obviously, this notation only works for
finite sets, although infinite sets can be informally indicated with
ellipses (as {1, 2, 3, . . .}) in cases where the rule for filling in the remain-
ing elements is sufficiently obvious to the reader.
An I N T E N S I O N A L set definition (given by describing all members
of the set rather than listing them) is notated by placing in braces a
schematic element of the set, followed by a vertical bar, followed by a
description of the range of any variables in the schema. For instance,
the set of all even numbers might be { x ∣ x mod 2 = 0 }, read “the set of
all x such that x is evenly divisible by 2.” Similarly, the set of all squares
of prime numbers would be { x 2 ∣ x is prime }. (Note the combination
of mathematical notation and natural language, a typical instance of
“code switching” in mathematical writing.)
The E M P T Y S E T , notated ∅, is the set containing no members.
Certain standard operations on sets are notated with infix operators:

Union: s ∪ t is the U N I O N of sets s and t , that is, the set containing all
the elements that are in either of the two sets;

Intersection: s ∩ t is the I N T E R S E C T I O N , containing just the elements

that are in both of the sets;

Difference: s − t is the set D I F F E R E N C E , all elements in s except for

those in t ; and

Membership: x ∈ s specifies M E M B E R S H I P , stating that x is a member

of the set s.

By way of example, the following are all true statements, expressed in

424 PROGRAMMING WELL

this notation:

{1, 2, 3} ∪ {3, 4} = {1, 2, 3, 4}

{1, 2, 3} ∩ {3, 4} = {3}
{1, 2, 3} − {3, 4} = {1, 2}
3 ∈ {1, 2, 3}
3 ∈/ {2, 4, 6}

Note the use of a slash through a symbol to indicate its N E G AT I O N : ∈/

for ‘is not a member of’.

A.5 Equality and identity

There are different notions of I D E N T I T Y used in mathematical no-

tation. The = symbol typically connotes two values being the same
“semantically”. The ≡ symbol connotes a stronger notion of syntactic
identity, so that x ≡ y means that x and y are (that is, represent) the
same syntactic entity (variable say) rather than that they have the same
value (in whatever context that might be appropriate). For instance,
consider these equations found in the definition of substitution:

x [x ↦ P ] = P
y [x ↦ P ] = y where x ≡
/y

Recall that P [x ↦ Q ] specifies the expression P with all free occur-

rences of x replaced by the expression Q (with care taken not to
capture any free occurrences of x in Q). Here x and y are variables
(metavariables) ranging over expressions that may themselves be
(object-level) variables. The notation x ≡
/ y indicates that the variable
y that constitutes the expression being substituted into is a different
variable from the variable x that is being substituted for.
B
A style guide

This guide provides some simple rules of good programming style,

both general and OCaml-specific, developed for the Harvard course
CS51. The rules presented here tend to follow from a small set of un-
derlying principles.1 1
This style guide is reworked from a
long line of style guides for courses at
Consistency Similar decisions should be made within similar contexts. Princeton, University of Pennsylvania,
and Cornell, including Cornell CS
312, U Penn CIS 500 and CIS 120, and
Brevity “Everything should be made as simple as possible, but no Princeton COS 326. All this shows the
simpler.” (attr. Albert Einstein) great power of recursion. (Also, the
joke about recursion was stolen from
COS 326. (Also, the joke about the joke
Clarity Code should be chosen so as to communicate clearly to the about recursion was stolen from Greg
human reader. Morrisett. I think. See the Preface.))

Transparency Appearance should summarize and reflect structure.

Like all rules, those below are not to be followed slavishly. Rather, they
should be seen as instances of these underlying principles. These
principles may sometimes be in conflict, in which case judgement is
required in finding the best way to write the code. This is one of the
many ways in which programming is an art, not (just) a science.
This guide is not complete. For more recommendations, from the
OCaml developers themselves, see the official OCaml guidelines.

Figure B.1: Yes, coding style is impor-

tant.
426 PROGRAMMING WELL

B.1 Formatting

Formatting concerns the layout of the text of a program on the screen

or page, such issues as vertical alignments and indentation, line
breaks, and whitespace. To allow for repeatable formatting, code is
typically presented with a fixed-width font in which all characters
including spaces take up the same horizontal pitch.

B.1.1 No tab characters

You may feel inclined to use tab characters ( A S C I I 0x09) to align text.
Do not do so; use spaces instead. The width of a tab is not uniform
across all renderings, and what looks good on your machine may look
terrible on another’s, especially if you have mixed spaces and tabs.
Some text editors map the tab key to a sequence of spaces rather than
a tab character; in this case, it’s fine to use the tab key.

B.1.2 80 column limit

No line of code should extend beyond 80 characters long. Using more

than 80 columns typically causes your code to wrap around to the next
line, which is devastating to readability.

B.1.3 No needless blank lines

The obvious way to stay within the 80 character limit imposed by the
rule above is to press the enter key every once in a while. However,
blank lines should only be used at major logical breaks in a program,
for instance, between value declarations, especially between function
declarations. Often it is not necessary to have blank lines between
other declarations unless you are separating the different types of
declarations (such as modules, types, exceptions and values). Unless
function declarations within a let block are long, there should be no
blank lines within a let block. There should absolutely never be a
blank line within an expression.

B.1.4 Use parentheses sparely

Parentheses have many purposes in OCaml, including constructing

tuples, specifying the unit value, grouping sequences of side-effect
expressions, forcing higher precedence on an expression for parsing,
and grouping structures for functor arguments. Clearly, parentheses
must be used with care, as they force the reader to disambiguate the
intended purpose of the parentheses, making code more difficult to
 A STYLE GUIDE 427

understand. You should therefore only use parentheses when neces-

sary or when doing so improves readability.

7 let x = function1 (arg1) (arg2) (function2 (arg3)) (arg4)

3 let x = function1 arg1 arg2 (function2 arg3) arg4

On the other hand, it is often useful to add parentheses to help

indentation algorithms, as in this example:

7 let x = "Long line ..."

^ "Another long line..."

3 let x = ("Long line ..."

^ "Another long line...")

Similarly, wrapping match expressions in parentheses helps avoid a

common (and confusing) error that you get when you have a nested
match expression. (See Section 10.3.2 for an example.)
Parentheses should never appear on a line by themselves, nor
should they be the first visible character; parentheses do not serve
the same purpose as brackets do in C or Java.

B.1.5 Delimiting code used for side effects

Imperative programs will often have sequences of expressions to be

evaluated primarily for side effect rather than value. When delimiting
the scope of such sequences, use begin ⟨⟩ end rather than parentheses,
for instance,

7 if condition then
(do this;
do that;
do the other)
else
(do something else entirely;
do this too);
do in any case

3 if condition then begin

do this;
do that;
do the other
end else begin
do something else entirely;
do this too
end;
do in any case
428 PROGRAMMING WELL

B.1.6 Spacing for operators and delimiters

Infix operators (arithmetic operators like + and *, the typing operator

:, type forming operators like * and ->, etc.) should be surrounded by
spaces. Delimiters (like the list item delimiter ; and the tuple element
delimiter ,) are followed but not preceded by a space.

3 let f (x : int) : int * int = 3 * x - 1, 3 * x + 1 ;;

7 let f (x: int): int*int = 3* x-1, 3* x+1 ;;

Judgement can be applied to vary from these rules for clarity’s sake,
for instance, when emphasizing precedence.

3 let f (x : int) : int * int = 3*x - 1, 3*x + 1 ;;

When expressions with operators get overly long, it may be desir-

able to add line breaks. Such line breaks should tend to be placed just
before, rather than just after, operators, so as to highlight the operator
at the beginning of the next line.

3 let price = base * (100 + tax_pct) / 100 ;;

7 let price = base *

(100 + tax_pct) /
100 ;;

3 let price = base

* (100 + tax_pct)
/ 100 ;;

It’s better to place breaks at operators higher in the abstract syntax tree,
to emphasize the structure.

7 let price = base * (100

+ tax_pct) / 100 ;;

In the case of delimiters, however, line breaks should occur after the
delimiter.

7 let r = { product = "Dynamite"

; company = "Acme"
; price = base * (100 + tax_pct) / 100} ;;

3 let r = {product = "Dynamite";

company = "Acme";
price = base * (100 + tax_pct) / 100} ;;

Of course, keep in mind that understanding of the code might be

enhanced by restructuring the code and naming partial results:

3 let tax = base * tax_pct / 100 ;;

let price = base + tax ;;
 A STYLE GUIDE 429

B.1.7 Indentation

Indentation should be used to encode the block structure of the code

as described in the following sections. It is typical to indent by two xor
four spaces. Choose one system for indentation, and be consistent
throughout your code.

Indenting if expressions Indent if expressions using one of the

following methods, depending on the sizes of the expressions. For very
short then and else branches, a single line may be sufficient.

3
if exp1 then veryshortexp2 else veryshortexp3

When the branches are too long for a single line, move the else onto its
own line.

3
if exp1 then exp2
else exp3

This style lends itself nicely to nested conditionals.

3
if exp1 then shortexp2
else if exp3 then shortexp4
else if exp5 then shortexp6
else exp8

For very long then or else branches, the branch expression can be
indented and use multiple lines.

3
if exp1 then
longexp2
else shortexp3

3
if exp1 then
longexp2
else
longexp3

Some use an alternative conditional layout, with the then and else
keywords starting their own lines.

7
if exp1
then exp2
else exp3

This approach is less attractive for nested conditionals and long

branches, though for unnested cases it can be acceptable.
430 PROGRAMMING WELL

Indenting let expressions Indent the body of a let expression the

same as the let keyword itself.

3 let x = definition in
code_that_uses_x

This is an exception to the rule of further indenting subexpression

blocks to manifest the nesting structure.

7 let x = definition in
code_that_uses_x

The intention is that let definitions be thought of like mathematical

assumptions that are listed before their use, leading to the following
attractive indentation for multiple definitions:

let x = x_definition in
let y = y_definition in
let z = z_definition in
block_that_uses_all_the_defined_notions

Indenting match expressions Indent match expressions so that the

patterns are aligned with the match keyword, always including the
initial (optional) |, as follows:

match expr with

| first_pattern -> ...
| second_pattern -> ...

Some disfavor aligning the arrows in a match, arguing that it makes

the code harder to maintain. However, where there is strong paral-
lelism among the patterns, this alignment (and others) can make the
parallelism easier to see, and hence the code easier to understand. Use
your judgement.

B.2 Documentation

B.2.1 Comments before code

Comments go above the code they reference. Consider the following:

7 let sum = List.fold_left (+) 0

(* Sums a list of integers. *)

3 (* Sums a list of integers. *)

let sum = List.fold_left (+) 0

The latter is the better style, although you may find some source code
that uses the first. Comments should be indented to the level of the
line of code that follows the comment.
 A STYLE GUIDE 431

B.2.2 Comment length should match abstraction level

Long comments, usually focused on overall structure and function

for a program, tend to appear at the top of a file. In that type of com-
ment, you should explain the overall design of the code and reference
any sources that have more information about the algorithms or data
structures. Comments can document the design and structure of a
class at some length. For individual functions or methods comments
should state the invariants, the non-obvious, or any references that
have more information about the code. Avoid comments that merely
restate the code they reference or state the obvious. All other com-
ments in the file should be as short as possible; after all, brevity is the
soul of wit. Rarely should you need to comment within a function;
expressive variable naming should be enough.

B.2.3 Multi-line commenting

There are several styles for demarcating multi-line comments in

OCaml. Some use this style:
(* This is one of those rare but long comments
* that need to span multiple lines because
* the code is unusually complex and requires
* extra explanation. *)
let complicated_function () = ...

arguing that the aligned asterisks demarcate the comment well when
it is viewed without syntax highlighting. Others find this style heavy-
handed and hard to maintain without good code editor support (for
instance, emacs Tuareg mode doesn’t support it well), leading to this
alternative:
(* This is one of those rare but long comments
that need to span multiple lines because
the code is unusually complex and requires
extra explanation.
*)
let complicated_function () = ...

Whichever you use, be consistent.

B.3 Naming and declarations

B.3.1 Naming conventions

Table B.1 provides the naming convention rules that are followed by
OCaml libraries. You should follow them too. Some of these naming
conventions are enforced by the compiler; these are shown in boldface
below. For example, it is not possible to have the name of a variable
start with an uppercase letter.
432 PROGRAMMING WELL

Token Convention Example

Variables and functions Symbolic or initial lower case. Use get_item

underscores for multiword names.
Constructors Initial upper case. Use embedded caps Node, EmptyQueue
for multiword names. Historical
exceptions are true and false.
Types All lower case. Use underscores for priority_queue
multiword names.
Module Types Initial upper case. Use embedded caps PriorityQueue or PRIORITY_QUEUE
for multiword names, or (as we do
here) use all uppercase with
underscores.
Modules Initial upper case. Use embedded caps PriorityQueue
for multiword names.
Functors Initial upper case. Use embedded caps PriorityQueue
for multiword names.
Table B.1: Naming conventions

B.3.2 Use meaningful names

Variable names should describe what the variables are for, in the form
of a word or sequence of words. Proper naming of a variable can be the
best form of documentation, obviating the need for any further doc-
umentation. By convention (Table B.1) the words in a variable name
are separated by underscores (multi_word_name), not (ironically)
distinguished by camel case (multiWordName).

3 let local_date = Unix.localtime (Unix.time ()) ;;

let total_cost = quantity * price_each ;;

7 let d = Unix.localtime (Unix.time ()) ;;

let c = n * at ;;

The length of a variable name is roughly correlated with how long a

reader of the code will have to remember its use. In short let blocks,
one letter variable names can sometimes be appropriate. The defini-
tion

fun the_optional_number -> the_optional_number <> None

is not better than

fun x -> x <> None

(Of course, this function can be specified even more compactly as (<>)
None.)
Often it is the case that a function used in a fold, filter, or map is
named f. Here is an example with appropriate variable names:
 A STYLE GUIDE 433

let local_date = Unix.localtime (Unix.time ()) in

let minutes = date.Unix.tm_min in
let seconds = date.Unix.tm_min in
let f n = (n mod 3) = 0 in
List.filter f [minutes; seconds]

Take advantage of the fact that OCaml allows the prime character ’
in variable names. Use it to make clear related functions:
let reverse (lst : 'a list) =
let rec reverse' remaining accum =
match remaining with
| [] -> accum
| hd :: tl -> reverse' tl (hd :: accum) in
reverse' lst [] ;;

B.3.3 Constants and magic numbers

M A G I C N U M B E R S are explicit values sprinkled in code that are used

without explanation, as 1.0625 in the following code:

7 let total_cost = (quantity *. price_each) *. 1.0625 ;;

Magic numbers are inscrutable, a nightmare for readers of the code.

Instead, give those constants an expressive name. If these defined con-
stants are global, we use the naming convention of using a variable in
all uppercase letters except for an initial lowercase ‘c’ (for “constant”).

3 let cTAX_RATE = .0625 ;;

(* ... some time later ... *)
let total_cost = (quantity *. price_each) *. (1. +. cTAX_RATE)
;;

Not only is this more explanatory – we understand that the final mul-
tiplication is to account for taxes – it allows for a single point of code
change if the tax rate changes.

B.3.4 Function declarations and type annotations

Top-level functions and values should be declared with explicit type

annotations to allow the compiler to verify the programmer’s inten-
tions. Use spaces around :, as with all operators.

7 let succ x = x + 1

3 let succ (x : int) : int = x + 1

When a function being declared has multiple arguments with compli-

cated types, so that the declaration doesn’t fit nicely on one line,

7 let rec zip3 (x : 'a list) (y : 'b list) (z : 'c list) : ('a * 'b *
'c) list option =
...
434 PROGRAMMING WELL

one of the following indentation conventions can be used:

3 let rec zip3 (x : 'a list)

(y : 'b list)
(z : 'c list)
: ('a * 'b * 'c) list option =
...

3 let rec zip3

(x : 'a list)
(y : 'b list)
(z : 'c list)
: ('a * 'b * 'c) list option =
...

B.3.5 Avoid global mutable variables

Mutable values, on the rare occasion that they are necessary at all,
should be local to functions and almost never declared as a structure’s
value. Making a mutable value global causes many problems. First,
an algorithm that mutates the value cannot be ensured that the value
is consistent with the algorithm, as it might be modified outside the
function or by a previous execution of the algorithm. Second, having
global mutable values makes it more likely that your code is nonreen-
trant. Without proper knowledge of the ramifications, declaring global
mutable values can easily lead not only to bad design but also to incor-
rect code.

B.3.6 When to rename variables

You should rarely need to rename values: in fact, this is a sure way to
obfuscate code. Renaming a value should be backed up with a very
good reason. One instance where renaming a variable is both common
and reasonable is aliasing modules. In these cases, other modules used
by functions within the current module are aliased to one or two letter
variables at the top of the struct block. This serves two purposes: it
shortens the name of the module and it documents the modules you
use. Here is an example:

module H = Hashtbl
module L = List
module A = Array
...

B.3.7 Order of declarations in a module

When declaring elements in a file (or nested module) you first alias
the modules you intend to use, then declare the types, then define
exceptions, and finally list all the value declarations for the module.
 A STYLE GUIDE 435

Separating each of these sections with a blank line is good practice

unless the whole is quite short. Here is an example:

module L = List
type foo = int
exception InternalError
let first list = L.nth list 0

Every declaration within the module should be indented the same

amount.

B.4 Pattern matching

B.4.1 No incomplete pattern matches

Incomplete pattern matches are flagged with compiler warnings,

and you should avoid them. In fact, it’s best if your code generates no
warnings at all. Even if you “know” that a certain match case can never
occur, it’s better to record that knowledge by adding the match case
with an action that raises an appropriate error.

B.4.2 Pattern match in the function arguments when possible

Tuples, records, and algebraic datatypes can be deconstructed using

pattern matching. If you simply deconstruct a function argument
before you do anything else substantive, it is better to pattern match in
the function argument itself. Consider these examples:

7 let f arg1 arg2 =

let x = fst arg1 in
let y = snd arg1 in
let z = fst arg2 in
...

3 let f (x, y) (z, _) =

...

7 let f arg1 =
let x = arg1.foo in
let y = arg1.bar in
let baz = arg1.baz in
...

3 let f {foo = x; bar = y; baz} =

...

See also the discussion of extraneous match expressions in let

definitions in Section B.4.4.
436 PROGRAMMING WELL

B.4.3 Pattern match with as few match expressions as necessary

Rather than nesting match expressions, you can sometimes combine

them by pattern matching against a tuple. Of course, this doesn’t
work if one of the nested match expressions matches against a value
obtained from a branch in another match expression. Nevertheless, if
all the values are independent of each other you should combine the
values in a tuple and match against that. Here is an example:

7 let d = Date.fromTimeLocal (Unix.time ()) in

match Date.month d with
| Date.Jan -> (match Date.day d with
| 1 -> print "Happy New Year"
| _ -> ())
| Date.Jul -> (match Date.day d with
| 4 -> print "Happy Independence Day"
| _ -> ())
| Date.Oct -> (match Date.day d with
| 10 -> print "Happy Metric Day"
| _ -> ())

3 let d = Date.fromTimeLocal (Unix.time ()) in

match Date.month d, Date.day d with
| Date.Jan, 1 -> print "Happy New Year"
| Date.Jul, 4 -> print "Happy Independence Day"
| Date.Oct, 10 -> print "Happy Metric Day"
| _ -> ()

(This example also provides a case where aligning arrows improves

clarity by emulating a table.)

B.4.4 Misusing match expressions

The match expression is misused in two common situations. First,

match should never be used with single atomic values in place of an if
expression. (That’s why if exists.) For instance,

7 match e with
| true -> x
| false -> y

3 if e then x else y

and

7 match e with
| c -> x (* c is a constant value *)
| _ -> y

3 if e = c then x else y
 A STYLE GUIDE 437

(Using a match to match against several atomic values may, however,

be preferable to nested conditionals.)
Second, match expressions should not be used when pattern match-
ing within an enclosing expression (like let, fun, function) allows
pattern-matching itself:

7 let x = match expr with

| y, z -> y in
...

3 let x, _ = expr in
...

B.4.5 Avoid using too many projection functions

Frequently projecting a value from a record or tuple causes your code

to become unreadable. This is especially a problem with tuple projec-
tion because the value is not documented by a mnemonic name. To
prevent projections, you should use pattern matching with a function
argument or a value declaration. Of course, using projections is okay as
long as use is infrequent and the meaning is clearly understood from
the context.

7 let v = some_function () in
let x = fst v in
let y = snd v in
x + y

3 let x, y = some_function () in
x + y

Don’t use List.hd or List.tl at all The functions hd and tl are used
to deconstruct list types; however, they raise exceptions on certain
arguments. You should never use these functions. In the case that you
find it absolutely necessary to use these (something that probably
won’t ever happen), you should explicitly handle any exceptions that
can be raised by these functions.

B.5 Verbosity

B.5.1 Reuse code where possible

The OCaml standard library has a great number of functions and

data structures. Unless told otherwise, use them! Become familiar
with the contents of the Stdlib module. Often students will recode
List.filter, List.map, and similar functions. A more subtle situa-
tion for recoding is all the fold functions. Functions that recursively
walk down lists should make vigorous use of List.fold_left or
438 PROGRAMMING WELL

List.fold_right. Other data structures often have a fold function;

use them whenever they are available. (In some exercises, we will ask
you to implement some constructs yourself rather than relying on a
library function. In such cases, we’ll specify that using library functions
is not allowed.)

B.5.2 Do not abuse if expressions

Remember that the type of the condition in an if expression is bool.

There is no reason to compare boolean values against boolean literals.

7 if e = true then x else y

3 if e then x else y

In general, the type of an if expression can be any ’a, but in the

case that the type is bool, you should probably not be using if at all.
Consider the following:

7 3
if e then true else false e
if e then false else true not e
if e then e else false e
if x then true else y x || y
if x then y else false x && y
if x then false else y not x && y

Also problematic is overly complex conditions such as extraneous

negation.

7 if not e then x else y

3 if e then y else x

The exception here is if the expression y is very long and complex, in

which case it may be more readable to have it placed at the end of the
if expression.

B.5.3 Don’t rewrap functions

Don’t fall for the misconception that functions passed as arguments

have to start with fun or function, which leads to the extraneous
rewrapping of functions like this:

7 List.map (fun x -> sqrt x) [1.0; 4.0; 9.0; 16.0]

Instead, just pass the function directly.

3
 A STYLE GUIDE 439

List.map sqrt [1.0; 4.0; 9.0; 16.0]

You can even do this when the function is an infix binary operator,
though you’ll need to place the operator in parentheses.

7 List.fold_left (fun x y -> x + y) 0

3 List.fold_left (+) 0

B.5.4 Avoid computing values twice

When computing values more than once, you may be wasting CPU
time (a design consideration) and making your program less clear (a
style consideration) and harder to maintain (a consideration of both
design and style). The best way to avoid computing things twice is to
create a let expression and bind the computed value to a variable
name. This has the added benefit of letting you document the purpose
of the value with a well-chosen variable name, which means less com-
menting. On the other hand, not every computed sub-value needs to
be let-bound.

7 f (calc_score (if cond then val1 else val2))

(calc_score (if cond then val1 else val2))

3 let score = calc_score (if cond then val1 else val2) in

f score score

B.6 Other common infelicities

Here is a compilation of some other common infelicities to watch out

for:

7 3
x :: [] [x]
length + 0 length
length * 1 length
big_expression * big_expression let x = big_expression in x * x
if x then f a b c1 else f a b c2 f a b (if x then c1 else c2)
String.compare x y = 0 x = y
String.compare x y < 0 x < y
String.compare y x < 0 x > y
C
Solutions to selected exercises

Solution to Exercise 3 ⟨nounphrase ⟩

⟨nounphrase ⟩ ⟨noun ⟩

⟨adjective ⟩ ⟨nounphrase ⟩ party

mad ⟨noun ⟩

tea

Solution to Exercise 4 There are three structures given the rules pro-
vided, corresponding to eaters of flying purple people, flying eaters of
purple people, and flying purple eaters of people.

Solution to Exercise 6

1. +

~- 6

2. ~-

4 6
442 PROGRAMMING WELL

3. +

/ 6

20 ~-

4. *

5 +

3 4

5. *

+ 5

4 3

6. *

+ 5

+ 0

3 4

Solution to Exercise 7 Among the concrete expressions of the abstract

syntax trees are these, though others are possible.

1. ~- (1 + 42)

2. 84 / (0 + 42)

3. 84 + 0 / 42 or 84 + (0 / 42)

Solution to Exercise 8 The value of the golden ratio is about 1.618.

Here’s the calculation using OCaml’s R E P L .

# (1. +. sqrt 5.) /. 2. ;;

- : float = 1.6180339887498949

Note the consistent use of floating point literals and operators, without
which you’d get errors like this:

# (1. + sqrt 5.) /. 2. ;;

Line 1, characters 1-3:
 SOLUTIONS TO SELECTED EXERCISES 443

1 | (1. + sqrt 5.) /. 2. ;;

^^
Error: This expression has type float but an expression was
expected of type
int

Solution to Exercise 9 The fourth and seventh might have struck you
as unusual.
Why does 3.1416 = 314.16 /. 100. turn out to be false? Float-
ing point arithmetic isn’t exact, so that the division 314.16 /. 100.
yields a value that is extremely close to, but not exactly, 3.1416, as
demonstrated here:

# 314.16 /. 100. ;;
- : float = 3.14160000000000039

Why is false less than true? It turns out that all values of a type are
ordered in this way. The decision to order false as less than true was
arbitrary. Universalizing orderings of values within a type allows for the
ordering operators to be polymorphic, which is quite useful, although
it does lead to these arbitrary decisions.

Solution to Exercise 10 Only the third of these typings holds, as shown

by the R E P L .

1. # (3 + 5 : float) ;;
Line 1, characters 1-6:
1 | (3 + 5 : float) ;;
^^^^^
Error: This expression has type int but an expression was expected
of type
float

2. # (3. + 5. : float) ;;
Line 1, characters 1-3:
1 | (3. + 5. : float) ;;
^^
Error: This expression has type float but an expression was
expected of type
int

3. # (3. +. 5. : float) ;;
- : float = 8.

4. # (3 : bool) ;;
Line 1, characters 1-2:
1 | (3 : bool) ;;
^
Error: This expression has type int but an expression was expected
of type
bool
444 PROGRAMMING WELL

5. # (3 || 5 : bool) ;;
Line 1, characters 1-2:
1 | (3 || 5 : bool) ;;
^
Error: This expression has type int but an expression was expected
of type
bool

6. # (3 || 5 : int) ;;
Line 1, characters 1-2:
1 | (3 || 5 : int) ;;
^
Error: This expression has type int but an expression was expected
of type
bool

Solution to Exercise 11 Since the unit type has only one value, there
is only one such typing:

() : unit

Solution to Exercise 12 No good comes of applying a function of type

float -> float to an argument of type bool.

# sqrt true ;;
Line 1, characters 5-9:
1 | sqrt true ;;
^^^^
Error: This expression has type bool but an expression was expected
of type
float

Solution to Exercise 13 As it turns out, the let construct itself has

low precedence so that the body of the let extends as far as it can.
Evaluating the expression without the parentheses demonstrates this,
as otherwise it would have generated an unbound variable error for the
second radius.

# 3.1416 *. let radius = 2.

# in radius *. radius ;;
- : float = 12.5664

Nonetheless, the parentheses arguably improve readability, and they

can help autoindenters that implement a less nuanced view of OCaml
syntax.

Solution to Exercise 14 The most direct approach uses two let bind-
ing for the two sides:

# let side1 = 1.88496 in

# let side2 = 2.51328 in
# sqrt (side1 *. side1 +. side2 *. side2) ;;
- : float = 3.1416
 SOLUTIONS TO SELECTED EXERCISES 445

However, by taking advantages of pattern-matching over pairs, which

will be introduced later in Section 7.2, a single let that binds both
variables using pattern matching is arguably more elegant:

# let side1, side2 = 1.88496, 2.51328 in

# sqrt (side1 *. side1 +. side2 *. side2) ;;
- : float = 3.1416

Solution to Exercise 15 Simply dropping the parentheses solves the

problem, since let has relatively low precedence, as described in
Exercise 13.

# let s = "hi ho " in

# s ^ s ^ s ;;
- : string = "hi ho hi ho hi ho "

Solution to Exercise 16 As shown in the solution to Exercise 15, the

REPL infers the type string for s.

Solution to Exercise 17

1. let x = 3 in
let y = 4 in
y * y ;;

2. let x = 3 in
let y = x + 2 in
y * y ;;

3. let x = 3 in
let y = 4 + (let z = 5 in z) + x in
y * y ;;

Solution to Exercise 18 The value for price at the end is 5. Surprise!

# let tax_rate = 0.05 ;;

val tax_rate : float = 0.05
# let price = 5. ;;
val price : float = 5.
# let price = price * (1. +. tax_rate) ;;
Line 1, characters 12-17:
1 | let price = price * (1. +. tax_rate) ;;
^^^^^
Error: This expression has type float but an expression was
expected of type
int
# price ;;
- : float = 5.

What was probably intended was

# let tax_rate = 0.05 ;;

val tax_rate : float = 0.05
# let price = 5. ;;
446 PROGRAMMING WELL

val price : float = 5.

# let price = price *. (1. +. tax_rate) ;;
val price : float = 5.25
# price ;;
- : float = 5.25

with a final value of price of 5.25. Thank goodness for strong static
typing, so that the R E P L was able to warn us of the error, rather than,
for instance, silently rounding the result or some such problematic
“correction” of the code.

Solution to Exercise 19 You can get the effect of this definition of

a global variable area by making use of local variables for pi and
radius by making sure to define the local variables within the global
definition:

# let area =
# let radius = 4. in
# let pi = 3.1416 in
# pi *. radius ** 2. ;;
val area : float = 50.2656

This way, the global let is at the top level.

Solution to Exercise 20

1. 2 : int

2. 2 : int

3. This sequence of tokens doesn’t parse, as - is a binary infix opera-

tor.

4. "OCaml" : string

5. "OCaml" : string

6. The expression evaluates to a function (unnamed) of type string

-> string.

7. Again, the expression evaluates to a function of type float ->

float -> float (the exponentiation function).

Solution to Exercise 21 A function that squares its floating point

argument is

# fun x -> x *. x ;;
- : float -> float = <fun>

and one to repeat a string is

# fun s -> s ^ s ;;
- : string -> string = <fun>
 SOLUTIONS TO SELECTED EXERCISES 447

Solution to Exercise 22

1. let foo (b : bool) (n : int) : bool = ...

2. let foo (b : bool) : (int -> bool) = ...

3. let foo (f : float -> int) (x : float) : bool = ...

Solution to Exercise 23 Typing them into the R E P L reveals their types:

1. # let greet y = "Hello" ^ y in greet "World!" ;;
- : string = "HelloWorld!"

2. # fun x -> let exp = 3. in x ** exp ;;

- : float -> float = <fun>

Solution to Exercise 24
# let square (x : float) : float =
# x *. x ;;
val square : float -> float = <fun>

Solution to Exercise 25
# let abs (n : int) : int =
# if n > 0 then n else ~- n ;;
val abs : int -> int = <fun>

Solution to Exercise 26 The type for string_of_bool is bool ->

string. It can be defined as

# let string_of_bool (condition : bool) : string =

# if condition then "true" else "false" ;;
val string_of_bool : bool -> string = <fun>

A common stylistic mistake (discussed in Section B.5.2) is to write the

test as if condition = true then..., but there’s no need for the
comparison. What goes in the test part of a conditional is a boolean,
and condition is already one.

Solution to Exercise 27 Using the compact notation:

# let even (n : int) : bool =
# n mod 2 = 0 ;;
val even : int -> bool = <fun>

(Did you try

# let even (n : int) : bool =
# if n mod 2 = 0 then true else false ;;
val even : int -> bool = <fun>

instead? That works, but the conditional is actually redundant. Re-

member, boolean expressions aren’t limited to use in the test part of
conditionals. Such extraneous conditionals are considered poor style.)
Using the explicit, desugared notation:
448 PROGRAMMING WELL

# let even : int -> bool =

# fun n -> n mod 2 = 0 ;;
val even : int -> bool = <fun>

Dropping the typing information, the R E P L still infers the correct

type.
# let even =
# fun n -> n mod 2 = 0 ;;
val even : int -> bool = <fun>

Nonetheless, the edict of intention argues for retaining the explicit

typing information.

Solution to Exercise 31 There are many possibilities. Here are some I

find especially nice.
1. let rec odd_terminate (n : int) : int =
if n < 0 then odd_terminate (~- n)
else if n = 1 then 0
else odd_terminate (n - 2) ;;

2. let rec small_terminate (n : int) : int =

if n = 5 then 0
else small_terminate (n + 1) ;;

3. let rec zero_terminate (n : int) : int =

if n = 0 then 0
else zero_terminate (n * 2) ;;

4. let rec true_terminate (b : bool) : bool =

b || (true_terminate b) ;;

Solution to Exercise 32 The most straightforward recursive solution is

simply
# let rec fib (i : int) : int =
# if i = 1 then 0
# else if i = 2 then 1
# else fib (i - 1) + fib (i - 2) ;;
val fib : int -> int = <fun>

Foreshadowing the discussion of error handling in Chapter 10, the

following definition verifies an assumption on the argument, before
calculating the number recursively.
# let rec fib (i : int) : int =
# assert (i >= 1);
# if i = 1 then 0
# else if i = 2 then 1
# else fib (i - 1) + fib (i - 2) ;;
val fib : int -> int = <fun>

As an alternative for the three way condition, a match statement might

be clearer:
 SOLUTIONS TO SELECTED EXERCISES 449

# let rec fib (i : int) : int =

# match i with
# | 1 -> 0
# | 2 -> 1
# | _ -> assert (i >= 1);
# fib (i - 1) + fib (i - 2) ;;
val fib : int -> int = <fun>

Solution to Exercise 33

# let fewer_divisors (n : int) (bound : int) : bool =

# let rec divisors_from (start : int) : int =
# if start > n / 2 then 1
# else divisors_from (start + 1)
# + (if n mod start = 0 then 1 else 0) in
# bound > divisors_from 1 ;;
val fewer_divisors : int -> int -> bool = <fun>

Solution to Exercise 34

1. bool * int

2. bool * bool

3. int * int

4. float * int

5. float * int

6. int * int

7. (int -> int) * (int -> int)

Solution to Exercise 35

# true, true ;;
- : bool * bool = (true, true)
# true, 42, 3.14 ;;
- : bool * int * float = (true, 42, 3.14)
# (true, 42), 3.14 ;;
- : (bool * int) * float = ((true, 42), 3.14)
# (1, 2), 3, 4 ;;
- : (int * int) * int * int = ((1, 2), 3, 4)
# succ, 0, 42 ;;
- : (int -> int) * int * int = (<fun>, 0, 42)
# fun (f, n) -> 1 + f (1 + n) ;;
- : (int -> int) * int -> int = <fun>

Solution to Exercise 36

# let div_mod (x : int) (y : int) : int * int =

# x / y, x mod y ;;
val div_mod : int -> int -> int * int = <fun>
450 PROGRAMMING WELL

Solution to Exercise 38
# let snd (pair : int * int) : int =
# match pair with
# | _x, y -> y ;;
val snd : int * int -> int = <fun>

Solution to Exercise 39
# let addpair (x, y : int * int) : int =
# x + y ;;
val addpair : int * int -> int = <fun>

# let fst (x, _y : int * int) : int = x ;;

val fst : int * int -> int = <fun>

Solution to Exercise 41
1. # 3 :: [] ;;
- : int list = [3]

2. # true :: false ;;
Line 1, characters 8-13:
1 | true :: false ;;
^^^^^
Error: This variant expression is expected to have type bool list
The constructor false does not belong to type list

3. # true :: [false] ;;
- : bool list = [true; false]

4. # [true] :: [false] ;;
Line 1, characters 11-16:
1 | [true] :: [false] ;;
^^^^^
Error: This variant expression is expected to have type bool list
The constructor false does not belong to type list

5. # [1; 2; 3.1416] ;;
Line 1, characters 7-13:
1 | [1; 2; 3.1416] ;;
^^^^^^
Error: This expression has type float but an expression was
expected of type
int

6. # [4; 2; -1; 1_000_000] ;;

- : int list = [4; 2; -1; 1000000]

7. # ([true], false) ;;
- : bool list * bool = ([true], false)

Solution to Exercise 42 The length function is of type int list ->

int; it expects an int list argument. However, we’ve applied it to
an argument of type int list list, that is, a list of integer lists. The
types are inconsistent, and OCaml reports the type mismatch.
 SOLUTIONS TO SELECTED EXERCISES 451

Solution to Exercise 43

# let rec sum (lst : int list) : int =

# match lst with
# | [] -> 0
# | hd :: tl -> hd + sum tl ;;
val sum : int list -> int = <fun>

It’s natural to return the additive identity 0 for the empty list to simplify
the recursion.
This function can also be implemented using the techniques of
Chapter 8 as a single fold.

Solution to Exercise 44

# let rec prod (lst : int list) : int =

# match lst with
# | [] -> 1
# | hd :: tl -> hd * prod tl ;;
val prod : int list -> int = <fun>

It’s natural to return the multiplicative identity 1 for the empty list to
simplify the recursion.
This function can also be implemented using the techniques of
Chapter 8 as a single fold.

Solution to Exercise 45

# let rec sums (lst : (int * int) list) : int list =

# match lst with
# | [] -> []
# | (x, y) :: tl -> (x + y) :: sums tl ;;
val sums : (int * int) list -> int list = <fun>

Solution to Exercise 46

# let rec inc_all lst =

# match lst with
# | [] -> []
# | hd :: tl -> (succ hd) :: inc_all tl ;;
val inc_all : int list -> int list = <fun>

Solution to Exercise 47

# let rec square_all lst =

# match lst with
# | [] -> []
# | hd :: tl -> (hd * hd) :: square_all tl ;;
val square_all : int list -> int list = <fun>

Solution to Exercise 48

# let rec append (x : int list) (y : int list)

# : int list =
452 PROGRAMMING WELL

# match x with
# | [] -> y
# | hd :: tl -> hd :: (append tl y) ;;
val append : int list -> int list -> int list = <fun>

Solution to Exercise 49

# let rec concat (sep : string) (lst : string list)

# : string =
# match lst with
# | [] -> ""
# | [hd] -> hd
# | hd :: tl -> hd ^ sep ^ (concat sep tl) ;;
val concat : string -> string list -> string = <fun>

Solution to Exercise 50

# let tesseract = power 4 ;;

val tesseract : int -> int = <fun>

If your definition was longer, you’ll want to review the partial applica-
tion discussion.

Solution to Exercise 51

# let double_all = map (( * ) 2) ;;

val double_all : int list -> int list = <fun>

Solution to Exercise 52

# let rec fold_left (f : int -> int -> int)

# (init : int)
# (xs : int list)
# : int =
# match xs with
# | [] -> init
# | hd :: tl -> fold_left f (f init hd) tl ;;
val fold_left : (int -> int -> int) -> int -> int list -> int =
<fun>

Solution to Exercise 53 The definition analogous to the one using

fold_right is

# let length lst = fold_left (fun tlval _hd -> 1 + tlval) 0 lst
# ;;
val length : int list -> int = <fun>

but again this can be further simplified by partial application:

# let length = fold_left (fun tlval _hd -> 1 + tlval) 0 ;;

val length : int list -> int = <fun>

Solution to Exercise 54 A simple solution is to use fold_left itself to

implement reduce:
 SOLUTIONS TO SELECTED EXERCISES 453

# let reduce (f : int -> int -> int) (list : int list) : int =
# match list with
# | hd :: tl -> List.fold_left f hd tl ;;
Lines 2-3, characters 0-36:
2 | match list with
3 | | hd :: tl -> List.fold_left f hd tl...
Warning 8: this pattern-matching is not exhaustive.
Here is an example of a case that is not matched:
[]
val reduce : (int -> int -> int) -> int list -> int = <fun>

This approach has the disadvantage that applying reduce to the empty
list yields an unintuitive “Match failure” error message. Looking ahead
to Section 10.3 on handling such errors explicitly, we can raise a more
appropriate exception, the Invalid_argument exception.

# let reduce (f : int -> int -> int) (list : int list) : int =
# match list with
# | hd :: tl -> List.fold_left f hd tl
# | [] -> raise (Invalid_argument "reduce: empty list") ;;
val reduce : (int -> int -> int) -> int list -> int = <fun>

Solution to Exercise 55 The filter function can be implemented

directly as a recursive function:

# let rec filter (test : int -> bool)

# (lst : int list)
# : int list =
# match lst with
# | [] -> []
# | hd :: tl -> if test hd then hd :: filter test tl
# else filter test tl ;;
val filter : (int -> bool) -> int list -> int list = <fun>

Looking ahead to the next chapter, it can be implemented using poly-

morphic fold_right (from the List module):

# let filter (test : int -> bool)

# : int list -> int list =
# List.fold_right (fun elt accum ->
# if test elt then elt :: accum
# else accum)
# [] ;;
val filter : (int -> bool) -> int list -> int list = <fun>

(You may want to revisit this latter solution after reading Chapter 9.)

Solution to Exercise 56 A first stab, maximizing partial application:

# let evens = filter (fun n -> n mod 2 = 0) ;;

val evens : int list -> int list = <fun>
# let odds = filter (fun n -> n mod 2 <> 0) ;;
val odds : int list -> int list = <fun>
# let positives = filter ((<) 0) ;;
454 PROGRAMMING WELL

val positives : int list -> int list = <fun>

# let negatives = filter ((>) 0) ;;
val negatives : int list -> int list = <fun>

The last two may be a bit confusing: Why ((<) 0) for the positives?
Don’t we want to accept only those that are greater than 0? The < func-
tion is curried with its prefix argument before its postfix argument, so
that the function ((<) 0) is equivalent to fun x -> 0 < x, that is,
the function that returns true for positive integers. Nonetheless, the
expression ((<) 0) doesn’t “read” that way, which is a good argument
for not being so cute and using instead the slightly more verbose but
transparent
# let positives = filter (fun n -> n > 0) ;;
val positives : int list -> int list = <fun>
# let negatives = filter (fun n -> n < 0) ;;
val negatives : int list -> int list = <fun>

Clarity trumps compactness.

Solution to Exercise 57 A list can be reversed by repeatedly append-

ing elements at the end of the accumulating reversal. A fold_right
implements this solution.
# let reverse (lst : int list) : int list =
# List.fold_right (fun elt accum -> accum @ [elt])
# lst [] ;;
val reverse : int list -> int list = <fun>

Alternatively, we can start at the left.

# let reverse (lst : int list) : int list =
# List.fold_left (fun accum elt -> elt :: accum)
# [] lst ;;
val reverse : int list -> int list = <fun>

Taking advantage of partial application, we have

# let reverse : int list -> int list =
# List.fold_left (fun accum elt -> elt :: accum)
# [] ;;
val reverse : int list -> int list = <fun>

# reverse [1; 2; 3] ;;
- : int list = [3; 2; 1]

Solution to Exercise 58
# let append (xs : int list) (ys : int list) : int list =
# List.fold_right cons xs ys ;;
Line 2, characters 16-20:
2 | List.fold_right cons xs ys ;;
^^^^
Error: Unbound value cons
Hint: Did you mean cos?
 SOLUTIONS TO SELECTED EXERCISES 455

Solution to Exercise 59 Here are some possible solutions, with com-

mentary on how to think through the problems.

1. You were asked to construct an expression that bears a particular

type as inferred by OCaml. The constraint that there be “no explicit
typing annotations” was intended to prevent trivial solutions such
as this:

# let (f : bool * bool -> bool) =

# fun _ -> true in
# f ;;
- : bool * bool -> bool = <fun>

or even

# ((fun _ -> failwith "") : bool * bool -> bool) ;;

- : bool * bool -> bool = <fun>

where the explicit type annotation does the work. The structure of
the code does little (respectively, nothing) to manifest the requested
type.
A simple solution relies on the insight that the required type is just
the uncurried version of the type for the (&&) operator.

# let f (x, y) =
# x && y in
# f ;;
- : bool * bool -> bool = <fun>

A typical approach to this problem is to use a top-level let defini-

tion of a function, such as this:

# let f (x, y) = x && y ;;

val f : bool * bool -> bool = <fun>

Strictly speaking, this is not an expression of OCaml that returns a

value, but rather a top-level command that names a value, though
the value is of the appropriate type. The value itself can be con-
structed as a self-contained expression either by using a local
let...in (as above) or an anonymous function:

# fun (x, y) -> x && y ;;

- : bool * bool -> bool = <fun>

2. In these problems that ask for a function of a given type, it makes

sense to start by building the first line of a let definition of the func-
tion with its arguments: let f x = ... and then figure out how
to force x and the result to be of the right types. Here, x should be
an ’a list, so we better not operate nontrivially on any of its el-
ements. Let’s match against the list as would typically happen in a
recursive function. This provides the skeleton of the code:
456 PROGRAMMING WELL

let f xs =
match xs with
| [] -> ...
| h :: t -> ... in
f ;;

Now, we need to make sure the result type is bool list, taking care
not to further instantiate ’a. We can insert any values of the right
type as return values, but to continue the verisimilitude, we use the
empty list for the first case and a recursive call for the second. (Note
the added rec to allow the recursive call.)

# let rec f xs =
# match xs with
# | [] -> []
# | _h :: t -> true :: (f t) in
# f ;;
- : 'a list -> bool list = <fun>

Of course, no recursion is really necessary. For instance, even some-

thing as simple as the following does the job.

# fun [] -> [true] ;;

Line 1, characters 0-16:
1 | fun [] -> [true] ;;
^^^^^^^^^^^^^^^^
Warning 8: this pattern-matching is not exhaustive.
Here is an example of a case that is not matched:
_::_
- : 'a list -> bool list = <fun>

3. A natural approach is to apply the first argument (a function) to a

pair composed of the second and third arguments, thereby enforc-
ing that the first argument is of type ’a * ’b -> ..., viz.,

# let f g a b =
# g (a, b) in
# f ;;
- : ('a * 'b -> 'c) -> 'a -> 'b -> 'c = <fun>

but this by itself does not guarantee that the result type of the func-
tion is ’a. Rather, f types as (’a * ’b -> ’c) -> ’a -> ’b ->
’c. (It’s the curry function from lab!) We can fix that by, say, com-
paring the result with a known value of the right type, namely a.

# let f g a b =
# if g (a, b) = a then a else a in
# f ;;
- : ('a * 'b -> 'a) -> 'a -> 'b -> 'a = <fun>

4. Again, we start with a let definition that just lays out the types of
the arguments in a pattern, and then make sure that each compo-
nent has the right type. One of many possibilities is
 SOLUTIONS TO SELECTED EXERCISES 457

# let f (i, a, b) alst =

# if i = 0 && (List.hd alst) = a then [b] else []
# in f ;;
- : int * 'a * 'b -> 'a list -> 'b list = <fun>

5. We force the argument to be a bool by placing it in the test part of a

conditional, and return the only value that we can.

# fun b -> if b then () else () ;;

- : bool -> unit = <fun>

6. We want to construct a polymorphic, higher-order function that

takes arguments of type ’a and ’a -> ’b; let’s call this function
f. Notice that the argument of type ’a -> ’b is also a function;
let’s call this argument function g. Conveniently, the input to the
argument function g is of the same type as the first input to the
higher-order function f, that is, of type ’a. Analogously, the output
of the argument function g is of the same type as the output of the
higher-order function f, that is, of type ’b. We can thus simply
apply g to the first argument of f and return the result:

# let f x g = g x ;;
val f : 'a -> ('a -> 'b) -> 'b = <fun>

The function f is the reverse application function!

# ( |> ) ;;
- : 'a -> ('a -> 'b) -> 'b = <fun>

7. This question is deceptively simple. The trick here is that the func-
tion is polymorphic in both its inputs and outputs, yet the argu-
ments and return type may be different. In fact, we circumvent this
issue by simply not returning a value at all. There are two ways to
approach this:

(a) Raise an exception (to be introduced in Section 10.3) instead of

returning:
# let f x y =
# if x = y then failwith "true" else failwith "false" ;;
val f : 'a -> 'a -> 'b = <fun>

(b) Recur indefinitely to prevent a return:

# let rec f x y =
# if x = y then f x y else f x y ;;
val f : 'a -> 'a -> 'b = <fun>

or even more elegantly:

# let rec f x y = f y x ;;
val f : 'a -> 'a -> 'b = <fun>
458 PROGRAMMING WELL

Solution to Exercise 60 All that needs to be changed from the

monomorphic version in the preceding chapter is the typing infor-
mation in the header. The definition itself naturally works polymorphi-
cally.

# let rec fold (f : 'a -> 'b -> 'b)

# (xs : 'a list)
# (init : 'b)
# : 'b =
# match xs with
# | [] -> init
# | hd :: tl -> f hd (fold f tl init) ;;
val fold : ('a -> 'b -> 'b) -> 'a list -> 'b -> 'b = <fun>

# let rec filter (test : 'a -> bool)

# (lst : 'a list)
# : 'a list =
# match lst with
# | [] -> []
# | hd :: tl -> if test hd then hd :: filter test tl
# else filter test tl ;;
val filter : ('a -> bool) -> 'a list -> 'a list = <fun>

Solution to Exercise 61 To implement map f lst with fold, we can

start with the empty list and at each step cons on f applied to each
element of the lst:

# let map (f : 'a -> 'b) (lst : 'a list) : 'b list =
# fold (fun elt accum -> (f elt) :: accum)
# lst [] ;;
val map : ('a -> 'b) -> 'a list -> 'b list = <fun>

A polymorphic version of this solution is found in Problem 63.

Solution to Exercise 62 1. Since x is an argument of a float operator,

it is of type float. The result is also of type float. Thus f is of
function type float -> float, as can be easily verified in the
R E P L:

# let f x =
# x +. 42. ;;
val f : float -> float = <fun>

2. The function f is clearly of a function type taking two (curried)

arguments, that is, of type ... -> ... -> .... The argument g is
also a function, apparently from integers to some result type ’a, so f
is of type (int -> ’a) -> int -> ’a.

# let f g x =
# g (x + 1) ;;
val f : (int -> 'a) -> int -> 'a = <fun>
 SOLUTIONS TO SELECTED EXERCISES 459

3. The argument type for f, that is, the type of x, must be a list, say, ’a
list. The result type can be gleaned from the two possible return
values x and h. Since h is an element of x, it must be of type ’a.
Thus the return type is both ’a and ’a list. But there is no type
that matches both. Thus, the expression does not type.

# let f x =
# match x with
# | [] -> x
# | h :: t -> h ;;
Line 4, characters 12-13:
4 | | h :: t -> h ;;
^
Error: This expression has type 'a but an expression was expected
of type
'a list
The type variable 'a occurs inside 'a list

4. The result type for f must be the same as the type of a since it re-
turns a in one of the match branches. Since x is matched as a list, it
must be of list type. So far, then, we have f of type ... list -> ’a
-> ’a. The elements of x (such as h) are apparently functions, as
shown in the second match branch where h is applied to something
of type ’a and returning also an ’a; so h is of type’a -> ’a. The
final typing is f : (’a -> ’a) list -> ’a -> ’a.

# let rec f x a =
# match x with
# | [] -> a
# | h :: t -> h (f t a) ;;
val f : ('a -> 'a) list -> 'a -> 'a = <fun>

5. The match tells us that the first argument x is a pair, whose element
w is used as a bool; we’ll take the type of the element z to be ’a. The
second argument y is applied to z (of type ’a) and returns a bool
(since the then and else branches of the conditional tell us that y z
and w are of the same type). Thus the type of f is given by the typing
f : bool * ’a -> (’a -> bool) -> bool.

let f x y =
match x with
| (w, z) -> if w then y z else w ;;

6. We can see that we apply y to x twice. There’s nothing else in this

function that would indicate a specific typing, so we know our
function is polymorphic. Let’s say the type of y is ’a. We know that
since we can apply x to two arguments of type ’a, and there are no
constraints on the output type of x, x must be of type ’a -> ’a ->
’b. Since f returns x y y, we know the output type of f must be the
460 PROGRAMMING WELL

same as the output type of x. The final typing is thus f : (’a ->
’a -> ’b) -> ’a -> ’b.

# let f x y =
# x y y ;;
val f : ('a -> 'a -> 'b) -> 'a -> 'b = <fun>

7. This definition does not type. y is here applied as a function, so its

type must be of the form ’a -> ’b. Yet the function y can take y as
an argument. This implies that ’a, the type of the input to y, must
be identical to ’a -> ’b, the type of y itself. There is no finite type
satisfying that constraint. A type cannot be a subpart of itself.

# let f x y =
# x (y y) ;;
Line 2, characters 5-6:
2 | x (y y) ;;
^
Error: This expression has type 'a -> 'b
but an expression was expected of type 'a
The type variable 'a occurs inside 'a -> 'b

8. The code matches x with option types formed with Some or None, so
we know that x must be of type ’a option for some ’a. We also see
that when deconstructing x into Some y, we perform subtraction
on y in the recursive function call: f (Some (y - 1)). We can thus
conclude y is of type int, and can further specify x to be of type
int option. Finally, note that the case None | Some 0 -> None
is the sole terminal case in this recursive function. Because this
case returns None, we know that if f terminates, f returns None. Our
function f therefore outputs a value of type ’a option. We cannot
infer a more specific type for ’a because we always return None and
thus have no constraints on ’a. The final typing is thus as follows: f
: int option -> ’a option.

# let rec f x =
# match x with
# | None
# | Some 0 -> None
# | Some y -> f (Some (y - 1)) ;;
val f : int option -> 'a option = <fun>

9. Since x is in the condition of an if statement (if x then ...), we

know that x must be of type bool. We also can see that both return
paths of the code return a list; these lists contain x, so we know f
returns a bool list. Since y appears in the list [not x; y], we
therefore know y must be of type bool as well. This gives us the
overall typing of f : bool -> bool -> bool list.
 SOLUTIONS TO SELECTED EXERCISES 461

# let f x y =
# if x then [x]
# else [not x; y] ;;
val f : bool -> bool -> bool list = <fun>

Solution to Exercise 63 Here are two solutions, implemented using

fold_left and fold_right, respectively.

let map (f : 'a -> 'b) (lst : 'a list) : 'b list =
List.fold_right (fun elt accum -> f elt :: accum)
lst [] ;;

let map (f : 'a -> 'b) (lst : 'a list) : 'b list =
List.fold_left (fun accum elt -> accum @ [f elt])
[] lst ;;

The latter can be simplified through use of partial application to

let map (f : 'a -> 'b) : 'a list -> 'b list =
List.fold_left (fun accum elt -> accum @ [f elt]) [] ;;

Solution to Exercise 64 An implementation of fold_right solely in

terms of a single call to map over the same list is not possible. The type
of fold_right makes clear that the output may be of any type. But map
always returns a list. So a single call to map cannot generate the range
of answers that fold_right can.
One can use map in an implementation of fold_right in a trivial
way, for instance, by vacuously mapping the identity function over
the list argument of fold_right before doing the real work, but that
misses the point of the question, which asks that the implementation
use only a call to List.map.

Solution to Exercise 65 We approach this problem similarly to how we

implemented filter. The distinction here is that the base case returns
two empty lists rather than one, so we have to deconstruct the tuple
created by the recursive function call. This results in two output lists –
the yeses and the nos – so we simply pass the current element into the
test function and append to the appropriate output list according to
the result.

# let rec partition (test : 'a -> bool)

# (lst : 'a list)
# : 'a list * 'a list =
# match lst with
# | [] -> [], []
# | hd :: tl ->
# let yeses, nos = partition test tl in
# if test hd then hd :: yeses, nos
# else yeses, hd :: nos ;;
val partition : ('a -> bool) -> 'a list -> 'a list * 'a list =
<fun>
462 PROGRAMMING WELL

Solution to Exercise 66
# let rec interleave (n : 'a) (lst : 'a list)
# : 'a list list =
# match lst with
# | [] -> [[n]]
# | x :: xs -> (n :: x :: xs)
# :: List.map (fun l -> x :: l)
# (interleave n xs) ;;
val interleave : 'a -> 'a list -> 'a list list = <fun>

# let rec permutations (lst : 'a list) : 'a list list =

# match lst with
# | [] -> [[]]
# | x :: xs -> List.concat (List.map (interleave x)
# (permutations xs)) ;;
val permutations : 'a list -> 'a list list = <fun>

Solution to Exercise 67 We start by providing implementations for sum

and prods making use of the higher-order polymorphic list processing
functions of the List module.
# let sum = List.fold_left (+) 0 ;;
val sum : int list -> int = <fun>
# let prods = List.map (fun (x, y) -> x * y) ;;
val prods : (int * int) list -> int list = <fun>

The composition function @+ is simply

# let (@+) f g x = f (g x) ;;
val ( @+ ) : ('a -> 'b) -> ('c -> 'a) -> 'c -> 'b = <fun>

We can use it to implement the weighted_sum function and test it out:

# let weighted_sum = sum @+ prods ;;
val weighted_sum : (int * int) list -> int = <fun>
# weighted_sum [(1, 3); (2, 4); (3, 5)] ;;
- : int = 26

Solution to Exercise 68 The R E P L response in the solution to Exer-

cise 67 reveals the polymorphic type of compose as (’a -> ’b) ->
(’c -> ’a) -> ’c ->’b, or equivalently but more intuitively, (’b ->
’c) -> (’a -> ’b) -> (’a -> ’c).

Solution to Exercise 69
# let rec odds (lst : 'a list) : 'a list =
# match lst with
# | [] -> []
# | [a] -> [a]
# | a :: _b :: rest -> a :: odds rest ;;
val odds : 'a list -> 'a list = <fun>

Solution to Exercise 70 The odds function is typed as odds : ’a

list -> ’a list. Note the polymorphic type.
 SOLUTIONS TO SELECTED EXERCISES 463

Solution to Exercise 71 The recursive definition analogous to that

used for odds is
# let rec evens (nums : 'a list) : 'a list =
# match nums with
# | [] -> []
# | [_a] -> []
# | _a :: b :: rest -> b :: evens rest ;;
val evens : 'a list -> 'a list = <fun>

Alternatively, evens can be written in terms of odds.

# let evens (nums : 'a list) : 'a list =
# match nums with
# | [] -> []
# | _hd :: tl -> odds tl ;;
val evens : 'a list -> 'a list = <fun>

Solution to Exercise 72 Taking advantage of partial application:

# let sum =
# List.fold_left (+) 0 ;;
val sum : int list -> int = <fun>

Solution to Exercise 73
# let luhn (nums : int list) : int =
# let s = sum ((List.map doublemod9 (odds nums))
# @ (evens nums)) in
# 10 - (s mod 10) ;;
val luhn : int list -> int = <fun>

Solution to Exercise 74 No and no.

Solution to Exercise 77 We can first check for the additional anoma-

lous condition.
# let rec nth_opt (lst : 'a list) (n : int) : 'a option =
# if n < 0 then None
# else
# match lst with
# | [] -> None
# | hd :: tl ->
# if n = 0 then Some hd
# else nth_opt tl (n - 1) ;;
val nth_opt : 'a list -> int -> 'a option = <fun>

Alternatively, the check could have been done inside the second match
statement. Why might this be the dispreferred choice?

Solution to Exercise 78
# let rec last_opt (lst : 'a list) : 'a option =
# match lst with
# | [] -> None
# | [elt] -> Some elt
# | _ :: tl -> last_opt tl ;;
val last_opt : 'a list -> 'a option = <fun>
464 PROGRAMMING WELL

Solution to Exercise 79 Here’s a solution that peforms all list calcula-

tions directly, making no use of the List library. Can you simplify this
using the List library?

# let variance (lst : float list) : float option =

# let rec sum_length (lst : float list) : float * int =
# match lst with
# | [] -> (0., 0)
# | hd :: tl -> let sum, len = sum_length tl in
# hd +. sum, 1 + len in
# let sum, length = sum_length lst in
# if length < 2
# then None
# else let flength = float length in
# let mean = sum /. flength in
# let rec residuals (lst : float list) : float =
# match lst with
# | [] -> 0.
# | hd :: tl -> (hd -. mean) ** 2.
# +. residuals tl in
# Some (residuals lst /. (flength -. 1.)) ;;
val variance : float list -> float option = <fun>

Solution to Exercise 81 If the first two patterns are [], [] and _, _,

the third branch of the match can never be reached. The R E P L gives an
appropriate warning to that effect:

# let rec zip_opt' (xs : 'a list)

# (ys : 'b list)
# : ('a * 'b) list option =
# match xs, ys with
# | [], [] -> Some []
# | _, _ -> None
# | xhd :: xtl, yhd :: ytl ->
# match zip_opt' xtl ytl with
# | None -> None
# | Some ztl -> Some ((xhd, yhd) :: ztl) ;;
Line 7, characters 2-24:
7 | | xhd :: xtl, yhd :: ytl ->
^^^^^^^^^^^^^^^^^^^^^^
Warning 11: this match case is unused.
val zip_opt' : 'a list -> 'b list -> ('a * 'b) list option = <fun>

Solution to Exercise 82 The function can be implemented as:

# let zip_safe (x : 'a list)

# (y : 'b list)
# : ('a * 'b) list =
# try
# zip x y
# with
# Invalid_argument _msg -> [] ;;
val zip_safe : 'a list -> 'b list -> ('a * 'b) list = <fun>
 SOLUTIONS TO SELECTED EXERCISES 465

However, this approach to handling anomalous conditions in zip uses

in-band error signaling, which we’d always want to avoid; the error
value also happens to be the value returned by the non-error call zip
[] [].

Solution to Exercise 83

# let rec fact (n : int) : int =

# if n < 0 then
# raise (Invalid_argument "fact: arg less than zero")
# else if n = 0 then 1
# else n * fact (n - 1) ;;
val fact : int -> int = <fun>

Solution to Exercise 85

1. # let f x y =
# Some (x + y) ;;
val f : int -> int -> int option = <fun>

2. # let f g =
# Some (1 + g 3) ;;
val f : (int -> int) -> int option = <fun>

3. # let f x g = g x ;;
val f : 'a -> ('a -> 'b) -> 'b = <fun>

or

# let f = ( |> ) ;;
val f : 'a -> ('a -> 'b) -> 'b = <fun>

4. # let rec f xl yl =
# match xl, yl with
# | (Some xhd :: xtl), (Some yhd :: ytl)
# -> (xhd, yhd) :: f xtl ytl
# | (None :: _), _
# | _, (None :: _)
# | [], _
# | _, [] -> [] ;;
val f : 'a option List.t -> 'b option List.t -> ('a * 'b) List.t =
<fun>

Solution to Exercise 86

1. No type exists for f. Assume that the type of f is some instantiation

of the function type ’a -> ’b. Since the first match clause returns
f, the result type ’b of f must be the entire type ’a -> ’b of f itself.
But a type can’t contain itself as a subpart. So no type for f exists.

2. The type of f is bool -> bool * bool. In fact, f always returns the
same value, the pair true, true.
466 PROGRAMMING WELL

Solution to Exercise 87 Taking advantage of the fact that f always

returns the same value:

let f (b : bool) = true, true ;;

Note that the explicit typing of b is required to force the function type
to be bool -> bool * bool instead of ’a -> bool * bool.

Solution to Exercise 95 The R E P L provides the answer:

# ( |> ) ;;
- : 'a -> ('a -> 'b) -> 'b = <fun>

Solution to Exercise 96

# let ( |> ) arg func = func arg ;;

val ( |> ) : 'a -> ('a -> 'b) -> 'b = <fun>

Making the types explicit:

# let ( |> ) (arg : 'a) (func : 'a -> 'b) : 'b =

# func arg ;;
val ( |> ) : 'a -> ('a -> 'b) -> 'b = <fun>

Solution to Exercise 97 There are only six card types, so one might be
inclined to just have an enumerated type with six constructors:

type card =
| KSpades
| QSpades
| JSpades
| KDiamonds
| QDiamonds
| JDiamonds ;;

The inelegance of this approach should be clear.

The crucial point here is that the information be kept in a structured
form (as specified in the problem), clearly keeping separate informa-
tion about the suit and the value of a card. This calls for enumerated
types for suits and values.
The type for cards can integrate a suit and a value, either by pairing
them or putting them into a record. Here, we take the latter approach.

type suit = Spades | Diamonds ;;

type cardval = King | Queen | Jack ;;
type card = {suit : suit; cardval : cardval} ;;

Note that the field names and type names can be identical, since they
are in different namespaces.
Using ints for the suits and card values, for instance,

type card = {suit : int; cardval : int} ;;

 SOLUTIONS TO SELECTED EXERCISES 467

is inferior as the convention for mapping between int and card suit
or value is obscure. At best it could be made clear in documentation,
but the enumerated type makes it clear in the constructors themselves.
Further, the int approach allows ints that don’t participate in the
mapping, and thus doesn’t let the language help with catching errors.
We have carefully ordered the constructors from better to worse
and ordered the record components from higher to lower order so that
comparisons on the data values will accord with the “better” relation,
as seen in the solution to Problem 99.

Solution to Exercise 98 The better function is supposed to take two

cards and return a truth value, so if the arguments are taken curried,
then

better : card -> card -> bool

Alternatively, but less idiomatically, the function could be uncurried:

better : card * card -> bool

Solution to Exercise 99 The following oh-so-clever approach works

if you carefully order the constructors and fields from best to worst
and higher order (suit) before lower order (cardval), as is done in the
solution to Problem 97.

let better (card1 : card) (card2 : card) : bool =

card1 < card2 ;;

This relies on the fact that the < operator has a kind of ad hoc poly-
morphism, which works on enumerated and variant types, pairs, and
records inductively to define an ordering on values of those types.
Relying on this property of variant types behooves you to explicitly
document the fact at the type definition so it gets preserved.
To not rely on the ad hoc polymorphism of <, we need a more ex-
plicit definition like this:

let better ({suit = suit1; cardval = cardval1} : card)

({suit = suit2; cardval = cardval2} : card)
: bool =
let to_int v = match v with
| King -> 3
| Queen -> 2
| Jack -> 1 in
if suit1 = suit2 then
(to_int cardval1) > (to_int cardval2)
else suit1 = Spades ;;

though this is hacky since it doesn’t generalize well to adding more

suits. Of course, a separate map of suits to an int value would solve
that problem. Many other approaches are possible.
468 PROGRAMMING WELL

Solution to Exercise 102

# let str_bintree =
# Node ("red",
#
#
#
# Node ("orange", Node ("green", Node ("blue", Empty, Empty),
# Node ("indigo", Empty,
# Empty)),
# Empty),
# Node ("yellow", Empty,
# Node ("violet", Empty, Empty))) ;;
val str_bintree : string bintree =
Node ("red",
Node ("orange",
Node ("green", Node ("blue", Empty, Empty),
Node ("indigo", Empty, Empty)),
Empty),
Node ("yellow", Empty, Node ("violet", Empty, Empty)))

Solution to Exercise 103

# let rec preorder (t : 'a bintree) : 'a list =
# match t with
# | Empty -> []
# | Node (n, left, right) ->
# n :: preorder left @ preorder right ;;
val preorder : 'a bintree -> 'a list = <fun>

Solution to Exercise 104 Let’s start with the tree input and the output
of the tree traversal. The third argument to foldbt is a binary tree of
type ’a bintree, say. The result of the traversal is a value of type, say,
’b. Then the first argument, which serves as the return value for empty
trees must also be of type ’b and the function calculating the values for
internal nodes is given the value stored at the node (’a) and the two
recursively returned values and returns a ’b; it must be of type ’a ->
’b -> ’b -> ’b. Overall, the appropriate type for foldbt is
'b -> ('a -> 'b -> 'b -> 'b) -> 'a bintree -> 'b

Solution to Exercise 105 A directly recursive implementation looks

like
# let rec foldbt (emptyval : 'b)
# (nodefn : 'a -> 'b -> 'b -> 'b)
# (t : 'a bintree)
# : 'b =
# match t with
# | Empty -> emptyval
# | Node (value, left, right) ->
# nodefn value (foldbt emptyval nodefn left)
# (foldbt emptyval nodefn right) ;;
val foldbt : 'b -> ('a -> 'b -> 'b -> 'b) -> 'a bintree -> 'b =
<fun>
 SOLUTIONS TO SELECTED EXERCISES 469

Notice that each time foldbt is recursively called, it passes along

the same first two arguments.The following version of foldbt uses a
local function to avoid this redundancy.

# let foldbt (emptyval : 'b)

# (nodefn : 'a -> 'b -> 'b -> 'b)
# (t : 'a bintree)
# : 'b =
# let rec foldbt' t =
# match t with
# | Empty -> emptyval
# | Node (value, left, right) ->
# nodefn value (foldbt' left) (foldbt' right) in
# foldbt' t ;;
val foldbt : 'b -> ('a -> 'b -> 'b -> 'b) -> 'a bintree -> 'b =
<fun>

Here’s a third slightly less attractive alternative, which introduces a

level of function application indirection and doesn’t take advantage of
the lexical scoping.

# let rec foldbt (emptyval : 'b)

# (nodefn : 'a -> 'b -> 'b -> 'b)
# (t : 'a bintree)
# : 'b =
# let foldbt' = foldbt emptyval nodefn in
# match t with
# | Empty -> emptyval
# | Node (value, left, right) ->
# nodefn value (foldbt' left)
# (foldbt' right) ;;
val foldbt : 'b -> ('a -> 'b -> 'b -> 'b) -> 'a bintree -> 'b =
<fun>

At least it uses the partial application of foldbt in the definition of

foldbt’.

Solution to Exercise 106 By abstracting out the generic tree walking,

this and other functions can be succinctly implemented. The value of
the sum for an empty tree is 0, and the function to be applied to the
value at a node and the values of the subtrees should just sum up those
three values.

# let sum_bintree =
# foldbt 0 (fun v l r -> v + l + r) ;;
val sum_bintree : int bintree -> int = <fun>
# preorder int_bintree ;;
- : int list = [16; 93; 3; 42]

Solution to Exercise 107

# let preorder tree =

# foldbt [] (fun v l r -> v :: l @ r) tree ;;
470 PROGRAMMING WELL

val preorder : 'a bintree -> 'a list = <fun>

# preorder int_bintree ;;
- : int list = [16; 93; 3; 42]

Why not partially apply foldbt, as in the sum_bintree example?

Because of the problem with weak type variables noted in Section 9.7.

Solution to Exercise 108

# let find (tree : 'a bintree) (value : 'a) : bool =

# foldbt false
# (fun v l r -> (value = v) || l || r)
# tree ;;
val find : 'a bintree -> 'a -> bool = <fun>

You’ll want to avoid redundant locutions like (l = true) in the sec-

ond to last line. See Section B.5.2 in the style guide.

Solution to Exercise 111 What we were looking for here is the proper
definition of a functor named MakeImaging taking an argument, where
the functor and argument are appropriately signature-constrained.

module MakeImaging (P : PIXEL)

: (IMAGING with type pixel = P.t) =
struct
(* ... the implementation would go here ... *)
end ;;

Typical problems are to leave out the : PIXEL, the : IMAGING, or the
sharing constraint.

Solution to Exercise 112 Applying the functor to the IntPixel mod-

ule is simply

module IntImaging = MakeImaging(IntPixel) ;;

Optionally, signature specifications can be added, so long as appropri-

ate sharing constraints are provided.

Solution to Exercise 113 Here, a local open simplifies things.

let open IntImaging in

depict (const (to_pixel 5000) (100, 100)) ;;

Solution to Exercise 114

module MakeInterval (Point : COMPARABLE)

: (INTERVAL with type point = Point.t) =
struct
(* ... the implementation would go here ... *)
end ;;

Solution to Exercise 115

 SOLUTIONS TO SELECTED EXERCISES 471

module DiscreteTime : (COMPARABLE with type t = int) =

struct
type t = int
type order = Less | Equal | Greater
let compare x y = if x < y then Less
else if x = y then Equal
else Greater
end ;;

Solution to Exercise 116

module DiscreteTimeInterval =
MakeInterval (DiscreteTime) ;;

Solution to Exercise 117

let intersection i j =
if relation i j = Disjoint then None
else let (x, y), (x', y') = endpoints i, endpoints j in
Some (interval (max x x') (min y y')) ;;

Solution to Exercise 118 There are myriad solutions here. The idea is
just to establish a few intervals and then test that you can recover some
endpoints or relations. Here are a few possibilities:

open Absbook ;;
let test () =
let open DiscreteTimeInterval in
let i1 = interval 1 3 in
let i2 = interval 2 6 in
let i3 = interval 0 7 in
let i4 = interval 4 5 in
unit_test (relation i1 i4 = Disjoint) "disjoint\ n";
unit_test (relation i1 i2 = Overlaps) "overlaps\ n";
unit_test (relation i1 i3 = Contains) "contains\ n";
unit_test
(relation (union i1 i2) i4 = Contains) "unioncontains\ n";
let i23 = intersection i1 i2 in
un
it_test (let
Some e23 = i23 in endpoints e23 = (2, 3)) "intersection";;
print_endline "tests completed" ;;

Solution to Exercise 119 Since we only need the float functionality for
weight, a simple definition is best.

# type weight = float ;;

type weight = float

Solution to Exercise 120 Since we want all shapes to be one of three

distinct types – either a circle OR an oval OR a fin – we want to use a
disjunctive type here. Variant types get the job done.
472 PROGRAMMING WELL

# type shape =
# | Circle
# | Oval
# | Fin ;;
type shape = Circle | Oval | Fin

Solution to Exercise 121 Since we want each object to have two at-
tributes – a weight AND a shape – we want to use conjunction here. We
can construct a record type obj to represent objects. This allows us to
ensure each object has a weight and shape that are of the appropriate
type.
# type obj = { weight : weight; shape : shape } ;;
type obj = { weight : weight; shape : shape; }

Solution to Exercise 122 In the header of the functor, we want to

explicate the name of the functor and the type of the input module, as
well as any sharing constraint. We want to transform any module of
type BINTREEARG into a module of type BINTREE. We also need to add
sharing constraints so that the types for leaft and nodet in the output
module of type BINTREE are of the same type as the leaft and nodet
in the Element module.
module MakeBintree (Element : BINTREE_ARG)
: (BINTREE with
type leaft = Element.leaft and
type nodet = Element.nodet) =
struct
..... (* the implementation would go here *)
end ;;

Solution to Exercise 123 To define a Mobile with objs as leaves

and weights as nodes, we just need to pass in a module of type
BINTREEARG. This argument module will also have leaves of type obj
and nodes of type weight:
module Mobile = MakeBinTree (struct
type leaft = obj
type nodet = weight
end) ;;

An alternative is to explicitly define the argument values:

module MobileArg =
struct
type leaft = obj
type nodet = weight
end ;;

module Mobile = MakeBintree (MobileArg) ;;

If a module type is given to the argument module, however, there need

to be sharing constraints. So the following won’t work:
 SOLUTIONS TO SELECTED EXERCISES 473

module MobileArg : BINTREE_ARG =

struct
type leaft = obj
type nodet = weight
end ;;

module Mobile = MakeBintree (MobileArg) ;;

It should be

module MobileArg : (BINTREE_ARG with type leaft = obj

and type nodet = weight) =
struct
type leaft = obj
type nodet = weight
end ;;

module Mobile = MakeBintree (MobileArg) ;;

Solution to Exercise 124 The only aspects pertinent to the use of a

module are manifest in the signature. A user need not know how a
module of type BINTREE, say, makes a leaf; a user only needs to know
the signature of the make_leaf function in order to use it. A user in fact
cannot access the implementation details because we’ve constrained
the module to the BINTREE interface Similarly, a user need not know
how the functor MakeBintree works, as implementation details would
not be accessible to the user anyway. So long as a user knows the
functor’s signature, they know if they pass in any module following
the BINTREE_ARG signature, the functor will return a module follwing
the BINTREE signature.

Solution to Exercise 125 We construct the mobile using the make_-

leaf and make_node functions in the Mobile module.

let mobile1 =
let open Mobile in
make_node
1.0
(make_leaf {shape = Oval; weight = 9.0})
(make_node
1.0
(make_leaf {shape = Fin; weight = 3.5})
(make_leaf {shape = Fin; weight = 4.5})) ;;

Solution to Exercise 126 The size function takes in a binary tree

representing a mobile and returns the number of leaves in that tree.
The type is thus Mobile.tree -> int.

Solution to Exercise 127 Notice that we pass in mobile as an ar-

gument to size, only to just pass it in again as the last argument to
Mobile.walk; partial application allows us to simplify as follows:
474 PROGRAMMING WELL

let size =
Mobile.walk (fun _leaf -> 1)
(fun _node left_size right_size ->
left_size + right_size) ;;

Solution to Exercise 128 Let’s first think about the signature of

shape_count. We want shape_count to take in a value of type shape
and output an int, so its type is shape -> int, leading to a first line of
let shape_count (s : shape) : int = ...

We’re told we want to use the walk function here. Since the walk func-
tion does the hard work of traversing the Mobile.tree for us, we just
need to pass in the proper arguments to walk in order to construct
the function shape_count. The walk function is of type (leaft ->
’a) -> (nodet -> ’a -> ’a -> ’a) -> tree -> ’a and takes in
two functions, one specifying behavior for leaves and one for nodes.
If we can define these two functions, we can easily define shape. Let’s
start with the function that specifies how we want to count leaves; we
need a function of type leaf -> ’a. The shape_count of a single leaf
should be 1 if the leaf matches the desired shape s and 0 otherwise. We
can construct an anonymous function that achieves this functionality
as follows:
fun leaf -> if leaf.shape = s then 1 else 0

We now want to address the case of nodes. Nodes don’t have shapes
themselves, but rather connect to other subtrees that might. To find
the shape count of a node, we just need to add the shape counts of its
subtrees.
fun _node l r -> l + r ;;

Putting it all together, we get

let shape_count (s : shape) =
Mobile.walk
(fun leaf -> if leaf.shape = s then 1 else 0)
(fun _node left_count right_count ->
left_count + right_count) ;;

Solution to Exercise 129 No, this mobile is not balanced. To deter-

mine whether the mobile is balanced, we can just sum the total weight
on each node. The right subtree connects two submobiles of different
weights (3.5 and 4.5).

Solution to Exercise 130 Again, we can use the walk function here
to avoid traversing the tree directly. We will again need to come up
with two functions to pass into walk, one for the leaves and one for the
nodes. Let’s look at the base case, leaves. A leaf is always balanced, so
we just ned to return Some w, where w is the weight of the leaf.
 SOLUTIONS TO SELECTED EXERCISES 475

fun leaf -> Some leaf.weight

Now, let’s look at the nodes. We want a function of the form nodet ->
’a -> ’a -> ’a, where the first argument is the node itself and the
remaining two are the results of walk on the left subtree and walk on
the right subtree, respectively. We want to ensure our node is balanced:
this requires that the left and right subtrees are each balanced and are
of equal weight. If these conditions are met we want to return If the
subtrees aren’t balanced or are of unequal weight, we want to return
Some w, where w is the sum of the weights of the connector and its
subtrees. We return None otherwise.

fun node left right ->

match left, right with
| Some wt1, Some wt2 ->
if wt1 = wt2 then Some (node +. wt1 +. wt2)
else None
| _, _ -> None) ;;

Putting it all together and passing in our mobile as an argument, we

get:

let balanced (mobile : Mobile.tree) =

Mobile.walk (fun leaf -> Some leaf.weight)
(fun node left right ->
match left, right with
| Some wt1, Some wt2 ->
if wt1 = wt2 then
Some (node +. wt1 +. wt2)
else None
| _, _ -> None)
mobile ;;

Note the redundancy of passing in mobile. We can use partial applica-

tion and arrive at the following final solution:

let balanced =
Mobile.walk (fun leaf -> Some leaf.weight)
(fun node l r ->
match l, r with
| Some wt1, Some wt2 ->
if wt1 = wt2 then
Some (node +. wt1 +. wt2)
else None
| _, _ -> None) ;;

Solution to Exercise 136 Since the + operator is left-associative, the

concrete syntax 3 + 5 + 7 corresponds to the same abstract syntax as
(3 + 5) + 7. The derivation is structured accordingly. The alternate
derivation provided in the exercise corresponds to the evaluation of
the concrete expression 3 + (5 + 7).
476 PROGRAMMING WELL

Solution to Exercise 149 R fun : “A function expression of the form fun

x -> B evaluates to itself.”
R app : “To evaluate an application of the form P Q, first evaluate
P to a function value of the form fun x -> B and Q to a value vQ .
Then evaluate the expression resulting from substituting vQ for free
occurrences of x in B to a value v B . The value of the full expression is
then v B .”

Solution to Exercise 152

(let x = Q in R )[x ↦ P ] = let x = Q [x ↦ P ] in R

(let y = Q in R )[x ↦ P ] = let y = Q [x ↦ P ] in R [x ↦ P ]

where x ≡
/ y and y ∈/ F V (P )
(let y = Q in R )[x ↦ P ] = let z = Q [x ↦ P ] in R [ y ↦ z ][x ↦ P ]
where x ≡
/ y and y ∈ F V (P ) and z is a fresh variable

Solution to Exercise 157

# module MergeSort : SORT =
# struct
# let rec split lst =
# match lst with
# | []
# | [_] -> lst, []
# | first :: second :: rest ->
# let first', second' = split rest in
# first :: first', second :: second'
#
# let rec merge lt xs ys =
# match xs, ys with
# | [], _ -> ys
# | _, [] -> xs
# | x :: xst, y :: yst ->
# if lt x y then x :: (merge lt xst ys)
# else y :: (merge lt xs yst)
#
# let rec sort (lt : 'a -> 'a -> bool)
# (xs : 'a list)
# : 'a list =
# match xs with
# | []
# | [_] -> xs
# | _ -> let first, second = split xs in
# merge lt (sort lt first) (sort lt second)
# end ;;
module MergeSort : SORT

Solution to Exercise 158 The claims in 1, 2, 4, and 5 hold.

Solution to Exercise 159 1. Big-O notation only gives you information

about the worst-case performance as the input size becomes very
 SOLUTIONS TO SELECTED EXERCISES 477

large. Because of this, it ignores lower-order terms and constants

that may have a large effect for small inputs. So A may be slower
than B for some inputs, and the statement is false.

2. Since big-O notation provides worst-case performance, and A is

polynomial in big-O, they can be guaranteed that for any input (ex-
cept for a finite set), A will run in polynomial time, so the statement
is true.

3. As a worst-case bound, big-O doesn’t say anything about average-

case performance, so the statement is false.

Solution to Exercise 160 Since length is linear in the length of its

argument, compare_lengths is linear in the sum of the lengths of its
two arguments. But that sum is less than or equal to twice the length
of the longer argument. Thus, compare_lengths is in O (2n ), where
n is the length of the longer argument, hence, dropping multiplicative
constants, O (n ).
An alternative implementation, which stops the recursion once the
shorter list is exhausted, is linear in the length of the shorter list.

# let rec compare_lengths xs ys =

# match xs, ys with
# | [], [] -> 0
# | _, [] -> 1
# | [], _ -> -1
# | _xhd :: xtl, _yhd :: ytl -> compare_lengths xtl ytl ;;
val compare_lengths : 'a list -> 'b list -> int = <fun>
# compare_lengths [1] [2;3;4] ;;
- : int = -1
# compare_lengths [1;2;3] [4] ;;
- : int = 1
# compare_lengths [1;2] [3;4] ;;
- : int = 0

Solution to Exercise 162

Tod d s (n ) = c + Tod d s (n − 2)

More detail in the equation in terms of constants for different bits is

unnecessary, but benign. Note the n − 2, though n − 1 yields the same
complexity.

Solution to Exercise 163 Linear – O (n ).

Solution to Exercise 164 The O classes are independent of multipli-

cation or division by constants, so each “triplet” of answers after the
first are equivalent. Since f is O (n ), it is also O (n 2 ) etc. for all higher
classes. Thus, all answers from the fifth on are correct.
478 PROGRAMMING WELL

Solution to Exercise 165 O (n ) – linear. The odds and evens function

are both linear and return a list of length linear in n. The append is
linear in the length of the odds list, so also linear in n. The sum is
linear in the length of its argument, which is identical in length to
(and thus linear in) n. The let body is constant time. Summing these
complexities up, we’re left with linear and constant terms, which is
dominated by the linear term. Hence the function is linear.

Solution to Exercise 169 Let’s start with two mutable values of type
int list ref that are structurally equal but physically distinct:

# let lstref1 = ref [1; 2; 3] ;;

val lstref1 : int list ref = {contents = [1; 2; 3]}
# let lstref2 = ref [1; 2; 3] ;;
val lstref2 : int list ref = {contents = [1; 2; 3]}
# lstref1 = lstref2 ;;
- : bool = true
# lstref1 == lstref2 ;;
- : bool = false

Modifying one of them makes them both structurally and physically

unequal:
# lstref1 := [4; 5] ;;
- : unit = ()
# lstref1 = lstref2 ;;
- : bool = false
# lstref1 == lstref2 ;;
- : bool = false

Now for two values that are physically equal (that is, aliases), and
therefore structurally equal as well:
# let lstref3 = ref [1; 2; 3] ;;
val lstref3 : int list ref = {contents = [1; 2; 3]}
# let lstref4 = lstref3 ;;
val lstref4 : int list ref = {contents = [1; 2; 3]}
# lstref3 = lstref4 ;;
- : bool = true
# lstref3 == lstref4 ;;
- : bool = true

Modifying one of them retains their physical, and hence structural

equality:
# lstref3 := [4; 5] ;;
- : unit = ()
# lstref3 = lstref4 ;;
- : bool = true
# lstref3 == lstref4 ;;
- : bool = true

Solution to Exercise 170 We evaluate the expressions in the R E P L to

show their types and values.
 SOLUTIONS TO SELECTED EXERCISES 479

1. # let a = ref 3 in
# let b = ref 5 in
# let a = ref b in
# !(!a) ;;
Line 1, characters 4-5:
1 | let a = ref 3 in
^
Warning 26: unused variable a.
- : int = 5

2. In this example, a is a reference to b,which is itself a reference to a.

If we take the type of a to be ’a then, b must be of type ’a ref and
a (of type ’a remember) must also be of type ’a ref ref, leading
to an infinite regress of types. The expression is thus not well-typed.
The R E P L reports accordingly.

# let rec a, b = ref b, ref a in

# !a ;;
Line 1, characters 22-27:
1 | let rec a, b = ref b, ref a in
^^^^^
Error: This expression has type 'a ref ref
but an expression was expected of type 'a
The type variable 'a occurs inside 'a ref ref

3. Note the warning that the inner definition of a is not used; the a
used in the definition of b is the outer one, as required by lexical
scoping. (The R E P L even reports that the inner b is unused.)

# let a = ref 1 in
# let b = ref a in
# let a = ref 2 in
# !(!b) ;;
Line 3, characters 4-5:
3 | let a = ref 2 in
^
Warning 26: unused variable a.
- : int = 1

4. # let a = 2 in
# let f = (fun b -> a * b) in
# let a = 3 in
# f (f a) ;;
- : int = 12

5. # let a = Cons(2, ref (Cons(3, ref Nil))) ;;

val a : int mlist = Cons (2, {contents = Cons (3, {contents =
Nil})})

Solution to Exercise 172

1. Since we’ve just declared p as a reference to the integer 11, p is of

type int ref
480 PROGRAMMING WELL

# let p = ref 11 ;;
val p : int ref = {contents = 11}

2. Our variable r is a reference to our variable p. We defined p as a

reference to an integer, so r is a reference to this reference, or an int
ref ref.

# let r = ref p ;;
val r : int ref ref = {contents = {contents = 11}}

3. (a) False. We know p is of type int ref. Since we declare s as s =

ref !r, we know that s is a reference to the same value that r
references. Since !r = p, we therefore know s is also a reference
to p, and thus also of type int ref ref. The types of p and r are
therefore not the same.
# let s = ref !r ;;
val s : int ref ref = {contents = {contents = 11}}

# p ;;
- : int ref = {contents = 11}

(b) True. The explanation here is the same as for (1): Since s is a
reference to !r, it’s of type int ref ref, the type of r.
# r ;;
- : int ref ref = {contents = {contents = 11}}

# s ;;
- : int ref ref = {contents = {contents = 11}}

(c) False. As explained in the solution to (1), p is a reference to 11,

whereas s contains a reference to that reference.
# p ;;
- : int ref = {contents = 11}

# s ;;
- : int ref ref = {contents = {contents = 11}}

(d) True. We see r and s are a reference to the same value – that
is, they both are references to p – they therefore are structurally
equivalent.
# r ;;
- : int ref ref = {contents = {contents = 11}}

# s ;;
- : int ref ref = {contents = {contents = 11}}

4. We know the starting values of p, r, and s: p is a reference to the

integer 11, and s and r are two different references to p.
To find the value of t, let’s track the value of each variable at each
step in 4–6. We first set the dereference of s to equal 14 with the
line !s = 14. We know that since s is a reference to p, as found in
question (2), !s points to the same address as p. When we reassign
 SOLUTIONS TO SELECTED EXERCISES 481

!s to 14, we’re thus changing the value at the block of memory to

which p points to store the value 14.
We now set t equivalent to the expression !p + !(!r) + !(!s); in
order to compute this we must first evaluate each of the addends:

• !p: As described above, since s is a reference to p, !s points to

the same address as p; by replacing the value at that block with
14, p is now a reference to the value 14. Dereferencing p with !p
thus gives us the integer 14.
• !(!r): As described in the explanation to (3), we know r is a
reference to p, so !r points to the same address as p. We know
!(!r), therefore, is equal to !p, which we found above to be 14.

• !(!s): Again, s is still a reference to p, so !s would point to

the address as p. By dereferencing s again, with !(!s), we’re
therefore returning the value to which p points, that is, 14.

Putting it all together, we can see that this evaluates to 14 + 14 +

14, so t = 42.

# let t =
# !s := 14;
# !p + !(!r) + !(!s) ;;
val t : int = 42
# t ;;
- : int = 42

5. Note how similar the code in 7–9 looks to the code in 4–6. Yet there
is in fact one key difference: we’re changing s itself rather than
!s. This means that instead of modifying our reference to p, we’re
replacing it. With the line s := ref 17, we’re declaring an entirely
new reference that points to an instance of the value 17, and setting
s to point to that reference. This effectively severs the tie between s
and p: s points to a to a completely separate reference to a block of
memory containing the value 17, while p continues to point to the
value 14.
As for r, note that while s and r started out structurally equivalent,
they were never physically equivalent. Think back to when we
defined s:

let s = ref !r ;;

When we dereference r with !r, we lose all association with the spe-
cific block of memory to which r refers and are only passed along
the value contained in that block. Thus while s is also a reference
to the value r references – that is, both s and r are references to p –
s and r are in fact distinct references pointing to distinct blocks in
482 PROGRAMMING WELL

memory. Because s and r are not structurally equivalent, s is still a

reference to p.
Putting it all together, we again evaluate each of the addends in the
expression !p + !(!r) + !(!s); !p and thus !(!r) each evaluate
to 14, while !(!s)) now evaluates to 17. We’re thus left with 14 +
14 + 17, and t = 45.

# let t =
# s := ref 17;
# !p + !(!r) + !(!s) ;;
val t : int = 45
# t ;;
- : int = 45

Solution to Exercise 173 A simple solution, subject to inexhaustive

match errors at run-time is

# let mhead (Cons (hd, _tl)) = hd ;;

Line 1, characters 10-31:
1 | let mhead (Cons (hd, _tl)) = hd ;;
^^^^^^^^^^^^^^^^^^^^^
Warning 8: this pattern-matching is not exhaustive.
Here is an example of a case that is not matched:
Nil
val mhead : 'a mlist -> 'a = <fun>
# let mtail (Cons (_hd, tl)) = !tl ;;
Line 1, characters 10-32:
1 | let mtail (Cons (_hd, tl)) = !tl ;;
^^^^^^^^^^^^^^^^^^^^^^
Warning 8: this pattern-matching is not exhaustive.
Here is an example of a case that is not matched:
Nil
val mtail : 'a mlist -> 'a mlist = <fun>

Better is a version that explicitly raises a Failure exception (a la

List.hd and List.tl):

# let mhead mlist =

# match mlist with
# | Nil -> raise (Failure "mhead of empty mlist")
# | (Cons (hd, _tl)) -> hd ;;
val mhead : 'a mlist -> 'a = <fun>
# let mtail mlist =
# match mlist with
# | Nil -> raise (Failure "mtail of empty mlist")
# | (Cons (_hd, tl)) -> !tl ;;
val mtail : 'a mlist -> 'a mlist = <fun>

Solution to Exercise 174 We evaluate the expressions in the R E P L to

show their types and values.

1. # let a = Cons(2, ref (Cons(3, ref Nil))) ;;

val a : int mlist = Cons (2, {contents = Cons (3, {contents =
 SOLUTIONS TO SELECTED EXERCISES 483

Nil})})
# let Cons(_h, t) = a in
# let b = Cons(1, ref a) in
# t := b;
# mhead (mtail (mtail b)) ;;
Lines 1-4, characters 0-23:
1 | let Cons(_h, t) = a in
2 | let b = Cons(1, ref a) in
3 | t := b;
4 | mhead (mtail (mtail b))...
Warning 8: this pattern-matching is not exhaustive.
Here is an example of a case that is not matched:
Nil
- : int = 1

Solution to Exercise 176

# let rec first (n: int) (mlst: 'a mlist) : 'a list =
# if n = 0 then []
# else match mlst with
# | Nil -> []
# | Cons(hd, tl) -> hd :: first (n-1) !tl ;;
val first : int -> 'a mlist -> 'a list = <fun>

Solution to Exercise 177

# let rec alternating =

# Cons(1, ref (Cons(2, ref alternating))) ;;
val alternating : int mlist =
Cons (1, {contents = Cons (2, {contents = <cycle>})})

Solution to Exercise 179 Let’s start with insertion. It will be useful

to have an auxiliary function that attempts to insert at a particular
location, carrying out the probing if that location is already used for a
different key.

let rec insert' dct target newvalue loc =

(* fallen off the end of the array; error *)
if loc >= D.size then raise Exit
else
match dct.(loc) with
| Empty ->
(* found an empty slot; fill it *)
dct.(loc) <- Element {key = target;
value = newvalue}
| Element {key; _} ->
if key = target then
(* found an existing pair for key; replace it *)
dct.(loc) <- Element {key = target;
value = newvalue}
else
(* hash collision; reprobe *)
insert' (succ loc) ;;
484 PROGRAMMING WELL

Now with this auxiliary function, we can implement insertion just by

attempting to insert at the location given by the hash function.
let insert dct target newvalue =
insert' dct target newvalue (D.hash_fn target) ;;

Of course, insert’ is only needed in the context of insert. Why not

make it a local function? Doing so also puts the definition of insert’
in the scope of the arguments of insert. Since these never change
in calls of insert’, we can drop them from the arguments list of
insert’.

let insert dct target newvalue =

let rec insert' loc =
(* fallen off the end of the array; error *)
if loc >= D.size then raise Exit
else
match dct.(loc) with
| Empty ->
(* found an empty slot; fill it *)
dct.(loc) <- Element {key = target;
value = newvalue}
| Element {key; _} ->
if key = target then
(* found an existing pair for key; replace it *)
dct.(loc) <- Element {key = target;
value = newvalue}
else
(* hash collision; reprobe *)
insert' (succ loc) in
insert' (D.hash_fn target) ;;

Next, we can look at the member function. Using the same approach,
we get
let member dct target =
let rec member' loc =
(* fallen off the end of the array; not found *)
if loc >= D.size then false
else
match dct.(loc) with
| Empty ->
(* found an empty slot; target not found *)
false
| Element {key; _} ->
if key = target then

(* found an existing pair for this key; target found *)

true
else
(* hash collision; reprobe *)
member' (succ loc) in
member' (D.hash_fn target) ;;
 SOLUTIONS TO SELECTED EXERCISES 485

Perhaps you see the problem. The code is nearly identical, once the
putative location for the target key is found. The same will be true for
lookup and remove. Rather than reimplement this search process in
each of the functions, we can abstract it into its own function, which
we’ll call findloc. This function returns the (optional) location (index)
where a particular target key is already or should go, or None if no such
location is found.

let findloc (dct : dict) (target : key) : int option =

let rec findloc' loc =
if loc >= D.size then None
else
match dct.(loc) with
| Empty -> Some loc
| Element {key; _} ->
(if key = target then Some loc
else findloc' (succ loc)) in
findloc' (D.hash_fn target) ;;

Using findloc, implementation of the other functions becomes much

simpler.

let member dct target =

match findloc dct target with
| None -> false
| Some loc ->
match dct.(loc) with
| Empty -> false
| Element {key; _} ->
assert (key = target);
true ;;

let lookup dct target =

match findloc dct target with
| None -> None
| Some loc ->
match dct.(loc) with
| Empty -> None
| Element {key; value} ->
assert (key = target);
Some value ;;

let insert dct target newvalue =

match findloc dct target with
| None -> raise Exit
| Some loc ->
dct.(loc) <- Element {key = target;
value = newvalue} ;;

let remove dct target =

match findloc dct target with
| None -> ()
| Some loc -> dct.(loc) <- Empty ;;
486 PROGRAMMING WELL

Furthermore, the code is more maintainable because all of the details

of collision handling are localized in the one findloc function. If we
want to change to, say, quadratic probing, only that one function need
be changed.
One might still think that there is more commonality among the
hashtable functions than is even getting captured by findloc. It seems
that in all cases, the result of the call to findloc is being tested for
three cases: (i) no location is available, (ii) an empty location is found,
or (iii) a non-empty location is found with the target key. Rather than
perform this triage in all of the various functions, why not do so in
findloc itself, which can be provided with appropriate functions,
called C A L L B A C K S , for each of the cases. The following version does
just this:

(* findloc dct key cb_unavailable cb_empty cb_samekey --

Finds the proper location for the key in the dct, and
calls the appropriate callback function:
cb_unavailable -- no element available for this key
cb_empty loc -- element available is empty at provided
loc
cb_samekey loc key value -- element available is non-empty
at provided loc and has the given key and value
*)
let findloc (dct : dict) (target : key)
(cb_unavailable : unit -> 'a)
(cb_empty : int -> 'a)
(cb_samekey : int -> key -> value -> 'a)
: 'a =
let rec findloc' loc =
if loc >= D.size then cb_unavailable ()
else
match dct.(loc) with
| Empty -> cb_empty loc
| Element {key; value} ->
(if key = target then cb_samekey loc key value
else findloc' (succ loc)) in
findloc' (D.hash_fn target) ;;

let member dct target =

findloc dct target
(fun () -> false)
(fun _ -> false)
(fun _ _ _ -> true) ;;

let lookup dct target =

findloc dct target
(fun () -> None)
(fun _ -> None)
(fun _ _ value -> Some value) ;;

let insert dct target newvalue =

 SOLUTIONS TO SELECTED EXERCISES 487

let newelt = Element {key = target;

value = newvalue} in
findloc dct target
(fun () -> raise Exit)
(fun loc -> dct.(loc) <- newelt)
(fun loc _ _ -> dct.(loc) <- newelt) ;;

let remove dct target =

findloc dct target
(fun () -> ())
(fun loc -> ())
(fun loc _ _ -> dct.(loc) <- Empty) ;;

Solution to Exercise 182 Here’s a simple implementation keeping an

internal counter of allocations since the last reset.

# module Metered : METERED = struct

# (* internal counter of allocations since last reset *)
# let constructor_count = ref 0
#
# let reset () =
# constructor_count := 0
#
# let count () =
# !constructor_count
#
# let cons hd tl =
# incr constructor_count;
# hd :: tl
#
# let pair first second =
# incr constructor_count;
# first, second
# end ;;
module Metered : METERED

Solution to Exercise 183

# let rec zip (xs : 'a list)

# (ys : 'b list)
# : ('a * 'b) list =
# match xs, ys with
# | [], [] -> []
# | [], _
# | _, [] -> raise (Invalid_argument
# "zip: unequal length lists")
# | xhd :: xtl, yhd :: ytl ->
# Metered.cons (Metered.pair xhd yhd) (zip xtl ytl) ;;
val zip : 'a list -> 'b list -> ('a * 'b) list = <fun>

Notice that the constructors in the patterns, which are merely used
to deconstruct values, are unchanged. Only the instances used to
construct new values are replaced with their metered counterparts.
488 PROGRAMMING WELL

Solution to Exercise 184 A metered version of quicksort replaces all

consing and pairing with the metered version. We’ve added a metered
version of append as well.

# module MeteredQuickSort : SORT =

# struct
# (* simplify access to the metering *)
# open Metered
#
# (* append xs ys -- A metered version of the (@) append
# function *)
# let rec append (xs : 'a list) (ys : 'a list) : 'a list =
# match xs with
# | [] -> ys
# | hd :: tl -> cons hd (append tl ys)
#
# (* partition lt pivot xs -- Returns two lists
# constituting all elements in `xs` less than (according
# to `lt`) than the `pivot` value and greater than the
# pivot `value`, respectively *)
# let rec partition lt pivot xs =
# match xs with
# | [] -> pair [] []
# | hd :: tl ->
# let first, second = partition lt pivot tl in
# if lt hd pivot then pair (cons hd first) second
# else pair first (cons hd second)
#
# (* sort lt xs -- Quicksorts `xs` according to the
# comparison function `lt` *)
# let rec sort (lt : 'a -> 'a -> bool)
# (xs : 'a list)
# : 'a list =
# match xs with
# | [] -> []
# | pivot :: rest ->
# let first, second = partition lt pivot rest in
# append (sort lt first)
# (append (cons pivot [])
# (sort lt second))
# end ;;
module MeteredQuickSort : SORT

With the metered version in hand, we can see the allocations more
clearly.

# Metered.reset () ;;
- : unit = ()
# MeteredQuickSort.sort (<)
# [1; 3; 5; 7; 9; 2; 4; 6; 8; 10] ;;
- : int list = [1; 2; 3; 4; 5; 6; 7; 8; ...]
# Metered.count () ;;
- : int = 92
 SOLUTIONS TO SELECTED EXERCISES 489

Solution to Exercise 186 New versions of the functions use

Lazy.force instead of application to unit and the lazy keyword
instead of wrapping a function. Notice that first is unchanged, as
it delays and forces only through its use of the other functions.

let tail (s : 'a stream) : 'a stream =

match Lazy.force s with
| Cons (_hd, tl) -> tl ;;

let rec smap (f : 'a -> 'b) (s : 'a stream)

: ('b stream) =
lazy (Cons (f (head s), smap f (tail s))) ;;

let rec smap2 f s1 s2 =

lazy (Cons (f (head s1) (head s2),
smap2 f (tail s1) (tail s2))) ;;

let rec first (n : int) (s : 'a stream) : 'a list =

if n = 0 then []
else head s :: first (n - 1) (tail s) ;;

Solution to Exercise 187 We start with a function to form ratios of

successive stream elements.

# let rec ratio_stream (s : float stream) : float stream =

# lazy (Cons ((head (tail s)) /. (head s),
# ratio_stream (tail s))) ;;
_
val ratio stream : float stream -> float stream = <fun>

Now we can generate the stream of ratios for the Fibonacci sequence
and find the required approximation:

# let fib_approx = ratio_stream (to_float fibs) ;;

val fib_approx : float stream = <lazy>
# within 0.000001 fib_approx ;;
- : float = 1.61803444782168193

Solution to Exercise 188

# let rec falses =

# lazy (Cons (false, falses)) ;;
val falses : bool stream = <lazy>

Solution to Exercise 189 As demonstrated by the OCaml R E P L type

inference in the previous exercise, the type of falses is bool stream.

Solution to Exercise 190 Here is a recursive implementation of

trueat:

# let rec trueat n =

# if n = 0 then lazy (Cons (true, falses))
# else lazy (Cons (false, trueat (n - 1))) ;;
val trueat : int -> bool stream = <fun>
490 PROGRAMMING WELL

Solution to Exercise 191 Here is a recursive implementation of

trueat:

# let circnot : bool stream -> bool stream =

# smap not ;;
val circnot : bool stream -> bool stream = <fun>

Note the use of the smap function and the use of partial application.

Solution to Exercise 192

# let circand : bool stream -> bool stream -> bool stream =
# smap2 (&&) ;;
val circand : bool stream -> bool stream -> bool stream = <fun>

Solution to Exercise 193

# let circnand (s: bool stream) (t: bool stream) : bool stream =
# circnot (circand s t) ;;
val circnand : bool stream -> bool stream -> bool stream = <fun>

Solution to Exercise 207

# class text (p : point) (s : string) : display_elt =

# object (this)
# inherit shape p
# method draw = let (w, h) = G.text_size s in
# G.set_color this#get_color ;
# G.moveto (this#get_pos.x - w/2)
# (this#get_pos.y - h/2);
# G.draw_string s
# end ;;
class text : point -> string -> display_elt

Solution to Exercise 208 There are many ways of implementing such

functions. Here’s one.

# let mono x = [x + 1] ;;
val mono : int -> int list = <fun>
# let poly x = [x] ;;
val poly : 'a -> 'a list = <fun>
# let need f =
# match f 3 with
# | [] -> []
# | hd :: tl -> hd + 1 :: tl ;;
val need : (int -> int list) -> int list = <fun>
# need mono ;;
- : int list = [5]
# need poly ;;
- : int list = [4]

Solution to Exercise 209 The solution here makes good use of inheri-
tance rather than reimplementation.
 SOLUTIONS TO SELECTED EXERCISES 491

# class loud_counter : counter_interface =

# object (this)
# inherit counter as super
# method! bump n =
# super#bump n;
# Printf.printf "State is now %d\ n" this#get_state
# end ;;
class loud_counter : counter_interface

The bump method is introduced with a ! to make clear our intention to

override the inherited method, and to avoid a warning.

Solution to Exercise 210

# class type reset_counter_interface =
# object
# inherit counter_interface
# method reset : unit
# end ;;
class type reset_counter_interface =
object
method bump : int -> unit
method get_state : int
method reset : unit
end

Solution to Exercise 211

# class loud_reset_counter : reset_counter_interface =
# object (this)
# inherit loud_counter
# method reset =
# this#bump (-this#get_state)
# end ;;
class loud_reset_counter : reset_counter_interface

Solution to Exercise 218

{} ⊢ let x = 3 in let y = 5 in x + y
⇓
RRR {} ⊢ 3 ⇓ 3
RRR
RRR {x ↦ 3} ⊢ let y = 5 in x + y
RRR
RRR ⇓
RRR
RRR RRR {x ↦ 3} ⊢ 5 ⇓ 5
RRR RRR
RRR RRR {x ↦ 3; y ↦ 5} ⊢ x + y
RRR RRR
RRR RRR ⇓
RRR RRR
RRR RRR {x ↦ 3; y ↦ 5} ⊢ x ⇓ 3
RRR RRR ∣
RRR RRR {x ↦ 3; y ↦ 5} ⊢ y ⇓ 5
RRR RRR
RRR RRR ⇓8
RRR
RR ⇓ 8

⇓8
492 PROGRAMMING WELL

Solution to Exercise 219

{} ⊢ let x = 3 in let x = 5 in x + y
⇓
RRR {} ⊢ 3 ⇓ 3
RRR
RRR {x ↦ 3} ⊢ let x = 5 in x + x
RRR
RRR ⇓
RRR
RRR RRR {x ↦ 3} ⊢ 5 ⇓ 5
RRR RRR
RRR RRR {x ↦ 5} ⊢ x + x
RRR RRR
RRR RRR ⇓
RRR RRR
RRR R
RRR {x ↦ 5} ⊢ x ⇓ 5
RRR RRR ∣
RRR RRR {x ↦ 5} ⊢ x ⇓ 5
RRR RRR
RRR RR ⇓ 10
RRR
RR ⇓ 10
⇓ 10

Solution to Exercise 220

• R fun : “A function expression of the form fun x -> B in an environ-

ment E evaluates to itself.”

• R app : “To evaluate an application of the form P Q in an environ-

ment E , first evaluate P in E to a function value of the form fun x
-> B and Q in E to a value v Q . Then evaluate the expression B in an
environment that augments E with a binding of x to vQ , resulting in
a value v B . The value of the full expression is then v B .”
 SOLUTIONS TO SELECTED EXERCISES 493

Solution to Exercise 222 The derivation under a lexical environment

semantics is as follows:

{} ⊢ let x = 1 in let f = fun y -> x + y in let x = 2 in f 3

⇓
RRR {} ⊢ 1 ⇓ 1
RRR
RRR {x ↦ 1} ⊢ let f = fun y -> x + y in let x = 2 in f 3
RRR
RRR ⇓
RRR
RRR RRR {x ↦ 1} ⊢ fun y -> x + y ⇓ [{x ↦ 1} ⊢ fun y -> x + y]
RRR RRR
RRR RRR {x ↦ 1; f ↦ [{x ↦ 1} ⊢ fun y -> x + y]} ⊢ let x = 2 in f 3
RRR RRR
RRR RRR ⇓
RRR RRR
RRR R
RRR RRR {x ↦ 1; f ↦ [{x ↦ 1} ⊢ fun y -> x + y]} ⊢ 2 ⇓ 2
RRR RRR RRR
RRR RRR RRR {f ↦ [{x ↦ 1} ⊢ fun y -> x + y]; x ↦ 2} ⊢ f 3
RRR RRR RRR
RRR RRR RRR ⇓
RRR RRR RRR
R
RRR RRR
RRR RRR
RRR RRR {f ↦ [{x ↦ 1} ⊢ fun y -> x + y]; x ↦ 2} ⊢ f ⇓ [ {x ↦ 1} ⊢ fun y -> x + y]
RRR RRR RRR
RRR RRR RRR RRR {f ↦ [{x ↦ 1} ⊢ fun y -> x + y]; x ↦ 2} ⊢ 3 ⇓ 3
RRR RRR R
RRR RRR
RRR RRR RRR RRR {x ↦ 1; y ↦ 3} ⊢ x + y
RRR RRR RRR RRR
RRR RRR RRR RRR ⇓
RRR RRR RRR RRR
RRR RRR RRR RRR { x ↦ 1; y ↦ 3} ⊢ x ⇓ 1
RRR RRR RRR RRR ∣
RRR RRR RRR RRR { x ↦ 1; y ↦ 3} ⊢ y ⇓ 3
RRR RRR RRR RRR
RRR RRR RRR RR ⇓4
RRR RRR RRR
RRR RRR RR ⇓ 4
RRR RRR
RRR RR ⇓4
RRR
RR ⇓4
⇓4

Notice that the body of the function is evaluated in an environment

constructed by taking the lexical environment of the function (the
first highlight in the derivation above) and augmenting it with the
argument binding to form the environment in which to evaluate the
body (the second highlight). In the lexical environment x has the value
1, so the entire expression evaluates to 4 rather than 5 (as under the
substitution semantics as well).

Solution to Exercise 224 Only (4) evaluates to a different value under

dynamic scoping. Under OCaml’s lexical scoping, the a in the body
of the f function is the lexically containing a that has value 2. The
expression thus has value 12 under lexical scoping:

# let a = 2 in
# let f = (fun b -> a * b) in
# let a = 3 in
# f (f a) ;;
- : int = 12
494 PROGRAMMING WELL

Under dynamic scoping, the a in the body of the f function is the

dynamically more recent a with value 3. The value of the expression is
thus 27 under dynamic scoping.

Solution to Exercise 225 Environment semantics rules for true and

false, appropriate for both lexical and dynamic variants, are

E ⊢ true ⇓ true (R true )

E ⊢ false ⇓ false (R false )

Solution to Exercise 226 Environment semantics rules for true and

false, appropriate for both lexical and dynamic variants, are

E ⊢ if C then T else F ⇓

E ⊢ C ⇓ true
∣ (R ifthen )
E ⊢ T ⇓ vT

⇓ vT

E ⊢ if C then T else F ⇓

E ⊢ C ⇓ false
∣ (R ifelse )
E ⊢ F ⇓ vF

⇓ vF

Solution to Exercise 227

E,S ⊢ ! P ⇓
∣ E , S ⊢ P ⇓ l , S′ (R deref )

⇓ S ′ (l ), S ′

The rule can be glossed “to evaluate an expression of the form ! P in

environment E and store S, evaluate P in E and S to a location l and
new store S ′ . The result is the value that l maps to in S ′ and new store
S ′ .”

Solution to Exercise 228 The following rule evaluates P to a unit,

passing the side-effected store S ′ on for the evaluation of Q. The re-
sult of the sequencing is then the value and store resulting from the
evaluation of Q.

E,S ⊢ P ; Q ⇓

E , S ⊢ P ⇓ (), S ′
∣ (R seq )
E , S ′ ⊢ Q ⇓ vQ , S ′′

⇓ vQ , S ′′
 SOLUTIONS TO SELECTED EXERCISES 495

Solution to Exercise 229 We start by taking

let rec x = D in B

to be equivalent to

′ ′
let x = ref unassigned in (x := D ); B

where for brevity we abbreviate D ′ ≡ D [x ↦ !x ], B ′ ≡ B [x ↦ !x ], and

U ≡ unassigned.
In order to develop the semantic rule for let rec x = D in B , we
carry out a schematic derivation of its desugared equivalent:

E , S ⊢ let x = ref U in (x := D ′ ); B ′

⇓
RRR E , S ⊢ ref U
RRR
RRR ⇓
RRR
RRR ∣ E,S ⊢U ⇓U,S
RRR
RRR ⇓ l , S {l ↦ U }
RRR
RRR E {x ↦ l }, S {l ↦ U } ⊢ (x := D ′ ); B ′
RRR
RRR ⇓
RRR RRR E {x ↦ l }, S {l ↦ U } ⊢ x := D ′
RRR RRR
RRR RRR
RRR RRR ⇓
RRR R RRR E {x ↦ l }, S {l ↦ U } ⊢ x ⇓ l , S {l ↦ U }
RRR RRR RRR
RRR RRR RRR E {x ↦ l }, S {l ↦ U } ⊢ D’ ⎫
RRR RRR RRR ⎪
⎪
RRR RRR ⎪
⎪
⎪
RRR R R
R
RRR ⇓ ⎪
⎪
RRR ⎬
RRR RRR R
R ∣ ⋯ ⎪
RRR RRR RRR ⎪
⎪
RRR RRR RRR ⎪
⎪
⎪
RRR RRR R
R ⇓ v D , S ′ ⎪
⎭
RRR RRR ⇓ , S ′
{ l ↦ v }
RRR RRR () D
RRR RRR E {x ↦ l }, S ′ {l ↦ v D } ⊢ B’ ⎫
⎪
RRR RRR ⎪
⎪
⎪
RRR RRR ⇓ ⎪
⎪
⎪
RRR RRR ⎬
RRR RRR ∣ ⋯ ⎪
⎪
RRR RRR ⎪
⎪
⎪
RRR RRR ⇓ vB , S ′′ ⎪
⎪
RRR ⎭
RRR ⇓ vB , S ′′

⇓ v B , S ′′

This schematic derivation is complete, except for the two highlighted

subderivations for D ′ and B ′ respectively. Thus, we can define a se-
mantic rule for the original construct let rec x = D in B (now
with abbreviations expanded) that incorporates these two subderiva-
496 PROGRAMMING WELL

tions as premises:

E , S ⊢ let rec x = D in B ⇓

E {x ↦ l }, S {l ↦ unassigned} ⊢ D [x ↦ !x ] ⇓ v D , S ′
∣
E {x ↦ l }, S ′ {l ↦ v D } ⊢ B [x ↦ !x ] ⇓ v B , S ′′

⇓ v B , S ′′
(R letrec )

This is just the semantic rule presented in Section 19.6.1.

Solution to Exercise 230 The fold implementation from the solution

to Exercise 104 is the following:

let rec foldbt (emptyval : 'b)

(nodefn : 'a -> 'b -> 'b -> 'b)
(t : 'a bintree)
: 'b =
match t with
| Empty -> emptyval
| Node (value, left, right) ->
nodefn value (foldbt emptyval nodefn left)
(foldbt emptyval nodefn right) ;;

As a first step, let’s isolate the two recursive calls.

let rec foldbt (emptyval : 'b)

(nodefn : 'a -> 'b -> 'b -> 'b)
(t : 'a bintree)
: 'b =
match t with
| Empty -> emptyval
| Node (value, left, right) ->
let left' = foldbt emptyval nodefn left in
let right' = foldbt emptyval nodefn right in
nodefn value left' right' ;;

Now, we can compute the left subtree in a separate thread using

future, remembering to force the value when it’s needed.

# let rec foldbt_conc (emptyval : 'b)

# (nodefn : 'a -> 'b -> 'b -> 'b)
# (t : 'a bintree)
# : 'b =
# match t with
# | Empty -> emptyval
# | Node (value, left, right) ->
# let left' =
# Future.future (foldbt_conc emptyval nodefn) left in
# let right' = foldbt_conc emptyval nodefn right in
# nodefn value (Future.force left') right' ;;
val foldbt_conc : 'b -> ('a -> 'b -> 'b -> 'b) -> 'a bintree -> 'b
= <fun>
 SOLUTIONS TO SELECTED EXERCISES 497

To demonstrate its operation, we can sum the values in a binary tree as

per Exercise 106.

# let sum_bintree =
# foldbt_conc 0 (fun v l r -> v + l + r) ;;
val sum_bintree : int bintree -> int = <fun>

# let int_bintree =
# Node (16, Node (93, Empty, Empty),
# Node (3, Node (42, Empty, Empty),
# Empty)) ;;
val int_bintree : int bintree =
Node (16, Node (93, Empty, Empty),
Node (3, Node (42, Empty, Empty), Empty))

# sum_bintree int_bintree ;;
- : int = 154

Solution to Exercise 232 Here’s one such interleaving. We adjust the

previous interleaving moving thread A’s balance update to after thread
B’s update.

thread A ($75 withdrawal) thread B ($50 withdrawal)

1. if balance >= amt then begin

2. let diff = balance - amt in
3. if balance >= amt then begin
4. let diff = balance - amt in
5. balance <- diff;
6. balance <- diff;
7. amt
8. ⋯ amt
⋯

Solution to Exercise 233 Here’s one such interleaving. We adjust the

previous interleaving so that thread B verifies the balance adequacy
(line 2) before thread A’s update (line 3-4), but computes its updated
balance afterwards (line 6).

thread A ($75 withdrawal) thread B ($50 withdrawal)

1. if balance >= amt then begin

2. if balance >= amt then begin
3. let diff = balance - amt in
4. balance <- diff;
5. amt
6. let diff = balance - amt in
7. ⋯ balance <- diff;
8. amt
⋯

Solution to Exercise 234 We wrap the computation of the critical

region f () in a try ⟨⟩ with to trap any exceptions and unlock on the
way out.

# (* with_lock l f -- Run thunk `f` in context of acquired lock `l`,

# unlocking on return or exceptions *)
# let with_lock (l : Mutex.t) (f : unit -> 'a) : 'a =
# Mutex.lock l;
498 PROGRAMMING WELL

# let res =
# try f ()
# with exn -> Mutex.unlock l;
# raise exn in
# Mutex.unlock l;
# res ;;
val with_lock : Mutex.t -> (unit -> 'a) -> 'a = <fun>
Bibliography

“A New York correspondent”. To find Easter. Nature, XIII(338):

487, April 20 1876. URL https://books.google.com/books?id=
H4ICAAAAIAAJ&pg=PA487#v=onepage&q&f=false.

Marianne Baudinet and David MacQueen. Tree pattern matching for

ML (extended abstract). Technical report, Stanford University, 1985.
URL http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.52.
9225&rep=rep1&type=pdf.

Jorge Luis Borges. Funes the memorious. In Donald A. Yates and

James E. Irby, editors, Labyrinths. New Directions, New York, New
York, 1962.

Richard G. Brown, Mary P. Dolciani, Robert H. Sorgenfrey, and

William L. Cole. Algebra: Structure and Method: Book 1. McDou-
gal Littell, Evanston, Illinois, California edition, 2000.

California Utilities Statewide Codes and Standards Team. Guest room

occupancy controls: 2013 California building energy efficiency
standards. Technical report, California Statewide Utility Codes and
Standards Program, October 2011. URL https://www.energy.ca.gov/
title24/2013standards/prerulemaking/documents/current/Reports/
Nonresidential/Lighting_Controls_Bldg_Power/2013_CASE_NR_
Guest_Room_Occupancy_Controls_Oct_2011.pdf.

Emmanuel Chailloux, Pascal Manoury, and Bruno Pagano. Développe-

ment d’applications avec Objective CAML (Developing Applica-
tions With Objective CAML). O’Reilly Media, 2000. URL https:
//caml.inria.fr/pub/docs/oreilly-book/html/book-ora168.html.

Alonzo Church. An unsolvable problem of elementary number the-

ory. American Journal of Mathematics, 58(2):345–363, 1936. ISSN
00029327, 10806377. URL http://www.jstor.org/stable/2371045.

Craig Conley. Wye’s Dictionary of Improbable Words: All-Vowel Words

and All-Consonant Words. CreateSpace Independent Publishing
500 PROGRAMMING WELL

Platform, 2009. ISBN 9781441455277. URL https://books.google.

com/books?id=yk1j1WLh80QC.

Jeffrey Dean and Sanjay Ghemawat. MapReduce: Simplified data

processing on large clusters. In Sixth Symposium on Operating
System Design and Implementation (OSDI’04), pages 137–150, San
Francisco, CA, 2004. URL https://research.google.com/archive/
mapreduce-osdi04.pdf.

Charles Antony Richard Hoare. The emperor’s old clothes. Com-

munications of the ACM, 24(2):75–83, February 1981. D O I :
10.1145/1283920.1283936. URL http://doi.acm.org/10.1145/
1283920.1283936.

Donald E. Knuth. Von Neumann’s first computer program. ACM

Computing Surveys, 2(4):247–260, December 1970. ISSN 0360-0300.
DOI: 10.1145/356580.356581. URL http://doi.acm.org/10.1145/
356580.356581.

Donald E. Knuth. Structured programming with go to statements.

Computing Surveys, 6(4):261–301, December 1974. URL https:
//dl.acm.org/citation.cfm?id=356640.

Eriola Kruja, Joe Marks, Ann Blair, and Richard Waters. A short note on
the history of graph drawing. In International Symposium on Graph
Drawing, pages 272–286. Springer, 2001.

Peter J. Landin. The mechanical evaluation of expressions. The Com-

puter Journal, 6(4):308–320, 1964. D O I : 10.1093/comjnl/6.4.308.
URL http://dx.doi.org/10.1093/comjnl/6.4.308.

Peter J. Landin. A correspondence between ALGOL 60 and Church’s

lambda-notation: Part I. Communications of the ACM, 8(2):89–101,
February 1965. ISSN 0001-0782. D O I : 10.1145/363744.363749. URL
http://doi.acm.org/10.1145/363744.363749.

Peter J. Landin. The next 700 programming languages. Communica-

tions of the ACM, 9(3):157–166, March 1966. ISSN 0001-0782. D O I :
10.1145/365230.365257. URL http://doi.acm.org/10.1145/365230.
365257.

Harry Lewis and Rachel Zax. Essential Discrete Mathematics for Com-
puter Science. Princeton University Press, Princeton, New Jersey,
2019.

John McCarthy. Recursive functions of symbolic expressions and their

computation by machine, part i. Communications of the ACM, 3(4):
184–195, April 1960. ISSN 0001-0782. D O I : 10.1145/367177.367199.
URL http://doi.acm.org/10.1145/367177.367199.
 BIBLIOGRAPHY 501

Luigi Federico Menabrea and Ada Lovelace. Sketch of the analytical

engine invented by charles babbage with notes by the translator.
translated by ada lovelace. In Richard Taylor, editor, Scientific Mem-
oirs, volume 3, pages 666–731. Richard and John E. Taylor, London,
1843. URL http://nrs.harvard.edu/urn-3:FHCL.HOUGH:33047333.

Gordon E. Moore. Cramming more components onto integrated

circuits. Electronics, 38(8):114–117, 19 April 1965.

Christopher Null. Hello, i’m mr. null. my name makes me invisible to

computers, November 2015. URL https://www.wired.com/2015/11/
null/.

Plato. Phaedrus. In Plato in Twelve Volumes, volume I. William Heine-

mann Ltd., London, 1927. URL https://hdl.handle.net/2027/uc1.
32106017211316?urlappend=%3Bseq=563. Translated by Harold N.
Fowler.

Jean E. Sammet. The beginning and development of FORMAC (FOR-

mula MAnipulation Compiler). In The Second ACM SIGPLAN Con-
ference on History of Programming Languages, HOPL-II, pages 209–
230, New York, NY, USA, 1993. ACM. ISBN 0-89791-570-4. D O I :
10.1145/154766.155372. URL http://doi.acm.org/10.1145/154766.
155372.

Moses Schönfinkel. Über die bausteine der mathematischen logik.

Mathematische Annalen, 9(2):305–316, 1924.

Alan Turing. Computability and λ-definability. J. Symbolic Logic, 2

(4):153–163, 12 1937. URL https://projecteuclid.org:443/euclid.jsl/
1183383711.
502 PROGRAMMING WELL
Index

Note: The page numbers for primary occurrences of indexed items are typeset as 123, other occurrences as
123.

^ (string concatenation), 39 binding construct, 47 cons, 77

bound, 225 control, 352
absolute value, 421 box and arrow diagrams, 275 controls, 353
abstract data type, 169 breadth-first search, 266 critical region, 399
abstract syntax, 32 buffer over-reads, 277 Curry, Haskell, 326
abstract syntax tree, 32 buffer overflows, 277 currying, 56
abstraction, 15 busy waiting, 391 cycle, 284
abstraction barrier, 169
algebraic data types, 141 callbacks, 486 decomposition, see edict of
alias, 276 characters, 39 decomposition
ambiguity, 29 checksum, 110 definiendum, 47
anomaly, 115 Church, Alonzo, 21, 248, 418 definiens, 47
anonymous function, 58 Church-Turing thesis, 21, 401 delay, 310
anonymous variable, 74 class, 341 denotational semantics, 217
application, 55 class interface, 341 dependent type systems, 65
argument, 55 closed form, 254 depth-first search, 266
artificial intelligence, 264 closed hashing, 291 dequeuing, 167
associativity, 31 closure, 372 dereference, 273
asymptotic, 248 comments, 34 design, 21
atomic type, 42 comparison operators, 39 difference, 423
compartmentalization, see Dikstra, Edsger, 64
Backus-Naur form, 28 edict of compart- disjunction, 422
backwards application, 148 mentalization distance, 422
best_first search, 266 composition, 109 divide-and-conquer, 261
big-O notation, 248 computus, 62 donation game, 87
bignums, 134 concatenation dynamic environment, 370
binary operators, 30 string, 39 dynamic environment seman-
binary search tree, 157, 209 concrete syntax, 32 tics, 370
binary tree, 155 concurrent computation, 385 dynamically typed, 41
bind, 225 conditional, 40
binding, 47, 409 conjunction, 422 eager, 309
504 PROGRAMMING WELL

Easter, 62, 64 value of, 55 intention, see edict of inten-

edges, 351 functors, 190 tion
edict future, 392 interpreter, 25
of compartmentaliza- intersection, 423
tion, 170, 281, 344 garbage, 277 interval, 202
of decomposition, 100, garbage collection, 277 invariant, 168
333, 340 Gilles Kahn, 216 representation, 135
of intention, 34, 48, 60, goal state, 264 invocation, 341
65, 74, 103, 106, 118, golden ratio, 39, 157, 320 irredundancy, see edict of
150, 182, 231, 448 Gorn addresses, 157 irredundancy
of irredundancy, 91–93, grammar, 28 ISWIM, 326
103, 104, 106, 107, graph, 351 iterated prisoner’s dilemma,
144, 185, 189, 231, graph drawing, 351 88
343, 348 graphical objects, 354
of prevention, 42, 151, greatest common divisor, 18 judgement, 217
152, 169, 400 greedy search, 266
lambda calculus, 21, 418, 421
empty set, 423
hash collision, 291 Landin, Peter, 59, 64, 325, 326
enqueueing, 167
hash function, 291 large-step, 217
environment, 364
hash table, 291 lazy evaluation, 310
error value, 116
Haskell, 326 left associative, 31
Euclid’s algorithm, 20
head, 77 length, 80
evaluation, 37
heartbleed, 277 lexical environment, 370
exception, 120
expressions, 27 higher-order functional pro- lexical environment seman-

extensional, 423 gramming, 44 tics, 370

higher-order functions, 44 library modules, 201
factorial, 62, 417 Hoare, C. A. R., 64 linear probing, 291
Fibonacci sequence, 65, 315 Hope, 141 Lists, 77
field, 84 hypotenuse, 422 literals, 25
field punning, 86 local, 49
filter, 99 identity, 424 local open, 172
first-class values, 44 identity function, 105 location, 377
first-in-first-out, 167 imperative programming, 272 lock, 398
fold, 97 implementation, 169 Luhn check, 110
force, 310 implicitly typed, 44
force-directed graph layout, in-band signaling, 116 magic number, 433
352 in-place, 302 map, 93
forces, 352 infix, 57 MapReduce, 99
fork, 388 inherit, 343 Marx, Groucho, 30
formal, 216 initial state, 264 masses, 352, 353
formal verification, 66 insertion sort, 243 median, 115
free, 225 instance variables, 341 membership, 423
free variables, 226 instantiation, 341 memoization, 316
function intensional, 423 memory corruption, 277
 INDEX 505

memory leak, 277 polymorphic function, 105 sequential computation, 385

merge sort, 244 polymorphic type, 105 series acceleration, 320
metacircular interpreter, 239, postfix, 77 shadowing, 50
401 precedence, 31 sharing constraints, 187
metalanguage, 220 prefix, 57 side effect, 271
methods, 341 premature optimization, 241 signature, 169
ML, 141 preorder, 156 slope, 422
modules, 170 prevention, see edict of pre- small-step semantics, 217
monad, 120 vention sorting, 242
move, 264 prisoner’s dilemma, 88 space efficiency, 295
mutex locks, 398 procedural programming, 295 stack frame, 297
procedure, 275 stages, 401
natural semantics, 216 prompt, 25 state, 264
Naur, Peter, 64 pure, 271 state variable, 17
negation, 422, 424 pyramid of doom, 120 statically typed, 41
neighbor, 266
store, 377
nil, 77 quadratic probing, 293
strategy, 88
nodes, 351 queue, 167
stream, 311
numerical solution, 157 quicksort, 64, 245, 302
strings
concatenation of, 39
object, 341 race condition, 386
strongly typed, 41
object language, 220 read-after-write dependency,
structural equality, 276
object-oriented, 340 386
structure-driven program-
object-oriented program- read-eval-print loop, 26
ming, 82
ming, 334 records, 84
subclass, 343
OCaml, 22 recur, 18
substitution semantics, 224
open expressions, 229 recurrence equations, 253
operational semantics, 217 recurse, see recur superclass, 343

operators recursion, 18 supertype, 348

defining new, 110 refactoring, 100 symbolic solution, 157

option poisoning, 120 reference, 273 syntactic sugar, 59
optional chaining, 120 reference types, 273 syntax, 27
order of operations, 31 R E P L , 26, see read-eval-print
ordered type, 175 loop tail, 77

representation invariant, 135 tail recursion, 299

parallel computation, 385 rest length, 352 optimization, 299
parentheses, 31 reverse index, 178 Taylor series, 318
partial application, 95 right associative, 31 tesseract, 95
partial sum, 319 right triangle, 422 tesseractic numbers, 95
physical equality, 276 thread, 386
π, 47 scope, 49 thunk, 316
π, 423 search, 264 timeout, 322
pointers, 276 search tree, 265 tit-for-tat, 88
points, 353 semantics, 29 truth values, 39
506 PROGRAMMING WELL

tuple, 71 unfolding, 254 weak type variables, 112

Turing Award, 17, 22, 64, 133, union, 423 weighted sum, 109
158, 169, 241 unit testing, 66 widgets, 333
Turing machine, 17, 17, 21 University of Edinburgh, 141 wild-card, 77
Turing, Alan, 17, 21 update, 273 worst-case complexity, 243
type constructor, 71 write-after-read dependency,
type expressions, 42 value 386
type inference, 44 of a function, 55
write-after-writedependency,
type variables, 105 value constructor, 71
386
weak, 112 values, 218
typed, 40 variable capture, 235
typing, 43 variant type, 142 zeros, 163
Image Credits

Figure 1.1. Trial model of a part of the Analytical Engine, built

by Babbage, as displayed at the Science Museum (London),
Bruno Barral. CC-BY-SA 2.5, courtesy of Wikipedia. . . . . . . . 16
Figure 1.3. Passport photo of Alan Turing. Public domain; cour-
tesy of Wikipedia. . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Figure 1.4. T. L. Heath, translator. 1908. The Thirteen books of
Euclid’s Elements. Cambridge: Cambridge University Press,
page 298. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Figure 1.6. Photo of Alonzo Church, copyright (presumed) Prince-
ton University. Used under fair use, courtesy of Wikipedia. . . 21
Figure 1.7. Photo of Robin Milner, University of Cambridge. . . . . 22
Figure 4.2. Explosion of first Ariane 5 flight, June 4, 1996. Copy-
right European Space Agency; used by permission. . . . . . . . 40
Figure 6.1. Photo of Moses Schönfinkel as a student in 1910. Pub-
lic domain, courtesy of Wikimedia Commons. . . . . . . . . . . 56
Figure 6.2. Photo of Haskell B. Curry, Gleb.svechnikov. CC-BY-SA
4.0, courtesy of Wikipedia. . . . . . . . . . . . . . . . . . . . . . 56
Figure 7.1. Photo of Marianne Baudinet, courtesy of LinkedIn. . . 73
Figure 9.1. Image of J. Roger Hindley, cached by Internet Archive
from University of Swansea Mathematics Department, August
6, 2002. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
Figure 10.1. Portrait of Whitfield Diffie by Duncan Hall, courtesy
of Wikimedia, licensed under CC BY-SA 4.0. Portrait of Martin
Hellman, courtesy of Wikimedia, licensed under CC BY-SA 3.0. 133
Figure 11.4. Photo of Saul Gorn courtesy of University of Pennsyl-
vania Archives and Records Center. Permission pending. . . . 157
Figure 11.5. Daguerrotype of Augusta Ada King, Countess of
Lovelace by Antoine Claudet about 1843. Work in the public
domain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
Figure 11.6. Photograph of John McCarthy. Used by implicit per-
mission of the author. . . . . . . . . . . . . . . . . . . . . . . . . 158
Figure 11.7. Scanned photograph of Jean Sammet. Copyright
2018 Mount Holyoke College, used under fair use. . . . . . . . . 159
508 PROGRAMMING WELL

Figure 12.2. Photo of Barbara Liskov by Kenneth C. Zirkel. CC-BY-

SA 3.0; courtesy of Wikimedia Commons. . . . . . . . . . . . . . 169
Figure 12.4. Alexander Calder, “L’empennage”, 1953. Photo by
Finlay McWalter, courtesy of Wikimedia Commons, used by
permission. CC-by-sa 3.0. . . . . . . . . . . . . . . . . . . . . . . 203
Figure 13.1. Portrait of Gottfried Wilhelm Leibniz by Christoph
Bernhard Francke in the Herzog Anton Ulrich-Museum
Braunschweig. Public domain, courtesy of Wikimedia Com-
mons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
Figure 13.2. Photograph of Gilles Kahn by F. Jannin. Copyright
INRIA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
Figure 14.1. Portrait of Donald Knuth, copyright 2010 Photo-
graphic Unit, University of Glasgow. . . . . . . . . . . . . . . . . 241
Figure 15.2. Heartbleed logo by Leena Snidate/Codenomicon, in
the public domain by CC0 dedication, courtesy of Wikipedia. . 277
Figure 17.3. Small copy of a portrait of Brook Taylor, the English
mathematician. 1720. National Portrait Gallery (London).
Public domain, courtesy of Wikipedia. . . . . . . . . . . . . . . 318
Figure 17.7. Photo of Peter Landin, courtesy of Wikipedia. . . . . . 325
Figure 18.3. Photo of Alan Kay, by Alan Kay, used under a CC-by
2.0 Generic license. Courtesy of Wikimedia Commons. Photo
of Adele Goldberg, by Terry Hancock, used under a CC-by-sa
2.5 Generic license. Courtesy of Wikimedia Commons. Photo
of Dan Ingalls, used under a CC-by-sa 3.0 Unported license.
Courtesy of Wikimedia Commons. . . . . . . . . . . . . . . . . . 334
Figure 20.1. Figure from Moore (1965). . . . . . . . . . . . . . . . . . 383
Figure 20.1. Computer performance data from Wikipedia. Used
by permission (CC-BY-SA). . . . . . . . . . . . . . . . . . . . . . 384
Figure A.0. Richard G. Brown, Mary P. Dolciani, Robert H. Sor-
genfrey, and William L. Cole. Algebra: Structure and Method:
Book 1: California Edition. Evanston Illinois: McDougal Lit-
tell, 2000. Page 379. . . . . . . . . . . . . . . . . . . . . . . . . . . 418
Figure A.2. Portrait of Leonhard Euler (1707-1783). Jakob
Emanuel Handmann. Public domain; courtesy of Wikipedia. . 418
Figure A.2. Selection from Leonhard Euler [Leonh. Eulero]. 1734-
5. Additamentum ad Dissertationem de Infinitis Curvis Eius-
dem Generis [An Addition to the Dissertation Concerning an
Infinite Number of Curves of the Same Kind]. In Commentarii
Academiae Scientarium Imperialis Petropolitanae [Memoirs
of the Imperial Academy of Sciences in St. Petersburg], Volume
VII (1734-35). Petropoli, Typis Academiae, 1740. Photo of the
author. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419
 I M AG E C R E D I TS 509

Figure B.0. XKCD comic 1513, “Code Quality”, by Randall Munro.

CC-BY-NC 2.5, courtesy of Randall Munro. . . . . . . . . . . . . 425