All Categories

Begin Reading

Table of Contents

About the Authors

Copyright Page

Thank you for buying this

Henry Holt and Company ebook.

To receive special offers, bonus content,

and info on new releases and other great reads,

sign up for our newsletters.

Or visit us online at

us.macmillan.com/newslettersignup

For email updates on Brian Christian, click here.

For email updates on Tom Griffiths, click here.

The author and publisher have provided this e-book to you for your personal use

only. You may not make this e-book publicly available in any way. Copyright

infringement is against the law. If you believe the copy of this e-book you

are reading infringes on the author’s copyright, please notify the publisher

at: us.macmillanusa.com/piracy.

For our families

Introduction

Algorithms to Live By

Imagine you’re searching for an apartment in San Francisco—arguably the most

harrowing American city in which to do so. The booming tech sector and tight

zoning laws limiting new construction have conspired to make the city just as

expensive as New York, and by many accounts more competitive. New listings

go up and come down within minutes, open houses are mobbed, and often the

keys end up in the hands of whoever can physically foist a deposit check on the

landlord first.

Such a savage market leaves little room for the kind of fact-finding and

deliberation that is theoretically supposed to characterize the doings of the

rational consumer. Unlike, say, a mall patron or an online shopper, who can

compare options before making a decision, the would-be San Franciscan has to

decide instantly either way: you can take the apartment you are currently looking

at, forsaking all others, or you can walk away, never to return.

Let’s assume for a moment, for the sake of simplicity, that you care only

about maximizing your chance of getting the very best apartment available. Your

goal is reducing the twin, Scylla-and-Charybdis regrets of the “one that got

away” and the “stone left unturned” to the absolute minimum. You run into a

dilemma right off the bat: How are you to know that an apartment is indeed the

best unless you have a baseline to judge it by? And how are you to establish that

baseline unless you look at (and lose) a number of apartments? The more

information you gather, the better you’ll know the right opportunity when you

see it—but the more likely you are to have already passed it by.

So what do you do? How do you make an informed decision when the very

act of informing it jeopardizes the outcome? It’s a cruel situation, bordering on

paradox.

When presented with this kind of problem, most people will intuitively say

something to the effect that it requires some sort of balance between looking and

leaping—that you must look at enough apartments to establish a standard, then

take whatever satisfies the standard you’ve established. This notion of balance

is, in fact, precisely correct. What most people don’t say with any certainty is

what that balance is. Fortunately, there’s an answer.

Thirty-seven percent.

If you want the best odds of getting the best apartment, spend 37% of your

apartment hunt (eleven days, if you’ve given yourself a month for the search)

noncommittally exploring options. Leave the checkbook at home; you’re just

calibrating. But after that point, be prepared to immediately commit—deposit

and all—to the very first place you see that beats whatever you’ve already seen.

This is not merely an intuitively satisfying compromise between looking and

leaping. It is the provably optimal solution.

We know this because finding an apartment belongs to a class of

mathematical problems known as “optimal stopping” problems. The 37% rule

defines a simple series of steps—what computer scientists call an “algorithm”—

for solving these problems. And as it turns out, apartment hunting is just one of

the ways that optimal stopping rears its head in daily life. Committing to or

forgoing a succession of options is a structure that appears in life again and

again, in slightly different incarnations. How many times to circle the block

before pulling into a parking space? How far to push your luck with a risky

business venture before cashing out? How long to hold out for a better offer on

that house or car?

The same challenge also appears in an even more fraught setting: dating.

Optimal stopping is the science of serial monogamy.

Simple algorithms offer solutions not only to an apartment hunt but to all such

situations in life where we confront the question of optimal stopping. People

grapple with these issues every day—although surely poets have spilled more

ink on the tribulations of courtship than of parking—and they do so with, in

some cases, considerable anguish. But the anguish is unnecessary.

Mathematically, at least, these are solved problems.

Every harried renter, driver, and suitor you see around you as you go through

a typical week is essentially reinventing the wheel. They don’t need a therapist;

they need an algorithm. The therapist tells them to find the right, comfortable

balance between impulsivity and overthinking.

The algorithm tells them the balance is thirty-seven percent.

*

There is a particular set of problems that all people face, problems that are a

direct result of the fact that our lives are carried out in finite space and time.

What should we do, and leave undone, in a day or in a decade? What degree of

mess should we embrace—and how much order is excessive? What balance

between new experiences and favored ones makes for the most fulfilling life?

These might seem like problems unique to humans; they’re not. For more than

half a century, computer scientists have been grappling with, and in many cases

solving, the equivalents of these everyday dilemmas. How should a processor

allocate its “attention” to perform all that the user asks of it, with the minimum

overhead and in the least amount of time? When should it switch between

different tasks, and how many tasks should it take on in the first place? What is

the best way for it to use its limited memory resources? Should it collect more

data, or take an action based on the data it already has? Seizing the day might be

a challenge for humans, but computers all around us are seizing milliseconds

with ease. And there’s much we can learn from how they do it.

Talking about algorithms for human lives might seem like an odd

juxtaposition. For many people, the word “algorithm” evokes the arcane and

inscrutable machinations of big data, big government, and big business:

increasingly part of the infrastructure of the modern world, but hardly a source

of practical wisdom or guidance for human affairs. But an algorithm is just a

finite sequence of steps used to solve a problem, and algorithms are much

broader—and older by far—than the computer. Long before algorithms were

ever used by machines, they were used by people.

The word “algorithm” comes from the name of Persian mathematician alKhwārizmī, author of a ninth-century book of techniques for doing mathematics

by hand. (His book was called al-Jabr wa’l-Muqābala—and the “al-jabr” of the

title in turn provides the source of our word “algebra.”) The earliest known

mathematical algorithms, however, predate even al-Khwārizmī’s work: a fourthousand-year-old Sumerian clay tablet found near Baghdad describes a scheme

for long division.

But algorithms are not confined to mathematics alone. When you cook bread

from a recipe, you’re following an algorithm. When you knit a sweater from a

pattern, you’re following an algorithm. When you put a sharp edge on a piece of

flint by executing a precise sequence of strikes with the end of an antler—a key

step in making fine stone tools—you’re following an algorithm. Algorithms

have been a part of human technology ever since the Stone Age.

*

In this book, we explore the idea of human algorithm design—searching for

better solutions to the challenges people encounter every day. Applying the lens

of computer science to everyday life has consequences at many scales. Most

immediately, it offers us practical, concrete suggestions for how to solve specific

problems. Optimal stopping tells us when to look and when to leap. The

explore/exploit tradeoff tells us how to find the balance between trying new

things and enjoying our favorites. Sorting theory tells us how (and whether) to

arrange our offices. Caching theory tells us how to fill our closets. Scheduling

theory tells us how to fill our time.

At the next level, computer science gives us a vocabulary for understanding

the deeper principles at play in each of these domains. As Carl Sagan put it,

“Science is a way of thinking much more than it is a body of knowledge.” Even

in cases where life is too messy for us to expect a strict numerical analysis or a

ready answer, using intuitions and concepts honed on the simpler forms of these

problems offers us a way to understand the key issues and make progress.

Most broadly, looking through the lens of computer science can teach us

about the nature of the human mind, the meaning of rationality, and the oldest

question of all: how to live. Examining cognition as a means of solving the

fundamentally computational problems posed by our environment can utterly

change the way we think about human rationality.

The notion that studying the inner workings of computers might reveal how to

think and decide, what to believe and how to behave, might strike many people

as not only wildly reductive, but in fact misguided. Even if computer science did

have things to say about how to think and how to act, would we want to listen?

We look at the AIs and robots of science fiction, and it seems like theirs is not a

life any of us would want to live.

In part, that’s because when we think about computers, we think about coldly

mechanical, deterministic systems: machines applying rigid deductive logic,

making decisions by exhaustively enumerating the options, and grinding out the

exact right answer no matter how long and hard they have to think. Indeed, the

person who first imagined computers had something essentially like this in mind.

Alan Turing defined the very notion of computation by an analogy to a human

mathematician who carefully works through the steps of a lengthy calculation,

yielding an unmistakably right answer.

So it might come as a surprise that this is not what modern computers are

actually doing when they face a difficult problem. Straightforward arithmetic, of

course, isn’t particularly challenging for a modern computer. Rather, it’s tasks

like conversing with people, fixing a corrupted file, or winning a game of Go—

problems where the rules aren’t clear, some of the required information is

missing, or finding exactly the right answer would require considering an

astronomical number of possibilities—that now pose the biggest challenges in

computer science. And the algorithms that researchers have developed to solve

the hardest classes of problems have moved computers away from an extreme

reliance on exhaustive calculation. Instead, tackling real-world tasks requires

being comfortable with chance, trading off time with accuracy, and using

approximations.

As computers become better tuned to real-world problems, they provide not

only algorithms that people can borrow for their own lives, but a better standard

against which to compare human cognition itself. Over the past decade or two,

behavioral economics has told a very particular story about human beings: that

we are irrational and error-prone, owing in large part to the buggy, idiosyncratic

hardware of the brain. This self-deprecating story has become increasingly

familiar, but certain questions remain vexing. Why are four-year-olds, for

instance, still better than million-dollar supercomputers at a host of cognitive

tasks, including vision, language, and causal reasoning?

The solutions to everyday problems that come from computer science tell a

different story about the human mind. Life is full of problems that are, quite

simply, hard. And the mistakes made by people often say more about the

intrinsic difficulties of the problem than about the fallibility of human brains.

Thinking algorithmically about the world, learning about the fundamental

structures of the problems we face and about the properties of their solutions,

can help us see how good we actually are, and better understand the errors that

we make.

In fact, human beings turn out to consistently confront some of the hardest

cases of the problems studied by computer scientists. Often, people need to make

decisions while dealing with uncertainty, time constraints, partial information,

and a rapidly changing world. In some of those cases, even cutting-edge

computer science has not yet come up with efficient, always-right algorithms.

For certain situations it appears that such algorithms might not exist at all.

Even where perfect algorithms haven’t been found, however, the battle

between generations of computer scientists and the most intractable real-world

problems has yielded a series of insights. These hard-won precepts are at odds

with our intuitions about rationality, and they don’t sound anything like the

narrow prescriptions of a mathematician trying to force the world into clean,

formal lines. They say: Don’t always consider all your options. Don’t

necessarily go for the outcome that seems best every time. Make a mess on

occasion. Travel light. Let things wait. Trust your instincts and don’t think too

long. Relax. Toss a coin. Forgive, but don’t forget. To thine own self be true.

Living by the wisdom of computer science doesn’t sound so bad after all. And

unlike most advice, it’s backed up by proofs.

*

Just as designing algorithms for computers was originally a subject that fell into

the cracks between disciplines—an odd hybrid of mathematics and engineering

—so, too, designing algorithms for humans is a topic that doesn’t have a natural

disciplinary home. Today, algorithm design draws not only on computer science,

math, and engineering but on kindred fields like statistics and operations

research. And as we consider how algorithms designed for machines might relate

to human minds, we also need to look to cognitive science, psychology,

economics, and beyond.

We, your authors, are familiar with this interdisciplinary territory. Brian

studied computer science and philosophy before going on to graduate work in

English and a career at the intersection of the three. Tom studied psychology and

statistics before becoming a professor at UC Berkeley, where he spends most of

his time thinking about the relationship between human cognition and

computation. But nobody can be an expert in all of the fields that are relevant to

designing better algorithms for humans. So as part of our quest for algorithms to

live by, we talked to the people who came up with some of the most famous

algorithms of the last fifty years. And we asked them, some of the smartest

people in the world, how their research influenced the way they approached their

own lives—from finding their spouses to sorting their socks.

The next pages begin our journey through some of the biggest challenges

faced by computers and human minds alike: how to manage finite space, finite

time, limited attention, unknown unknowns, incomplete information, and an

unforeseeable future; how to do so with grace and confidence; and how to do so

in a community with others who are all simultaneously trying to do the same.

We will learn about the fundamental mathematical structure of these challenges

and about how computers are engineered—sometimes counter to what we

imagine—to make the most of them. And we will learn about how the mind

works, about its distinct but deeply related ways of tackling the same set of

issues and coping with the same constraints. Ultimately, what we can gain is not

only a set of concrete takeaways for the problems around us, not only a new way

to see the elegant structures behind even the hairiest human dilemmas, not only a

recognition of the travails of humans and computers as deeply conjoined, but

something even more profound: a new vocabulary for the world around us, and a

chance to learn something truly new about ourselves.

1 Optimal Stopping

When to Stop Looking

Though all Christians start a wedding invitation by solemnly declaring

their marriage is due to special Divine arrangement, I, as a philosopher,

would like to talk in greater detail about this … —JOHANNES KEPLER

If you prefer Mr. Martin to every other person; if you think him the most

agreeable man you have ever been in company with, why should you

hesitate?

—JANE AUSTEN, EMMA

It’s such a common phenomenon that college guidance counselors even have a

slang term for it: the “turkey drop.” High-school sweethearts come home for

Thanksgiving of their freshman year of college and, four days later, return to

campus single.

An angst-ridden Brian went to his own college guidance counselor his

freshman year. His high-school girlfriend had gone to a different college several

states away, and they struggled with the distance. They also struggled with a

stranger and more philosophical question: how good a relationship did they

have? They had no real benchmark of other relationships by which to judge it.

Brian’s counselor recognized theirs as a classic freshman-year dilemma, and was

surprisingly nonchalant in her advice: “Gather data.”

The nature of serial monogamy, writ large, is that its practitioners are

confronted with a fundamental, unavoidable problem. When have you met

enough people to know who your best match is? And what if acquiring the data

costs you that very match? It seems the ultimate Catch-22 of the heart.

As we have seen, this Catch-22, this angsty freshman cri de coeur, is what

mathematicians call an “optimal stopping” problem, and it may actually have an

answer: 37%.

Of course, it all depends on the assumptions you’re willing to make about

love.

The Secretary Problem

In any optimal stopping problem, the crucial dilemma is not which option to

pick, but how many options to even consider. These problems turn out to have

implications not only for lovers and renters, but also for drivers, homeowners,

burglars, and beyond.

The 37% Rule* derives from optimal stopping’s most famous puzzle, which

has come to be known as the “secretary problem.” Its setup is much like the

apartment hunter’s dilemma that we considered earlier. Imagine you’re

interviewing a set of applicants for a position as a secretary, and your goal is to

maximize the chance of hiring the single best applicant in the pool. While you

have no idea how to assign scores to individual applicants, you can easily judge

which one you prefer. (A mathematician might say you have access only to the

ordinal numbers—the relative ranks of the applicants compared to each other—

but not to the cardinal numbers, their ratings on some kind of general scale.)

You interview the applicants in random order, one at a time. You can decide to

offer the job to an applicant at any point and they are guaranteed to accept,

terminating the search. But if you pass over an applicant, deciding not to hire

them, they are gone forever.

The secretary problem is widely considered to have made its first appearance

in print—sans explicit mention of secretaries—in the February 1960 issue of

Scientific American, as one of several puzzles posed in Martin Gardner’s

beloved column on recreational mathematics. But the origins of the problem are

surprisingly mysterious. Our own initial search yielded little but speculation,

before turning into unexpectedly physical detective work: a road trip down to the

archive of Gardner’s papers at Stanford, to haul out boxes of his midcentury

correspondence. Reading paper correspondence is a bit like eavesdropping on

someone who’s on the phone: you’re only hearing one side of the exchange, and

must infer the other. In our case, we only had the replies to what was apparently

Gardner’s own search for the problem’s origins fiftysome years ago. The more

we read, the more tangled and unclear the story became.

Harvard mathematician Frederick Mosteller recalled hearing about the

problem in 1955 from his colleague Andrew Gleason, who had heard about it

from somebody else. Leo Moser wrote from the University of Alberta to say that

he read about the problem in “some notes” by R. E. Gaskell of Boeing, who

himself credited a colleague. Roger Pinkham of Rutgers wrote that he first heard

of the problem in 1955 from Duke University mathematician J. Shoenfield, “and

I believe he said that he had heard the problem from someone at Michigan.”

“Someone at Michigan” was almost certainly someone named Merrill Flood.

Though he is largely unheard of outside mathematics, Flood’s influence on

computer science is almost impossible to avoid. He’s credited with popularizing

the traveling salesman problem (which we discuss in more detail in chapter 8),

devising the prisoner’s dilemma (which we discuss in chapter 11), and even with

possibly coining the term “software.” It’s Flood who made the first known

discovery of the 37% Rule, in 1958, and he claims to have been considering the

problem since 1949—but he himself points back to several other

mathematicians.

Suffice it to say that wherever it came from, the secretary problem proved to

be a near-perfect mathematical puzzle: simple to explain, devilish to solve,

succinct in its answer, and intriguing in its implications. As a result, it moved

like wildfire through the mathematical circles of the 1950s, spreading by word of

mouth, and thanks to Gardner’s column in 1960 came to grip the imagination of

the public at large. By the 1980s the problem and its variations had produced so

much analysis that it had come to be discussed in papers as a subfield unto itself.

As for secretaries—it’s charming to watch each culture put its own

anthropological spin on formal systems. We think of chess, for instance, as

medieval European in its imagery, but in fact its origins are in eighth-century

India; it was heavy-handedly “Europeanized” in the fifteenth century, as its

shahs became kings, its viziers turned to queens, and its elephants became

bishops. Likewise, optimal stopping problems have had a number of

incarnations, each reflecting the predominating concerns of its time. In the

nineteenth century such problems were typified by baroque lotteries and by

women choosing male suitors; in the early twentieth century by holidaying

motorists searching for hotels and by male suitors choosing women; and in the

paper-pushing, male-dominated mid-twentieth century, by male bosses choosing

female assistants. The first explicit mention of it by name as the “secretary

problem” appears to be in a 1964 paper, and somewhere along the way the name

stuck.

Whence 37%?

In your search for a secretary, there are two ways you can fail: stopping early

and stopping late. When you stop too early, you leave the best applicant

undiscovered. When you stop too late, you hold out for a better applicant who

doesn’t exist. The optimal strategy will clearly require finding the right balance

between the two, walking the tightrope between looking too much and not

enough.

If your aim is finding the very best applicant, settling for nothing less, it’s

clear that as you go through the interview process you shouldn’t even consider

hiring somebody who isn’t the best you’ve seen so far. However, simply being

the best yet isn’t enough for an offer; the very first applicant, for example, will

of course be the best yet by definition. More generally, it stands to reason that

the rate at which we encounter “best yet” applicants will go down as we proceed

in our interviews. For instance, the second applicant has a 50/50 chance of being

the best we’ve yet seen, but the fifth applicant only has a 1-in-5 chance of being

the best so far, the sixth has a 1-in-6 chance, and so on. As a result, best-yet

applicants will become steadily more impressive as the search continues (by

definition, again, they’re better than all those who came before)—but they will

also become more and more infrequent.

Okay, so we know that taking the first best-yet applicant we encounter (a.k.a.

the first applicant, period) is rash. If there are a hundred applicants, it also seems

hasty to make an offer to the next one who’s best-yet, just because she was better

than the first. So how do we proceed?

Intuitively, there are a few potential strategies. For instance, making an offer

the third time an applicant trumps everyone seen so far—or maybe the fourth

time. Or perhaps taking the next best-yet applicant to come along after a long

“drought”—a long streak of poor ones.

But as it happens, neither of these relatively sensible strategies comes out on

top. Instead, the optimal solution takes the form of what we’ll call the LookThen-Leap Rule: You set a predetermined amount of time for “looking”—that

is, exploring your options, gathering data—in which you categorically don’t

choose anyone, no matter how impressive. After that point, you enter the “leap”

phase, prepared to instantly commit to anyone who outshines the best applicant

you saw in the look phase.

We can see how the Look-Then-Leap Rule emerges by considering how the

secretary problem plays out in the smallest applicant pools. With just one

applicant the problem is easy to solve—hire her! With two applicants, you have

a 50/50 chance of success no matter what you do. You can hire the first applicant

(who’ll turn out to be the best half the time), or dismiss the first and by default

hire the second (who is also best half the time).

Add a third applicant, and all of a sudden things get interesting. The odds if

we hire at random are one-third, or 33%. With two applicants we could do no

better than chance; with three, can we? It turns out we can, and it all comes

down to what we do with the second interviewee. When we see the first

applicant, we have no information—she’ll always appear to be the best yet.

When we see the third applicant, we have no agency—we have to make an offer

to the final applicant, since we’ve dismissed the others. But when we see the

second applicant, we have a little bit of both: we know whether she’s better or

worse than the first, and we have the freedom to either hire or dismiss her. What

happens when we just hire her if she’s better than the first applicant, and dismiss

her if she’s not? This turns out to be the best possible strategy when facing three

applicants; using this approach it’s possible, surprisingly, to do just as well in the

three-applicant problem as with two, choosing the best applicant exactly half the

time.*

Enumerating these scenarios for four applicants tells us that we should still

begin to leap as soon as the second applicant; with five applicants in the pool, we

shouldn’t leap before the third.

As the applicant pool grows, the exact place to draw the line between looking

and leaping settles to 37% of the pool, yielding the 37% Rule: look at the first

37% of the applicants,* choosing none, then be ready to leap for anyone better

than all those you’ve seen so far.

How to optimally choose a secretary.

As it turns out, following this optimal strategy ultimately gives us a 37%

chance of hiring the best applicant; it’s one of the problem’s curious

mathematical symmetries that the strategy itself and its chance of success work

out to the very same number. The table above shows the optimal strategy for the

secretary problem with different numbers of applicants, demonstrating how the

chance of success—like the point to switch from looking to leaping—converges

on 37% as the number of applicants increases.

A 63% failure rate, when following the best possible strategy, is a sobering

fact. Even when we act optimally in the secretary problem, we will still fail most

of the time—that is, we won’t end up with the single best applicant in the pool.

This is bad news for those of us who would frame romance as a search for “the

one.” But here’s the silver lining. Intuition would suggest that our chances of

picking the single best applicant should steadily decrease as the applicant pool

grows. If we were hiring at random, for instance, then in a pool of a hundred

applicants we’d have a 1% chance of success, and in a pool of a million

applicants we’d have a 0.0001% chance. Yet remarkably, the math of the

secretary problem doesn’t change. If you’re stopping optimally, your chance of

finding the single best applicant in a pool of a hundred is 37%. And in a pool of

a million, believe it or not, your chance is still 37%. Thus the bigger the

applicant pool gets, the more valuable knowing the optimal algorithm becomes.

It’s true that you’re unlikely to find the needle the majority of the time, but

optimal stopping is your best defense against the haystack, no matter how large.

Lover’s Leap

The passion between the sexes has appeared in every age to be so nearly

the same that it may always be considered, in algebraic language, as a

given quantity.

—THOMAS MALTHUS

I married the first man I ever kissed. When I tell this to my children they

just about throw up.

—BARBARA BUSH

Before he became a professor of operations research at Carnegie Mellon,

Michael Trick was a graduate student, looking for love. “It hit me that the

problem has been studied: it is the Secretary Problem! I had a position to fill

[and] a series of applicants, and my goal was to pick the best applicant for the

position.” So he ran the numbers. He didn’t know how many women he could

expect to meet in his lifetime, but there’s a certain flexibility in the 37% Rule: it

can be applied to either the number of applicants or the time over which one is

searching. Assuming that his search would run from ages eighteen to forty, the

37% Rule gave age 26.1 years as the point at which to switch from looking to

leaping. A number that, as it happened, was exactly Trick’s age at the time. So

when he found a woman who was a better match than all those he had dated so

far, he knew exactly what to do. He leapt. “I didn’t know if she was Perfect (the

assumptions of the model don’t allow me to determine that), but there was no

doubt that she met the qualifications for this step of the algorithm. So I

proposed,” he writes.

“And she turned me down.”

Mathematicians have been having trouble with love since at least the

seventeenth century. The legendary astronomer Johannes Kepler is today

perhaps best remembered for discovering that planetary orbits are elliptical and

for being a crucial part of the “Copernican Revolution” that included Galileo and

Newton and upended humanity’s sense of its place in the heavens. But Kepler

had terrestrial concerns, too. After the death of his first wife in 1611, Kepler

embarked on a long and arduous quest to remarry, ultimately courting a total of

eleven women. Of the first four, Kepler liked the fourth the best (“because of her

tall build and athletic body”) but did not cease his search. “It would have been

settled,” Kepler wrote, “had not both love and reason forced a fifth woman on

me. This one won me over with love, humble loyalty, economy of household,

diligence, and the love she gave the stepchildren.”

“However,” he wrote, “I continued.”

Kepler’s friends and relations went on making introductions for him, and he

kept on looking, but halfheartedly. His thoughts remained with number five.

After eleven courtships in total, he decided he would search no further. “While

preparing to travel to Regensburg, I returned to the fifth woman, declared

myself, and was accepted.” Kepler and Susanna Reuttinger were wed and had

six children together, along with the children from Kepler’s first marriage.

Biographies describe the rest of Kepler’s domestic life as a particularly peaceful

and joyous time.

Both Kepler and Trick—in opposite ways—experienced firsthand some of the

ways that the secretary problem oversimplifies the search for love. In the

classical secretary problem, applicants always accept the position, preventing the

rejection experienced by Trick. And they cannot be “recalled” once passed over,

contrary to the strategy followed by Kepler.

In the decades since the secretary problem was first introduced, a wide range

of variants on the scenario have been studied, with strategies for optimal

stopping worked out under a number of different conditions. The possibility of

rejection, for instance, has a straightforward mathematical solution: propose

early and often. If you have, say, a 50/50 chance of being rejected, then the same

kind of mathematical analysis that yielded the 37% Rule says you should start

making offers after just a quarter of your search. If turned down, keep making

offers to every best-yet person you see until somebody accepts. With such a

strategy, your chance of overall success—that is, proposing and being accepted

by the best applicant in the pool—will also be 25%. Not such terrible odds,

perhaps, for a scenario that combines the obstacle of rejection with the general

difficulty of establishing one’s standards in the first place.

Kepler, for his part, decried the “restlessness and doubtfulness” that pushed

him to keep on searching. “Was there no other way for my uneasy heart to be

content with its fate,” he bemoaned in a letter to a confidante, “than by realizing

the impossibility of the fulfillment of so many other desires?” Here, again,

optimal stopping theory provides some measure of consolation. Rather than

being signs of moral or psychological degeneracy, restlessness and doubtfulness

actually turn out to be part of the best strategy for scenarios where second

chances are possible. If you can recall previous applicants, the optimal algorithm

puts a twist on the familiar Look-Then-Leap Rule: a longer noncommittal

period, and a fallback plan.

For example, assume an immediate proposal is a sure thing but belated

proposals are rejected half the time. Then the math says you should keep looking

noncommittally until you’ve seen 61% of applicants, and then only leap if

someone in the remaining 39% of the pool proves to be the best yet. If you’re

still single after considering all the possibilities—as Kepler was—then go back

to the best one that got away. The symmetry between strategy and outcome

holds in this case once again, with your chances of ending up with the best

applicant under this second-chances-allowed scenario also being 61%.

For Kepler, the difference between reality and the classical secretary problem

brought with it a happy ending. In fact, the twist on the classical problem worked

out well for Trick, too. After the rejection, he completed his degree and took a

job in Germany. There, he “walked into a bar, fell in love with a beautiful

woman, moved in together three weeks later, [and] invited her to live in the

United States ‘for a while.’” She agreed—and six years later, they were wed.

Knowing a Good Thing When You See It: Full

Information

The first set of variants we considered—rejection and recall—altered the

classical secretary problem’s assumptions that timely proposals are always

accepted, and tardy proposals, never. For these variants, the best approach

remained the same as in the original: look noncommittally for a time, then be

ready to leap.

But there’s an even more fundamental assumption of the secretary problem

that we might call into question. Namely, in the secretary problem we know

nothing about the applicants other than how they compare to one another. We

don’t have an objective or preexisting sense of what makes for a good or a bad

applicant; moreover, when we compare two of them, we know which of the two

is better, but not by how much. It’s this fact that gives rise to the unavoidable

“look” phase, in which we risk passing up a superb early applicant while we

calibrate our expectations and standards. Mathematicians refer to this genre of

optimal stopping problems as “no-information games.”

This setup is arguably a far cry from most searches for an apartment, a

partner, or even a secretary. Imagine instead that we had some kind of objective

criterion—if every secretary, for instance, had taken a typing exam scored by

percentile, in the fashion of the SAT or GRE or LSAT. That is, every applicant’s

score will tell us where they fall among all the typists who took the test: a 51stpercentile typist is just above average, a 75th-percentile typist is better than three

test takers out of four, and so on.

Suppose that our applicant pool is representative of the population at large and

isn’t skewed or self-selected in any way. Furthermore, suppose we decide that

typing speed is the only thing that matters about our applicants. Then we have

what mathematicians call “full information,” and everything changes. “No

buildup of experience is needed to set a standard,” as the seminal 1966 paper on

the problem put it, “and a profitable choice can sometimes be made

immediately.” In other words, if a 95th-percentile applicant happens to be the

first one we evaluate, we know it instantly and can confidently hire her on the

spot—that is, of course, assuming we don’t think there’s a 96th-percentile

applicant in the pool.

And there’s the rub. If our goal is, again, to get the single best person for the

job, we still need to weigh the likelihood that there’s a stronger applicant out

there. However, the fact that we have full information gives us everything we

need to calculate those odds directly. The chance that our next applicant is in the

96th percentile or higher will always be 1 in 20, for instance. Thus the decision

of whether to stop comes down entirely to how many applicants we have left to

see. Full information means that we don’t need to look before we leap. We can

instead use the Threshold Rule, where we immediately accept an applicant if

she is above a certain percentile. We don’t need to look at an initial group of

candidates to set this threshold—but we do, however, need to be keenly aware of

how much looking remains available.

The math shows that when there are a lot of applicants left in the pool, you

should pass up even a very good applicant in the hopes of finding someone still

better than that—but as your options dwindle, you should be prepared to hire

anyone who’s simply better than average. It’s a familiar, if not exactly inspiring,

message: in the face of slim pickings, lower your standards. It also makes clear

the converse: with more fish in the sea, raise them. In both cases, crucially, the

math tells you exactly by how much.

The easiest way to understand the numbers for this scenario is to start at the

end and think backward. If you’re down to the last applicant, of course, you are

necessarily forced to choose her. But when looking at the next-to-last applicant,

the question becomes: is she above the 50th percentile? If yes, then hire her; if

not, it’s worth rolling the dice on the last applicant instead, since her odds of

being above the 50th percentile are 50/50 by definition. Likewise, you should

choose the third-to-last applicant if she’s above the 69th percentile, the fourth-tolast applicant if she’s above the 78th, and so on, being more choosy the more

applicants are left. No matter what, never hire someone who’s below average

unless you’re totally out of options. (And since you’re still interested only in

finding the very best person in the applicant pool, never hire someone who isn’t

the best you’ve seen so far.)

The chance of ending up with the single best applicant in this full-information

version of the secretary problem comes to 58%—still far from a guarantee, but

considerably better than the 37% success rate offered by the 37% Rule in the noinformation game. If you have all the facts, you can succeed more often than not,

even as the applicant pool grows arbitrarily large.

Optimal stopping thresholds in the full-information secretary problem.

The full-information game thus offers an unexpected and somewhat bizarre

takeaway. Gold digging is more likely to succeed than a quest for love. If you’re

evaluating your partners based on any kind of objective criterion—say, their

income percentile—then you’ve got a lot more information at your disposal than

if you’re after a nebulous emotional response (“love”) that might require both

experience and comparison to calibrate.

Of course, there’s no reason that net worth—or, for that matter, typing speed

—needs to be the thing that you’re measuring. Any yardstick that provides full

information on where an applicant stands relative to the population at large will

change the solution from the Look-Then-Leap Rule to the Threshold Rule and

will dramatically boost your chances of finding the single best applicant in the

group.

There are many more variants of the secretary problem that modify its other

assumptions, perhaps bringing it more in line with the real-world challenges of

finding love (or a secretary). But the lessons to be learned from optimal stopping

aren’t limited to dating or hiring. In fact, trying to make the best choice when

options only present themselves one by one is also the basic structure of selling a

house, parking a car, and quitting when you’re ahead. And they’re all, to some

degree or other, solved problems.

When to Sell

If we alter two more aspects of the classical secretary problem, we find ourselves

catapulted from the realm of dating to the realm of real estate. Earlier we talked

about the process of renting an apartment as an optimal stopping problem, but

owning a home has no shortage of optimal stopping either.

Imagine selling a house, for instance. After consulting with several real estate

agents, you put your place on the market; a new coat of paint, some landscaping,

and then it’s just a matter of waiting for the offers to come in. As each offer

arrives, you typically have to decide whether to accept it or turn it down. But

turning down an offer comes at a cost—another week (or month) of mortgage

payments while you wait for the next offer, which isn’t guaranteed to be any

better.

Selling a house is similar to the full-information game. We know the objective

dollar value of the offers, telling us not only which ones are better than which,

but also by how much. What’s more, we have information about the broader

state of the market, which enables us to at least roughly predict the range of

offers to expect. (This gives us the same “percentile” information about each

offer that we had with the typing exam above.) The difference here, however, is

that our goal isn’t actually to secure the single best offer—it’s to make the most

money through the process overall. Given that waiting has a cost measured in

dollars, a good offer today beats a slightly better one several months from now.

Having this information, we don’t need to look noncommittally to set a

threshold. Instead, we can set one going in, ignore everything below it, and take

the first option to exceed it. Granted, if we have a limited amount of savings that

will run out if we don’t sell by a certain time, or if we expect to get only a

limited number of offers and no more interest thereafter, then we should lower

our standards as such limits approach. (There’s a reason why home buyers look

for “motivated” sellers.) But if neither concern leads us to believe that our backs

are against the wall, then we can simply focus on a cost-benefit analysis of the

waiting game.

Here we’ll analyze one of the simplest cases: where we know for certain the

price range in which offers will come, and where all offers within that range are

equally likely. If we don’t have to worry about the offers (or our savings)

running out, then we can think purely in terms of what we can expect to gain or

lose by waiting for a better deal. If we decline the current offer, will the chance

of a better one, multiplied by how much better we expect it to be, more than

compensate for the cost of the wait? As it turns out, the math here is quite clean,

giving us an explicit function for stopping price as a function of the cost of

waiting for an offer.

This particular mathematical result doesn’t care whether you’re selling a

mansion worth millions or a ramshackle shed. The only thing it cares about is

the difference between the highest and lowest offers you’re likely to receive. By

plugging in some concrete figures, we can see how this algorithm offers us a

considerable amount of explicit guidance. For instance, let’s say the range of

offers we’re expecting runs from $400,000 to $500,000. First, if the cost of

waiting is trivial, we’re able to be almost infinitely choosy. If the cost of getting

another offer is only a dollar, we’ll maximize our earnings by waiting for

someone willing to offer us $499,552.79 and not a dime less. If waiting costs

$2,000 an offer, we should hold out for an even $480,000. In a slow market

where waiting costs $10,000 an offer, we should take anything over $455,279.

Finally, if waiting costs half or more of our expected range of offers—in this

case, $50,000—then there’s no advantage whatsoever to holding out; we’ll do

best by taking the very first offer that comes along and calling it done. Beggars

can’t be choosers.

Optimal stopping thresholds in the house-selling problem.

The critical thing to note in this problem is that our threshold depends only on

the cost of search. Since the chances of the next offer being a good one—and the

cost of finding out—never change, our stopping price has no reason to ever get

lower as the search goes on, regardless of our luck. We set it once, before we

even begin, and then we quite simply hold fast.

The University of Wisconsin–Madison’s Laura Albert McLay, an

optimization expert, recalls turning to her knowledge of optimal stopping

problems when it came time to sell her own house. “The first offer we got was

great,” she explains, “but it had this huge cost because they wanted us to move

out a month before we were ready. There was another competitive offer … [but]

we just kind of held out until we got the right one.” For many sellers, turning

down a good offer or two can be a nerve-racking proposition, especially if the

ones that immediately follow are no better. But McLay held her ground and

stayed cool. “That would have been really, really hard,” she admits, “if I didn’t

know the math was on my side.”

This principle applies to any situation where you get a series of offers and pay

a cost to seek or wait for the next. As a consequence, it’s relevant to cases that

go far beyond selling a house. For example, economists have used this algorithm

to model how people look for jobs, where it handily explains the otherwise

seemingly paradoxical fact of unemployed workers and unfilled vacancies

existing at the same time.

In fact, these variations on the optimal stopping problem have another, even

more surprising property. As we saw, the ability to “recall” a past opportunity

was vital in Kepler’s quest for love. But in house selling and job hunting, even if

it’s possible to reconsider an earlier offer, and even if that offer is guaranteed to

still be on the table, you should nonetheless never do so. If it wasn’t above your

threshold then, it won’t be above your threshold now. What you’ve paid to keep

searching is a sunk cost. Don’t compromise, don’t second-guess. And don’t look

back.

When to Park

I find that the three major administrative problems on a campus are sex for

the students, athletics for the alumni, and parking for the faculty.

—CLARK KERR, PRESIDENT OF UC BERKELEY, 1958–1967

Another domain where optimal stopping problems abound—and where looking

back is also generally ill-advised—is the car. Motorists feature in some of the

earliest literature on the secretary problem, and the framework of constant

forward motion makes almost every car-trip decision into a stopping problem:

the search for a restaurant; the search for a bathroom; and, most acutely for

urban drivers, the search for a parking space. Who better to talk to about the ins

and outs of parking than the man described by the Los Angeles Times as “the

parking rock star,” UCLA Distinguished Professor of Urban Planning Donald

Shoup? We drove down from Northern California to visit him, reassuring Shoup

that we’d be leaving plenty of time for unexpected traffic. “As for planning on

‘unexpected traffic,’ I think you should plan on expected traffic,” he replied.

Shoup is perhaps best known for his book The High Cost of Free Parking, and

he has done much to advance the discussion and understanding of what really

happens when someone drives to their destination.

We should pity the poor driver. The ideal parking space, as Shoup models it,

is one that optimizes a precise balance between the “sticker price” of the space,

the time and inconvenience of walking, the time taken seeking the space (which

varies wildly with destination, time of day, etc.), and the gas burned in doing so.

The equation changes with the number of passengers in the car, who can split the

monetary cost of a space but not the search time or the walk. At the same time,

the driver needs to consider that the area with the most parking supply may also

be the area with the most demand; parking has a game-theoretic component, as

you try to outsmart the other drivers on the road while they in turn are trying to

outsmart you.* That said, many of the challenges of parking boil down to a

single number: the occupancy rate. This is the proportion of all parking spots

that are currently occupied. If the occupancy rate is low, it’s easy to find a good

parking spot. If it’s high, finding anywhere at all to park is a challenge.

Shoup argues that many of the headaches of parking are consequences of

cities adopting policies that result in extremely high occupancy rates. If the cost

of parking in a particular location is too low (or—horrors!—nothing at all), then

there is a high incentive to park there, rather than to park a little farther away and

walk. So everybody tries to park there, but most of them find the spaces are

already full, and people end up wasting time and burning fossil fuel as they

cruise for a spot.

Shoup’s solution involves installing digital parking meters that are capable of

adaptive prices that rise with demand. (This has now been implemented in

downtown San Francisco.) The prices are set with a target occupancy rate in

mind, and Shoup argues that this rate should be somewhere around 85%—a

radical drop from the nearly 100%-packed curbs of most major cities. As he

notes, when occupancy goes from 90% to 95%, it accommodates only 5% more

cars but doubles the length of everyone’s search.

The key impact that occupancy rate has on parking strategy becomes clear

once we recognize that parking is an optimal stopping problem. As you drive

along the street, every time you see the occasional empty spot you have to make

a decision: should you take this spot, or go a little closer to your destination and

try your luck?

Assume you’re on an infinitely long road, with parking spots evenly spaced,

and your goal is to minimize the distance you end up walking to your

destination. Then the solution is the Look-Then-Leap Rule. The optimally

stopping driver should pass up all vacant spots occurring more than a certain

distance from the destination and then take the first space that appears thereafter.

And the distance at which to switch from looking to leaping depends on the

proportion of spots that are likely to be filled—the occupancy rate. The table on

the next page gives the distances for some representative proportions.

How to optimally find parking.

If this infinite street has a big-city occupancy rate of 99%, with just 1% of

spots vacant, then you should take the first spot you see starting at almost 70

spots—more than a quarter mile—from your destination. But if Shoup has his

way and occupancy rates drop to just 85%, you don’t need to start seriously

looking until you’re half a block away.

Most of us don’t drive on perfectly straight, infinitely long roads. So as with

other optimal stopping problems, researchers have considered a variety of

tweaks to this basic scenario. For instance, they have studied the optimal parking

strategy for cases where the driver can make U-turns, where fewer parking

spaces are available the closer one gets to the destination, and where the driver is

in competition against rival drivers also heading to the same destination. But

whatever the exact parameters of the problem, more vacant spots are always

going to make life easier. It’s something of a policy reminder to municipal

governments: parking is not as simple as having a resource (spots) and

maximizing its utilization (occupancy). Parking is also a process—an optimal

stopping problem—and it’s one that consumes attention, time, and fuel, and

generates both pollution and congestion. The right policy addresses the whole

problem. And, counterintuitively, empty spots on highly desirable blocks can be

the sign that things are working correctly.

We asked Shoup if his research allows him to optimize his own commute,

through the Los Angeles traffic to his office at UCLA. Does arguably the

world’s top expert on parking have some kind of secret weapon?

He does: “I ride my bike.”

When to Quit

In 1997, Forbes magazine identified Boris Berezovsky as the richest man in

Russia, with a fortune of roughly $3 billion. Just ten years earlier he had been

living on a mathematician’s salary from the USSR Academy of Sciences. He

made his billions by drawing on industrial relationships he’d formed through his

research to found a company that facilitated interaction between foreign

carmakers and the Soviet car manufacturer AvtoVAZ. Berezovky’s company

then became a large-scale dealer for the cars that AvtoVAZ produced, using a

payment installment scheme to take advantage of hyperinflation in the ruble.

Using the funds from this partnership he bought partial ownership of AvtoVAZ

itself, then the ORT Television network, and finally the Sibneft oil company.

Becoming one of a new class of oligarchs, he participated in politics, supporting

Boris Yeltsin’s re-election in 1996 and the choice of Vladimir Putin as his

successor in 1999.

But that’s when Berezovsky’s luck turned. Shortly after Putin’s election,

Berezovsky publicly objected to proposed constitutional reforms that would

expand the power of the president. His continued public criticism of Putin led to

the deterioration of their relationship. In October 2000, when Putin was asked

about Berezovsky’s criticisms, he replied, “The state has a cudgel in its hands

that you use to hit just once, but on the head. We haven’t used this cudgel yet.…

The day we get really angry, we won’t hesitate.” Berezovsky left Russia

permanently the next month, taking up exile in England, where he continued to

criticize Putin’s regime.

How did Berezovsky decide it was time to leave Russia? Is there a way,

perhaps, to think mathematically about the advice to “quit while you’re ahead”?

Berezovsky in particular might have considered this very question himself, since

the topic he had worked on all those years ago as a mathematician was none

other than optimal stopping; he authored the first (and, so far, the only) book

entirely devoted to the secretary problem.

The problem of quitting while you’re ahead has been analyzed under several

different guises, but perhaps the most appropriate to Berezovsky’s case—with

apologies to Russian oligarchs—is known as the “burglar problem.” In this

problem, a burglar has the opportunity to carry out a sequence of robberies. Each

robbery provides some reward, and there’s a chance of getting away with it each

time. But if the burglar is caught, he gets arrested and loses all his accumulated

gains. What algorithm should he follow to maximize his expected take?

The fact that this problem has a solution is bad news for heist movie

screenplays: when the team is trying to lure the old burglar out of retirement for

one last job, the canny thief need only crunch the numbers. Moreover, the results

are pretty intuitive: the number of robberies you should carry out is roughly

equal to the chance you get away, divided by the chance you get caught. If

you’re a skilled burglar and have a 90% chance of pulling off each robbery (and

a 10% chance of losing it all), then retire after 90/10 = 9 robberies. A ham-fisted

amateur with a 50/50 chance of success? The first time you have nothing to lose,

but don’t push your luck more than once.

Despite his expertise in optimal stopping, Berezovsky’s story ends sadly. He

died in March 2013, found by a bodyguard in the locked bathroom of his house

in Berkshire with a ligature around his neck. The official conclusion of a

postmortem examination was that he had committed suicide, hanging himself

after losing much of his wealth through a series of high-profile legal cases

involving his enemies in Russia. Perhaps he should have stopped sooner—

amassing just a few tens of millions of dollars, say, and not getting into politics.

But, alas, that was not his style. One of his mathematician friends, Leonid

Boguslavsky, told a story about Berezovsky from when they were both young

researchers: on a water-skiing trip to a lake near Moscow, the boat they had

planned to use broke down. Here’s how David Hoffman tells it in his book The

Oligarchs:

While their friends went to the beach and lit a bonfire, Boguslavsky and Berezovsky headed to the

dock to try to repair the motor.… Three hours later, they had taken apart and reassembled the motor.

It was still dead. They had missed most of the party, yet Berezovsky insisted they had to keep trying.

“We tried this and that,” Boguslavsky recalled. Berezovsky would not give up.

Surprisingly, not giving up—ever—also makes an appearance in the optimal

stopping literature. It might not seem like it from the wide range of problems we

have discussed, but there are sequential decision-making problems for which

there is no optimal stopping rule. A simple example is the game of “triple or

nothing.” Imagine you have $1.00, and can play the following game as many

times as you want: bet all your money, and have a 50% chance of receiving

triple the amount and a 50% chance of losing your entire stake. How many times

should you play? Despite its simplicity, there is no optimal stopping rule for this

problem, since each time you play, your average gains are a little higher. Starting

with $1.00, you will get $3.00 half the time and $0.00 half the time, so on

average you expect to end the first round with $1.50 in your pocket. Then, if you

were lucky in the first round, the two possibilities from the $3.00 you’ve just

won are $9.00 and $0.00—for an average return of $4.50 from the second bet.

The math shows that you should always keep playing. But if you follow this

strategy, you will eventually lose everything. Some problems are better avoided

than solved.

Always Be Stopping

I expect to pass through this world but once. Any good therefore that I can

do, or any kindness that I can show to any fellow creature, let me do it now.

Let me not defer or neglect it, for I shall not pass this way again.

—STEPHEN GRELLET

Spend the afternoon. You can’t take it with you.

—ANNIE DILLARD

We’ve looked at specific cases of people confronting stopping problems in their

lives, and it’s clear that most of us encounter these kinds of problems, in one

form or another, daily. Whether it involves secretaries, fiancé(e)s, or apartments,

life is full of optimal stopping. So the irresistible question is whether—by

evolution or education or intuition—we actually do follow the best strategies.

At first glance, the answer is no. About a dozen studies have produced the

same result: people tend to stop early, leaving better applicants unseen. To get a

better sense for these findings, we talked to UC Riverside’s Amnon Rapoport,

who has been running optimal stopping experiments in the laboratory for more

than forty years.

The study that most closely follows the classical secretary problem was run in

the 1990s by Rapoport and his collaborator Darryl Seale. In this study people

went through numerous repetitions of the secretary problem, with either 40 or 80

applicants each time. The overall rate at which people found the best possible

applicant was pretty good: about 31%, not far from the optimal 37%. Most

people acted in a way that was consistent with the Look-Then-Leap Rule, but

they leapt sooner than they should have more than four-fifths of the time.

Rapoport told us that he keeps this in mind when solving optimal stopping

problems in his own life. In searching for an apartment, for instance, he fights

his own urge to commit quickly. “Despite the fact that by nature I am very

impatient and I want to take the first apartment, I try to control myself!”

But that impatience suggests another consideration that isn’t taken into

account in the classical secretary problem: the role of time. After all, the whole

time you’re searching for a secretary, you don’t have a secretary. What’s more,

you’re spending the day conducting interviews instead of getting your own work

done.

This type of cost offers a potential explanation for why people stop early

when solving a secretary problem in the lab. Seale and Rapoport showed that if

the cost of seeing each applicant is imagined to be, for instance, 1% of the value

of finding the best secretary, then the optimal strategy would perfectly align with

where people actually switched from looking to leaping in their experiment.

The mystery is that in Seale and Rapoport’s study, there wasn’t a cost for

search. So why might people in the laboratory be acting like there was one?

Because for people there’s always a time cost. It doesn’t come from the design

of the experiment. It comes from people’s lives.

The “endogenous” time costs of searching, which aren’t usually captured by

optimal stopping models, might thus provide an explanation for why human

decision-making routinely diverges from the prescriptions of those models. As

optimal stopping researcher Neil Bearden puts it, “After searching for a while,

we humans just tend to get bored. It’s not irrational to get bored, but it’s hard to

model that rigorously.”

But this doesn’t make optimal stopping problems less important; it actually

makes them more important, because the flow of time turns all decision-making

into optimal stopping.

“The theory of optimal stopping is concerned with the problem of choosing a

time to take a given action,” opens the definitive textbook on optimal stopping,

and it’s hard to think of a more concise description of the human condition. We

decide the right time to buy stocks and the right time to sell them, sure; but also

the right time to open the bottle of wine we’ve been keeping around for a special

occasion, the right moment to interrupt someone, the right moment to kiss them.

Viewed this way, the secretary problem’s most fundamental yet most

unbelievable assumption—its strict seriality, its inexorable one-way march—is

revealed to be the nature of time itself. As such, the explicit premise of the

optimal stopping problem is the implicit premise of what it is to be alive. It’s this

that forces us to decide based on possibilities we’ve not yet seen, this that forces

us to embrace high rates of failure even when acting optimally. No choice

recurs. We may get similar choices again, but never that exact one. Hesitation—

inaction—is just as irrevocable as action. What the motorist, locked on the one-

way road, is to space, we are to the fourth dimension: we truly pass this way but

once.

Intuitively, we think that rational decision-making means exhaustively

enumerating our options, weighing each one carefully, and then selecting the

best. But in practice, when the clock—or the ticker—is ticking, few aspects of

decision-making (or of thinking more generally) are as important as this one:

when to stop.

2 Explore/Exploit

The Latest vs. the Greatest

Your stomach rumbles. Do you go to the Italian restaurant that you know and

love, or the new Thai place that just opened up? Do you take your best friend, or

reach out to a new acquaintance you’d like to get to know better? This is too

hard—maybe you’ll just stay home. Do you cook a recipe that you know is

going to work, or scour the Internet for new inspiration? Never mind, how about

you just order a pizza? Do you get your “usual,” or ask about the specials?

You’re already exhausted before you get to the first bite. And the thought of

putting on a record, watching a movie, or reading a book—which one?—no

longer seems quite so relaxing.

Every day we are constantly forced to make decisions between options that

differ in a very specific dimension: do we try new things or stick with our

favorite ones? We intuitively understand that life is a balance between novelty

and tradition, between the latest and the greatest, between taking risks and

savoring what we know and love. But just as with the look-or-leap dilemma of

the apartment hunt, the unanswered question is: what balance?

In the 1974 classic Zen and the Art of Motorcycle Maintenance, Robert Pirsig

decries the conversational opener “What’s new?”—arguing that the question, “if

pursued exclusively, results only in an endless parade of trivia and fashion, the

silt of tomorrow.” He endorses an alternative as vastly superior: “What’s best?”

But the reality is not so simple. Remembering that every “best” song and

restaurant among your favorites began humbly as something merely “new” to

you is a reminder that there may be yet-unknown bests still out there—and thus

that the new is indeed worthy of at least some of our attention.

Age-worn aphorisms acknowledge this tension but don’t solve it. “Make new

friends, but keep the old Those are silver, these are gold,” and “There is no life

so rich and rare But one more friend could enter there” are true enough;

certainly their scansion is unimpeachable. But they fail to tell us anything useful

about the ratio of, say, “silver” and “gold” that makes the best alloy of a life well

lived.

Computer scientists have been working on finding this balance for more than

fifty years. They even have a name for it: the explore/exploit tradeoff.

Explore/Exploit

In English, the words “explore” and “exploit” come loaded with completely

opposite connotations. But to a computer scientist, these words have much more

specific and neutral meanings. Simply put, exploration is gathering information,

and exploitation is using the information you have to get a known good result.

It’s fairly intuitive that never exploring is no way to live. But it’s also worth

mentioning that never exploiting can be every bit as bad. In the computer science

definition, exploitation actually comes to characterize many of what we consider

to be life’s best moments. A family gathering together on the holidays is

exploitation. So is a bookworm settling into a reading chair with a hot cup of

coffee and a beloved favorite, or a band playing their greatest hits to a crowd of

adoring fans, or a couple that has stood the test of time dancing to “their song.”

What’s more, exploration can be a curse.

Part of what’s nice about music, for instance, is that there are constantly new

things to listen to. Or, if you’re a music journalist, part of what’s terrible about

music is that there are constantly new things to listen to. Being a music journalist

means turning the exploration dial all the way to 11, where it’s nothing but new

things all the time. Music lovers might imagine working in music journalism to

be paradise, but when you constantly have to explore the new you can never

enjoy the fruits of your connoisseurship—a particular kind of hell. Few people

know this experience as deeply as Scott Plagenhoef, the former editor in chief of

Pitchfork. “You try to find spaces when you’re working to listen to something

that you just want to listen to,” he says of a critic’s life. His desperate urges to

stop wading through unheard tunes of dubious quality and just listen to what he

loved were so strong that Plagenhoef would put only new music on his iPod, to

make himself physically incapable of abandoning his duties in those moments

when he just really, really, really wanted to listen to the Smiths. Journalists are

martyrs, exploring so that others may exploit.

In computer science, the tension between exploration and exploitation takes

its most concrete form in a scenario called the “multi-armed bandit problem.”

The odd name comes from the colloquial term for a casino slot machine, the

“one-armed bandit.” Imagine walking into a casino full of different slot

machines, each one with its own odds of a payoff. The rub, of course, is that you

aren’t told those odds in advance: until you start playing, you won’t have any

idea which machines are the most lucrative (“loose,” as slot-machine aficionados

call it) and which ones are just money sinks.

Naturally, you’re interested in maximizing your total winnings. And it’s clear

that this is going to involve some combination of pulling the arms on different

machines to test them out (exploring), and favoring the most promising

machines you’ve found (exploiting).

To get a sense for the problem’s subtleties, imagine being faced with only two

machines. One you’ve played a total of 15 times; 9 times it paid out, and 6 times

it didn’t. The other you’ve played only twice, and it once paid out and once did

not. Which is more promising?

Simply dividing the wins by the total number of pulls will give you the

machine’s “expected value,” and by this method the first machine clearly comes

out ahead. Its 9–6 record makes for an expected value of 60%, whereas the

second machine’s 1–1 record yields an expected value of only 50%. But there’s

more to it than that. After all, just two pulls aren’t really very many. So there’s a

sense in which we just don’t yet know how good the second machine might

actually be.

Choosing a restaurant or an album is, in effect, a matter of deciding which arm

to pull in life’s casino. But understanding the explore/exploit tradeoff isn’t just a

way to improve decisions about where to eat or what to listen to. It also provides

fundamental insights into how our goals should change as we age, and why the

most rational course of action isn’t always trying to choose the best. And it turns

out to be at the heart of, among other things, web design and clinical trials—two

topics that normally aren’t mentioned in the same sentence.

People tend to treat decisions in isolation, to focus on finding each time the

outcome with the highest expected value. But decisions are almost never

isolated, and expected value isn’t the end of the story. If you’re thinking not just

about the next decision, but about all the decisions you are going to make about

the same options in the future, the explore/exploit tradeoff is crucial to the

process. In this way, writes mathematician Peter Whittle, the bandit problem

“embodies in essential form a conflict evident in all human action.”

So which of those two arms should you pull? It’s a trick question. It

completely depends on something we haven’t discussed yet: how long you plan

to be in the casino.

Seize the Interval

“Carpe diem,” urges Robin Williams in one of the most memorable scenes of the

1989 film Dead Poets Society. “Seize the day, boys. Make your lives

extraordinary.”

It’s incredibly important advice. It’s also somewhat self-contradictory. Seizing

a day and seizing a lifetime are two entirely different endeavors. We have the

expression “Eat, drink, and be merry, for tomorrow we die,” but perhaps we

should also have its inverse: “Start learning a new language or an instrument,

and make small talk with a stranger, because life is long, and who knows what

joy could blossom over many years’ time.” When balancing favorite experiences

and new ones, nothing matters as much as the interval over which we plan to

enjoy them.

“I’m more likely to try a new restaurant when I move to a city than when I’m

leaving it,” explains data scientist and blogger Chris Stucchio, a veteran of

grappling with the explore/exploit tradeoff in both his work and his life. “I

mostly go to restaurants I know and love now, because I know I’m going to be

leaving New York fairly soon. Whereas a couple years ago I moved to Pune,

India, and I just would eat friggin’ everywhere that didn’t look like it was gonna

kill me. And as I was leaving the city I went back to all my old favorites, rather

than trying out new stuff.… Even if I find a slightly better place, I’m only going

to go there once or twice, so why take the risk?”

A sobering property of trying new things is that the value of exploration, of

finding a new favorite, can only go down over time, as the remaining

opportunities to savor it dwindle. Discovering an enchanting café on your last

night in town doesn’t give you the opportunity to return.

The flip side is that the value of exploitation can only go up over time. The

loveliest café that you know about today is, by definition, at least as lovely as the

loveliest café you knew about last month. (And if you’ve found another favorite

since then, it might just be more so.) So explore when you will have time to use

the resulting knowledge, exploit when you’re ready to cash in. The interval

makes the strategy.

Interestingly, since the interval makes the strategy, then by observing the

strategy we can also infer the interval. Take Hollywood, for instance: Among the

ten highest-grossing movies of 1981, only two were sequels. In 1991, it was

three. In 2001, it was five. And in 2011, eight of the top ten highest-grossing

films were sequels. In fact, 2011 set a record for the greatest percentage of

sequels among major studio releases. Then 2012 immediately broke that record;

the next year would break it again. In December 2012, journalist Nick Allen

looked ahead with palpable fatigue to the year to come:

Audiences will be given a sixth helping of X-Men plus Fast and Furious 6, Die Hard 5, Scary Movie

5 and Paranormal Activity 5. There will also be Iron Man 3, The Hangover 3, and second outings for

The Muppets, The Smurfs, GI Joe and Bad Santa.

From a studio’s perspective, a sequel is a movie with a guaranteed fan base: a

cash cow, a sure thing, an exploit. And an overload of sure things signals a shorttermist approach, as with Stucchio on his way out of town. The sequels are more

likely than brand-new movies to be hits this year, but where will the beloved

franchises of the future come from? Such a sequel deluge is not only lamentable

(certainly critics think so); it’s also somewhat poignant. By entering an almost

purely exploit-focused phase, the film industry seems to be signaling a belief

that it is near the end of its interval.

A look into the economics of Hollywood confirms this hunch. Profits of the

largest film studios declined by 40% between 2007 and 2011, and ticket sales

have declined in seven of the past ten years. As the Economist puts it, “Squeezed

between rising costs and falling revenues, the big studios have responded by

trying to make more films they think will be hits: usually sequels, prequels, or

anything featuring characters with name recognition.” In other words, they’re

pulling the arms of the best machines they’ve got before the casino turns them

out.

Win-Stay

Finding optimal algorithms that tell us exactly how to handle the multi-armed

bandit problem has proven incredibly challenging. Indeed, as Peter Whittle

recounts, during World War II efforts to solve the question “so sapped the

energies and minds of Allied analysts … that the suggestion was made that the

problem be dropped over Germany, as the ultimate instrument of intellectual

sabotage.”

The first steps toward a solution were taken in the years after the war, when

Columbia mathematician Herbert Robbins showed that there’s a simple strategy

that, while not perfect, comes with some nice guarantees.

Robbins specifically considered the case where there are exactly two slot

machines, and proposed a solution called the Win-Stay, Lose-Shift algorithm:

choose an arm at random, and keep pulling it as long as it keeps paying off. If

the arm doesn’t pay off after a particular pull, then switch to the other one.

Although this simple strategy is far from a complete solution, Robbins proved in

1952 that it performs reliably better than chance.

Following Robbins, a series of papers examined the “stay on a winner”

principle further. Intuitively, if you were already willing to pull an arm, and it

has just paid off, that should only increase your estimate of its value, and you

should be only more willing to pull it again. And indeed, win-stay turns out to be

an element of the optimal strategy for balancing exploration and exploitation

under a wide range of conditions.

But lose-shift is another story. Changing arms each time one fails is a pretty

rash move. Imagine going to a restaurant a hundred times, each time having a

wonderful meal. Would one disappointment be enough to induce you to give up

on it? Good options shouldn’t be penalized too strongly for being imperfect.

More significantly, Win-Stay, Lose-Shift doesn’t have any notion of the

interval over which you are optimizing. If your favorite restaurant disappointed

you the last time you ate there, that algorithm always says you should go to

another place—even if it’s your last night in town.

Still, Robbins’s initial work on the multi-armed bandit problem kicked off a

substantial literature, and researchers made significant progress over the next

few years. Richard Bellman, a mathematician at the RAND Corporation, found

an exact solution to the problem for cases where we know in advance exactly

how many options and opportunities we’ll have in total. As with the fullinformation secretary problem, Bellman’s trick was essentially to work

backward, starting by imagining the final pull and considering which slot

machine to choose given all the possible outcomes of the previous decisions.

Having figured that out, you’d then turn to the second-to-last opportunity, then

the previous one, and the one before that, all the way back to the start.

The answers that emerge from Bellman’s method are ironclad, but with many

options and a long casino visit it can require a dizzying—or impossible—amount

of work. What’s more, even if we are able to calculate all possible futures, we of

course don’t always know exactly how many opportunities (or even how many

options) we’ll have. For these reasons, the multi-armed bandit problem

effectively stayed unsolved. In Whittle’s words, “it quickly became a classic,

and a byword for intransigence.”

The Gittins Index

As so often happens in mathematics, though, the particular is the gateway to the

universal. In the 1970s, the Unilever corporation asked a young mathematician

named John Gittins to help them optimize some of their drug trials.

Unexpectedly, what they got was the answer to a mathematical riddle that had

gone unsolved for a generation.

Gittins, who is now a professor of statistics at Oxford, pondered the question

posed by Unilever. Given several different chemical compounds, what is the

quickest way to determine which compound is likely to be effective against a

disease? Gittins tried to cast the problem in the most general form he could:

multiple options to pursue, a different probability of reward for each option, and

a certain amount of effort (or money, or time) to be allocated among them. It

was, of course, another incarnation of the multi-armed bandit problem.

Both the for-profit drug companies and the medical profession they serve are

constantly faced with the competing demands of the explore/exploit tradeoff.

Companies want to invest R & D money into the discovery of new drugs, but

also want to make sure their profitable current product lines are flourishing.

Doctors want to prescribe the best existing treatments so that patients get the

care they need, but also want to encourage experimental studies that may turn up

even better ones.

In both cases, notably, it’s not entirely clear what the relevant interval ought

to be. In a sense, drug companies and doctors alike are interested in the indefinite

future. Companies want to be around theoretically forever, and on the medical

side a breakthrough could go on to help people who haven’t even been born yet.

Nonetheless, the present has a higher priority: a cured patient today is taken to

be more valuable than one cured a week or a year from now, and certainly the

same holds true of profits. Economists refer to this idea, of valuing the present

more highly than the future, as “discounting.”

Unlike previous researchers, Gittins approached the multi-armed bandit

problem in those terms. He conceived the goal as maximizing payoffs not for a

fixed interval of time, but for a future that is endless yet discounted.

Such discounting is not unfamiliar to us from our own lives. After all, if you

visit a town for a ten-day vacation, then you should be making your restaurant

decisions with a fixed interval in mind; but if you live in the town, this doesn’t

make as much sense. Instead, you might imagine the value of payoffs decreasing

the further into the future they are: you care more about the meal you’re going to

eat tonight than the meal you’re going to eat tomorrow, and more about

tomorrow’s meal than one a year from now, with the specifics of how much

more depending on your particular “discount function.” Gittins, for his part,

made the assumption that the value assigned to payoffs decreases geometrically:

that is, each restaurant visit you make is worth a constant fraction of the last one.

If, let’s say, you believe there is a 1% chance you’ll get hit by a bus on any given

day, then you should value tomorrow’s dinner at 99% of the value of tonight’s, if

only because you might never get to eat it.

Working with this geometric-discounting assumption, Gittins investigated a

strategy that he thought “at least would be a pretty good approximation”: to

think about each arm of the multi-armed bandit separately from the others, and

try to work out the value of that arm on its own. He did this by imagining

something rather ingenious: a bribe.

In the popular television game show Deal or No Deal, a contestant chooses

one of twenty-six briefcases, which contain prizes ranging from a penny to a

million dollars. As the game progresses, a mysterious character called the

Banker will periodically call in and offer the contestant various sums of money

to not open the chosen briefcase. It’s up to the contestant to decide at what price

they’re willing to take a sure thing over the uncertainty of the briefcase prize.

Gittins (albeit many years before the first episode of Deal or No Deal aired)

realized that the multi-armed bandit problem is no different. For every slot

machine we know little or nothing about, there is some guaranteed payout rate

which, if offered to us in lieu of that machine, will make us quite content never

to pull its handle again. This number—which Gittins called the “dynamic

allocation index,” and which the world now knows as the Gittins index—

suggests an obvious strategy on the casino floor: always play the arm with the

highest index.*

In fact, the index strategy turned out to be more than a good approximation. It

completely solves the multi-armed bandit with geometrically discounted payoffs.

The tension between exploration and exploitation resolves into the simpler task

of maximizing a single quantity that accounts for both. Gittins is modest about

the achievement—“It’s not quite Fermat’s Last Theorem,” he says with a

chuckle—but it’s a theorem that put to rest a significant set of questions about

the explore/exploit dilemma.

Now, actually calculating the Gittins index for a specific machine, given its

track record and our discounting rate, is still fairly involved. But once the Gittins

index for a particular set of assumptions is known, it can be used for any

problem of that form. Crucially, it doesn’t even matter how many arms are

involved, since the index for each arm is calculated separately.

In the table on the next page we provide the Gittins index values for up to nine

successes and failures, assuming that a payoff on our next pull is worth 90% of a

payoff now. These values can be used to resolve a variety of everyday multiarmed bandit problems. For example, under these assumptions you should, in

fact, choose the slot machine that has a track record of 1–1 (and an expected

value of 50%) over the one with a track record of 9–6 (and an expected value of

60%). Looking up the relevant coordinates in the table shows that the lesserknown machine has an index of 0.6346, while the more-played machine scores

only a 0.6300. Problem solved: try your luck this time, and explore.

Looking at the Gittins index values in the table, there are a few other

interesting observations. First, you can see the win-stay principle at work: as you

go from left to right in any row, the index scores always increase. So if an arm is

ever the correct one to pull, and that pull is a winner, then (following the chart to

the right) it can only make more sense to pull the same arm again. Second, you

can see where lose-shift would get you into trouble. Having nine initial wins

followed by a loss gets you an index of 0.8695, which is still higher than most of

the other values in the table—so you should probably stay with that arm for at

least another pull.

Gittins index values as a function of wins and losses, assuming that a payoff next time is worth 90% of a

payoff now.

But perhaps the most interesting part of the table is the top-left entry. A record

of 0–0—an arm that’s a complete unknown—has an expected value of 0.5000

but a Gittins index of 0.7029. In other words, something you have no experience

with whatsoever is more attractive than a machine that you know pays out seven

times out of ten! As you go down the diagonal, notice that a record of 1–1 yields

an index of 0.6346, a record of 2–2 yields 0.6010, and so on. If such 50%-

successful performance persists, the index does ultimately converge on 0.5000,

as experience confirms that the machine is indeed nothing special and takes

away the “bonus” that spurs further exploration. But the convergence happens

fairly slowly; the exploration bonus is a powerful force. Indeed, note that even a

failure on the very first pull, producing a record of 0–1, makes for a Gittins

index that’s still above 50%.

We can also see how the explore/exploit tradeoff changes as we change the

way we’re discounting the future. The following table presents exactly the same

information as the preceding one, but assumes that a payoff next time is worth

99% of one now, rather than 90%. With the future weighted nearly as heavily as

the present, the value of making a chance discovery, relative to taking a sure

thing, goes up even more. Here, a totally untested machine with a 0–0 record is

worth a guaranteed 86.99% chance of a payout!

Gittins index values as a function of wins and losses, assuming that a payoff next time is worth 99% of a

payoff now.

The Gittins index, then, provides a formal, rigorous justification for preferring

the unknown, provided we have some opportunity to exploit the results of what

we learn from exploring. The old adage tells us that “the grass is always greener

on the other side of the fence,” but the math tells us why: the unknown has a

chance of being better, even if we actually expect it to be no different, or if it’s

just as likely to be worse. The untested rookie is worth more (early in the season,

anyway) than the veteran of seemingly equal ability, precisely because we know

less about him. Exploration in itself has value, since trying new things increases

our chances of finding the best. So taking the future into account, rather than

focusing just on the present, drives us toward novelty.

The Gittins index thus provides an amazingly straightforward solution to the

multi-armed bandit problem. But it doesn’t necessarily close the book on the

puzzle, or help us navigate all the explore/exploit tradeoffs of everyday life. For

one, the Gittins index is optimal only under some strong assumptions. It’s based

on geometric discounting of future reward, valuing each pull at a constant

fraction of the previous one, which is something that a variety of experiments in

behavioral economics and psychology suggest people don’t do. And if there’s a

cost to switching among options, the Gittins strategy is no longer optimal either.

(The grass on the other side of the fence may look a bit greener, but that doesn’t

necessarily warrant climbing the fence—let alone taking out a second mortgage.)

Perhaps even more importantly, it’s hard to compute the Gittins index on the fly.

If you carry around a table of index values you can optimize your dining

choices, but the time and effort involved might not be worth it. (“Wait, I can

resolve this argument. That restaurant was good 29 times out of 35, but this other

one has been good 13 times out of 16, so the Gittins indices are … Hey, where

did everybody go?”)

In the time since the development of the Gittins index, such concerns have

sent computer scientists and statisticians searching for simpler and more flexible

strategies for dealing with multi-armed bandits. These strategies are easier for

humans (and machines) to apply in a range of situations than crunching the

optimal Gittins index, while still providing comparably good performance. They

also engage with one of our biggest human fears regarding decisions about

which chances to take.

Regret and Optimism

Regrets, I’ve had a few. But then again, too few to mention.

—FRANK SINATRA

For myself I am an optimist. It does not seem to be much use being anything

else.

—WINSTON CHURCHILL

If the Gittins index is too complicated, or if you’re not in a situation well

characterized by geometric discounting, then you have another option: focus on

regret. When we choose what to eat, who to spend time with, or what city to live

in, regret looms large—presented with a set of good options, it is easy to torture

ourselves with the consequences of making the wrong choice. These regrets are

often about the things we failed to do, the options we never tried. In the

memorable words of management theorist Chester Barnard, “To try and fail is at

least to learn; to fail to try is to suffer the inestimable loss of what might have

been.”

Regret can also be highly motivating. Before he decided to start Amazon.com,

Jeff Bezos had a secure and well-paid position at the investment company D. E.

Shaw & Co. in New York. Starting an online bookstore in Seattle was going to

be a big leap—something that his boss (that’s D. E. Shaw) advised him to think

about carefully. Says Bezos:

The framework I found, which made the decision incredibly easy, was what I called—which only a

nerd would call—a “regret minimization framework.” So I wanted to project myself forward to age

80 and say, “Okay, now I’m looking back on my life. I want to have minimized the number of regrets

I have.” I knew that when I was 80 I was not going to regret having tried this. I was not going to

regret trying to participate in this thing called the Internet that I thought was going to be a really big

deal. I knew that if I failed I wouldn’t regret that, but I knew the one thing I might regret is not ever

having tried. I knew that that would haunt me every day, and so, when I thought about it that way it

was an incredibly easy decision.

Computer science can’t offer you a life with no regret. But it can, potentially,

offer you just what Bezos was looking for: a life with minimal regret.

Regret is the result of comparing what we actually did with what would have

been best in hindsight. In a multi-armed bandit, Barnard’s “inestimable loss” can

in fact be measured precisely, and regret assigned a number: it’s the difference

between the total payoff obtained by following a particular strategy and the total

payoff that theoretically could have been obtained by just pulling the best arm

every single time (had we only known from the start which one it was). We can

calculate this number for different strategies, and search for those that minimize

it.

In 1985, Herbert Robbins took a second shot at the multi-armed bandit

problem, some thirty years after his initial work on Win-Stay, Lose-Shift. He

and fellow Columbia mathematician Tze Leung Lai were able to prove several

key points about regret. First, assuming you’re not omniscient, your total amount

of regret will probably never stop increasing, even if you pick the best possible

strategy—because even the best strategy isn’t perfect every time. Second, regret

will increase at a slower rate if you pick the best strategy than if you pick others;

what’s more, with a good strategy regret’s rate of growth will go down over

time, as you learn more about the problem and are able to make better choices.

Third, and most specifically, the minimum possible regret—again assuming nonomniscience—is regret that increases at a logarithmic rate with every pull of the

handle.

Logarithmically increasing regret means that we’ll make as many mistakes in

our first ten pulls as in the following ninety, and as many in our first year as in

the rest of the decade combined. (The first decade’s mistakes, in turn, are as

many as we’ll make for the rest of the century.) That’s some measure of

consolation. In general we can’t realistically expect someday to never have any

more regrets. But if we’re following a regret-minimizing algorithm, every year

we can expect to have fewer new regrets than we did the year before.

Starting with Lai and Robbins, researchers in recent decades have set about

looking for algorithms that offer the guarantee of minimal regret. Of the ones

they’ve discovered, the most popular are known as Upper Confidence Bound

algorithms.

Visual displays of statistics often include so-called error bars that extend

above and below any data point, indicating uncertainty in the measurement; the

error bars show the range of plausible values that the quantity being measured

could actually have. This range is known as the “confidence interval,” and as we

gain more data about something the confidence interval will shrink, reflecting an

increasingly accurate assessment. (For instance, a slot machine that has paid out

once out of two pulls will have a wider confidence interval, though the same

expected value, as a machine that has paid out five times on ten pulls.) In a

multi-armed bandit problem, an Upper Confidence Bound algorithm says, quite

simply, to pick the option for which the top of the confidence interval is highest.

Like the Gittins index, therefore, Upper Confidence Bound algorithms assign

a single number to each arm of the multi-armed bandit. And that number is set to

the highest value that the arm could reasonably have, based on the information

available so far. So an Upper Confidence Bound algorithm doesn’t care which

arm has performed best so far; instead, it chooses the arm that could reasonably

perform best in the future. If you have never been to a restaurant before, for

example, then for all you know it could be great. Even if you have gone there

once or twice, and tried a couple of their dishes, you might not have enough

information to rule out the possibility that it could yet prove better than your

regular favorite. Like the Gittins index, the Upper Confidence Bound is always

greater than the expected value, but by less and less as we gain more experience

with a particular option. (A restaurant with a single mediocre review still retains

a potential for greatness that’s absent in a restaurant with hundreds of such

reviews.) The recommendations given by Upper Confidence Bound algorithms

will be similar to those provided by the Gittins index, but they are significantly

easier to compute, and they don’t require the assumption of geometric

discounting.

Upper Confidence Bound algorithms implement a principle that has been

dubbed “optimism in the face of uncertainty.” Optimism, they show, can be

perfectly rational. By focusing on the best that an option could be, given the

evidence obtained so far, these algorithms give a boost to possibilities we know

less about. As a consequence, they naturally inject a dose of exploration into the

decision-making process, leaping at new options with enthusiasm because any

one of them could be the next big thing. The same principle has been used, for

instance, by MIT’s Leslie Kaelbling, who builds “optimistic robots” that explore

the space around them by boosting the value of uncharted terrain. And it clearly

has implications for human lives as well.

The success of Upper Confidence Bound algorithms offers a formal

justification for the benefit of the doubt. Following the advice of these

algorithms, you should be excited to meet new people and try new things—to

assume the best about them, in the absence of evidence to the contrary. In the

long run, optimism is the best prevention for regret.

Bandits Online

In 2007, Google product manager Dan Siroker took a leave of absence to join

the presidential campaign of then senator Barack Obama in Chicago. Heading

the “New Media Analytics” team, Siroker brought one of Google’s web

practices to bear on the campaign’s bright-red DONATE button. The result was

nothing short of astonishing: $57 million of additional donations were raised as a

direct result of his work.

What exactly did he do to that button?

He A/B tested it.

A/B testing works as follows: a company drafts several different versions of a

particular webpage. Perhaps they try different colors or images, or different

headlines for a news article, or different arrangements of items on the screen.

Then they randomly assign incoming users to these various pages, usually in

equal numbers. One user may see a red button, while another user may see a

blue one; one may see DONATE and another may see CONTRIBUTE. The relevant

metrics (e.g., click-through rate or average revenue per visitor) are then

monitored. After a period of time, if statistically significant effects are observed,

the “winning” version is typically locked into place—or becomes the control for

another round of experiments.

In the case of Obama’s donation page, Siroker’s A/B tests were revealing. For

first-time visitors to the campaign site, a DONATE AND GET A GIFT button turned

out to be the best performer, even after the cost of sending the gifts was taken

into account. For longtime newsletter subscribers who had never given money,

PLEASE DONATE worked the best, perhaps appealing to their guilt. For visitors

who had already donated in the past, CONTRIBUTE worked best at securing

follow-up donations—the logic being perhaps that the person had already

“donated” but could always “contribute” more. And in all cases, to the

astonishment of the campaign team, a simple black-and-white photo of the

Obama family outperformed any other photo or video the team could come up

with. The net effect of all these independent optimizations was gigantic.

If you’ve used the Internet basically at all over the past decade, then you’ve

been a part of someone else’s explore/exploit problem. Companies want to

discover the things that make them the most money while simultaneously

making as much of it as they can—explore, exploit. Big tech firms such as

Amazon and Google began carrying out live A/B tests on their users starting in

about 2000, and over the following years the Internet has become the world’s

largest controlled experiment. What are these companies exploring and

exploiting? In a word, you: whatever it is that makes you move your mouse and

open your wallet.

Companies A/B test their site navigation, the subject lines and timing of their

marketing emails, and sometimes even their actual features and pricing. Instead

of “the” Google search algorithm and “the” Amazon checkout flow, there are

now untold and unfathomably subtle permutations. (Google infamously tested

forty-one shades of blue for one of its toolbars in 2009.) In some cases, it’s

unlikely that any pair of users will have the exact same experience.

Data scientist Jeff Hammerbacher, former manager of the Data group at

Facebook, once told Bloomberg Businessweek that “the best minds of my

generation are thinking about how to make people click ads.” Consider it the

millennials’ Howl—what Allen Ginsberg’s immortal “I saw the best minds of

my generation destroyed by madness” was to the Beat Generation.

Hammerbacher’s take on the situation was that this state of affairs “sucks.” But

regardless of what one makes of it, the web is allowing for an experimental

science of the click the likes of which had never even been dreamed of by

marketers of the past.

We know what happened to Obama in the 2008 election, of course. But what

happened to his director of analytics, Dan Siroker? After the inauguration,

Siroker returned west, to California, and with fellow Googler Pete Koomen cofounded the website optimization firm Optimizely. By the 2012 presidential

election cycle, their company counted among its clients both the Obama reelection campaign and the campaign of Republican challenger Mitt Romney.

Within a decade or so after its first tentative use, A/B testing was no longer a

secret weapon. It has become such a deeply embedded part of how business and

politics are conducted online as to be effectively taken for granted. The next time

you open your browser, you can be sure that the colors, images, text, perhaps

even the prices you see—and certainly the ads—have come from an

explore/exploit algorithm, tuning itself to your clicks. In this particular multiarmed bandit problem, you’re not the gambler; you’re the jackpot.

The process of A/B testing itself has become increasingly refined over time.

The most canonical A/B setup—splitting the traffic evenly between two options,

running the test for a set period of time, and thereafter giving all the traffic to the

winner—might not necessarily be the best algorithm for solving the problem,

since it means half the users are stuck getting the inferior option as long as the

test continues. And the rewards for finding a better approach are potentially very

high. More than 90% of Google’s approximately $50 billion in annual revenue

currently comes from paid advertising, and online commerce comprises

hundreds of billions of dollars a year. This means that explore/exploit algorithms

effectively power, both economically and technologically, a significant fraction

of the Internet itself. The best algorithms to use remain hotly contested, with

rival statisticians, engineers, and bloggers endlessly sparring about the optimal

way to balance exploration and exploitation in every possible business scenario.

Debating the precise distinctions among various takes on the explore/exploit

problem may seem hopelessly arcane. In fact, these distinctions turn out to

matter immensely—and it’s not just presidential elections and the Internet

economy that are at stake.

It’s also human lives.

Clinical Trials on Trial

Between 1932 and 1972, several hundred African-American men with syphilis

in Macon County, Alabama, went deliberately untreated by medical

professionals, as part of a forty-year experiment by the US Public Health Service

known as the Tuskegee Syphilis Study. In 1966, Public Health Service employee

Peter Buxtun filed a protest. He filed a second protest in 1968. But it was not

until he broke the story to the press—it appeared in the Washington Star on July

25, 1972, and was the front-page story in the New York Times the next day—that

the US government finally halted the study.

What followed the public outcry, and the subsequent congressional hearing,

was an initiative to formalize the principles and standards of medical ethics. A

commission held at the pastoral Belmont Conference Center in Maryland

resulted in a 1979 document known as the Belmont Report. The Belmont Report

lays out a foundation for the ethical practice of medical experiments, so that the

Tuskegee experiment—an egregious, unambiguously inappropriate breach of the

health profession’s duty to its patients—might never be repeated. But it also

notes the difficulty, in many other cases, of determining exactly where the line

should be drawn.

“The Hippocratic maxim ‘do no harm’ has long been a fundamental principle

of medical ethics,” the report points out. “[The physiologist] Claude Bernard

extended it to the realm of research, saying that one should not injure one person

regardless of the benefits that might come to others. However, even avoiding

harm requires learning what is harmful; and, in the process of obtaining this

information, persons may be exposed to risk of harm.”

The Belmont Report thus acknowledges, but does not resolve, the tension that

exists between acting on one’s best knowledge and gathering more. It also

makes it clear that gathering knowledge can be so valuable that some aspects of

normal medical ethics can be suspended. Clinical testing of new drugs and

treatments, the report notes, often requires risking harm to some patients, even if

steps are taken to minimize that risk.

The principle of beneficence is not always so unambiguous. A difficult ethical problem remains, for

example, about research [on childhood diseases] that presents more than minimal risk without

immediate prospect of direct benefit to the children involved. Some have argued that such research is

inadmissible, while others have pointed out that this limit would rule out much research promising

great benefit to children in the future. Here again, as with all hard cases, the different claims covered

by the principle of beneficence may come into conflict and force difficult choices.

One of the fundamental questions that has arisen in the decades since the

Belmont Report is whether the standard approach to conducting clinical trials

really does minimize risk to patients. In a conventional clinical trial, patients are

split into groups, and each group is assigned to receive a different treatment for

the duration of the study. (Only in exceptional cases does a trial get stopped

early.) This procedure focuses on decisively resolving the question of which

treatment is better, rather than on providing the best treatment to each patient in

the trial itself. In this way it operates exactly like a website’s A/B test, with a

certain fraction of people receiving an experience during the experiment that will

eventually be proven inferior. But doctors, like tech companies, are gaining

some information about which option is better while the trial proceeds—

information that could be used to improve outcomes not only for future patients

beyond the trial, but for the patients currently in it.

Millions of dollars are at stake in experiments to find the optimal

configuration of a website, but in clinical trials, experimenting to find optimal

treatments has direct life-or-death consequences. And a growing community of

doctors and statisticians think that we’re doing it wrong: that we should be

treating the selection of treatments as a multi-armed bandit problem, and trying

to get the better treatments to people even while an experiment is in progress.

In 1969, Marvin Zelen, a biostatistician who is now at Harvard, proposed

conducting “adaptive” trials. One of the ideas he suggested was a randomized

“play the winner” algorithm—a version of Win-Stay, Lose-Shift, in which the

chance of using a given treatment is increased by each win and decreased by

each loss. In Zelen’s procedure, you start with a hat that contains one ball for

each of the two treatment options being studied. The treatment for the first

patient is selected by drawing a ball at random from the hat (the ball is put back

afterward). If the chosen treatment is a success, you put another ball for that

treatment into the hat—now you have three balls, two of which are for the

successful treatment. If it fails, then you put another ball for the other treatment

into the hat, making it more likely you’ll choose the alternative.

Zelen’s algorithm was first used in a clinical trial sixteen years later, for a

study of extracorporeal membrane oxygenation, or “ECMO”—an audacious

approach to treating respiratory failure in infants. Developed in the 1970s by

Robert Bartlett of the University of Michigan, ECMO takes blood that’s heading

for the lungs and routes it instead out of the body, where it is oxygenated by a

machine and returned to the heart. It is a drastic measure, with risks of its own

(including the possibility of embolism), but it offered a possible approach in

situations where no other options remained. In 1975 ECMO saved the life of a

newborn girl in Orange County, California, for whom even a ventilator was not

providing enough oxygen. That girl has now celebrated her fortieth birthday and

is married with children of her own. But in its early days the ECMO technology

and procedure were considered highly experimental, and early studies in adults

showed no benefit compared to conventional treatments.

From 1982 to 1984, Bartlett and his colleagues at the University of Michigan

performed a study on newborns with respiratory failure. The team was clear that

they wanted to address, as they put it, “the ethical issue of withholding an

unproven but potentially lifesaving treatment,” and were “reluctant to withhold a

lifesaving treatment from alternate patients simply to meet conventional random

assignment technique.” Hence they turned to Zelen’s algorithm. The strategy

resulted in one infant being assigned the “conventional” treatment and dying,

and eleven infants in a row being assigned the experimental ECMO treatment,

all of them surviving. Between April and November of 1984, after the end of the

official study, ten additional infants met the criteria for ECMO treatment. Eight

were treated with ECMO, and all eight survived. Two were treated

conventionally, and both died.

These are eye-catching numbers, yet shortly after the University of Michigan

study on ECMO was completed it became mired in controversy. Having so few

patients in a trial receive the conventional treatment deviated significantly from

standard methodology, and the procedure itself was highly invasive and

potentially risky. After the publication of the paper, Jim Ware, professor of

biostatistics at the Harvard School of Public Health, and his medical colleagues

examined the data carefully and concluded that they “did not justify routine use

of ECMO without further study.” So Ware and his colleagues designed a second

clinical trial, still trying to balance the acquisition of knowledge with the

effective treatment of patients but using a less radical design. They would

randomly assign patients to either ECMO or the conventional treatment until a

prespecified number of deaths was observed in one of the groups. Then they

would switch all the patients in the study to the more effective treatment of the

two.

In the first phase of Ware’s study, four of ten infants receiving conventional

treatment died, and all nine of nine infants receiving ECMO survived. The four

deaths were enough to trigger a transition to the second phase, where all twenty

patients were treated with ECMO and nineteen survived. Ware and colleagues

were convinced, concluding that “it is difficult to defend further randomization

ethically.”

But some had already concluded this before the Ware study, and were vocal

about it. The critics included Don Berry, one of the world’s leading experts on

multi-armed bandits. In a comment that was published alongside the Ware study

in Statistical Science, Berry wrote that “randomizing patients to non-ECMO

therapy as in the Ware study was unethical.… In my view, the Ware study

should not have been conducted.”

And yet even the Ware study was not conclusive for all in the medical

community. In the 1990s yet another study on ECMO was conducted, enrolling

nearly two hundred infants in the United Kingdom. Instead of using adaptive

algorithms, this study followed the traditional methods, splitting the infants

randomly into two equal groups. The researchers justified the experiment by

saying that ECMO’s usefulness “is controversial because of varying

interpretation of the available evidence.” As it turned out, the difference between

the treatments wasn’t as pronounced in the United Kingdom as it had been in the

two American studies, but the results were nonetheless declared “in accord with

the earlier preliminary findings that a policy of ECMO support reduces the risk

of death.” The cost of that knowledge? Twenty-four more infants died in the

“conventional” group than in the group receiving ECMO treatment.

The widespread difficulty with accepting results from adaptive clinical trials

might seem incomprehensible. But consider that part of what the advent of

statistics did for medicine, at the start of the twentieth century, was to transform

it from a field in which doctors had to persuade each other in ad hoc ways about

every new treatment into one where they had clear guidelines about what sorts of

evidence were and were not persuasive. Changes to accepted standard statistical

practice have the potential to upset this balance, at least temporarily.

After the controversy over ECMO, Don Berry moved from the statistics

department at the University of Minnesota to the MD Anderson Cancer Center in

Houston, where he has used methods developed by studying multi-armed bandits

to design clinical trials for a variety of cancer treatments. While he remains one

of the more vocal critics of randomized clinical trials, he is by no means the only

one. In recent years, the ideas he’s been fighting for are finally beginning to

come into the mainstream. In February 2010, the FDA released a “guidance”

document, “Adaptive Design Clinical Trials for Drugs and Biologics,” which

suggests—despite a long history of sticking to an option they trust—that they

might at last be willing to explore alternatives.

The Restless World

Once you become familiar with them, it’s easy to see multi-armed bandits just

about everywhere we turn. It’s rare that we make an isolated decision, where the

outcome doesn’t provide us with any information that we’ll use to make other

decisions in the future. So it’s natural to ask, as we did with optimal stopping,

how well people generally tend to solve these problems—a question that has

been extensively explored in the laboratory by psychologists and behavioral

economists.

In general, it seems that people tend to over-explore—to favor the new

disproportionately over the best. In a simple demonstration of this phenomenon,

published in 1966, Amos Tversky and Ward Edwards conducted experiments

where people were shown a box with two lights on it and told that each light

would turn on a fixed (but unknown) percentage of the time. They were then

given 1,000 opportunities either to observe which light came on, or to place a bet

on the outcome without getting to observe it. (Unlike a more traditional bandit

problem setup, here one could not make a “pull” that was both wager and

observation at once; participants would not learn whether their bets had paid off

until the end.) This is pure exploration vs. exploitation, pitting the gaining of

information squarely against the use of it. For the most part, people adopted a

sensible strategy of observing for a while, then placing bets on what seemed like

the best outcome—but they consistently spent a lot more time observing than

they should have. How much more time? In one experiment, one light came on

60% of the time and the other 40% of the time, a difference neither particularly

blatant nor particularly subtle. In that case, people chose to observe 505 times,

on average, placing bets the other 495 times. But the math says they should have

started to bet after just 38 observations—leaving 962 chances to cash in.

Other studies have produced similar conclusions. In the 1990s, Robert Meyer

and Yong Shi, researchers at Wharton, ran a study where people were given a

choice between two options, one with a known payoff chance and one unknown

—specifically two airlines, an established carrier with a known on-time rate and

a new company without a track record yet. Given the goal of maximizing the

number of on-time arrivals over some period of time, the mathematically optimal

strategy is to initially only fly the new airline, as long as the established one isn’t

clearly better. If at any point it’s apparent that the well-known carrier is better—

that is, if the Gittins index of the new option falls below the on-time rate of the

familiar carrier—then you should switch hard to the familiar one and never look

back. (Since in this setup you can’t get any more information about the new

company once you stop flying it, there is no opportunity for it to redeem itself.)

But in the experiment, people tended to use the untried airline too little when it

was good and too much when it was bad. They also didn’t make clean breaks

away from it, often continuing to alternate, particularly when neither airline was

departing on time. All of this is consistent with tending to over-explore.

Finally, psychologists Mark Steyvers, Michael Lee, and E.-J. Wagenmakers

have run an experiment with a four-armed bandit, asking a group of people to

choose which arm to play over a sequence of fifteen opportunities. They then

classified the strategies that participants seemed to use. The results suggested

that 30% were closest to the optimal strategy, 47% most resembled Win-Stay,

Lose-Shift, and 22% seemed to move at random between selecting a new arm

and playing the best arm found so far. Again, this is consistent with overexploring, as Win-Stay, Lose-Shift and occasionally trying an arm at random are

both going to lead people to try things other than the best option late in the

game, when they should be purely exploiting.

So, while we tend to commit to a new secretary too soon, it seems like we

tend to stop trying new airlines too late. But just as there’s a cost to not having a

secretary, there’s a cost to committing too soon to a particular airline: the world

might change.

The standard multi-armed bandit problem assumes that the probabilities with

which the arms pay off are fixed over time. But that’s not necessarily true of

airlines, restaurants, or other contexts in which people have to make repeated

choices. If the probabilities of a payoff on the different arms change over time—

what has been termed a “restless bandit”—the problem becomes much harder.

(So much harder, in fact, that there’s no tractable algorithm for completely

solving it, and it’s believed there never will be.) Part of this difficulty is that it is

no longer simply a matter of exploring for a while and then exploiting: when the

world can change, continuing to explore can be the right choice. It might be

worth going back to that disappointing restaurant you haven’t visited for a few

years, just in case it’s under new management.

In his celebrated essay “Walking,” Henry David Thoreau reflected on how he

preferred to do his traveling close to home, how he never tired of his

surroundings and always found something new or surprising in the

Massachusetts landscape. “There is in fact a sort of harmony discoverable

between the capabilities of the landscape within a circle of ten miles’ radius, or

the limits of an afternoon walk, and the threescore years and ten of human life,”

he wrote. “It will never become quite familiar to you.”

To live in a restless world requires a certain restlessness in oneself. So long as

things continue to change, you must never fully cease exploring.

Still, the algorithmic techniques honed for the standard version of the multiarmed bandit problem are useful even in a restless world. Strategies like the

Gittins index and Upper Confidence Bound provide reasonably good

approximate solutions and rules of thumb, particularly if payoffs don’t change

very much over time. And many of the world’s payoffs are arguably more static

today than they’ve ever been. A berry patch might be ripe one week and rotten

the next, but as Andy Warhol put it, “A Coke is a Coke.” Having instincts tuned

by evolution for a world in constant flux isn’t necessarily helpful in an era of

industrial standardization.

Perhaps most importantly, thinking about versions of the multi-armed bandit

problem that do have optimal solutions doesn’t just offer algorithms, it also

offers insights. The conceptual vocabulary derived from the classical form of the

problem—the tension of explore/exploit, the importance of the interval, the high

value of the 0–0 option, the minimization of regret—gives us a new way of

making sense not only of specific problems that come before us, but of the entire

arc of human life.

Explore …

While laboratory studies can be illuminating, the interval of many of the most

important problems people face is far too long to be studied in the lab. Learning

the structure of the world around us and forming lasting social relationships are

both lifelong tasks. So it’s instructive to see how the general pattern of early

exploration and late exploitation appears over the course of a lifetime.

One of the curious things about human beings, which any developmental

psychologist aspires to understand and explain, is that we take years to become

competent and autonomous. Caribou and gazelles must be prepared to run from

predators the day they’re born, but humans take more than a year to make their

first steps. Alison Gopnik, professor of developmental psychology at UC

Berkeley and author of The Scientist in the Crib, has an explanation for why

human beings have such an extended period of dependence: “it gives you a

developmental way of solving the exploration/exploitation tradeoff.” As we have

seen, good algorithms for playing multi-armed bandits tend to explore more

early on, exploiting the resulting knowledge later. But as Gopnik points out, “the

disadvantage of that is that you don’t get good payoffs when you are in the

exploration stage.” Hence childhood: “Childhood gives you a period in which

you can just explore possibilities, and you don’t have to worry about payoffs

because payoffs are being taken care of by the mamas and the papas and the

grandmas and the babysitters.”

Thinking about children as simply being at the transitory exploration stage of

a lifelong algorithm might provide some solace for parents of preschoolers.

(Tom has two highly exploratory preschool-age daughters, and hopes they are

following an algorithm that has minimal regret.) But it also provides new

insights about the rationality of children. Gopnik points out that “if you look at

the history of the way that people have thought about children, they have

typically argued that children are cognitively deficient in various ways—because

if you look at their exploit capacities, they look terrible. They can’t tie their

shoes, they’re not good at long-term planning, they’re not good at focused

attention. Those are all things that kids are really awful at.” But pressing buttons

at random, being very interested in new toys, and jumping quickly from one

thing to another are all things that kids are really great at. And those are exactly

what they should be doing if their goal is exploration. If you’re a baby, putting

every object in the house into your mouth is like studiously pulling all the

handles at the casino.

More generally, our intuitions about rationality are too often informed by

exploitation rather than exploration. When we talk about decision-making, we

usually focus just on the immediate payoff of a single decision—and if you treat

every decision as if it were your last, then indeed only exploitation makes sense.

But over a lifetime, you’re going to make a lot of decisions. And it’s actually

rational to emphasize exploration—the new rather than the best, the exciting

rather than the safe, the random rather than the considered—for many of those

choices, particularly earlier in life.

What we take to be the caprice of children may be wiser than we know.

… And Exploit

I had reached a juncture in my reading life that is familiar to those who

have been there: in the allotted time left to me on earth, should I read more

and more new books, or should I cease with that vain consumption—vain

because it is endless—and begin to reread those books that had given me

the intensest pleasure in my past.

—LYDIA DAVIS

At the other extreme from toddlers we have the elderly. And thinking about

aging from the perspective of the explore/exploit dilemma also provides some

surprising insights into how we should expect our lives to change as time goes

on.

Laura Carstensen, a professor of psychology at Stanford, has spent her career

challenging our preconceptions about getting older. Particularly, she has

investigated exactly how, and why, people’s social relationships change as they

age. The basic pattern is clear: the size of people’s social networks (that is, the

number of social relationships they engage in) almost invariably decreases over

time. But Carstensen’s research has transformed how we should think about this

phenomenon.

The traditional explanation for the elderly having smaller social networks is

that it’s just one example of the decrease in quality of life that comes with aging

—the result of diminished ability to contribute to social relationships, greater

fragility, and general disengagement from society. But Carstensen has argued

that, in fact, the elderly have fewer social relationships by choice. As she puts it,

these decreases are “the result of lifelong selection processes by which people

strategically and adaptively cultivate their social networks to maximize social

and emotional gains and minimize social and emotional risks.”

What Carstensen and her colleagues found is that the shrinking of social

networks with aging is due primarily to “pruning” peripheral relationships and

focusing attention instead on a core of close friends and family members. This

process seems to be a deliberate choice: as people approach the end of their

lives, they want to focus more on the connections that are the most meaningful.

In an experiment testing this hypothesis, Carstensen and her collaborator

Barbara Fredrickson asked people to choose who they’d rather spend thirty

minutes with: an immediate family member, the author of a book they’d recently

read, or somebody they had met recently who seemed to share their interests.

Older people preferred the family member; young people were just as excited to

meet the author or make a new friend. But in a critical twist, if the young people

were asked to imagine that they were about to move across the country, they

preferred the family member too. In another study, Carstensen and her

colleagues found the same result in the other direction as well: if older people

were asked to imagine that a medical breakthrough would allow them to live

twenty years longer, their preferences became indistinguishable from those of

young people. The point is that these differences in social preference are not

about age as such—they’re about where people perceive themselves to be on the

interval relevant to their decision.

Being sensitive to how much time you have left is exactly what the computer

science of the explore/exploit dilemma suggests. We think of the young as

stereotypically fickle; the old, stereotypically set in their ways. In fact, both are

behaving completely appropriately with respect to their intervals. The deliberate

honing of a social network down to the most meaningful relationships is the

rational response to having less time to enjoy them.

Recognizing that old age is a time of exploitation helps provide new

perspectives on some of the classic phenomena of aging. For example, while

going to college—a new social environment filled with people you haven’t met

—is typically a positive, exciting time, going to a retirement home—a new

social environment filled with people you haven’t met—can be painful. And that

difference is partly the result of where we are on the explore/exploit continuum

at those stages of our lives.

The explore/exploit tradeoff also tells us how to think about advice from our

elders. When your grandfather tells you which restaurants are good, you should

listen—these are pearls gleaned from decades of searching. But when he only

goes to the same restaurant at 5:00 p.m. every day, you should feel free to

explore other options, even though they’ll likely be worse.

Perhaps the deepest insight that comes from thinking about later life as a

chance to exploit knowledge acquired over decades is this: life should get better

over time. What an explorer trades off for knowledge is pleasure. The Gittins

index and the Upper Confidence Bound, as we’ve seen, inflate the appeal of

lesser-known options beyond what we actually expect, since pleasant surprises

can pay off many times over. But at the same time, this means that exploration

necessarily leads to being let down on most occasions. Shifting the bulk of one’s

attention to one’s favorite things should increase quality of life. And it seems

like it does: Carstensen has found that older people are generally more satisfied

with their social networks, and often report levels of emotional well-being that

are higher than those of younger adults.

So there’s a lot to look forward to in being that late-afternoon restaurant

regular, savoring the fruits of a life’s explorations.

3 Sorting

Making Order

Nowe if the word, which thou art desirous to finde, begin with (a) then

looke in the beginning of this Table, but if with (v) looke towards the end.

Againe, if thy word beginne with (ca) looke in the beginning of the letter (c)

but if with (cu) then looke toward the end of that letter. And so of all the

rest. &c.

—ROBERT CAWDREY, A TABLE ALPHABETICALL (1604)

Before Danny Hillis founded the Thinking Machines corporation, before he

invented the famous Connection Machine parallel supercomputer, he was an

MIT undergraduate, living in the student dormitory, and horrified by his

roommate’s socks.

What horrified Hillis, unlike many a college undergraduate, wasn’t his

roommate’s hygiene. It wasn’t that the roommate didn’t wash the socks; he did.

The problem was what came next.

The roommate pulled a sock out of the clean laundry hamper. Next he pulled

another sock out at random. If it didn’t match the first one, he tossed it back in.

Then he continued this process, pulling out socks one by one and tossing them

back until he found a match for the first.

With just 10 different pairs of socks, following this method will take on

average 19 pulls merely to complete the first pair, and 17 more pulls to complete

the second. In total, the roommate can expect to go fishing in the hamper 110

times just to pair 20 socks.

It was enough to make any budding computer scientist request a room

transfer.

Now, just how socks should be sorted is a good way get computer scientists

talking at surprising length. A question about socks posted to the programming

website Stack Overflow in 2013 prompted some twelve thousand words of

debate.

“Socks confound me!” confessed legendary cryptographer and Turing Award–

winning computer scientist Ron Rivest to the two of us when we brought up the

topic.

He was wearing sandals at the time.

The Ecstasy of Sorting

Sorting is at the very heart of what computers do. In fact, in many ways it was

sorting that brought the computer into being.

In the late nineteenth century, the American population was growing by 30%

every decade, and the number of “subjects of inquiry” in the US Census had

gone from just five in 1870 to more than two hundred in 1880. The tabulation of

the 1880 census took eight years—just barely finishing by the time the 1890

census began. As a writer at the time put it, it was a wonder “the clerks who

toiled at the irritating slips of tally paper … did not go blind and crazy.” The

whole enterprise was threatening to collapse under its own weight. Something

had to be done.

Inspired by the punched railway tickets of the time, an inventor by the name

of Herman Hollerith devised a system of punched manila cards to store

information, and a machine, which he called the Hollerith Machine, to count and

sort them. Hollerith was awarded a patent in 1889, and the government adopted

the Hollerith Machine for the 1890 census. No one had ever seen anything like

it. Wrote one awestruck observer, “The apparatus works as unerringly as the

mills of the Gods, but beats them hollow as to speed.” Another, however,

reasoned that the invention was of limited use: “As no one will ever use it but

governments, the inventor will not likely get very rich.” This prediction, which

Hollerith clipped and saved, would not prove entirely correct. Hollerith’s firm

merged with several others in 1911 to become the Computing-TabulatingRecording Company. A few years later it was renamed—to International

Business Machines, or IBM.

Sorting continued to spur the development of the computer through the next

century. The first code ever written for a “stored program” computer was a

program for efficient sorting. In fact, it was the computer’s ability to outsort

IBM’s dedicated card-sorting machines that convinced the US government their

enormous financial investment in a general-purpose machine was justified. By

the 1960s, one study estimated that more than a quarter of the computing

resources of the world were being spent on sorting. And no wonder—sorting is

essential to working with almost any kind of information. Whether it’s finding

the largest or the smallest, the most common or the rarest, tallying, indexing,

flagging duplicates, or just plain looking for the thing you want, they all

generally begin under the hood with a sort.

But sorting is more pervasive, even, than this. After all, one of the main

reasons things get sorted is to be shown in useful form to human eyes, which

means that sorting is also key to the human experience of information. Sorted

lists are so ubiquitous that—like the fish who asks, “What is water?”—we must

consciously work to perceive them at all. And then we perceive them

everywhere.

Our email inbox typically displays the top fifty messages of potentially

thousands, sorted by time of receipt. When we look for restaurants on Yelp

we’re shown the top dozen or so of hundreds, sorted by proximity or by rating.

A blog shows a cropped list of articles, sorted by date. The Facebook news feed,

Twitter stream, and Reddit homepage all present themselves as lists, sorted by

some proprietary measure. We refer to things like Google and Bing as “search

engines,” but that is something of a misnomer: they’re really sort engines. What

makes Google so dominant as a means of accessing the world’s information is

less that it finds our text within hundreds of millions of webpages—its 1990s

competitors could generally do that part well enough—but that it sorts those

webpages so well, and only shows us the most relevant ten.

The truncated top of an immense, sorted list is in many ways the universal

user interface.

Computer science gives us a way to understand what’s going on behind the

scenes in all of these cases, which in turn can offer us some insight for those

times when we are the one stuck making order—with our bills, our papers, our

books, our socks, probably more times each day than we realize. By quantifying

the vice (and the virtue) of mess, it also shows us the cases where we actually

shouldn’t make order at all.

What’s more, when we begin looking, we see that sorting isn’t just something

we do with information. It’s something we do with people. Perhaps the place

where the computer science of establishing rank is most unexpectedly useful is

on the sporting field and in the boxing ring—which is why knowing a little about

sorting might help explain how human beings are able to live together while

only occasionally coming to blows. That is to say, sorting offers some surprising

clues about the nature of society—that other, larger, and more important kind of

order that we make.

The Agony of Sorting

“To lower costs per unit of output, people usually increase the size of their

operations,” wrote J. C. Hosken in 1955, in the first scientific article published

on sorting. This is the economy of scale familiar to any business student. But

with sorting, size is a recipe for disaster: perversely, as a sort grows larger, “the

unit cost of sorting, instead of falling, rises.” Sorting involves steep

diseconomies of scale, violating our normal intuitions about the virtues of doing

things in bulk. Cooking for two is typically no harder than cooking for one, and

it’s certainly easier than cooking for one person twice. But sorting, say, a shelf

of a hundred books will take you longer than sorting two bookshelves of fifty

apiece: you have twice as many things to organize, and there are twice as many

places each of them could go. The more you take on, the worse it gets.

This is the first and most fundamental insight of sorting theory. Scale hurts.

From this we might infer that minimizing our pain and suffering when it

comes to sorting is all about minimizing the number of things we have to sort.

It’s true: one of the best preventives against the computational difficulty of sock

sorting is just doing your laundry more often. Doing laundry three times as

frequently, say, could reduce your sorting overhead by a factor of nine. Indeed,

if Hillis’s roommate stuck with his peculiar procedure but went thirteen days

between washes instead of fourteen, that alone would save him twenty-eight

pulls from the hamper. (And going just a single day longer between washes

would cost him thirty pulls more.)

Even at such a modest, fortnightly scope we can see the scale of sorting

beginning to grow untenable. Computers, though, must routinely sort millions of

items in a single go. For that, as the line from Jaws puts it, we’re going to need a

bigger boat—and a better algorithm.

But to answer the question of just how we ought to be sorting, and which

methods come out on top, we need to figure out something else first: how we’re

going to keep score.

Big-O: A Yardstick for the Worst Case

The Guinness Book of World Records attributes the record for sorting a deck of

cards to the Czech magician Zdeněk Bradáč. On May 15, 2008, Bradáč sorted a

52-card deck in just 36.16 seconds.* How did he do it? What sorting technique

delivered him the title? Though the answer would shed interesting light on

sorting theory, Bradáč declined to comment.

While we have nothing but respect for Bradáč’s skill and dexterity, we are

100% certain of the following: we can personally break his record. In fact, we

are 100% certain that we can attain an unbreakable record. All we need are

about

80,658,175,170,943,878,571,660,636,856,403,766,975,289,505,440,883,277,824,000,000,000,000

attempts at the title. This number, a bit over 80 unvigintillion, is 52 factorial, or

“52!” in mathematical notation—the number of ways that a deck of 52 cards can

possibly be ordered. By taking roughly that many attempts, sooner or later we

are bound to start with a shuffled deck that is in fact completely sorted by

chance. At that point we can proudly enter Christian-Griffiths into The Guinness

Book alongside a not-too-shabby sort time of 0m00s.

To be fair, we’d almost certainly be trying until the heat death of the universe

before we got our perfect record attempt. Nonetheless, this highlights the biggest

fundamental difference between record keepers and computer scientists. The fine

folks at Guinness care only about best-case performance (and beer). They’re

hardly blameworthy, of course: all records in sports reflect the single best

performance. Computer science, however, almost never cares about the best

case. Instead, computer scientists might want to know the average sort time of

someone like Bradáč: get him to sort all of the 80 unvigintillion deck orders, or a

reasonably sized sample, and score him on his average speed across all attempts.

(You can see why they don’t let computer scientists run these things.)

Moreover, a computer scientist would want to know the worst sort time.

Worst-case analysis lets us make hard guarantees: that a critical process will

finish in time, that deadlines won’t be blown. So for the rest of this chapter—and

the rest of this book, actually—we will be discussing only algorithms’ worstcase performance, unless noted otherwise.

Computer science has developed a shorthand specifically for measuring

algorithmic worst-case scenarios: it’s called “Big-O” notation. Big-O notation

has a particular quirk, which is that it’s inexact by design. That is, rather than

expressing an algorithm’s performance in minutes and seconds, Big-O notation

provides a way to talk about the kind of relationship that holds between the size

of the problem and the program’s running time. Because Big-O notation

deliberately sheds fine details, what emerges is a schema for dividing problems

into different broad classes.

Imagine you’re hosting a dinner party with n guests. The time required to

clean the house for their arrival doesn’t depend on the number of guests at all.

This is the rosiest class of problems there is: called “Big-O of one,” written O(1),

it is also known as “constant time.” Notably, Big-O notation doesn’t care a whit

how long the cleaning actually takes—just that it’s always the same, totally

invariant of the guest list. You’ve got the same work to do if you have one guest

as if you have ten, a hundred, or any other n.

Now, the time required to pass the roast around the table will be “Big-O of n,”

written O(n), also known as “linear time”—with twice the guests, you’ll wait

twice as long for the dish to come around. And again, Big-O notation couldn’t

care less about the number of courses that get served, or whether they go around

for second helpings. In each case, the time still depends linearly on the guest list

size—if you drew a graph of the number of guests vs. the time taken, it would be

a straight line. What’s more, the existence of any linear-time factors will, in BigO notation, swamp all constant-time factors. That is to say, passing the roast

once around the table, or remodeling your dining room for three months and then

passing the roast once around the table, are both, to a computer scientist,

effectively equivalent. If that seems crazy, remember that computers deal with

values of n that could easily be in the thousands, millions, or billions. In other

words, computer scientists are thinking about very, very big parties. With a guest

list in the millions, passing the roast once around would indeed make the home

remodel seem dwarfed to the point of insignificance.

What if, as the guests arrived, each one hugged the others in greeting? Your

first guest hugs you; your second guest has two hugs to give; your third guest,

three. How many hugs will there be in total? This turns out to be “Big-O of nsquared,” written O(n

2

) and also known as “quadratic time.” Here again, we only

care about the basic contours of the relationship between n and time. There’s no

O(2n

2

) for two hugs apiece, or O(n

2 + n) for hugs plus passing the food around,

or O(n

2 + 1) for hugs plus home cleaning. It’s all quadratic time, so O(n

2

) covers

everything.

Constant time, written O(1); linear time, written O(n); and quadratic time, written O(n

2

).

It gets worse from there. There’s “exponential time,” O(2

n

), where each

additional guest doubles your work. Even worse is “factorial time,” O(n!), a

class of problems so truly hellish that computer scientists only talk about it when

they’re joking—as we were in imagining shuffling a deck until it’s sorted—or

when they really, really wish they were.

The Squares: Bubble Sort and Insertion Sort

When then senator Obama visited Google in 2007, CEO Eric Schmidt jokingly

began the Q&A like a job interview, asking him, “What’s the best way to sort a

million thirty-two-bit integers?” Without missing a beat, Obama cracked a wry

smile and replied, “I think the Bubble Sort would be the wrong way to go.” The

crowd of Google engineers erupted in cheers. “He had me at Bubble Sort,” one

later recalled.

Obama was right to eschew Bubble Sort, an algorithm which has become

something of a punching bag for computer science students: it’s simple, it’s

intuitive, and it’s extremely inefficient.

Imagine you want to alphabetize your unsorted collection of books. A natural

approach would be just to scan across the shelf looking for out-of-order pairs—

Wallace followed by Pynchon, for instance—and flipping them around. Put

Pynchon ahead of Wallace, then continue your scan, looping around to the

beginning of the shelf each time you reach the end. When you make a complete

pass without finding any more out-of-order pairs on the entire shelf, then you

know the job is done.

This is Bubble Sort, and it lands us in quadratic time. There are n books out of

order, and each scan through the shelf can move each one at most one position.

(We spot a tiny problem, make a tiny fix.) So in the worst case, where the shelf

is perfectly backward, at least one book will need to be moved n positions. Thus

a maximum of n passes through n books, which gives us O(n

2

) in the worst

case.* It’s not terrible—for one thing, it’s worlds better than our O(n!) shuffletill-it’ssorted idea from earlier (in case you needed computer science to confirm

that). But all the same, that squared term can get daunting quickly. For instance,

it means that sorting five shelves of books will take not five times as long as

sorting a single shelf, but twenty-five times as long.

You might take a different tack—pulling all the books off the shelf and

putting them back in place one by one. You’d put the first book in the middle of

the shelf, then take the second and compare it to the first, inserting it either to the

right or to the left. Picking up the third book, you’d run through the books on the

shelf from left to right until you found the right spot to tuck it in. Repeating this

process, gradually all of the books would end up sorted on the shelf and you’d be

done.

Computer scientists call this, appropriately enough, Insertion Sort. The good

news is that it’s arguably even more intuitive than Bubble Sort and doesn’t have

quite the bad reputation. The bad news is that it’s not actually that much faster.

You still have to do one insertion for each book. And each insertion still

involves moving past about half the books on the shelf, on average, to find the

correct place. Although in practice Insertion Sort does run a bit faster than

Bubble Sort, again we land squarely, if you will, in quadratic time. Sorting

anything more than a single bookshelf is still an unwieldy prospect.

Breaking the Quadratic Barrier: Divide and Conquer

At this point, having seen two entirely sensible approaches fall into

unsustainable quadratic time, it’s natural to wonder whether faster sorting is

even possible.

The question sounds like it’s about productivity. But talk to a computer

scientist and it turns out to be closer to metaphysics—akin to thinking about the

speed of light, time travel, superconductors, or thermodynamic entropy. What

are the universe’s fundamental rules and limits? What is possible? What is

allowed? In this way computer scientists are glimpsing God’s blueprints every

bit as much as the particle physicists and cosmologists. What is the minimum

effort requred to make order?

Could we find a constant-time sort, O(1), one that (like cleaning the house

before the bevy of guests arrive) can sort a list of any size in the same amount of

time? Well, even just confirming that a shelf of n books is sorted cannot be done

in constant time, since it requires checking all n of them. So actually sorting the

books in constant time seems out of the question.

What about a linear-time sort, O(n), as efficient as passing a dish around a

table, where doubling the number of items to sort merely doubles the work?

Thinking about the examples above, it’s tough to imagine how that might work

either. The n

2

in each case comes from the fact that you need to move n books,

and the work required in each move scales with n as well. How would we get

from n moves of size n down to just n by itself? In Bubble Sort, our O(n

2

)

running time came from handling each of the n books and moving them as many

as n places each. In Insertion Sort, quadratic running time came from handling

each of the n books and comparing them to as many as n others before inserting

them. A linear-time sort means handling each book for constant time regardless

of how many others it needs to find its place among. Doesn’t seem likely.

So we know that we can do at least as well as quadratic time, but probably not

as well as linear time. Perhaps our limit lies somewhere between linear time and

quadratic time. Are there any algorithms between linear and quadratic, between

n and n × n?

There are—and they were hiding in plain sight.

As we mentioned earlier, information processing began in the US censuses of

the nineteenth century, with the development, by Herman Hollerith and later by

IBM, of physical punch-card sorting devices. In 1936, IBM began producing a

line of machines called “collators” that could merge two separately ordered

stacks of cards into one. As long as the two stacks were themselves sorted, the

procedure of merging them into a single sorted stack was incredibly

straightforward and took linear time: simply compare the two top cards to each

other, move the smaller of them to the new stack you’re creating, and repeat

until finished.

The program that John von Neumann wrote in 1945 to demonstrate the power

of the stored-program computer took the idea of collating to its beautiful and

ultimate conclusion. Sorting two cards is simple: just put the smaller one on top.

And given a pair of two-card stacks, both of them sorted, you can easily collate

them into an ordered stack of four. Repeating this trick a few times, you’d build

bigger and bigger stacks, each one of them already sorted. Soon enough, you

could collate yourself a perfectly sorted full deck—with a final climactic merge,

like a riffle shuffle’s order-creating twin, producing the desired result.

This approach is known today as Mergesort, one of the legendary algorithms

in computer science. As a 1997 paper put it, “Mergesort is as important in the

history of sorting as sorting in the history of computing.”

The power of Mergesort comes from the fact that it indeed ends up with a

complexity between linear and quadratic time—specifically, O(n log n), known

as “linearithmic” time. Each pass through the cards doubles the size of the sorted

stacks, so to completely sort n cards you’ll need to make as many passes as it

takes for the number 2, multiplied by itself, to equal n: the base-two logarithm,

in other words. You can sort up to four cards in two collation passes, up to eight

cards with a third pass, and up to sixteen cards with a fourth. Mergesort’s divideand-conquer approach inspired a host of other linearithmic sorting algorithms

that quickly followed on its heels. And to say that linearithmic complexity is an

improvement on quadratic complexity is a titanic understatement. In the case of

sorting, say, a census-level number of items, it’s the difference between making

twenty-nine passes through your data set … and three hundred million. No

wonder it’s the method of choice for large-scale industrial sorting problems.

Mergesort also has real applications in small-scale domestic sorting problems.

Part of the reason why it’s so widely used is that it can easily be parallelized. If

you’re still strategizing about that bookshelf, the Mergesort solution would be to

order a pizza and invite over a few friends. Divide the books evenly, and have

each person sort their own stack. Then pair people up and have them merge their

stacks. Repeat this process until there are just two stacks left, and merge them

one last time onto the shelf. Just try to avoid getting pizza stains on the books.

Beyond Comparison: Outsmarting the Logarithm

In an inconspicuous industrial park near the town of Preston, Washington,

tucked behind one nondescript gray entrance of many, lies the 2011 and 2013

National Library Sorting Champion. A long, segmented conveyor belt moves

167 books a minute—85,000 a day—through a bar code scanner, where they are

automatically diverted into bomb-bay doors that drop into one of 96 bins.

A Mergesort in action. Given a shelf of eight unsorted books, start by putting adjacent books into sorted

pairs. Then collate the pairs into ordered sets of four, and finally collate those sets to get a fully sorted

shelf.

The Preston Sort Center is one of the biggest and most efficient book-sorting

facilities in the world. It’s run by the King County Library System, which has

begun a healthy rivalry with the similarly equipped New York Public Library,

with the title going back and forth over four closely contested years. “King

County Library beating us this year?” said the NYPL’s deputy director of

BookOps, Salvatore Magaddino, before the 2014 showdown. “Fuhgeddaboutit.”

There’s something particularly impressive about the Preston Sort Center from

a theoretician’s point of view, too. The books going through its system are sorted

in O(n)—linear time.

In an important sense, the O(n log n) linearithmic time offered by Mergesort is

truly the best we can hope to achieve. It’s been proven that if we want to fully

sort n items via a series of head-to-head comparisons, there’s just no way to

compare them any fewer than O(n log n) times. It’s a fundamental law of the

universe, and there are no two ways around it.

But this doesn’t, strictly speaking, close the book on sorting. Because

sometimes you don’t need a fully ordered set—and sometimes sorting can be

done without any item-to-item comparisons at all. These two principles, taken

together, allow for rough practical sorts in faster than linearithmic time. This is

beautifully demonstrated by an algorithm known as Bucket Sort—of which the

Preston Sort Center is a perfect illustration.

In Bucket Sort, items are grouped together into a number of sorted categories,

with no regard for finer, intracategory sorting; that can come later. (In computer

science the term “bucket” simply refers to a chunk of unsorted data, but some of

the most powerful real-world uses of Bucket Sort, as at the KCLS, take the name

entirely literally.) Here’s the kicker: if you want to group n items into m buckets,

the grouping can be done in O(nm) time—that is, the time is simply proportional

to the number of items times the number of buckets. And as long as the number

of buckets is relatively small compared to the number of items, Big-O notation

will round that to O(n), or linear time.

The key to actually breaking the linearithmic barrier is knowing the

distribution from which the items you’re sorting are drawn. Poorly chosen

buckets will leave you little better than when you started; if all the books end up

in the same bin, for instance, you haven’t made any progress at all. Well-chosen

buckets, however, will divide your items into roughly equal-sized groups, which

—given sorting’s fundamental “scale hurts” nature—is a huge step toward a

complete sort. At the Preston Sort Center, whose job is to sort books by their

destination branch, rather than alphabetically, the choice of buckets is driven by

circulation statistics. Some branches have a greater circulation volume than

others, so they may have two bins allocated to them, or even three.

A similar knowledge of the material is useful to human sorters too. To see

sorting experts in action, we took a field trip to UC Berkeley’s Doe and Moffitt

Libraries, where there are no less than fifty-two miles of bookshelves to keep in

order—and it’s all done by hand. Books returned to the library are first placed in

a behind-the-scenes area, allocated to shelves designated by Library of Congress

call numbers. For example, one set of shelves there contains a jumble of all the

recently returned books with call numbers PS3000–PS9999. Then student

assistants load those books onto carts, putting up to 150 books in proper order so

they can be returned to the library shelves. The students get some basic training

in sorting, but develop their own strategies over time. After a bit of experience,

they can sort a full cart of 150 books in less than 40 minutes. And a big part of

that experience involves knowing what to expect.

Berkeley undergraduate Jordan Ho, a chemistry major and star sorter, talked

us through his process as he went through an impressive pile of books on the

PS3000–PS9999 shelves:

I know from experience that there’s a lot of 3500s, so I want to look for any books that are below

3500 and rough-sort those out. And once I do that, then I sort those more finely. After I sort the ones

under 3500, I know 3500 itself is a big section—3500–3599—so I want to make that a section itself.

If there are a lot of those I might want to fine-tune it even more: 3510s, 3520s, 3530s.

Jordan aims to get a group of 25 or so books onto his cart before putting them in

final order, which he does using an Insertion Sort. And his carefully developed

strategy is exactly the right way to get there: a Bucket Sort, with his wellinformed forecast of how many books he’ll have with various call numbers

telling him what his buckets should be.

Sort Is Prophylaxis for Search

Knowing all these sorting algorithms should come in handy next time you decide

to alphabetize your bookshelf. Like President Obama, you’ll know not to use

Bubble Sort. Instead, a good strategy—ratified by human and machine librarians

alike—is to Bucket Sort until you get down to small enough piles that Insertion

Sort is reasonable, or to have a Mergesort pizza party.

But if you actually asked a computer scientist to help implement this process,

the first question they would ask is whether you should be sorting at all.

Computer science, as undergraduates are taught, is all about tradeoffs. We’ve

already seen this in the tensions between looking and leaping, between exploring

and exploiting. And one of the most central tradeoffs is between sorting and

searching. The basic principle is this: the effort expended on sorting materials is

just a preemptive strike against the effort it’ll take to search through them later.

What the precise balance should be depends on the exact parameters of the

situation, but thinking about sorting as valuable only to support future search

tells us something surprising:

Err on the side of messiness.

Sorting something that you will never search is a complete waste; searching

something you never sorted is merely inefficient.

The question, of course, becomes how to estimate ahead of time what your

future usage will be.

The poster child for the advantages of sorting would be an Internet search

engine like Google. It seems staggering to think that Google can take the search

phrase you typed in and scour the entire Internet for it in less than half a second.

Well, it can’t—but it doesn’t need to. If you’re Google, you are almost certain

that (a) your data will be searched, (b) it will be searched not just once but

repeatedly, and (c) the time needed to sort is somehow “less valuable” than the

time needed to search. (Here, sorting is done by machines ahead of time, before

the results are needed, and searching is done by users for whom time is of the

essence.) All of these factors point in favor of tremendous up-front sorting,

which is indeed what Google and its fellow search engines do.

So, should you alphabetize your bookshelves? For most domestic

bookshelves, almost none of the conditions that make sorting worthwhile are

true. It’s fairly rare that we find ourselves searching for a particular title. The

costs of an unsorted search are pretty low: for every book, if we know roughly

where it is we can put our hands on it quickly. And the difference between the

two seconds it would take to find the book on a sorted shelf and the ten seconds

it would take to scan for it on an unsorted one is hardly a deal breaker. We rarely

need to find a title so urgently that it’s worth spending preparatory hours up front

to shave off seconds later on. What’s more, we search with our quick eyes and

sort with slow hands.

The verdict is clear: ordering your bookshelf will take more time and energy

than scanning through it ever will.

Your unsorted bookshelf might not be an everyday preoccupation, but your

email inbox almost certainly is—and it’s another domain where searching beats

sorting handily. Filing electronic messages by hand into folders takes about the

same amount of time as filing physical papers in the real world, but emails can

be searched much more efficiently than their physical counterparts. As the cost

of searching drops, sorting becomes less valuable.

Steve Whittaker is one of the world’s experts on how people handle their

email. A research scientist at IBM and professor at UC Santa Cruz, Whittaker,

for almost two decades, has been studying how people manage personal

information. (He wrote a paper on “email overload” in 1996, before many people

even had email.) In 2011, Whittaker led a study of the searching and sorting

habits of email users, resulting in a paper titled “Am I Wasting My Time

Organizing Email?” Spoiler alert: the conclusion was an emphatic Yes. “It’s

empirical, but it’s also experiential,” Whittaker points out. “When I interview

people about these kinds of organizational problems, that’s something that they

characteristically talk about, is that they sort of wasted a part of their life.”

Computer science shows that the hazards of mess and the hazards of order are

quantifiable and that their costs can be measured in the same currency: time.

Leaving something unsorted might be thought of as an act of procrastination—

passing the buck to one’s future self, who’ll have to pay off with interest what

we chose not to pay up front. But the whole story is subtler than that. Sometimes

mess is more than just the easy choice. It’s the optimal choice.

Sorts and Sports

The search-sort tradeoff suggests that it’s often more efficient to leave a mess.

Saving time isn’t the only reason we sort things, though: sometimes producing a

final order is an end in itself. And nowhere is that clearer than on the sporting

field.

In 1883, Charles Lutwidge Dodgson developed incredibly strong feelings

about the state of British lawn tennis. As he explains:

At a Lawn Tennis Tournament, where I chanced, some while ago, to be a spectator, the present

method of assigning prizes was brought to my notice by the lamentations of one of the Players, who

had been beaten (and had thus lost all chance of a prize) early in the contest, and who had had the

mortification of seeing the 2nd prize carried off by a Player whom he knew to be quite inferior to

himself.

Normal spectators might chalk up such “lamentations” to little more than the

sting of defeat, but Dodgson was no ordinary sympathetic ear. He was an Oxford

lecturer in mathematics, and the sportsman’s complaints sent him on a deep

investigation of the nature of sports tournaments.

Dodgson was more than just an Oxford mathematician—in fact, he’s barely

remembered as having been one. Today he’s best known by his pen name, Lewis

Carroll, under which he wrote Alice’s Adventures in Wonderland and many other

beloved works of nineteenth-century literature. Fusing his mathematical and

literary talents, Dodgson produced one of his lesser-known works: “Lawn

Tennis Tournaments: The True Method of Assigning Prizes with a Proof of the

Fallacy of the Present Method.”

Dodgson’s complaint was directed at the structure of the Single Elimination

tournament, where players are paired off with one another and eliminated from

competition as soon as they lose a single match. As Dodgson forcefully argued,

the true second-best player could be any of the players eliminated by the best—

not just the last-eliminated one. Ironically, in the Olympics we do hold bronze

medal matches, by which we appear to acknowledge that the Single Elimination

format doesn’t give us enough information to determine third place.* But in fact

this format doesn’t tell us enough to determine second place either—or, indeed,

anything except the winner. As Dodgson put it, “The present method of

assigning prizes is, except in the case of the first prize, entirely unmeaning.”

Said plainly, the silver medal is a lie.

“As a mathematical fact,” he continued, “the chance that the 2nd best Player

will get the prize he deserves is only 16/31sts; while the chance that the best 4

shall get their proper prizes is so small, that the odds are 12 to 1 against its

happening!”

Despite the powers of his pen, it appears that Dodgson had little impact on the

world of lawn tennis. His solution, an awkward take on triple elimination where

the defeat of someone who had defeated you could also eliminate you, never

caught on. But if Dodgson’s solution was cumbersome, his critique of the

problem was nevertheless spot on. (Alas, silver medals are still being handed out

in Single Elimination tournaments to this day.)

But there’s also a deeper insight in Dodgson’s logic. We humans sort more

than our data, more than our possessions. We sort ourselves.

The World Cup, the Olympics, the NCAA, NFL, NHL, NBA, and MLB—all

of these implicitly implement sorting procedures. Their seasons, ladders, and

playoffs are algorithms for producing rank order.

One of the most familiar algorithms in sports is the Round-Robin format,

where each of n teams eventually plays every one of the other n − 1 teams.

While arguably the most comprehensive, it’s also one of the most laborious.

Having every team grapple with every other team is like having guests exchange

hugs at our dinner party: the dreaded O(n

2

), quadratic time.

Ladder tournaments—popular in sports like badminton, squash, and

racquetball—put players in a linear ranking, with each player allowed to issue a

direct challenge to the player immediately above them, exchanging places if they

prevail. Ladders are the Bubble Sorts of the athletic world and are thus also

quadratic, requiring O(n

2

) games to reach a stable ranking.

Perhaps the most prevalent tournament format, however, is a bracket

tournament—as in the famous NCAA basketball “March Madness,” among

many others. The March Madness tournament progresses from the “Round of

64” and the “Round of 32” to the “Sweet 16,” “Elite Eight,” “Final Four,” and

the finals. Each round divides the field in half: does that sound familiarly

logarithmic? These tournaments are effectively Mergesort, beginning with

unsorted pairs of teams and collating, collating, collating them.

We know that Mergesort operates in linearithmic time—O(n log n)—and so,

given that there are 64 teams, we can expect to only need something like 6

rounds (192 games), rather than the whopping 63 rounds (2,016 games) it would

take to do a ladder or Round-Robin. That’s a huge improvement: algorithm

design at work.

Six rounds of March Madness sounds about right, but wait a second: 192

games? The NCAA tournament is only 63 games long.

In fact, March Madness is not a complete Mergesort—it doesn’t produce a full

ordering of all 64 teams. To truly rank the teams, we’d need an extra set of

games to determine second place, another for third, and so on—taking a

linearithmic number of games in sum. But March Madness doesn’t do that.

Instead, just like the lawn tennis tournament that Dodgson complained about, it

uses a Single Elimination format where the eliminated teams are left unsorted.

The advantage is that it runs in linear time: since every game eliminates exactly

one team, in order to have one team left standing you need just n − 1 games—a

linear number. The disadvantage is that, well, you never really figure out the

standings aside from first place.

Ironically, in Single Elimination no tournament structure is actually necessary

at all. Any 63 games will yield a single undefeated champion. For instance, you

could simply have a single “king of the hill” team take on challengers one by

one until it is dethroned, at which point whoever defeated it takes over its spot

and continues. This format would have the drawback of needing 63 separate

rounds, however, as games couldn’t happen in parallel; also, one team could

potentially have to play as many as 63 games in a row, which might not be ideal

from a fatigue standpoint.

Though born well over a century after Dodgson, perhaps no one carries

forward his mathematical take on sporting into the twenty-first century as

strongly as Michael Trick. We met Trick back in our discussion of optimal

stopping, but in the decades since his hapless application of the 37% Rule to his

love life he’s become not only a husband and a professor of operations research

—he’s now also one of the principal schedulers for Major League Baseball and

for NCAA conferences like the Big Ten and the ACC, using computer science to

decide the year’s matchups.

As Trick points out, sports leagues aren’t concerned with determining the

rankings as quickly and expeditiously as possible. Instead, sports calendars are

explicitly designed to maintain tension throughout the season, something that

has rarely been a concern of sorting theory.

For instance in Major League Baseball, you often have races to see who is going to win the division.

Now, if we ignored the divisional setup, some of those races might get resolved fairly early in the

season. But instead what we do is we make certain in the last five weeks, everybody plays everybody

else within their division. The purpose of that is it doesn’t matter who’s in a divisional race: they’re

going to have to play their next closest opponent at least six games in the final five weeks of the

season. That allows for more interest in the schedule or interest in the season because in this case,

uncertainty is delayed in its resolution.

What’s more, sports are not, of course, always designed strictly to minimize

the number of games. Without remembering this, some aspects of sports

scheduling would otherwise seem mysterious to a computer scientist. As Trick

says of baseball’s regular season of 2,430 games, “We know that n log n is the

right number of comparisons to do a full sort. That can get you everybody. Why

do they do n

2

in order to just get, in some sense, the top, if that’s all they care

about?” In other words, why do a full O(n

2

) Round-Robin and then some, if we

know we can do a full sort in linearithmic time, and can crown an undefeated

Single Elimination champion in less than n games? Well, minimizing the

number of games isn’t actually in the league’s interest. In computer science

unnecessary comparisons are always bad, a waste of time and effort. But in

sports that’s far from the case. In many respects, after all, the games themselves

are the point.

Griping Rights: Noise and Robustness

Another, perhaps even more important way of training an algorithmic lens on

sports is to ask not what confidence we should have in the silver medal, but what

confidence we should have in the gold.

As Michael Trick explains, in some sports, “for instance baseball, a team is

going to lose 30% of their games and a team is going to win 30% of their games

practically no matter who they are.” This has disturbing implications for the

Single Elimination format. If NCAA basketball games, say, are won by the

stronger team 70% of the time, and winning the tournament involves prevailing

in 6 straight games, then the best team has only a 0.70 to the 6th power—less

than 12%—chance of winning the tournament! Put another way, the tournament

would crown the league’s truly best team just once a decade.

It may be that in some sports, having even 70% confidence in a game’s

outcome might be putting too much stock in the final score. UCSD physicist

Tom Murphy applied numerical modeling techniques to soccer and concluded

that soccer’s low scores make game outcomes much closer to random than most

fans would prefer to imagine. “A 3:2 score gives the winning team only a 5-in-8

chance of actually being a better team … Personally, I don’t find this to be very

impressive. Even a 6:1 blowout leaves a 7% chance that it was a statistical

fluke.”

Computer scientists call this phenomenon noise. All of the sorting algorithms

that we’ve considered thus far assume perfect, flawless, foolproof comparisons,

ones that never mess up and mistakenly judge the lesser of two quantities to be

the greater. Once you allow for a “noisy comparator,” some of computer

science’s most hallowed algorithms go out the window—and some of its most

maligned have their day of redemption.

Dave Ackley, professor of computer science at the University of New Mexico,

works at the intersection of computer science and “artificial life”—he believes

computers can stand to learn a few things from biology. For starters, organisms

live in a world where few processes have anywhere near the level of reliability

that computers depend on, so they are built from the ground up for what

researchers call robustness. It’s time, argues Ackley, that we started recognizing

the virtues of robustness in algorithms too.

Thus, while the authoritative programming tome Sorting and Searching boldly

declares that “bubble sort has no apparent redeeming features,” the research of

Ackley and his collaborators suggests that there may be a place for algorithms

like Bubble Sort after all. Its very inefficiency—moving items only one position

at a time—makes it fairly robust against noise, far more robust than faster

algorithms like Mergesort, in which each comparison potentially moves an item

a long way. Mergesort’s very efficiency makes it brittle. An early error in a

Mergesort is like a fluke loss in the first round of a Single Elimination

tournament, which can not only dash a favored team’s championship hopes but

also permanently relegate them to the bottom half of the results.* In a Ladder

tournament, on the other hand, as in a Bubble Sort, a fluke loss would only set a

player back a single place in the standings.

But in fact it isn’t Bubble Sort that emerges as the single best algorithm in the

face of a noisy comparator. The winner of that particular honor is an algorithm

called Comparison Counting Sort. In this algorithm, each item is compared to

all the others, generating a tally of how many items it is bigger than. This

number can then be used directly as the item’s rank. Since it compares all pairs,

Comparison Counting Sort is a quadratic-time algorithm, like Bubble Sort. Thus

it’s not a popular choice in traditional computer science applications, but it’s

exceptionally fault-tolerant.

This algorithm’s workings should sound familiar. Comparison Counting Sort

operates exactly like a Round-Robin tournament. In other words, it strongly

resembles a sports team’s regular season—playing every other team in the

division and building up a win-loss record by which they are ranked.

That Comparison Counting Sort is the single most robust sorting algorithm

known, quadratic or better, should offer something very specific to sports fans: if

your team doesn’t make the playoffs, don’t whine. The Mergesort postseason is

chancy, but the Comparison Counting regular season is not; championship rings

aren’t robust, but divisional standings are literally as robust as it gets. Put

differently, if your team is eliminated early in the postseason, it’s tough luck.

But if your team fails to get to the postseason, it’s tough truth. You may get

sports-bar sympathy from your fellow disappointed fans, but you won’t get any

from a computer scientist.

Blood Sort: Pecking Orders and Dominance Hierarchies

In all the examples we’ve considered so far, the sorting process in every case has

been imposed from the top down: a librarian shelving books, the NCAA telling

teams whom to play and when. But what if head-to-head comparisons happened

only voluntarily? What does sorting look like when it emerges organically, from

the bottom up?

It might look something like online poker.

Unlike most sports, which are governed by a ruling body of some kind, poker

remains somewhat anarchic despite exploding in popularity over the past decade.

Though some high-profile tournaments do explicitly sort their contestants (and

remunerate them accordingly), a substantial portion of poker is still played in

what are known as “cash games,” where two or more players spontaneously

agree to play with real money on the line with every hand.

Virtually no one knows this world more deeply than Isaac Haxton, one of the

world’s best cash-game poker players. In most sports it’s sufficient to be as good

as possible, and the less self-conscious one is about one’s skills the better. But,

Haxton explains, “In some ways the most important skill as a professional poker

player is to be able to evaluate how good you are. If you’re anything short of the

very best poker player in the world, you can be pretty much assured of going

broke if you are endlessly willing to play people better than you.”

Haxton is a heads-up, no-limit specialist: “heads-up” meaning one-on-one

poker, and “no-limit” meaning just that—the highest stakes, limited only by

what they can bankroll and stomach. In multi-handed poker cash games, there

will often be one weak player—a wealthy amateur, for instance—feeding a table

full of professionals, who then don’t much care who among them is better than

whom. In the world of heads-up, it’s different. “There has to be a disagreement

between you and them about who’s better—or somebody has to be willingly

losing.”

So what happens when there’s a fairly established consensus and no one’s

willing to play anyone better than they are? You get something that looks a lot

like players simply jockeying for seats. Most online poker sites have only a finite

number of tables available. “So if you want to play heads-up no-limit, with

blinds of fifty and one hundred dollars, there are only ten available tables for

that,” says Haxton, “and so only the consensus ten best players who are out right

now … sit and wait for someone to show up who wants to play.” And if a

superior player arrives and sits down at one of these tables? If the person sitting

isn’t willing to ante up, they scram.

“Imagine two monkeys,” says Christof Neumann. “One is sitting and feeding

in its spot, very peacefully, and another one is coming up [to] where the other

guy is sitting. And that guy would then stand up and leave.”

Neumann isn’t making a poker metaphor. He’s a behavioral biologist at the

University of Neuchâtel who studies dominance in macaques. What he’s just

described is known as displacement.

Displacement happens when an animal uses its knowledge of the hierarchy to

determine that a particular confrontation simply isn’t worth it. In many animal

societies, resources and opportunities—food, mates, preferred spaces, and so

forth—are scarce, and somehow it must be decided who gets what. Establishing

an order ahead of time is less violent than coming to blows every time a mating

opportunity or a prime spot of grass becomes available. Though we may cringe

when we see creatures turning their claws and beaks on each other, biologists

tend to think of pecking orders as the violence that preempts violence.

Sound familiar? It’s the search-sort tradeoff.

The creation of a pecking order is a pugilistic solution to a fundamentally

computational problem. For this reason, incidentally, debeaking chickens on

farms may be a well-intentioned but counterproductive approach: it removes the

authority of individual fights to resolve the order, and therefore makes it much

harder for the flock to run any sorting procedure at all. So the amount of

antagonism within the flock in many cases actually increases.

Looking at animal behavior from the perspective of computer science suggests

several things. For one, it implies that the number of hostile confrontations

encountered by each individual will grow substantially—at least logarithmically,

and perhaps quadratically—as the group gets bigger. Indeed, studies of

“agonistic behavior” in hens have found that “aggressive acts per hen increased

as group size increased.” Sorting theory thus suggests that the ethical raising of

livestock may include limiting the size of the flock or herd. (In the wild, feral

chickens roam in groups of ten to twenty, far smaller than flock sizes on

commercial farms.) The studies also show that aggression appears to go away

after a period of some weeks, unless new members are added to the flock—

corroborating the idea that the group is sorting itself.

The key to thinking about decentralized sorting in nature, argues Jessica

Flack, codirector of the Center for Complexity and Collective Computation at

UW–Madison, is that dominance hierarchies are ultimately information

hierarchies. There’s a significant computational burden to these decentralized

sorting systems, Flack points out. The number of fights in, say, a group of

macaques is minimized only to the extent that every monkey has a detailed—and

similar—understanding of the hierarchy. Otherwise violence will ensue.

If it comes down to how good the protagonists are at keeping track of the

current order, we might expect to see fewer confrontations as animals become

better able to reason and remember. And perhaps humans do come closest to

optimally efficient sorting. As Haxton says of the poker world, “I’m one of the

top heads-up, no-limit hold ’em players in the world, and in my head I have a

fairly specific ranking of who I think the twenty or so best players are, and I

think each of them has a similar ranking in their mind. I think there is a pretty

high degree of consensus about what the list looks like.” Only when these

rankings differ will cash games ensue.

A Race Instead of a Fight

We’ve now seen two separate downsides to the desire of any group to sort itself.

You have, at minimum, a linearithmic number of confrontations, making

everyone’s life more combative as the group grows—and you also oblige every

competitor to keep track of the ever-shifting status of everyone else, otherwise

they’ll find themselves fighting battles they didn’t need to. It taxes not only the

body but the mind.

But it doesn’t have to be that way. There are ways of making order without

the costs.

There’s one sporting contest, for instance, where tens of thousands of

competitors are completely sorted within the time that it takes to hold just a

single event. (A Round-Robin tournament with ten thousand players, on the

other hand, would require a hundred million matchups.) The only caveat is that

the time required for the event is determined by its slowest competitors. This

sporting contest is the marathon, and it suggests something critical: a race is

fundamentally different from a fight.

Consider the difference between boxers and skiers, between fencers and

runners. An Olympic boxer must risk concussion O(log n) times, usually from 4

to 6, to make it to the podium; allowing a greater number of athletes into the

games would imperil the health of all. But a skeleton racer or ski jumper or

halfpipe specialist needs to make only a constant number of gambles with

gravity, no matter the size of the field. A fencer puts herself at her opponent’s

mercy O(log n) times, but a marathoner must endure only one race. Being able to

assign a simple numerical measure of performance results in a constant-time

algorithm for status.

This move from “ordinal” numbers (which only express rank) to “cardinal”

ones (which directly assign a measure to something’s caliber) naturally orders a

set without requiring pairwise comparisons. Accordingly, it makes possible

dominance hierarchies that don’t require direct head-to-head matchups. The

Fortune 500 list, to the extent that it creates a kind of corporate hierarchy, is one

of these. To find the most valuable company in the United States, analysts don’t

need to perform due diligence comparing Microsoft to General Motors, then

General Motors to Chevron, Chevron to Walmart, and so forth. These seemingly

apples-to-oranges contests (how many enterprise software installations equal

how many oil futures?) become apples-to-apples in the medium of dollars.

Having a benchmark—any benchmark—solves the computational problem of

scaling up a sort.

In Silicon Valley, for instance, there’s an adage about meetings: “You go to

the money, the money doesn’t come to you.” Vendors go to founders, founders

go to venture capitalists, venture capitalists go to their limited partners. It’s

possible for the individuals to resent the basis of this hierarchy, but not really to

contest its verdict. As a result, individual pairwise interactions take place with a

minimum of jockeying for status. By and large, any pair of people can tell,

without needing to negotiate, who is supposed to show what level of respect to

whom. Everyone knows where to meet.

Likewise, while maritime right-of-way is governed in theory by an extremely

elaborate set of conventions, in practice one straightforward principle determines

which ships give way to which: the “Law of Gross Tonnage.” Quite simply, the

smaller ship gets out of the way of the larger one. Some animals are also lucky

enough to have such clear-cut dominance hierarchies. As Neumann observes,

“Look at fish, for example: the bigger one is the dominant one. It’s very simple.”

And because it’s so simple, it’s peaceful. Unlike chickens and primates, fish

make order without shedding blood.

When we think about the factors that make large-scale human societies

possible, it’s easy to focus on technologies: agriculture, metals, machinery. But

the cultural practice of measuring status with quantifiable metrics might be just

as important. Money, of course, need not be the criterion; a rule like “respect

your elders,” for instance, likewise settles questions of people’s status by

reference to a common quantity. And the same principle is at work between

nations as within them. It is often noted that a benchmark like national GDP—

which underlies the invite lists to diplomatic summits such as the G20—is a

crude, imperfect measurement. But the existence of any benchmark at all

transforms the question of national status from one demanding at least a

linearithmic number of tussles and resolutions into something with a single

reference point that ranks all. Given that nation-to-nation status disputes often

take military form, this saves not only time but lives.

Linearithmic numbers of fights might work fine for small-scale groups; they

do in nature. But in a world where status is established through pairwise

comparisons—whether they involve exchanging rhetoric or gunfire—the amount

of confrontation quickly spirals out of control as society grows. Operating at

industrial scale, with many thousands or millions of individuals sharing the same

space, requires a leap beyond. A leap from ordinal to cardinal.

Much as we bemoan the daily rat race, the fact that it’s a race rather than a

fight is a key part of what sets us apart from the monkeys, the chickens—and, for

that matter, the rats.

4 Caching

Forget About It

In the practical use of our intellect, forgetting is as important a function as

remembering.

—WILLIAM JAMES

You have a problem. Your closet is overflowing, spilling shoes, shirts, and

underwear onto the floor. You think, “It’s time to get organized.” Now you have

two problems.

Specifically, you first need to decide what to keep, and second, how to arrange

it. Fortunately, there is a small industry of people who think about these twin

problems for a living, and they are more than happy to offer their advice.

On what to keep, Martha Stewart says to ask yourself a few questions: “How

long have I had it? Does it still function? Is it a duplicate of something I already

own? When was the last time I wore it or used it?” On how to organize what you

keep, she recommends “grouping like things together,” and her fellow experts

agree. Francine Jay, in The Joy of Less, stipulates, “Hang all your skirts together,

pants together, dresses together, and coats together.” Andrew Mellen, who bills

himself as “The Most Organized Man in America,” dictates, “Items will be

sorted by type—all slacks together, shirts together, coats, etc. Within each type,

they’re further sorted by color and style—long-sleeved or short-sleeved, by

neckline, etc.” Other than the sorting problem this could entail, it looks like good

advice; it certainly seems unanimous.

Except that there is another, larger industry of professionals who also think

obsessively about storage—and they have their own ideas.

Your closet presents much the same challenge that a computer faces when

managing its memory: space is limited, and the goal is to save both money and

time. For as long as there have been computers, computer scientists have

grappled with the dual problems of what to keep and how to arrange it. The

results of these decades of effort reveal that in her four-sentence advice about

what to toss, Martha Stewart actually makes several different, and not fully

compatible, recommendations—one of which is much more critical than the

others.

The computer science of memory management also reveals exactly how your

closet (and your office) ought to be arranged. At first glance, computers appear

to follow Martha Stewart’s maxim of “grouping like things together.” Operating

systems encourage us to put our files into folders, like with like, forming

hierarchies that branch as their contents become ever more specific. But just as

the tidiness of a scholar’s desk may hide the messiness of their mind, so does the

apparent tidiness of a computer’s file system obscure the highly engineered

chaos of how data is actually being stored underneath the nested-folder veneer.

What’s really happening is called caching.

Caching plays a critical role in the architecture of memory, and it underlies

everything from the layout of processor chips at the millimeter scale to the

geography of the global Internet. It offers a new perspective on all the various

storage systems and memory banks of human life—not only our machines, but

also our closets, our offices, our libraries. And our heads.

The Memory Hierarchy

A certain woman had a very sharp consciousness but almost no memory.…

She remembered enough to work, and she worked hard.

—LYDIA DAVIS

Starting roughly around 2008, anyone in the market for a new computer has

encountered a particular conundrum when choosing their storage option. They

must make a tradeoff between size and speed. The computer industry is currently

in transition from hard disk drives to solid-state drives; at the same price point, a

hard disk will offer dramatically greater capacity, but a solid-state drive will

offer dramatically better performance—as most consumers now know, or soon

discover when they begin to shop.

What casual consumers may not know is that this exact tradeoff is being made

within the machine itself at a number of scales—to the point where it’s

considered one of the fundamental principles of computing.

In 1946, Arthur Burks, Herman Goldstine, and John von Neumann, working

at the Institute for Advanced Study in Princeton, laid out a design proposal for

what they called an electrical “memory organ.” In an ideal world, they wrote, the

machine would of course have limitless quantities of lightning-fast storage, but

in practice this wasn’t possible. (It still isn’t.) Instead, the trio proposed what

they believed to be the next best thing: “a hierarchy of memories, each of which

has greater capacity than the preceding but which is less quickly accessible.” By

having effectively a pyramid of different forms of memory—a small, fast

memory and a large, slow one—maybe we could somehow get the best of both.

The basic idea behind a memory hierarchy should be intuitive to anyone who

has ever used a library. If you are researching a topic for a paper, let’s say, there

are some books you might need to refer to on multiple occasions. Rather than go

back to the library each time, you of course check out the relevant books and

take them home to your desk, where you can access them more easily.

In computing, this idea of a “memory hierarchy” remained just a theory until

the development in 1962 of a supercomputer in Manchester, England, called

Atlas. Its principal memory consisted of a large drum that could be rotated to

read and write information, not unlike a wax phonograph cylinder. But Atlas

also had a smaller, faster “working” memory built from polarized magnets. Data

could be read from the drum to the magnets, manipulated there with ease, and

the results then written back to the drum.

Shortly after the development of Atlas, Cambridge mathematician Maurice

Wilkes realized that this smaller and faster memory wasn’t just a convenient

place to work with data before saving it off again. It could also be used to

deliberately hold on to pieces of information likely to be needed later,

anticipating similar future requests—and dramatically speeding up the operation

of the machine. If what you needed was still in the working memory, you

wouldn’t have to load it from the drum at all. As Wilkes put it, the smaller

memory “automatically accumulates to itself words that come from a slower

main memory, and keeps them available for subsequent use without it being

necessary for the penalty of main memory access to be incurred again.”

The key, of course, would be managing that small, fast, precious memory so it

had what you were looking for as often as possible. To continue the library

analogy, if you’re able to make just one trip to the stacks to get all the books you

need, and then spend the rest of the week working at home, that’s almost as good

as if every book in the library had already been available at your desk. The more

trips back to the library you make, the slower things go, and the less your desk is

really doing for you.

Wilkes’s proposal was implemented in the IBM 360/85 supercomputer later in

the 1960s, where it acquired the name of the “cache.” Since then, caches have

appeared everywhere in computer science. The idea of keeping around pieces of

information that you refer to frequently is so powerful that it is used in every

aspect of computation. Processors have caches. Hard drives have caches.

Operating systems have caches. Web browsers have caches. And the servers that

deliver content to those browsers also have caches, making it possible to

instantly show you the same video of a cat riding a vacuum cleaner that millions

of … But we’re getting ahead of ourselves a bit.

The story of the computer over the past fifty-plus years has been painted as

one of exponential growth year after year—referencing, in part, the famously

accurate “Moore’s Law” prediction, made by Intel’s Gordon Moore in 1975, that

the number of transistors in CPUs would double every two years. What hasn’t

improved at that rate is the performance of memory, which means that relative to

processing time, the cost of accessing memory is also increasing exponentially.

The faster you can write your papers, for instance, the greater the loss of

productivity from each trip to the library. Likewise, a factory that doubles its

manufacturing speed each year—but has the same number of parts shipped to it

from overseas at the same sluggish pace—will mean little more than a factory

that’s twice as idle. For a while it seemed that Moore’s Law was yielding little

except processors that twiddled their thumbs ever faster and ever more of the

time. In the 1990s this began to be known as the “memory wall.”

Computer science’s best defense against hitting that wall has been an ever

more elaborate hierarchy: caches for caches for caches, all the way down.

Modern consumer laptops, tablets, and smartphones have on the order of a sixlayer memory hierarchy, and managing memory smartly has never been as

important to computer science as it is today.

So let’s start with the first question that comes to mind about caches (or

closets, for that matter). What do we do when they get full?

Eviction and Clairvoyance

Depend upon it there comes a time when for every addition of knowledge

you forget something that you knew before. It is of the highest importance,

therefore, not to have useless facts elbowing out the useful ones.

—SHERLOCK HOLMES

When a cache fills up, you are obviously going to need to make room if you

want to store anything else, and in computer science this making of room is

called “cache replacement” or “cache eviction.” As Wilkes wrote, “Since the

[cache] can only be a fraction of the size of the main memory, words cannot be

preserved in it indefinitely, and there must be wired into the system an algorithm

by which they are progressively overwritten.” These algorithms are known as

“replacement policies” or “eviction policies,” or simply as caching algorithms.

IBM, as we’ve seen, played an early role in the deployment of caching

systems in the 1960s. Unsurprisingly, it was also the home of seminal early

research on caching algorithms—none, perhaps, as important as that of László

“Les” Bélády. Bélády was born in 1928 in Hungary, where he studied as a

mechanical engineer before fleeing to Germany during the 1956 Hungarian

Revolution with nothing but a satchel containing “one change of underwear and

my graduation paper.” From Germany he went to France, and in 1961

immigrated to the United States, bringing his wife, “an infant son and $1,000 in

my pocket, and that’s it.” It seems he had acquired a finely tuned sense of what

to keep and what to leave behind by the time he found himself at IBM, working

on cache eviction.

Bélády’s 1966 paper on caching algorithms would become the most cited

piece of computer science research for fifteen years. As it explains, the goal of

cache management is to minimize the number of times you can’t find what

you’re looking for in the cache and must go to the slower main memory to find

it; these are known as “page faults” or “cache misses.” The optimal cache

eviction policy—essentially by definition, Bélády wrote—is, when the cache is

full, to evict whichever item we’ll need again the longest from now.

Of course, knowing exactly when you’ll need something again is easier said

than done.

The hypothetical all-knowing, prescient algorithm that would look ahead and

execute the optimal policy is known today in tribute as Bélády’s Algorithm.

Bélády’s Algorithm is an instance of what computer scientists call a

“clairvoyant” algorithm: one informed by data from the future. It’s not

necessarily as crazy as it sounds—there are cases where a system might know

what to expect—but in general clairvoyance is hard to come by, and software

engineers joke about encountering “implementation difficulties” when they try

to deploy Bélády’s Algorithm in practice. So the challenge is to find an

algorithm that comes as close to clairvoyance as we can get, for all those times

when we’re stuck firmly in the present and can only guess at what lies ahead.

We could just try Random Eviction, adding new data to the cache and

overwriting old data at random. One of the startling early results in caching

theory is that, while far from perfect, this approach is not half bad. As it

happens, just having a cache at all makes a system more efficient, regardless of

how you maintain it. Items you use often will end up back in the cache soon

anyway. Another simple strategy is First-In, First-Out (FIFO), where you evict

or overwrite whatever has been sitting in the cache the longest (as in Martha

Stewart’s question “How long have I had it?”). A third approach is Least

Recently Used (LRU): evicting the item that’s gone the longest untouched

(Stewart’s “When was the last time I wore it or used it?”).

It turns out that not only do these two mantras of Stewart’s suggest very

different policies, one of her suggestions clearly outperforms the other. Bélády

compared Random Eviction, FIFO, and variants of LRU in a number of

scenarios and found that LRU consistently performed the closest to

clairvoyance. The LRU principle is effective because of something computer

scientists call “temporal locality”: if a program has called for a particular piece

of information once, it’s likely to do so again in the near future. Temporal

locality results in part from the way computers solve problems (for example,

executing a loop that makes a rapid series of related reads and writes), but it

emerges in the way people solve problems, too. If you are working on your

computer, you might be switching among your email, a web browser, and a word

processor. The fact that you accessed one of these recently is a clue that you’re

likely to do so again, and, all things being equal, the program that you haven’t

been using for the longest time is also probably the one that won’t be used for

some time to come.

In fact, this principle is even implicit in the interface that computers show to

their users. The windows on your computer screen have what’s called a “Zorder,” a simulated depth that determines which programs are overlaid on top of

which. The least recently used end up at the bottom. As former creative lead for

Firefox, Aza Raskin, puts it, “Much of your time using a modern browser

(computer) is spent in the digital equivalent of shuffling papers.” This

“shuffling” is also mirrored exactly in the Windows and Mac OS task switching

interfaces: when you press Alt + Tab or Command + Tab, you see your

applications listed in order from the most recently to the least recently used.

The literature on eviction policies goes about as deep as one can imagine—

including algorithms that account for frequency as well as recency of use,

algorithms that track the time of the next-to-last access rather than the last one,

and so on. But despite an abundance of innovative caching schemes, some of

which can beat LRU under the right conditions, LRU itself—and minor tweaks

thereof—is the overwhelming favorite of computer scientists, and is used in a

wide variety of deployed applications at a variety of scales. LRU teaches us that

the next thing we can expect to need is the last one we needed, while the thing

we’ll need after that is probably the second-most-recent one. And the last thing

we can expect to need is the one we’ve already gone longest without.

Unless we have good reason to think otherwise, it seems that our best guide to

the future is a mirror image of the past. The nearest thing to clairvoyance is to

assume that history repeats itself—backward.

Turning the Library Inside Out

Deep within the underground Gardner Stacks at the University of California,

Berkeley, behind a locked door and a prominent “Staff Only” notice, totally offlimits to patrons, is one of the jewels of the UC library system. Cormac

McCarthy, Thomas Pynchon, Elizabeth Bishop, and J. D. Salinger; Anaïs Nin,

Susan Sontag, Junot Díaz, and Michael Chabon; Annie Proulx, Mark Strand, and

Philip K. Dick; William Carlos Williams, Chuck Palahniuk, and Toni Morrison;

Denis Johnson, Juliana Spahr, Jorie Graham, and David Sedaris; Sylvia Plath,

David Mamet, David Foster Wallace, and Neil Gaiman … It isn’t the library’s

rare book collection; it’s its cache.

As we have already discussed, libraries are a natural example of a memory

hierarchy when used in concert with our own desk space. In fact, libraries in

themselves, with their various sections and storage facilities, are a great example

of a memory hierarchy with multiple levels. As a consequence, they face all

sorts of caching problems. They have to decide which books to put in the limited

display space at the front of the library, which to keep in their stacks, and which

to consign to offsite storage. The policy for which books to shunt offsite varies

from library to library, but almost all use a version of LRU. “For the Main

Stacks, for example,” says Beth Dupuis, who oversees the process in the UC

Berkeley libraries, “if an item hasn’t been used in twelve years, that’s the

cutoff.”

At the other end of the spectrum from the books untouched in a dozen years is

the library’s “rough sorting” area, which we visited in the previous chapter. This

is where books go just after they are returned, before they’re fully sorted and

shelved once again in the stacks. The irony is that the hardworking assistants

putting them back on their shelves might, in some sense, be making them less

ordered.

Here’s why: if temporal locality holds, then the rough-sorting shelves contain

the most important books in the whole building. These are the books that were

most recently used, so they are the ones that patrons are most likely to be

looking for. It seems a crime that arguably the juiciest and most browseworthy

shelf of the libraries’ miles of stacks is both hidden away and constantly eroded

by earnest library staff just doing their jobs.

Meanwhile, the lobby of the Moffit Undergraduate Library—the location of

the most prominent and accessible shelves—showcases the library’s most

recently acquired books. This is instantiating a kind of FIFO cache, privileging

the items that were last added to the library, not last read.

The dominant performance of the LRU algorithm in most tests that computer

scientists have thrown at it leads to a simple suggestion: turn the library inside

out. Put acquisitions in the back, for those who want to find them. And put the

most recently returned items in the lobby, where they are ripe for the browsing.

Humans are social creatures, and presumably the undergraduate body would

find it interesting to peruse its own reading habits. It would nudge the campus

toward a more organic and free-form version of what colleges strive for when

they assign “common books”: the facilitation of intellectual common points of

reference. Here, the books being read on campus, whatever they happened to be,

would become the books most likely to be serendipitously encountered by other

students. A kind of grassroots, bottom-up analogue of the common book

program.

But a system like this wouldn’t only be more socially positive. Since the items

most recently returned are the ones most likely to be next checked out, it would

also be more efficient. It’s true that students might be puzzled by the fact that

popular books will sometimes be found in the stacks and sometimes in the lobby.

However, recently returned books that await reshelving are missing from the

stacks either way. It’s just that currently they are off-limits during this brief

limbo. Allowing the returned books to adorn the lobby instead would give

students a chance to short-circuit the shelving process entirely. No employees

would have to venture into the stacks to deposit the volumes, and no students

would have to venture into the stacks to get them back out. That’s exactly how

caching is meant to work.

The Cloud at the End of the Street

“We actually made a map of the country, on the scale of a mile to the

mile!”

“Have you used it much?” I enquired.

“It has never been spread out, yet,” said Mein Herr: “the farmers

objected: they said it would cover the whole country, and shut out the

sunlight! So we now use the country itself, as its own map, and I assure you

it does nearly as well.”

—LEWIS CARROLL

We often think of the Internet as a flat, independent, and loosely connected

network. In fact, it’s none of those things. A quarter of all Internet traffic at

present is handled by a single corporation, one that manages to stay almost

entirely out of the headlines. This Massachusetts-based company is called

Akamai, and they’re in the caching business.

We also think of the Internet as abstract, dematerial, post-geographic. We’re

told our data is “in the cloud,” which is meant to suggest a diffuse, distant place.

Again, none of these are true. The reality is that the Internet is all about bundles

of physical wires and racks of metal. And it’s much more closely tied to

geography than you might expect.

Engineers think about geography on a tiny scale when they design computer

hardware: faster memory is usually placed closer to the processor, minimizing

the length of the wires that information has to travel along. Today’s processor

cycles are measured in gigahertz, which is to say they are performing operations

in fractions of nanoseconds. For reference, that’s the time it takes light to travel a

few inches—so the physical layout of a computer’s internals is very much a

concern. And applying the same principle at a dramatically larger scale, actual

geography turns out to be critical for the functioning of the web, where the wires

span not inches but potentially thousands of miles.

If you can create a cache of webpage content that is physically, geographically

closer to the people who want it, you can serve those pages faster. Much of the

traffic on the Internet is now handled by “content distribution networks”

(CDNs), which have computers around the world that maintain copies of popular

websites. This allows users requesting those pages to get their data from a

computer that’s nearby, without having to make the long haul across continents

to the original server.

The largest of these CDNs is managed by Akamai: content providers pay for

their websites to be “Akamaized” for better performance. An Australian who

streams video from the BBC, for instance, is probably hitting local Akamai

servers in Sydney; the request never makes it to London at all. It doesn’t have to.

Says Akamai’s chief architect, Stephen Ludin, “It’s our belief—and we build the

company around the fact—that distance matters.”

In our earlier discussion, we noted that certain types of computer memory

have faster performance but cost more per unit of storage, leading to a “memory

hierarchy” that tries to get the best of both. But it’s not actually necessary to

have memory made of different materials for caching to make sense. Caching is

just as useful when it’s proximity, rather than performance, that’s the scarce

resource.

This fundamental insight—that in-demand files should be stored near the

location where they are used—also translates into purely physical environments.

For example, Amazon’s enormous fulfillment centers generally eschew any type

of human-comprehensible organization, of the kind you’d find in a library or a

department store. Instead, employees are told to place incoming items wherever

they can find space in the warehouse—batteries cheek by jowl with pencil

sharpeners, diapers, barbecue grills, and learn-the-dobro DVDs—and tag the

location of each item in a central database using bar codes. But this deliberately

disorganized-looking storage system still has one visible exception: highdemand items are placed in a different area, more quickly accessible than the

rest. That area is Amazon’s cache.

Recently, Amazon was granted a patent for an innovation that pushes this

principle one step further. The patent talks about “anticipatory package

shipping,” which the press seized upon as though Amazon could somehow mail

you something before you bought it. Amazon, like any technology company,

would love to have that kind of Bélády-like clairvoyance—but for the next best

thing, it turns to caching. Their patent is actually for shipping items that have

been recently popular in a given region to a staging warehouse in that region—

like having their own CDN for physical goods. Then, when somebody places an

order, the item is just down the street. Anticipating the purchases of individuals

is challenging, but when predicting the purchases of a few thousand people, the

law of large numbers kicks in. Somebody in Berkeley is going to order, say,

recycled toilet paper in a given day, and when they do it’s already most of the

way there.

When the things popular in an area are also from that area, an even more

interesting geography of the cloud emerges. In 2011, film critic Micah Mertes

created a map of the United States using each state’s “Local Favorites” from

Netflix—highlighting the movies uncommonly popular in each of those states.

Overwhelmingly, it turned out, people love watching movies set where they live.

Washingtonians favor Singles, set in Seattle; Louisianans watch The Big Easy,

set in New Orleans; Angelinos unsurprisingly enjoy L.A. Story; Alaskans love

Braving Alaska; and Montanans, Montana Sky.* And because nothing benefits

quite so much from local caching as the enormous files that comprise full-length

HD video, it’s certain that Netflix has arranged it so the files for, say, L.A. Story

live right in Los Angeles, just like its characters—and, more importantly, its

fans.

Caching on the Home Front

While caching began as a scheme for organizing digital information inside

computers, it’s clear that it is just as applicable to organizing physical objects in

human environments. When we spoke to John Hennessy—president of Stanford

University, and a pioneering computer architect who helped develop modern

caching systems—he immediately saw the link:

Caching is such an obvious thing because we do it all the time. I mean, the amount of information I

get … certain things I have to keep track of right now, a bunch of things I have on my desk, and then

other things are filed away, and then eventually filed away into the university archives system where

it takes a whole day to get stuff out of it if I wanted. But we use that technique all the time to try to

organize our lives.

The direct parallel between these problems means that there’s the potential to

consciously apply the solutions from computer science to the home.

First, when you are deciding what to keep and what to throw away, LRU is

potentially a good principle to use—much better than FIFO. You shouldn’t

necessarily toss that T-shirt from college if you still wear it every now and then.

But the plaid pants you haven’t worn in ages? Those can be somebody else’s

thrift-store bonanza.

Second, exploit geography. Make sure things are in whatever cache is closest

to the place where they’re typically used. This isn’t a concrete recommendation

in most home-organization books, but it consistently turns up in the schemes that

actual people describe as working well for them. “I keep running and exercise

gear in a crate on the floor of my front coat closet,” says one person quoted in

Julie Morgenstern’s Organizing from the Inside Out, for instance. “I like having

it close to the front door.”

A slightly more extreme example appears in the book Keeping Found Things

Found, by William Jones:

A doctor told me about her approach to keeping things. “My kids think I’m whacky, but I put things

where I think I’ll need them again later, even if it doesn’t make much sense.” As an example of her

system, she told me that she keeps extra vacuum cleaner bags behind the couch in the living room.

Behind the couch in the living room? Does that make any sense?… It turns out that when the vacuum

cleaner is used, it is usually used for the carpet in the living room.… When a vacuum cleaner bag

gets full and a new one is needed, it’s usually in the living room. And that’s just where the vacuum

cleaner bags are.

A final insight, which hasn’t yet made it into guides on closet organization, is

that of the multi-level memory hierarchy. Having a cache is efficient, but having

multiple levels of caches—from smallest and fastest to largest and slowest—can

be even better. Where your belongings are concerned, your closet is one cache

level, your basement another, and a self-storage locker a third. (These are in

decreasing order of access speed, of course, so you should use the LRU principle

as the basis for deciding what gets evicted from each level to the next.) But you

might also be able to speed things up by adding yet another level of caching: an

even smaller, faster, closer one than your closet.

Tom’s otherwise extremely tolerant wife objects to a pile of clothes next to

the bed, despite his insistence that it’s in fact a highly efficient caching scheme.

Fortunately, our conversations with computer scientists revealed a solution to

this problem too. Rik Belew of UC San Diego, who studies search engines from

a cognitive perspective, recommended the use of a valet stand. Though you don’t

see too many of them these days, a valet stand is essentially a one-outfit closet, a

compound hanger for jacket, tie, and slacks—the perfect piece of hardware for

your domestic caching needs. Which just goes to show that computer scientists

won’t only save you time; they might also save your marriage.

Filing and Piling

After deciding what to keep and where it should go, the final challenge is

knowing how to organize it. We’ve talked about what goes in the closet and

where the closet should be, but how should things be arranged inside?

One of the constants across all pieces of home-organization advice we’ve seen

so far is the idea of grouping “like with like”—and perhaps no one so directly

flies in the face of that advice as Yukio Noguchi. “I have to emphasize,” says

Noguchi, “that a very fundamental principle in my method is not to group files

according to content.” Noguchi is an economist at the University of Tokyo, and

the author of a series of books that offer “super” tricks for sorting out your office

and your life. Their titles translate roughly to Super Persuasion Method, Super

Work Method, Super Study Method—and, most relevantly for us, Super

Organized Method.

Early in his career as an economist, Noguchi found himself constantly

inundated with information—correspondence, data, manuscripts—and losing a

significant portion of each day just trying to organize it all. So he looked for an

alternative. He began by simply putting each document into a file labeled with

the document’s title and date, and putting all the files into one big box. That

saved time—he didn’t have to think about the right place to put each document

—but it didn’t result in any form of organization. Then, sometime in the early

1990s, he had a breakthrough: he started to insert the files exclusively at the lefthand side of the box. And thus the “super” filing system was born.

The left-side insertion rule, Noguchi specifies, has to be followed for old files

as well as new ones: every time you pull out a file to use its contents, you must

put it back as the leftmost file when you return it to the box. And when you

search for a file, you always start from the left-hand side as well. The most

recently accessed files are thus the fastest to find.

This practice began, Noguchi explains, because returning every file to the left

side was just easier than trying to reinsert it at the same spot it came from. Only

gradually did he realize that this procedure was not only simple but also

startlingly efficient.

The Noguchi Filing System clearly saves time when you’re replacing

something after you’re done using it. There’s still the question, however, of

whether it’s a good way to find the files you need in the first place. After all, it

certainly goes against the recommendations of other efficiency gurus, who tell us

that we should put similar things together. Indeed, even the etymology of the

word “organized” evokes a body composed of organs—which are nothing if not

cells grouped “like with like,” marshalled together by similar form and function.

But computer science gives us something that most efficiency gurus don’t:

guarantees.

Though Noguchi didn’t know it at the time, his filing system represents an

extension of the LRU principle. LRU tells us that when we add something to our

cache we should discard the oldest item—but it doesn’t tell us where we should

put the new item. The answer to that question comes from a line of research

carried out by computer scientists in the 1970s and ’80s. Their version of the

problem is called “self-organizing lists,” and its setup almost exactly mimics

Noguchi’s filing dilemma. Imagine that you have a set of items in a sequence,

and you must periodically search through them to find specific items. The search

itself is constrained to be linear—you must look through the items one by one,

starting at the beginning—but once you find the item you’re looking for, you can

put it back anywhere in the sequence. Where should you replace the items to

make searching as efficient as possible?

The definitive paper on self-organizing lists, published by Daniel Sleator and

Robert Tarjan in 1985, examined (in classic computer science fashion) the

worst-case performance of various ways to organize the list given all possible

sequences of requests. Intuitively, since the search starts at the front, you want to

arrange the sequence so that the items most likely to be searched for appear

there. But which items will those be? We’re back to wishing for clairvoyance

again. “If you know the sequence ahead of time,” says Tarjan, who splits his

time between Princeton and Silicon Valley, “you can customize the data

structure to minimize the total time for the entire sequence. That’s the optimum

offline algorithm: God’s algorithm if you will, or the algorithm in the sky. Of

course, nobody knows the future, so the question is, if you don’t know the

future, how close can you come to this optimum algorithm in the sky?” Sleator

and Tarjan’s results showed that some “very simple self-adjusting schemes,

amazingly, come within a constant factor” of clairvoyance. Namely, if you

follow the LRU principle—where you simply always put an item back at the

very front of the list—then the total amount of time you spend searching will

never be more than twice as long as if you’d known the future. That’s not a

guarantee any other algorithm can make.

Recognizing the Noguchi Filing System as an instance of the LRU principle in

action tells us that it is not merely efficient. It’s actually optimal.

Sleator and Tarjan’s results also provide us with one further twist, and we get

it by turning the Noguchi Filing System on its side. Quite simply, a box of files

on its side becomes a pile. And it’s the very nature of piles that you search them

from top to bottom, and that each time you pull out a document it goes back not

where you found it, but on top.*

In short, the mathematics of self-organizing lists suggests something radical:

the big pile of papers on your desk, far from being a guilt-inducing fester of

chaos, is actually one of the most well-designed and efficient structures

available. What might appear to others to be an unorganized mess is, in fact, a

self-organizing mess. Tossing things back on the top of the pile is the very best

you can do, shy of knowing the future. In the previous chapter we examined

cases where leaving something unsorted was more efficient than taking the time

to sort everything; here, however, there’s a very different reason why you don’t

need to organize it.

You already have.

The Forgetting Curve

Of course, no discussion of memory could be complete without mention of the

“memory organ” closest to home: the human brain. Over the past few decades,

the influence of computer science has brought about something of a revolution in

how psychologists think about memory.

The science of human memory is said to have begun in 1879, with a young

psychologist at the University of Berlin named Hermann Ebbinghaus.

Ebbinghaus wanted to get to the bottom of how human memory worked, and to

show that it was possible to study the mind with all the mathematical rigor of the

physical sciences. So he began to experiment on himself.

Each day, Ebbinghaus would sit down and memorize a list of nonsense

syllables. Then he would test himself on lists from previous days. Pursuing this

habit over the course of a year, he established many of the most basic results in

human memory research. He confirmed, for instance, that practicing a list

multiple times makes it persist longer in memory, and that the number of items

one can accurately recall goes down as time passes. His results mapped out a

graph of how memory fades over time, known today by psychologists as “the

forgetting curve.”

Ebbinghaus’s results established the credibility of a quantitative science of

human memory, but they left open something of a mystery. Why this particular

curve? Does it suggest that human memory is good or bad? What’s the

underlying story here? These questions have stimulated psychologists’

speculation and research for more than a hundred years.

In 1987, Carnegie Mellon psychologist and computer scientist John Anderson

found himself reading about the information retrieval systems of university

libraries. Anderson’s goal—or so he thought—was to write about how the design

of those systems could be informed by the study of human memory. Instead, the

opposite happened: he realized that information science could provide the

missing piece in the study of the mind.

“For a long time,” says Anderson, “I had felt that there was something

missing in the existing theories of human memory, including my own. Basically,

all of these theories characterize memory as an arbitrary and non-optimal

configuration.… I had long felt that the basic memory processes were quite

adaptive and perhaps even optimal; however, I had never been able to see a

framework in which to make this point. In the computer science work on

information retrieval, I saw that framework laid out before me.”

A natural way to think about forgetting is that our minds simply run out of

space. The key idea behind Anderson’s new account of human memory is that

the problem might be not one of storage, but of organization. According to his

theory, the mind has essentially infinite capacity for memories, but we have only

a finite amount of time in which to search for them. Anderson made the analogy

to a library with a single, arbitrarily long shelf—the Noguchi Filing System at

Library of Congress scale. You can fit as many items as you want on that shelf,

but the closer something is to the front the faster it will be to find.

The key to a good human memory then becomes the same as the key to a good

computer cache: predicting which items are most likely to be wanted in the

future.

Barring clairvoyance, the best approach to making such predictions in the

human world requires understanding the world itself. With his collaborator Lael

Schooler, Anderson set out to perform Ebbinghaus-like studies not on human

minds, but on human society. The question was straightforward: what patterns

characterize the way the world itself “forgets”—the way that events and

references fade over time? Anderson and Schooler analyzed three human

environments: headlines from the New York Times, recordings of parents talking

to their children, and Anderson’s own email inbox. In all domains, they found

that a word is most likely to appear again right after it had just been used, and

that the likelihood of seeing it again falls off as time goes on.

In other words, reality itself has a statistical structure that mimics the

Ebbinghaus curve.

This suggests something remarkable. If the pattern by which things fade from

our minds is the very pattern by which things fade from use around us, then there

may be a very good explanation indeed for the Ebbinghaus forgetting curve—

namely, that it’s a perfect tuning of the brain to the world, making available

precisely the things most likely to be needed.

Human memory and human environments. The left panel shows the percentage of nonsense syllables

Ebbinghaus correctly recalled from a list, as a function of the number of hours he waited after first

memorizing the list. The right panel shows the chance that a word appears in the headlines of the New

York Times on a given day, as a function of the time since its previous appearance in print.

In putting the emphasis on time, caching shows us that memory involves

unavoidable tradeoffs, and a certain zero-sumness. You can’t have every library

book at your desk, every product on display at the front of the store, every

headline above the fold, every paper at the top of the pile. And in the same way,

you can’t have every fact or face or name at the front of your mind.

“Many people hold the bias that human memory is anything but optimal,”

wrote Anderson and Schooler. “They point to the many frustrating failures of

memory. However, these criticisms fail to appreciate the task before human

memory, which is to try to manage a huge stockpile of memories. In any system

responsible for managing a vast data base there must be failures of retrieval. It is

just too expensive to maintain access to an unbounded number of items.”

This understanding has in turn led to a second revelation about human

memory. If these tradeoffs really are unavoidable, and the brain appears to be

optimally tuned to the world around it, then what we refer to as the inevitable

“cognitive decline” that comes with age may in fact be something else.

The Tyranny of Experience

A big book is a big nuisance.

—CALLIMACHUS (305–410 BC), LIBRARIAN AT ALEXANDRIA

Why don’t they make the whole plane out of that black box stuff?

—STEVEN WRIGHT

The need for a computer memory hierarchy, in the form of a cascade of caches,

is in large part the result of our inability to afford making the entire memory out

of the most expensive type of hardware. The fastest cache on current computers,

for instance, is made with what’s called SRAM, which costs roughly a thousand

times as much per byte as the flash memory in solid-state drives. But the true

motivation for caching goes deeper than that. In fact, even if we could get a

bespoke machine that used exclusively the fastest form of memory possible,

we’d still need caches.

As John Hennessy explains, size alone is enough to impair speed:

When you make something bigger, it’s inherently slower, right? If you make a city bigger, it takes

longer to get from point A to point B. If you make a library bigger, it takes longer to find a book in

the library. If you have a stack of papers on your desk that’s bigger, it takes longer to find the paper

you’re looking for, right? Caches are actually a solution to that problem.… For example, right now,

if you go to buy a processor, what you’ll get is a Level 1 cache and a Level 2 cache on the chip. The

reason that there are—even just on the chip there are two caches!—is that in order to keep up with

the cycle rate of the processor, the first-level cache is limited in size.

Unavoidably, the larger a memory is, the more time it takes to search for and

extract a piece of information from it.

Brian and Tom, in their thirties, already find themselves more frequently

stalling a conversation as, for instance, they wait for the name of someone “on

the tip of the tongue” to come to mind. Then again, Brian at age ten had two

dozen schoolmates; twenty years later he has hundreds of contacts in his phone

and thousands on Facebook, and has lived in four cities, each with its own

community of friends, acquaintances, and colleagues. Tom, by this point in his

academic career, has worked with hundreds of collaborators and taught

thousands of students. (In fact, this very book involved meeting with about a

hundred people and citing a thousand.) Such effects are by no means limited to

social connections, of course: a typical two-year-old knows two hundred words;

a typical adult knows thirty thousand. And when it comes to episodic memory,

well, every year adds a third of a million waking minutes to one’s total lived

experience.

Considered this way, it’s a wonder that the two of us—or anyone—can

mentally keep up at all. What’s surprising is not memory’s slowdown, but the

fact that the mind can possibly stay afloat and responsive as so much data

accumulates.

If the fundamental challenge of memory really is one of organization rather

than storage, perhaps it should change how we think about the impact of aging

on our mental abilities. Recent work by a team of psychologists and linguists led

by Michael Ramscar at the University of Tübingen has suggested that what we

call “cognitive decline”—lags and retrieval errors—may not be about the search

process slowing or deteriorating, but (at least partly) an unavoidable

consequence of the amount of information we have to navigate getting bigger

and bigger. Regardless of whatever other challenges aging brings, older brains—

which must manage a greater store of memories—are literally solving harder

computational problems with every passing day. The old can mock the young for

their speed: “It’s because you don’t know anything yet!”

Ramscar’s group demonstrated the impact of extra information on human

memory by focusing on the case of language. Through a series of simulations,

the researchers showed that simply knowing more makes things harder when it

comes to recognizing words, names, and even letters. No matter how good your

organization scheme is, having to search through more things will inevitably

take longer. It’s not that we’re forgetting; it’s that we’re remembering. We’re

becoming archives.

An understanding of the unavoidable computational demands of memory,

Ramscar says, should help people come to terms with the effects of aging on

cognition. “I think the most important tangible thing seniors can do is to try to

get a handle on the idea that their minds are natural information processing

devices,” he writes. “Some things that might seem frustrating as we grow older

(like remembering names!) are a function of the amount of stuff we have to sift

through … and are not necessarily a sign of a failing mind.” As he puts it, “A lot

of what is currently called decline is simply learning.”

Caching gives us the language to understand what’s happening. We say “brain

fart” when we should really say “cache miss.” The disproportionate occasional

lags in information retrieval are a reminder of just how much we benefit the rest

of the time by having what we need at the front of our minds.

So as you age, and begin to experience these sporadic latencies, take heart: the

length of a delay is partly an indicator of the extent of your experience. The

effort of retrieval is a testament to how much you know. And the rarity of those

lags is a testament to how well you’ve arranged it: keeping the most important

things closest to hand.

5 Scheduling

First Things First

How we spend our days is, of course, how we spend our lives.

—ANNIE DILLARD

“Why don’t we write a book on scheduling theory?” I asked.… “It

shouldn’t take much time!” Book-writing, like war-making, often entails

grave miscalculations. Fifteen years later, Scheduling is still unfinished.

—EUGENE LAWLER

It’s Monday morning, and you have an as-yet blank schedule and a long list of

tasks to complete. Some can be started only after others are finished (you can’t

load the dishwasher unless it’s unloaded first), and some can be started only after

a certain time (the neighbors will complain if you put the trash out on the curb

before Tuesday night). Some have sharp deadlines, others can be done

whenever, and many are fuzzily in between. Some are urgent, but not important.

Some are important, but not urgent. “We are what we repeatedly do,” you seem

to recall Aristotle saying—whether it’s mop the floor, spend more time with

family, file taxes on time, learn French.

So what to do, and when, and in what order? Your life is waiting.

Though we always manage to find some way to order the things we do in our

days, as a rule we don’t consider ourselves particularly good at it—hence the

perennial bestseller status of time-management guides. Unfortunately, the

guidance we find in them is frequently divergent and inconsistent. Getting

Things Done advocates a policy of immediately doing any task of two minutes

or less as soon as it comes to mind. Rival bestseller Eat That Frog! advises

beginning with the most difficult task and moving toward easier and easier

things. The Now Habit suggests first scheduling one’s social engagements and

leisure time and then filling the gaps with work—rather than the other way

around, as we so often do. William James, the “father of American psychology,”

asserts that “there’s nothing so fatiguing as the eternal hanging on of an

uncompleted task,” but Frank Partnoy, in Wait, makes the case for deliberately

not doing things right away.

Every guru has a different system, and it’s hard to know who to listen to.

Spending Time Becomes a Science

Though time management seems a problem as old as time itself, the science of

scheduling began in the machine shops of the industrial revolution. In 1874,

Frederick Taylor, the son of a wealthy lawyer, turned down his acceptance at

Harvard to become an apprentice machinist at Enterprise Hydraulic Works in

Philadelphia. Four years later, he completed his apprenticeship and began

working at the Midvale Steel Works, where he rose through the ranks from lathe

operator to machine shop foreman and ultimately to chief engineer. In the

process, he came to believe that the time of the machines (and people) he

oversaw was not being used very well, leading him to develop a discipline he

called “Scientific Management.”

Taylor created a planning office, at the heart of which was a bulletin board

displaying the shop’s schedule for all to see. The board depicted every machine

in the shop, showing the task currently being carried out by that machine and all

the tasks waiting for it. This practice would be built upon by Taylor’s colleague

Henry Gantt, who in the 1910s developed the Gantt charts that would help

organize many of the twentieth century’s most ambitious construction projects,

from the Hoover Dam to the Interstate Highway System. A century later, Gantt

charts still adorn the walls and screens of project managers at firms like

Amazon, IKEA, and SpaceX.

Taylor and Gantt made scheduling an object of study, and they gave it visual

and conceptual form. But they didn’t solve the fundamental problem of

determining which schedules were best. The first hint that this problem even

could be solved wouldn’t appear until several decades later, in a 1954 paper

published by RAND Corporation mathematician Selmer Johnson.

The scenario Johnson examined was bookbinding, where each book needs to

be printed on one machine and then bound on another. But the most common

instance of this two-machine setup is much closer to home: the laundry. When

you wash your clothes, they have to pass through the washer and the dryer in

sequence, and different loads will take different amounts of time in each. A

heavily soiled load might take longer to wash but the usual time to dry; a large

load might take longer to dry but the usual time to wash. So, Johnson asked, if

you have several loads of laundry to do on the same day, what’s the best way to

do them?

His answer was that you should begin by finding the single step that takes the

least amount of time—the load that will wash or dry the quickest. If that shortest

step involves the washer, plan to do that load first. If it involves the dryer, plan

to do it last. Repeat this process for the remaining loads, working from the two

ends of the schedule toward the middle.

Intuitively, Johnson’s algorithm works because regardless of how you

sequence the loads, there’s going to be some time at the start when the washer is

running but not the dryer, and some time at the end when the dryer is running

but not the washer. By having the shortest washing times at the start, and the

shortest drying times at the end, you maximize the amount of overlap—when the

washer and dryer are running simultaneously. Thus you can keep the total

amount of time spent doing laundry to the absolute minimum. Johnson’s analysis

had yielded scheduling’s first optimal algorithm: start with the lightest wash, end

with the smallest hamper.

Beyond its immediate applications, Johnson’s paper revealed two deeper

points: first, that scheduling could be expressed algorithmically, and second, that

optimal scheduling solutions existed. This kicked off what has become a

sprawling literature, exploring strategies for a vast menagerie of hypothetical

factories with every conceivable number and kind of machines.

We’re going to focus on a tiny subset of this literature: the part that, unlike

bookbinding or laundry, deals with scheduling for a single machine. Because the

scheduling problem that matters the most involves just one machine: ourselves.

Handling Deadlines

With single-machine scheduling, we run into something of a problem right off

the bat. Johnson’s work on bookbinding was based on minimizing the total time

required for the two machines to complete all of their jobs. In the case of singlemachine scheduling, however, if we are going to do all the tasks assigned, then

all schedules will take equally long to complete; the order is irrelevant.

This is a sufficiently fundamental and counterintuitive point that it’s worth

repeating. If you have only a single machine, and you’re going to do all of your

tasks, then any ordering of the tasks will take you the same amount of time.

Thus we encounter the first lesson in single-machine scheduling literally

before we even begin: make your goals explicit. We can’t declare some schedule

a winner until we know how we’re keeping score. This is something of a theme

in computer science: before you can have a plan, you must first choose a metric.

And as it turns out, which metric we pick here will directly affect which

scheduling approaches fare best.

The first papers on single-machine scheduling followed quickly on the heels

of Johnson’s bookbinding work and offered several plausible metrics to

consider. For each metric, they discovered a simple, optimal strategy.

It is of course common, for instance, for tasks to have a due date, with the

lateness of a task being how far it has gone overdue. So we can think of the

“maximum lateness” of a set of tasks as the lateness of whatever task has gone

furthest past its due date—the kind of thing your employer might care about in a

performance review. (Or what your customers might care about in a retail or

service setting, where the “maximally late” task corresponds to the customer

subjected to the longest wait time.)

If you’re concerned with minimizing maximum lateness, then the best strategy

is to start with the task due soonest and work your way toward the task due last.

This strategy, known as Earliest Due Date, is fairly intuitive. (For instance, in a

service-sector context, where each arriving patron’s “due date” is effectively the

instant they walk in the door, it just means serving customers in order of arrival.)

But some of its implications are surprising. For example, how long each task

will take to complete is entirely irrelevant: it doesn’t change the plan, so in fact

you don’t even need to know. All that matters is when the tasks are due.

You might already be using Earliest Due Date to tackle your workload, in

which case you probably don’t need computer science to tell you that it’s a

sensible strategy. What you may not have known, though, is that it’s the optimal

strategy. More precisely, it is optimal assuming that you’re only interested in one

metric in particular: reducing your maximum lateness. If that’s not your goal,

however, then another strategy might be more applicable.

For instance, consider the refrigerator. If you’re one of the many people who

have a community-supported agriculture (CSA) subscription, then every week or

two you’ve got a lot of fresh produce coming to your doorstep all at once. Each

piece of produce is set to spoil on a different date—so eating them by Earliest

Due Date, in order of their spoilage schedule, seems like a reasonable starting

point. It’s not, however, the end of the story. Earliest Due Date is optimal for

reducing maximum lateness, which means it will minimize the rottenness of the

single most rotten thing you’ll have to eat; that may not be the most appetizing

metric to eat by.

Maybe instead we want to minimize the number of foods that spoil. Here a

strategy called Moore’s Algorithm gives us our best plan. Moore’s Algorithm

says that we start out just like with Earliest Due Date—by scheduling out our

produce in order of spoilage date, earliest first, one item at a time. However, as

soon as it looks like we won’t get to eating the next item in time, we pause, look

back over the meals we’ve already planned, and throw out the biggest item (that

is, the one that would take the most days to consume). For instance, that might

mean forgoing the watermelon that would take a half dozen servings to eat; not

even attempting it will mean getting to everything that follows a lot sooner. We

then repeat this pattern, laying out the foods by spoilage date and tossing the

largest already scheduled item any time we fall behind. Once everything that

remains can be eaten in order of spoilage date without anything spoiling, we’ve

got our plan.

Moore’s Algorithm minimizes the number of items you’ll need to throw away.

Of course, you’re also welcome to compost the food, donate it to the local food

bank, or give it to your neighbor. In an industrial or bureaucratic context where

you can’t simply discard a project, but in which the number—rather than the

severity—of late projects is still your biggest concern, Moore’s Algorithm is just

as indifferent about how those late tasks are handled. Anything booted from the

main portion of your schedule can get done at the very end, in any order; it

doesn’t matter, as they’re all already late.

Getting Things Done

Do the difficult things while they are easy and do the great things while

they are small.

—LAO TZU

Sometimes due dates aren’t our primary concern and we just want to get stuff

done: as much stuff, as quickly as possible. It turns out that translating this

seemingly simple desire into an explicit scheduling metric is harder than it

sounds.

One approach is to take an outsider’s perspective. We’ve noted that in singlemachine scheduling, nothing we do can change how long it will take us to finish

all of our tasks—but if each task, for instance, represents a waiting client, then

there is a way to take up as little of their collective time as possible. Imagine

starting on Monday morning with a four-day project and a one-day project on

your agenda. If you deliver the bigger project on Thursday afternoon (4 days

elapsed) and then the small one on Friday afternoon (5 days elapsed), the clients

will have waited a total of 4 + 5 = 9 days. If you reverse the order, however, you

can finish the small project on Monday and the big one on Friday, with the

clients waiting a total of only 1 + 5 = 6 days. It’s a full workweek for you either

way, but now you’ve saved your clients three days of their combined time.

Scheduling theorists call this metric the “sum of completion times.”

Minimizing the sum of completion times leads to a very simple optimal

algorithm called Shortest Processing Time: always do the quickest task you

can.

Even if you don’t have impatient clients hanging on every job, Shortest

Processing Time gets things done. (Perhaps it’s no surprise that it is compatible

with the recommendation in Getting Things Done to immediately perform any

task that takes less than two minutes.) Again, there’s no way to change the total

amount of time your work will take you, but Shortest Processing Time may ease

your mind by shrinking the number of outstanding tasks as quickly as possible.

Its sum-of-completion-times metric can be expressed another way: it’s like

focusing above all on reducing the length of your to-do list. If each piece of

unfinished business is like a thorn in your side, then racing through the easiest

items may bring some measure of relief.

Of course, not all unfinished business is created equal. Putting out an actual

fire in the kitchen should probably be done before “putting out a fire” with a

quick email to a client, even if the former takes a bit longer. In scheduling, this

difference of importance is captured in a variable known as weight. When you’re

going through your to-do list, this weight might feel literal—the burden you get

off your shoulders by finishing each task. A task’s completion time shows how

long you carry that burden, so minimizing the sum of weighted completion times

(that is, each task’s duration multiplied by its weight) means minimizing your

total oppression as you work through your entire agenda.

The optimal strategy for that goal is a simple modification of Shortest

Processing Time: divide the weight of each task by how long it will take to

finish, and then work in order from the highest resulting importance-per-unittime (call it “density” if you like, to continue the weight metaphor) to the lowest.

And while it might be hard to assign a degree of importance to each one of your

daily tasks, this strategy nonetheless offers a nice rule of thumb: only prioritize a

task that takes twice as long if it’s twice as important.

In business contexts, “weight” might easily be translated to the amount of

money each task will bring in. The notion of dividing reward by duration

translates, therefore, to assigning each task an hourly rate. (If you’re a consultant

or freelancer, that might in effect already be done for you: simply divide each

project’s fee by its size, and work your way from the highest hourly rate to the

lowest.) Interestingly, this weighted strategy also shows up in studies of animal

foraging, with nuts and berries taking the place of dollars and cents. Animals,

seeking to maximize the rate at which they accumulate energy from food, should

pursue foods in order of the ratio of their caloric energy to the time required to

get and eat them—and indeed appear to do so.

When applied to debts rather than incomes, the same principle yields a

strategy for getting in the black that’s come to be called the “debt avalanche.”

This debt-reduction strategy says to ignore the number and size of your debts

entirely, and simply funnel your money toward the debt with the single highest

interest rate. This corresponds rather neatly to working through jobs in order of

importance per unit time. And it’s the strategy that will reduce the total burden

of your debt as quickly as possible.

If, on the other hand, you’re more concerned with reducing the number of

debts than the amount of debt—if, for instance, the hassle of numerous bills and

collection phone calls is a bigger deal than the difference in interest rates—then

you’re back to the unweighted, “just get stuff done” flavor of Shortest

Processing Time, paying off the smallest debts first simply to get them out of the

way. In debt-reduction circles, this approach is known as the “debt snowball.”

Whether people, in practice, ought to prioritize lowering the dollar amount of

their debts or the quantity of them remains an active controversy, both in the

popular press as well as in economics research.

Picking Our Problems

This brings us back to the note on which we began our discussion of singlemachine scheduling. It’s said that “a man with one watch knows what time it is;

a man with two watches is never sure.” Computer science can offer us optimal

algorithms for various metrics available in single-machine scheduling, but

choosing the metric we want to follow is up to us. In many cases, we get to

decide what problem we want to be solving.

This offers a radical way to rethink procrastination, the classic pathology of

time management. We typically think of it as a faulty algorithm. What if it’s

exactly the opposite? What if it’s an optimal solution to the wrong problem?

There’s an episode of The X-Files where the protagonist Mulder, bedridden

and about to be consumed by an obsessive-compulsive vampire, spills a bag of

sunflower seeds on the floor in self-defense. The vampire, powerless against his

compulsion, stoops to pick them up one by one, and ultimately the sun rises

before he can make a meal of Mulder. Computer scientists would call this a

“ping attack” or a “denial of service” attack: give a system an overwhelming

number of trivial things to do, and the important things get lost in the chaos.

We typically associate procrastination with laziness or avoidance behavior,

but it can just as easily spring up in people (or computers, or vampires) who are

trying earnestly and enthusiastically to get things done as quickly as possible. In

a 2014 study led by Penn State’s David Rosenbaum, for example, participants

were asked to bring either one of two heavy buckets to the opposite end of a

hallway. One of the buckets was right next to the participant; the other was

partway down the hall. To the experimenters’ surprise, people immediately

picked up the bucket near them and lugged it all the way down—passing the

other bucket on the way, which they could have carried a fraction of the

distance. As the researchers write, “this seemingly irrational choice reflected a

tendency to pre-crastinate, a term we introduce to refer to the hastening of

subgoal completion, even at the expense of extra physical effort.” Putting off

work on a major project by attending instead to various trivial matters can

likewise be seen as “the hastening of subgoal completion”—which is another

way of saying that procrastinators are acting (optimally!) to reduce as quickly as

possible the number of outstanding tasks on their minds. It’s not that they have a

bad strategy for getting things done; they have a great strategy for the wrong

metric.

Working on a computer brings with it an additional hazard when it comes to

being conscious and deliberate about our scheduling metrics: the user interface

may subtly (or not so subtly) force its own metric upon us. A modern

smartphone user, for instance, is accustomed to seeing “badges” hovering over

application icons, ominous numbers in white-on-red signaling exactly how many

tasks each particular app expects us to complete. If it’s an email inbox blaring

the figure of unread messages, then all messages are implicitly being given equal

weight. Can we be blamed, then, for applying the unweighted Shortest

Processing Time algorithm to the problem—dealing with all of the easiest emails

first and deferring the hardest ones till last—to lower this numeral as quickly as

possible?

Live by the metric, die by the metric. If all tasks are indeed of equal weight,

then that’s exactly what we should be doing. But if we don’t want to become

slaves to minutiae, then we need to take measures toward that end. This starts

with making sure that the single-machine problem we’re solving is the one we

want to be solving. (In the case of app badges, if we can’t get them to reflect our

actual priorities, and can’t overcome the impulse to optimally reduce any

numerical figure thrown in our face, then perhaps the next best thing is simply to

turn the badges off.)

Staying focused not just on getting things done but on getting weighty things

done—doing the most important work you can at every moment—sounds like a

surefire cure for procrastination. But as it turns out, even that is not enough. And

a group of computer scheduling experts would encounter this lesson in the most

dramatic way imaginable: on the surface of Mars, with the whole world

watching.

Priority Inversion and Precedence Constraints

It was the summer of 1997, and humanity had a lot to celebrate. For the first time

ever, a rover was navigating the surface of Mars. The $150 million Mars

Pathfinder spacecraft had accelerated to a speed of 16,000 miles per hour,

traveled across 309 million miles of empty space, and landed with space-grade

airbags onto the rocky red Martian surface.

And now it was procrastinating.

Back on Earth, Jet Propulsion Laboratory (JPL) engineers were both worried

and stumped. Pathfinder’s highest priority task (to move data into and out of its

“information bus”) was mysteriously being neglected as the robot whiled away

its time on tasks of middling importance. What was going on? Didn’t the robot

know any better?

Suddenly Pathfinder registered that the information bus hadn’t been dealt with

for an unacceptably long time, and, lacking a subtler recourse, initiated a

complete restart, costing the mission the better part of a day’s work. A day or so

later, the same thing happened again.

Working feverishly, the JPL team finally managed to reproduce and then

diagnose the behavior. The culprit was a classic scheduling hazard called

priority inversion. What happens in a priority inversion is that a low-priority task

takes possession of a system resource (access to a database, let’s say) to do some

work, but is then interrupted partway through that work by a timer, which pauses

it and invokes the system scheduler. The scheduler tees up a high-priority task,

but it can’t run because the database is occupied. And so the scheduler moves

down the priority list, running various unblocked medium-priority tasks instead

—rather than the high-priority one (which is blocked), or the low-priority one

that’s blocking it (which is stuck in line behind all the medium-priority work). In

these nightmarish scenarios, the system’s highest priority can sometimes be

neglected for arbitrarily long periods of time.*

Once JPL engineers had identified the Pathfinder problem as a case of priority

inversion, they wrote up a fix and beamed the new code across millions of miles

to Pathfinder. What was the solution they sent flying across the solar system?

Priority inheritance. If a low-priority task is found to be blocking a high-priority

resource, well, then all of a sudden that low-priority task should momentarily

become the highest-priority thing on the system, “inheriting” the priority of the

thing it’s blocking.

The comedian Mitch Hedberg recounts a time when “I was at a casino, I was

minding my own business, and this guy came up and said, ‘You’re gonna have

to move, you’re blocking the fire exit.’ As though if there was a fire, I wasn’t

gonna run.” The bouncer’s argument was priority inversion; Hedberg’s rebuttal

was priority inheritance. Hedberg lounging casually in front of a fleeing mob

puts his low-priority loitering ahead of their high-priority running for their lives

—but not if he inherits their priority. And an onrushing mob has a way of

making one inherit their priority rather quickly. As Hedberg explains, “If you’re

flammable and have legs, you are never blocking a fire exit.”

The moral here is that a love of getting things done isn’t enough to avoid

scheduling pitfalls, and neither is a love of getting important things done. A

commitment to fastidiously doing the most important thing you can, if pursued

in a head-down, myopic fashion, can lead to what looks for all the world like

procrastination. As with a car spinning its tires, the very desire to make

immediate progress is how one gets stuck. “Things which matter most must

never be at the mercy of things which matter least,” Goethe allegedly

proclaimed; but while that has the ring of wisdom about it, sometimes it’s just

not true. Sometimes that which matters most cannot be done until that which

matters least is finished, so there’s no choice but to treat that unimportant thing

as being every bit as important as whatever it’s blocking.

When a certain task can’t be started until another one is finished, scheduling

theorists call that a “precedence constraint.” For operations research expert

Laura Albert McLay, explicitly remembering this principle has made the

difference on more than one occasion in her own household. “It can be really

helpful if you can see these things. Of course, getting through the day with three

kids, there’s a lot of scheduling.… We can’t get out the door unless the kids get

breakfast first, and they can’t get breakfast first if I don’t remember to give them

a spoon. Sometimes there’s something very simple that you forget that just

delays everything. In terms of scheduling algorithms, just knowing what [that]

is, and keeping that moving, is incredibly helpful. That’s how I get things done

every day.”

In 1978, scheduling researcher Jan Karel Lenstra was able to use the same

principle while helping his friend Gene move into a new house in Berkeley.

“Gene was postponing something that had to be finished before we could start

something else which was urgent.” As Lenstra recalls, they needed to return a

van, but needed the van to return a piece of equipment, but needed the

equipment to fix something in the apartment. The apartment fix didn’t feel

urgent (hence its postponement), but the van return did. Says Lenstra, “I

explained to him that the former task should be considered even more urgent.”

While Lenstra is a central figure in scheduling theory, and thus was well

positioned to give this advice to his friend, it came with a particularly delicious

irony. This was a textbook case of priority inversion caused by precedence

constraints. And arguably the twentieth century’s single greatest expert on

precedence constraints was none other than his friend, Eugene “Gene” Lawler.

The Speed Bump

Considering he spent much of his life thinking about how to most efficiently

complete a sequence of tasks, Lawler took an intriguingly circuitous route to his

own career. He studied mathematics at Florida State University before beginning

graduate work at Harvard in 1954, though he left before finishing a doctorate.

After time in law school, the army, and (thematically enough) working in a

machine shop, he went back to Harvard in 1958, finishing his PhD and taking a

position at the University of Michigan. Visiting Berkeley on sabbatical in 1969,

he was arrested at a notorious Vietnam War protest. He became a member of the

faculty at Berkeley the following year, and acquired a reputation there for being

“the social conscience” of the computer science department. After his death in

1994, the Association for Computing Machinery established an award in

Lawler’s name, honoring people who demonstrate the humanitarian potential of

computer science.

Lawler’s first investigation into precedence constraints suggested that they

could be handled quite easily. For instance, take the Earliest Due Date algorithm

that minimizes the maximum lateness of a set of tasks. If your tasks have

precedence constraints, that makes things trickier—you can’t just plow forward

in order of due date if some tasks can’t be started until others are finished. But in

1968, Lawler proved that this is no trouble as long as you build the schedule

back to front: look only at the tasks that no other tasks depend on, and put the

one with the latest due date at the end of the schedule. Then simply repeat this

process, again considering at each step only those tasks that no other (as-yet

unscheduled) tasks depend upon as a prerequisite.

But as Lawler looked more deeply into precedence constraints, he found

something curious. The Shortest Processing Time algorithm, as we saw, is the

optimal policy if you want to cross off as many items as quickly as possible from

your to-do list. But if some of your tasks have precedence constraints, there isn’t

a simple or obvious tweak to Shortest Processing Time to adjust for that.

Although it looked like an elementary scheduling problem, neither Lawler nor

any other researcher seemed to be able to find an efficient way to solve it.

In fact, it was much worse than this. Lawler himself would soon discover that

this problem belongs to a class that most computer scientists believe has no

efficient solution—it’s what the field calls “intractable.”* Scheduling theory’s

first speed bump turned out to be a brick wall.

As we saw with the “triple or nothing” scenario for which optimal stopping

theory has no sage words, not every problem that can be formally articulated has

an answer. In scheduling, it’s clear by definition that every set of tasks and

constraints has some schedule that’s the best, so scheduling problems aren’t

unanswerable, per se—but it may simply be the case that there’s no

straightforward algorithm that can find you the optimal schedule in a reasonable

amount of time.

This led researchers like Lawler and Lenstra to an irresistible question. Just

what proportion of scheduling problems was intractable, anyway? Twenty years

after scheduling theory was kick-started by Selmer Johnson’s bookbinding

paper, the search for individual solutions was about to become something much

grander and more ambitious by far: a quest to map the entire landscape of

scheduling theory.

What the researchers found was that even the subtlest change to a scheduling

problem often tips it over the fine and irregular line between tractable and

intractable. For example, Moore’s Algorithm minimizes the number of late tasks

(or rotten fruits) when they’re all of equal value—but if some are more

important than others, the problem becomes intractable and no algorithm can

readily provide the optimal schedule. Likewise, having to wait until a certain

time to start some of your tasks makes nearly all of the scheduling problems for

which we otherwise have efficient solutions into intractable problems. Not being

able to put out the trash until the night before collection might be a reasonable

municipal bylaw, but it will send your calendar headlong into intractability.

The drawing of the borders of scheduling theory continues to this day. A

recent survey showed that the status of about 7% of all problems is still

unknown, scheduling’s terra incognita. Of the 93% of the problems that we do

understand, however, the news isn’t great: only 9% can be solved efficiently,

and the other 84% have been proven intractable.* In other words, most

scheduling problems admit no ready solution. If trying to perfectly manage your

calendar feels overwhelming, maybe that’s because it actually is. Nonetheless,

the algorithms we have discussed are often the starting point for tackling those

hard problems—if not perfectly, then at least as well as can be expected.

Drop Everything: Preemption and Uncertainty

The best time to plant a tree is twenty years ago. The second best time is

now.

—PROVERB

So far we have considered only factors that make scheduling harder. But there is

one twist that can make it easier: being able to stop one task partway through

and switch to another. This property, “preemption,” turns out to change the game

dramatically.

Minimizing maximum lateness (for serving customers in a coffee shop) or the

sum of completion times (for rapidly shortening your to-do list) both cross the

line into intractability if some tasks can’t be started until a particular time. But

they return to having efficient solutions once preemption is allowed. In both

cases, the classic strategies—Earliest Due Date and Shortest Processing Time,

respectively—remain the best, with a fairly straightforward modification. When

a task’s starting time comes, compare that task to the one currently under way. If

you’re working by Earliest Due Date and the new task is due even sooner than

the current one, switch gears; otherwise stay the course. Likewise, if you’re

working by Shortest Processing Time, and the new task can be finished faster

than the current one, pause to take care of it first; otherwise, continue with what

you were doing.

Now, on a good week a machine shop might know everything expected of

them in the next few days, but most of us are usually flying blind, at least in part.

We might not even be sure, for instance, when we’ll be able to start a particular

project (when will so-and-so give me a solid answer on the such-and-such?).

And at any moment our phone can ring or an email can pop up with news of a

whole new task to add to our agenda.

It turns out, though, that even if you don’t know when tasks will begin,

Earliest Due Date and Shortest Processing Time are still optimal strategies, able

to guarantee you (on average) the best possible performance in the face of

uncertainty. If assignments get tossed on your desk at unpredictable moments,

the optimal strategy for minimizing maximum lateness is still the preemptive

version of Earliest Due Date—switching to the job that just came up if it’s due

sooner than the one you’re currently doing, and otherwise ignoring it. Similarly,

the preemptive version of Shortest Processing Time—compare the time left to

finish the current task to the time it would take to complete the new one—is still

optimal for minimizing the sum of completion times.

In fact, the weighted version of Shortest Processing Time is a pretty good

candidate for best general-purpose scheduling strategy in the face of uncertainty.

It offers a simple prescription for time management: each time a new piece of

work comes in, divide its importance by the amount of time it will take to

complete. If that figure is higher than for the task you’re currently doing, switch

to the new one; otherwise stick with the current task. This algorithm is the

closest thing that scheduling theory has to a skeleton key or Swiss Army knife,

the optimal strategy not just for one flavor of problem but for many. Under

certain assumptions it minimizes not just the sum of weighted completion times,

as we might expect, but also the sum of the weights of the late jobs and the sum

of the weighted lateness of those jobs.

Intriguingly, optimizing all these other metrics is intractable if we know the

start times and durations of jobs ahead of time. So considering the impact of

uncertainty in scheduling reveals something counterintuitive: there are cases

where clairvoyance is a burden. Even with complete foreknowledge, finding the

perfect schedule might be practically impossible. In contrast, thinking on your

feet and reacting as jobs come in won’t give you as perfect a schedule as if

you’d seen into the future—but the best you can do is much easier to compute.

That’s some consolation. As business writer and coder Jason Fried says, “Feel

like you can’t proceed until you have a bulletproof plan in place? Replace ‘plan’

with ‘guess’ and take it easy.” Scheduling theory bears this out.

When the future is foggy, it turns out you don’t need a calendar—just a to-do

list.

Preemption Isn’t Free: The Context Switch

The hurrieder I go / The behinder I get

—NEEDLEPOINT SEEN IN BOONVILLE, CA

Programmers don’t talk because they must not be interrupted.… To

synchronize with other people (or their representation in telephones,

buzzers and doorbells) can only mean interrupting the thought train.

Interruptions mean certain bugs. You must not get off the train.

—ELLEN ULLMAN

Scheduling theory thus tells a reasonably encouraging story after all. There are

simple, optimal algorithms for solving many scheduling problems, and those

problems are tantalizingly close to situations we encounter daily in human lives.

But when it comes to actually carrying out single-machine scheduling in the real

world, things get complicated.

First of all, people and computer operating systems alike face a curious

challenge: the machine that is doing the scheduling and the machine being

scheduled are one and the same. Which makes straightening out your to-do list

an item on your to-do list—needing, itself, to get prioritized and scheduled.

Second, preemption isn’t free. Every time you switch tasks, you pay a price,

known in computer science as a context switch. When a computer processor

shifts its attention away from a given program, there’s always a certain amount

of necessary overhead. It needs to effectively bookmark its place and put aside

all of its information related to that program. Then it needs to figure out which

program to run next. Finally it must haul out all the relevant information for that

program, find its place in the code, and get in gear.

None of this switching back and forth is “real work”—that is, none of it

actually advances the state of any of the various programs the computer is

switching between. It’s metawork. Every context switch is wasted time.

Humans clearly have context-switching costs too. We feel them when we

move papers on and off our desk, close and open documents on our computer,

walk into a room without remembering what had sent us there, or simply say out

loud, “Now, where was I?” or “What was I saying?” Psychologists have shown

that for us, the effects of switching tasks can include both delays and errors—at

the scale of minutes rather than microseconds. To put that figure in perspective,

anyone you interrupt more than a few times an hour is in danger of doing no

work at all.

Personally, we have found that both programming and writing require keeping

in mind the state of the entire system, and thus carry inordinately large contextswitching costs. A friend of ours who writes software says that the normal

workweek isn’t well suited to his workflow, since for him sixteen-hour days are

more than twice as productive as eight-hour days. Brian, for his part, thinks of

writing as a kind of blacksmithing, where it takes a while just to heat up the

metal before it’s malleable. He finds it somewhat useless to block out anything

less than ninety minutes for writing, as nothing much happens in the first half

hour except loading a giant block of “Now, where was I?” into his head.

Scheduling expert Kirk Pruhs, of the University of Pittsburgh, has had the same

experience. “If it’s less than an hour I’ll just do errands instead, because it’ll take

me the first thirty-five minutes to really figure out what I want to do and then I

might not have time to do it.”

Rudyard Kipling’s celebrated 1910 poem “If—” ends with an exuberant call

for time management: “If you can fill the unforgiving minute / With sixty

seconds’ worth of distance run…”

If only. The truth is, there’s always overhead—time lost to metawork, to the

logistics of bookkeeping and task management. This is one of the fundamental

tradeoffs of scheduling. And the more you take on, the more overhead there is.

At its nightmarish extreme, this turns into a phenomenon called thrashing.

Thrashing

Gage: Mr. Zuckerberg, do I have your full attention?…

Zuckerberg: You have part of my attention—you have the minimum amount.

—THE SOCIAL NETWORK

Computers multitask through a process called “threading,” which you can think

of as being like juggling a set of balls. Just as a juggler only hurls one ball at a

time into the air but keeps three aloft, a CPU only works on one program at a

time, but by swapping between them quickly enough (on the scale of tenthousandths of a second) it appears to be playing a movie, navigating the web,

and alerting you to incoming email all at once.

In the 1960s, computer scientists began thinking about how to automate the

process of sharing computer resources between different tasks and users. It was

an exciting time, recounts Peter Denning, now one of the top experts on

computer multitasking, who was then working on his doctorate at MIT. Exciting,

and uncertain: “How do you partition a main memory among a bunch of jobs

that are in there when some of them want to grow and some might want to shrink

and they’re going to interact with each other, trying to steal [memory] and all

these kinds of things?… How do you manage that whole set of interactions?

Nobody knew anything about that.”

Not surprisingly, given that the researchers didn’t really know yet what they

were doing, the effort encountered difficulties. And there was one in particular

that caught their attention. As Denning explains, under certain conditions a

dramatic problem “shows up as you add more jobs to the multiprogramming

mix. At some point you pass a critical threshold—unpredictable exactly where it

is, but you’ll know it when you get there—and all of a sudden the system seems

to die.”

Think again about our image of a juggler. With one ball in the air, there’s

enough spare time while that ball is aloft for the juggler to toss some others

upward as well. But what if the juggler takes on one more ball than he can

handle? He doesn’t drop that ball; he drops everything. The whole system, quite

literally, goes down. As Denning puts it, “The presence of one additional

program has caused a complete collapse of service.… The sharp difference

between the two cases at first defies intuition, which might lead us to expect a

gradual degradation of service as new programs are introduced into crowded

main memory.” Instead, catastrophe. And while we can understand a human

juggler being overwhelmed, what could cause something like this to happen to a

machine?

Here scheduling theory intersects caching theory. The whole idea of caches is

to keep the “working set” of needed items available for quick access. One way

this is done is by keeping the information the computer is currently using in fast

memory rather than on the slow hard disk. But if a task requires keeping track of

so many things that they won’t all fit into memory, then you might well end up

spending more time swapping information in and out of memory than doing the

actual work. What’s more, when you switch tasks, the newly active task might

make space for its working set by evicting portions of other working sets from

memory. The next task, upon reactivation, would then reacquire parts of its

working set from the hard disk and muscle them back into memory, again

displacing others. This problem—tasks stealing space from each other—can get

even worse in systems with hierarchies of caches between the processor and the

memory. As Peter Zijlstra, one of the head developers on the Linux operating

system scheduler, puts it, “The caches are warm for the current workload, and

when you context switch you pretty much invalidate all caches. And that hurts.”

At the extreme, a program may run just long enough to swap its needed items

into memory, before giving way to another program that runs just long enough to

overwrite them in turn.

This is thrashing: a system running full-tilt and accomplishing nothing at all.

Denning first diagnosed this phenomenon in a memory-management context, but

computer scientists now use the term “thrashing” to refer to pretty much any

situation where the system grinds to a halt because it’s entirely preoccupied with

metawork. A thrashing computer’s performance doesn’t bog down gradually. It

falls off a cliff. “Real work” has dropped to effectively zero, which also means

it’s going to be nearly impossible to get out.

Thrashing is a very recognizable human state. If you’ve ever had a moment

where you wanted to stop doing everything just to have the chance to write down

everything you were supposed to be doing, but couldn’t spare the time, you’ve

thrashed. And the cause is much the same for people as for computers: each task

is a draw on our limited cognitive resources. When merely remembering

everything we need to be doing occupies our full attention—or prioritizing every

task consumes all the time we had to do them—or our train of thought is

continually interrupted before those thoughts can translate to action—it feels like

panic, like paralysis by way of hyperactivity. It’s thrashing, and computers know

it well.

If you’ve ever wrestled with a system in a state of thrashing—and if you’ve

ever been in such a state—then you might be curious about the computer science

of getting out. In his landmark 1960s paper on the subject, Denning noted that an

ounce of prevention is worth a pound of cure. The easiest thing to do is simply to

get more memory: enough RAM, for instance, to fit the working sets of all the

running programs into memory at once and reduce the time taken by a context

switch. But preventive advice for thrashing doesn’t help you when you find

yourself in the midst of it. Besides, when it comes to human attention, we’re

stuck with what we’ve got.

Another way to avert thrashing before it starts is to learn the art of saying no.

Denning advocated, for instance, that a system should simply refuse to add a

program to its workload if it didn’t have enough free memory to hold its working

set. This prevents thrashing in machines, and is sensible advice for anyone with

a full plate. But this, too, might seem like an unattainable luxury to those of us

who find ourselves already overloaded—or otherwise unable to throttle the

demands being placed on us.

In these cases there’s clearly no way to work any harder, but you can work …

dumber. Along with considerations of memory, one of the biggest sources of

metawork in switching contexts is the very act of choosing what to do next. This,

too, can at times swamp the actual doing of the work. Faced with, say, an

overflowing inbox of n messages, we know from sorting theory that repeatedly

scanning it for the most important one to answer next will take O(n

2

) operations

—n scans of n messages apiece. This means that waking up to an inbox that’s

three times as full as usual could take you nine times as long to process. What’s

more, scanning through those emails means swapping every message into your

mind, one after another, before you respond to any of them: a surefire recipe for

memory thrashing.

In a thrashing state, you’re making essentially no progress, so even doing

tasks in the wrong order is better than doing nothing at all. Instead of answering

the most important emails first—which requires an assessment of the whole

picture that may take longer than the work itself—maybe you should sidestep

that quadratic-time quicksand by just answering the emails in random order, or

in whatever order they happen to appear on-screen. Thinking along the same

lines, the Linux core team, several years ago, replaced their scheduler with one

that was less “smart” about calculating process priorities but more than made up

for it by taking less time to calculate them.

If you still want to maintain your priorities, though, there’s a different and

even more interesting bargain you can strike to get your productivity back.

Interrupt Coalescing

Part of what makes real-time scheduling so complex and interesting is that it is

fundamentally a negotiation between two principles that aren’t fully compatible.

These two principles are called responsiveness and throughput: how quickly you

can respond to things, and how much you can get done overall. Anyone who’s

ever worked in an office environment can readily appreciate the tension between

these two metrics. It’s part of the reason there are people whose job it is to

answer the phone: they are responsive so that others may have throughput.

Again, life is harder when—like a computer—you must make the

responsiveness/throughput tradeoff yourself. And the best strategy for getting

things done might be, paradoxically, to slow down.

Operating system schedulers typically define a “period” in which every

program is guaranteed to run at least a little bit, with the system giving a “slice”

of that period to each program. The more programs are running, the smaller

those slices become, and the more context switches are happening every period,

maintaining responsiveness at the cost of throughput. Left unchecked, however,

this policy of guaranteeing each process at least some attention every period

could lead to catastrophe. With enough programs running, a task’s slice would

shrink to the point that the system was spending the entire slice on context

switching, before immediately context-switching again to the next task.

The culprit is the hard responsiveness guarantee. So modern operating

systems in fact set a minimum length for their slices and will refuse to subdivide

the period any more finely. (In Linux, for instance, this minimum useful slice

turns out to be about three-quarters of a millisecond, but in humans it might

realistically be at least several minutes.) If more processes are added beyond that

point, the period will simply get longer. This means that processes will have to

wait longer to get their turn, but the turns they get will at least be long enough to

do something.

Establishing a minimum amount of time to spend on any one task helps to

prevent a commitment to responsiveness from obliterating throughput entirely: if

the minimum slice is longer than the time it takes to context-switch, then the

system can never get into a state where context switching is the only thing it’s

doing. It’s also a principle that is easy to translate into a recommendation for

human lives. Methods such as “timeboxing” or “pomodoros,” where you literally

set a kitchen timer and commit to doing a single task until it runs out, are one

embodiment of this idea.

But what slice size should you aim for? Faced with the question of how long

to wait between intervals of performing a recurring task, like checking your

email, the answer from the perspective of throughput is simple: as long as

possible. But that’s not the end of the story; higher throughput, after all, also

means lower responsiveness.

For your computer, the annoying interruption that it has to check on regularly

isn’t email—it’s you. You might not move the mouse for minutes or hours, but

when you do, you expect to see the pointer on the screen move immediately,

which means the machine expends a lot of effort simply checking in on you. The

more frequently it checks on the mouse and keyboard, the quicker it can react

when there is input, but the more context switches it has to do. So the rule that

computer operating systems follow when deciding how long they can afford to

dedicate themselves to some task is simple: as long as possible without seeming

jittery or slow to the user.

When we humans leave the house to run a quick errand, we might say

something like, “You won’t even notice I’m gone.” When our machines contextswitch into a computation, they must literally return to us before we notice

they’re gone. To find this balancing point, operating systems programmers have

turned to psychology, mining papers in psychophysics for the exact number of

milliseconds of delay it takes for a human brain to register lag or flicker. There is

no point in attending to the user any more often than that.

Thanks to these efforts, when operating systems are working right you don’t

even notice how hard your computer is exerting itself. You continue to be able to

move your mouse around the screen fluidly even when your processor is hauling

full-tilt. The fluidity is costing you some throughput, but that’s a design tradeoff

that has been explicitly made by the system engineers: your system spends as

much time as it possibly can away from interacting with you, then gets around to

redrawing the mouse just in time.

And again, this is a principle that can be transferred to human lives. The moral

is that you should try to stay on a single task as long as possible without

decreasing your responsiveness below the minimum acceptable limit. Decide

how responsive you need to be—and then, if you want to get things done, be no

more responsive than that.

If you find yourself doing a lot of context switching because you’re tackling a

heterogeneous collection of short tasks, you can also employ another idea from

computer science: “interrupt coalescing.” If you have five credit card bills, for

instance, don’t pay them as they arrive; take care of them all in one go when the

fifth bill comes. As long as your bills are never due less than thirty-one days

after they arrive, you can designate, say, the first of each month as “bill-paying

day,” and sit down at that point to process every bill on your desk, no matter

whether it came three weeks or three hours ago. Likewise, if none of your email

correspondents require you to respond in less than twenty-four hours, you can

limit yourself to checking your messages once a day. Computers themselves do

something like this: they wait until some fixed interval and check everything,

instead of context-switching to handle separate, uncoordinated interrupts from

their various subcomponents.*

On occasion, computer scientists notice the absence of interrupt coalescing in

their own lives. Says Google director of research Peter Norvig: “I had to go

downtown three times today to run errands, and I said, ‘Oh, well, that’s just a

one-line bug in your algorithm. You should have just waited, or added it to the

to-do queue, rather than executing them sequentially as they got added one at a

time.’”

At human scale, we get interrupt coalescing for free from the postal system,

just as a consequence of their delivery cycle. Because mail gets delivered only

once a day, something mailed only a few minutes late might take an extra

twenty-four hours to reach you. Considering the costs of context switching, the

silver lining to this should by now be obvious: you can only get interrupted by

bills and letters at most once a day. What’s more, the twenty-four-hour postal

rhythm demands minimal responsiveness from you: it doesn’t make any

difference whether you mail your reply five minutes or five hours after receiving

a letter.

In academia, holding office hours is a way of coalescing interruptions from

students. And in the private sector, interrupt coalescing offers a redemptive view

of one of the most maligned office rituals: the weekly meeting. Whatever their

drawbacks, regularly scheduled meetings are one of our best defenses against the

spontaneous interruption and the unplanned context switch.

Perhaps the patron saint of the minimal-context-switching lifestyle is the

legendary programmer Donald Knuth. “I do one thing at a time,” he says. “This

is what computer scientists call batch processing—the alternative is swapping in

and out. I don’t swap in and out.” Knuth isn’t kidding. On January 1, 2014, he

embarked on “The TeX Tuneup of 2014,” in which he fixed all of the bugs that

had been reported in his TeX typesetting software over the previous six years.

His report ends with the cheery sign-off “Stay tuned for The TeX Tuneup of

2021!” Likewise, Knuth has not had an email address since 1990. “Email is a

wonderful thing for people whose role in life is to be on top of things. But not

for me; my role is to be on the bottom of things. What I do takes long hours of

studying and uninterruptible concentration.” He reviews all his postal mail every

three months, and all his faxes every six.

But one does not need to take things to Knuth’s extreme to wish that more of

our lives used interrupt coalescing as a design principle. The post office gives it

to us almost by accident; elsewhere, we need to build it, or demand it, for

ourselves. Our beeping and buzzing devices have “Do Not Disturb” modes,

which we could manually toggle on and off throughout the day, but that is too

blunt an instrument. Instead, we might agitate for settings that would provide an

explicit option for interrupt coalescing—the same thing at a human timescale

that the devices are doing internally. Alert me only once every ten minutes, say;

then tell me everything.

6 Bayes’s Rule

Predicting the Future

All human knowledge is uncertain, inexact, and partial.

—BERTRAND RUSSELL

The sun’ll come out tomorrow. You can bet your bottom dollar there’ll be

sun.

—ANNIE

In 1969, before embarking on a doctorate in astrophysics at Princeton, J. Richard

Gott III took a trip to Europe. There he saw the Berlin Wall, which had been

built eight years earlier. Standing in the shadow of the wall, a stark symbol of

the Cold War, he began to wonder how much longer it would continue to divide

the East and West.

On the face of it, there’s something absurd about trying to make this kind of

prediction. Even setting aside the impossibility of forecasting geopolitics, the

question seems mathematically laughable: it’s trying to make a prediction from a

single data point.

But as ridiculous as this might seem on its face, we make such predictions all

the time, by necessity. You arrive at a bus stop in a foreign city and learn,

perhaps, that the other tourist standing there has been waiting seven minutes.

When is the next bus likely to arrive? Is it worthwhile to wait—and if so, how

long should you do so before giving up?

Or perhaps a friend of yours has been dating somebody for a month and wants

your advice: is it too soon to invite them along to an upcoming family wedding?

The relationship is off to a good start—but how far ahead is it safe to make

plans?

A famous presentation made by Peter Norvig, Google’s director of research,

carried the title “The Unreasonable Effectiveness of Data” and enthused about

“how billions of trivial data points can lead to understanding.” The media

constantly tell us that we’re living in an “age of big data,” when computers can

sift through those billions of data points and find patterns invisible to the naked

eye. But often the problems most germane to daily human life are at the opposite

extreme. Our days are full of “small data.” In fact, like Gott standing at the

Berlin Wall, we often have to make an inference from the smallest amount of

data we could possibly have: a single observation.

So how do we do it? And how should we?

The story begins in eighteenth-century England, in a domain of inquiry

irresistible to great mathematical minds of the time, even those of the clergy:

gambling.

Reasoning Backward with the Reverend Bayes

If we be, therefore, engaged by arguments to put trust in past experience,

and make it the standard of our future judgement, these arguments must be

probable only.

—DAVID HUME

More than 250 years ago, the question of making predictions from small data

weighed heavily on the mind of the Reverend Thomas Bayes, a Presbyterian

minister in the charming spa town of Tunbridge Wells, England.

If we buy ten tickets for a new and unfamiliar raffle, Bayes imagined, and five

of them win prizes, then it seems relatively easy to estimate the raffle’s chances

of a win: 5/10, or 50%. But what if instead we buy a single ticket and it wins a

prize? Do we really imagine the probability of winning to be 1/1, or 100%? That

seems too optimistic. Is it? And if so, by how much? What should we actually

guess?

For somebody who has had such an impact on the history of reasoning under

uncertainty, Bayes’s own history remains ironically uncertain. He was born in

1701, or perhaps 1702, in the English county of Hertfordshire, or maybe it was

London. And in either 1746, ’47, ’48, or ’49 he would write one of the most

influential papers in all of mathematics, abandon it unpublished, and move on to

other things.

Between those two events we have a bit more certainty. The son of a minister,

Bayes went to the University of Edinburgh to study theology, and was ordained

like his father. He had mathematical as well as theological interests, and in 1736

he wrote an impassioned defense of Newton’s newfangled “calculus” in

response to an attack by Bishop George Berkeley. This work resulted in his

election in 1742 as a Fellow of the Royal Society, to whom he was

recommended as “a Gentleman … well skilled in Geometry and all parts of

Mathematical and Philosophical Learning.”

After Bayes died in 1761, his friend Richard Price was asked to review his

mathematical papers to see if they contained any publishable material. Price

came upon one essay in particular that excited him—one he said “has great

merit, and deserves to be preserved.” The essay concerned exactly the kind of

raffle problem under discussion:

Let us then imagine a person present at the drawing of a lottery, who knows nothing of its scheme or

of the proportion of Blanks to Prizes in it. Let it further be supposed, that he is obliged to infer this

from the number of blanks he hears drawn compared with the number of prizes; and that it is

enquired what conclusions in these circumstances he may reasonably make.

Bayes’s critical insight was that trying to use the winning and losing tickets we

see to figure out the overall ticket pool that they came from is essentially

reasoning backward. And to do that, he argued, we need to first reason forward

from hypotheticals. In other words, we need to first determine how probable it is

that we would have drawn the tickets we did if various scenarios were true. This

probability—known to modern statisticians as the “likelihood”—gives us the

information we need to solve the problem.

For instance, imagine we bought three tickets and all three were winners.

Now, if the raffle was of the particularly generous sort where all the tickets are

winners, then our three-for-three experience would of course happen all of the

time; it has a 100% chance in that scenario. If, instead, only half of the raffle’s

tickets were winners, our three-for-three experience would happen

1

⁄2 ×

1

⁄2 ×

1

⁄2

of the time, or in other words

1

⁄8 of the time. And if the raffle rewarded only one

ticket in a thousand, our outcome would have been incredibly unlikely:

1

⁄1,000 ×

1

⁄1,000 ×

1

⁄1,000

, or one in a billion.

Bayes argued that we should accordingly judge it to be more probable that all

the raffle tickets are winners than that half of them are, and in turn more

probable that half of them are than that only one in a thousand is. Perhaps we

had already intuited as much, but Bayes’s logic offers us the ability to quantify

that intuition. All things being equal, we should imagine it to be exactly eight

times likelier that all the tickets are winners than that half of them are—because

the tickets we drew are exactly eight times likelier (100% versus one-in-eight) in

that scenario. Likewise, it’s exactly 125 million times likelier that half the raffle

tickets are winners than that there’s only one winning ticket per thousand, which

we know by comparing one-in-eight to one-in-a-billion.

This is the crux of Bayes’s argument. Reasoning forward from hypothetical

pasts lays the foundation for us to then work backward to the most probable one.

It was an ingenious and innovative approach, but it did not manage to provide

a full answer to the raffle problem. In presenting Bayes’s results to the Royal

Society, Price was able to establish that if you buy a single raffle ticket and it’s a

winner, then there’s a 75% chance that at least half the tickets are winners. But

thinking about the probabilities of probabilities can get a bit head-spinning.

What’s more, if someone pressed us, “Well, fine, but what do you think the

raffle odds actually are?” we still wouldn’t know what to say.

The answer to this question—how to distill all the various possible hypotheses

into a single specific expectation—would be discovered only a few years later,

by the French mathematician Pierre-Simon Laplace.

Laplace’s Law

Laplace was born in Normandy in 1749, and his father sent him to a Catholic

school with the intent that he join the clergy. Laplace went on to study theology

at the University of Caen, but unlike Bayes—who balanced spiritual and

scientific devotions his whole life—he ultimately abandoned the cloth entirely

for mathematics.

In 1774, completely unaware of the previous work by Bayes, Laplace

published an ambitious paper called “Treatise on the Probability of the Causes of

Events.” In it, Laplace finally solved the problem of how to make inferences

backward from observed effects to their probable causes.

Bayes, as we saw, had found a way to compare the relative probability of one

hypothesis to another. But in the case of a raffle, there is literally an infinite

number of hypotheses: one for every conceivable proportion of winning tickets.

Using calculus, the once-controversial mathematics of which Bayes had been an

important defender, Laplace was able to prove that this vast spectrum of

possibilities could be distilled down to a single estimate, and a stunningly

concise one at that. If we really know nothing about our raffle ahead of time, he

showed, then after drawing a winning ticket on our first try we should expect

that the proportion of winning tickets in the whole pool is exactly 2/3. If we buy

three tickets and all of them are winners, the expected proportion of winning

tickets is exactly 4/5. In fact, for any possible drawing of w winning tickets in n

attempts, the expectation is simply the number of wins plus one, divided by the

number of attempts plus two:

(w+1)

⁄(n+2)

.

This incredibly simple scheme for estimating probabilities is known as

Laplace’s Law, and it is easy to apply in any situation where you need to assess

the chances of an event based on its history. If you make ten attempts at

something and five of them succeed, Laplace’s Law estimates your overall

chances to be 6/12 or 50%, consistent with our intuitions. If you try only once

and it works out, Laplace’s estimate of 2/3 is both more reasonable than

assuming you’ll win every time, and more actionable than Price’s guidance

(which would tell us that there is a 75% metaprobability of a 50% or greater

chance of success).

Laplace went on to apply his statistical approach to a wide range of problems

of his time, including assessing whether babies are truly equally likely to be born

male or female. (He established, to a virtual certainty, that male infants are in

fact slightly more likely than female ones.) He also wrote the Philosophical

Essay on Probabilities, arguably the first book about probability for a general

audience and still one of the best, laying out his theory and considering its

applications to law, the sciences, and everyday life.

Laplace’s Law offers us the first simple rule of thumb for confronting small

data in the real world. Even when we’ve made only a few observations—or only

one—it offers us practical guidance. Want to calculate the chance your bus is

late? The chance your softball team will win? Count the number of times it has

happened in the past plus one, then divide by the number of opportunities plus

two. And the beauty of Laplace’s Law is that it works equally well whether we

have a single data point or millions of them. Little Annie’s faith that the sun will

rise tomorrow is justified, it tells us: with an Earth that’s seen the sun rise for

about 1.6 trillion days in a row, the chance of another sunrise on the next

“attempt” is all but indistinguishable from 100%.

Bayes’s Rule and Prior Beliefs

All these suppositions are consistent and conceivable. Why should we give

the preference to one, which is no more consistent or conceivable than the

rest?

—DAVID HUME

Laplace also considered another modification to Bayes’s argument that would

prove crucial: how to handle hypotheses that are simply more probable than

others. For instance, while it’s possible that a lottery might give away prizes to

99% of the people who buy tickets, it’s more likely—we’d assume—that they

would give away prizes to only 1%. That assumption should be reflected in our

estimates.

To make things concrete, let’s say a friend shows you two different coins. One

is a normal, “fair” coin with a 50–50 chance of heads and tails; the other is a

two-headed coin. He drops them into a bag and then pulls one out at random. He

flips it once: heads. Which coin do you think your friend flipped?

Bayes’s scheme of working backward makes short work of this question. A

flip coming up heads happens 50% of the time with a fair coin and 100% of the

time with a two-headed coin. Thus we can assert confidently that it’s

100%⁄50%, or

exactly twice as probable, that the friend had pulled out the two-headed coin.

Now consider the following twist. This time, the friend shows you nine fair

coins and one two-headed coin, puts all ten into a bag, draws one at random, and

flips it: heads. Now what do you suppose? Is it a fair coin or the two-headed

one?

Laplace’s work anticipated this wrinkle, and here again the answer is

impressively simple. As before, a fair coin is exactly half as likely to come up

heads as a two-headed coin. But now, a fair coin is also nine times as likely to

have been drawn in the first place. It turns out that we can just take these two

different considerations and multiply them together: it is exactly four and a half

times more likely that your friend is holding a fair coin than the two-headed one.

The mathematical formula that describes this relationship, tying together our

previously held ideas and the evidence before our eyes, has come to be known—

ironically, as the real heavy lifting was done by Laplace—as Bayes’s Rule. And

it gives a remarkably straightforward solution to the problem of how to combine

preexisting beliefs with observed evidence: multiply their probabilities together.

Notably, having some preexisting beliefs is crucial for this formula to work. If

your friend simply approached you and said, “I flipped one coin from this bag

and it came up heads. How likely do you think it is that this is a fair coin?,” you

would be totally unable to answer that question unless you had at least some

sense of what coins were in the bag to begin with. (You can’t multiply the two

probabilities together when you don’t have one of them.) This sense of what was

“in the bag” before the coin flip—the chances for each hypothesis to have been

true before you saw any data—is known as the prior probabilities, or “prior” for

short. And Bayes’s Rule always needs some prior from you, even if it’s only a

guess. How many two-headed coins exist? How easy are they to get? How much

of a trickster is your friend, anyway?

The fact that Bayes’s Rule is dependent on the use of priors has at certain

points in history been considered controversial, biased, even unscientific. But in

reality, it is quite rare to go into a situation so totally unfamiliar that our mind is

effectively a blank slate—a point we’ll return to momentarily.

When you do have some estimate of prior probabilities, meanwhile, Bayes’s

Rule applies to a wide range of prediction problems, be they of the big-data

variety or the more common small-data sort. Computing the probability of

winning a raffle or tossing heads is only the beginning. The methods developed

by Bayes and Laplace can offer help any time you have uncertainty and a bit of

data to work with. And that’s exactly the situation we face when we try to

predict the future.

The Copernican Principle

It’s difficult to make predictions, especially about the future.

—DANISH PROVERB

When J. Richard Gott arrived at the Berlin Wall, he asked himself a very simple

question: Where am I? That is to say, where in the total life span of this artifact

have I happened to arrive? In a way, he was asking the temporal version of the

spatial question that had obsessed the astronomer Nicolaus Copernicus four

hundred years earlier: Where are we? Where in the universe is the Earth?

Copernicus would make the radical paradigm shift of imagining that the Earth

was not the bull’s-eye center of the universe—that it was, in fact, nowhere

special in particular. Gott decided to take the same step with regard to time.

He made the assumption that the moment when he encountered the Berlin

Wall wasn’t special—that it was equally likely to be any moment in the wall’s

total lifetime. And if any moment was equally likely, then on average his arrival

should have come precisely at the halfway point (since it was 50% likely to fall

before halfway and 50% likely to fall after). More generally, unless we know

better we can expect to have shown up precisely halfway into the duration of any

given phenomenon.* And if we assume that we’re arriving precisely halfway

into something’s duration, the best guess we can make for how long it will last

into the future becomes obvious: exactly as long as it’s lasted already. Gott saw

the Berlin Wall eight years after it was built, so his best guess was that it would

stand for eight years more. (It ended up being twenty.)

This straightforward reasoning, which Gott named the Copernican Principle,

results in a simple algorithm that can be used to make predictions about all sorts

of topics. Without any preconceived expectations, we might use it to obtain

predictions for the end of not only the Berlin Wall but any number of other

short-and long-lived phenomena. The Copernican Principle predicts that the

United States of America will last as a nation until approximately the year 2255,

that Google will last until roughly 2032, and that the relationship your friend

began a month ago will probably last about another month (maybe tell him not to

RSVP to that wedding invitation just yet). Likewise, it tells us to be skeptical

when, for instance, a recent New Yorker cover depicts a man holding a six-inch

smartphone with a familiar grid of square app icons, and the caption reads

“2525.” Doubtful. The smartphone as we know it is barely a decade old, and the

Copernican Principle tells us that it isn’t likely to be around in 2025, let alone

five centuries later. By 2525 it’d be mildly surprising if there were even a New

York City.

More practically, if we’re considering employment at a construction site

whose signage indicates that it’s been “7 days since the last industrial accident,”

we might want to stay away, unless it’s a particularly short job we plan to do.

And if a municipal transit system cannot afford the incredibly useful but

expensive real-time signs that tell riders when the next bus is going to arrive, the

Copernican Principle suggests that there might be a dramatically simpler and

cheaper alternative. Simply displaying how long it’s been since the previous bus

arrived at that stop offers a substantial hint about when the next one will.

But is the Copernican Principle right? After Gott published his conjecture in

Nature, the journal received a flurry of critical correspondence. And it’s easy to

see why when we try to apply the rule to some more familiar examples. If you

meet a 90-year-old man, the Copernican Principle predicts he will live to 180.

Every 6-year-old boy, meanwhile, is predicted to face an early death at the

tender age of 12.

To understand why the Copernican Principle works, and why it sometimes

doesn’t, we need to return to Bayes. Because despite its apparent simplicity, the

Copernican Principle is really an instance of Bayes’s Rule.

Bayes Meets Copernicus

When predicting the future, such as the longevity of the Berlin Wall, the

hypotheses we need to evaluate are all the possible durations of the phenomenon

at hand: will it last a week, a month, a year, a decade? To apply Bayes’s Rule, as

we have seen, we first need to assign a prior probability to each of these

durations. And it turns out that the Copernican Principle is exactly what results

from applying Bayes’s Rule using what is known as an uninformative prior.

At first this may seem like a contradiction in terms. If Bayes’s Rule always

requires us to specify our prior expectations and beliefs, how could we tell it that

we don’t have any? In the case of a raffle, one way to plead ignorance would be

to assume what’s called the “uniform prior,” which considers every proportion

of winning tickets to be equally likely.* In the case of the Berlin Wall, an

uninformative prior means saying that we don’t know anything about the time

span we’re trying to predict: the wall could equally well come down in the next

five minutes or last for five millennia.

Aside from that uninformative prior, the only piece of data we supply to

Bayes’s Rule, as we’ve seen, is the fact that we’ve encountered the Berlin Wall

when it is eight years old. Any hypothesis that would have predicted a less than

eight-year life span for the wall is thereby ruled out immediately, since those

hypotheses can’t account for our situation at all. (Similarly, a two-headed coin is

ruled out by the first appearance of tails.) Anything longer than eight years is

within the realm of possibility—but if the wall were going to be around for a

million years, it would be a big coincidence that we happened to bump into it so

very close to the start of its existence. Therefore, even though enormously long

life spans cannot be ruled out, neither are they very likely.

When Bayes’s Rule combines all these probabilities—the more-probable short

time spans pushing down the average forecast, the less-probable yet still possible

long ones pushing it up—the Copernican Principle emerges: if we want to

predict how long something will last, and have no other knowledge about it

whatsoever, the best guess we can make is that it will continue just as long as it’s

gone on so far.

In fact, Gott wasn’t even the first to propose something like the Copernican

Principle. In the mid-twentieth century, the Bayesian statistician Harold Jeffreys

had looked into determining the number of tramcars in a city given the serial

number on just one tramcar, and came up with the same answer: double the

serial number. And a similar problem had arisen even earlier, during World War

II, when the Allies sought to estimate the number of tanks being produced by

Germany. Purely mathematical estimates based on captured tanks’ serial

numbers predicted that the Germans were producing 246 tanks every month,

while estimates obtained by extensive (and highly risky) aerial reconnaissance

suggested the figure was more like 1,400. After the war, German records

revealed the true figure: 245.

Recognizing that the Copernican Principle is just Bayes’s Rule with an

uninformative prior answers a lot of questions about its validity. The Copernican

Principle seems reasonable exactly in those situations where we know nothing at

all—such as looking at the Berlin Wall in 1969, when we’re not even sure what

timescale is appropriate. And it feels completely wrong in those cases where we

do know something about the subject matter. Predicting that a 90-year-old man

will live to 180 years seems unreasonable precisely because we go into the

problem already knowing a lot about human life spans—and so we can do better.

The richer the prior information we bring to Bayes’s Rule, the more useful the

predictions we can get out of it.

Real-World Priors …

In the broadest sense, there are two types of things in the world: things that tend

toward (or cluster around) some kind of “natural” value, and things that don’t.

Human life spans are clearly in the former category. They roughly follow

what’s termed a “normal” distribution—also known as the “Gaussian”

distribution, after the German mathematician Carl Friedrich Gauss, and

informally called the “bell curve” for its characteristic shape. This shape does a

good job of characterizing human life spans; the average life span for men in the

United States, for instance, is centered at about 76 years, and the probabilities

fall off fairly sharply to either side. Normal distributions tend to have a single

appropriate scale: a one-digit life span is considered tragic, a three-digit one

extraordinary. Many other things in the natural world are normally distributed as

well, from human height, weight, and blood pressure to the noontime

temperature in a city and the diameter of fruits in an orchard.

There are a number of things in the world that don’t look normally distributed,

however—not by a long shot. The average population of a town in the United

States, for instance, is 8,226. But if you were to make a graph of the number of

towns by population, you wouldn’t see anything remotely like a bell curve.

There would be way more towns smaller than 8,226 than larger. At the same

time, the larger ones would be way bigger than the average. This kind of pattern

typifies what are called “power-law distributions.” These are also known as

“scale-free distributions” because they characterize quantities that can plausibly

range over many scales: a town can have tens, hundreds, thousands, tens of

thousands, hundreds of thousands, or millions of residents, so we can’t pin down

a single value for how big a “normal” town should be.

The power-law distribution characterizes a host of phenomena in everyday life

that have the same basic quality as town populations: most things below the

mean, and a few enormous ones above it. Movie boxoffice grosses, which can

range from four to ten figures, are another example. Most movies don’t make

much money at all, but the occasional Titanic makes … well, titanic amounts.

In fact, money in general is a domain full of power laws. Power-law

distributions characterize both people’s wealth and people’s incomes. The mean

income in America, for instance, is $55,688—but because income is roughly

power-law distributed, we know, again, that many more people will be below

this mean than above it, while those who are above might be practically off the

charts. So it is: two-thirds of the US population make less than the mean income,

but the top 1% make almost ten times the mean. And the top 1% of the 1% make

ten times more than that.

It’s often lamented that “the rich get richer,” and indeed the process of

“preferential attachment” is one of the surest ways to produce a power-law

distribution. The most popular websites are the most likely to get incoming

links; the most followed online celebrities are the ones most likely to gain new

fans; the most prestigious firms are the ones most likely to attract new clients;

the biggest cities are the ones most likely to draw new residents. In every case, a

power-law distribution will result.

Bayes’s Rule tells us that when it comes to making predictions based on

limited evidence, few things are as important as having good priors—that is, a

sense of the distribution from which we expect that evidence to have come.

Good predictions thus begin with having good instincts about when we’re

dealing with a normal distribution and when with a power-law distribution. As it

turns out, Bayes’s Rule offers us a simple but dramatically different predictive

rule of thumb for each.

… and Their Prediction Rules

Did you mean “this could go on forever” in a good way?

—BEN LERNER

Examining the Copernican Principle, we saw that when Bayes’s Rule is given an

uninformative prior, it always predicts that the total life span of an object will be

exactly double its current age. In fact, the uninformative prior, with its wildly

varying possible scales—the wall that might last for months or for millennia—is

a power-law distribution. And for any power-law distribution, Bayes’s Rule

indicates that the appropriate prediction strategy is a Multiplicative Rule:

multiply the quantity observed so far by some constant factor. For an

uninformative prior, that constant factor happens to be 2, hence the Copernican

prediction; in other power-law cases, the multiplier will depend on the exact

distribution you’re working with. For the grosses of movies, for instance, it

happens to be about 1.4. So if you hear a movie has made $6 million so far, you

can guess it will make about $8.4 million overall; if it’s made $90 million, guess

it will top out at $126 million.

This multiplicative rule is a direct consequence of the fact that power-law

distributions do not specify a natural scale for the phenomenon they’re

describing. The only thing that gives us a sense of scale for our prediction,

therefore, is the single data point we have—such as the fact that the Berlin Wall

has stood for eight years. The larger the value of that single data point, the larger

the scale we’re probably dealing with, and vice versa. It’s possible that a movie

that’s grossed $6 million is actually a blockbuster in its first hour of release, but

it’s far more likely to be just a single-digit-millions kind of movie.

When we apply Bayes’s Rule with a normal distribution as a prior, on the

other hand, we obtain a very different kind of guidance. Instead of a

multiplicative rule, we get an Average Rule: use the distribution’s “natural”

average—its single, specific scale—as your guide. For instance, if somebody is

younger than the average life span, then simply predict the average; as their age

gets close to and then exceeds the average, predict that they’ll live a few years

more. Following this rule gives reasonable predictions for the 90-year-old and

the 6-year-old: 94 and 77, respectively. (The 6-year-old gets a tiny edge over the

population average of 76 by virtue of having made it through infancy: we know

he’s not in the distribution’s left tail.)

Movie running times, like human lifetimes, also follow a normal distribution:

most films cluster right around a hundred minutes or so, with diminishing

numbers of exceptions tailing off to either side. But not all human activities are

so well behaved. The poet Dean Young once remarked that whenever he’s

listening to a poem in numbered sections, his heart sinks if the reader announces

the start of section four: if there are more than three parts, all bets are off, and

Young needs to hunker down for an earful. It turns out that Young’s dismay is,

in fact, perfectly Bayesian. An analysis of poems shows that, unlike movie

running times, poems follow something closer to a power-law than a normal

distribution: most poems are short, but some are epics. So when it comes to

poetry, make sure you’ve got a comfortable seat. Something normally distributed

that’s gone on seemingly too long is bound to end shortly; but the longer

something in a power-law distribution has gone on, the longer you can expect it

to keep going.

Between those two extremes, there’s actually a third category of things in life:

those that are neither more nor less likely to end just because they’ve gone on

for a while. Sometimes things are simply … invariant. The Danish

mathematician Agner Krarup Erlang, who studied such phenomena, formalized

the spread of intervals between independent events into the function that now

carries his name: the Erlang distribution. The shape of this curve differs from

both the normal and the power-law: it has a winglike contour, rising to a gentle

hump, with a tail that falls off faster than a power-law but more slowly than a

normal distribution. Erlang himself, working for the Copenhagen Telephone

Company in the early twentieth century, used it to model how much time could

be expected to pass between successive calls on a phone network. Since then, the

Erlang distribution has also been used by urban planners and architects to model

car and pedestrian traffic, and by networking engineers designing infrastructure

for the Internet. There are a number of domains in the natural world, too, where

events are completely independent from one another and the intervals between

them thus fall on an Erlang curve. Radioactive decay is one example, which

means that the Erlang distribution perfectly models when to expect the next ticks

of a Geiger counter. It also turns out to do a pretty good job of describing certain

human endeavors—such as the amount of time politicians stay in the House of

Representatives.

The Erlang distribution gives us a third kind of prediction rule, the Additive

Rule: always predict that things will go on just a constant amount longer. The

familiar refrain of “Just five more minutes!… [five minutes later] Five more

minutes!” that so often characterizes human claims regarding, say, one’s

readiness to leave the house or office, or the time until the completion of some

task, may seem indicative of some chronic failure to make realistic estimates.

Well, in the cases where one’s up against an Erlang distribution, anyway, that

refrain happens to be correct.

If a casino card-playing enthusiast tells his impatient spouse, for example, that

he’ll quit for the day after hitting one more blackjack (the odds of which are

about 20 to 1), he might cheerily predict, “I’ll be done in about twenty more

hands!” If, an unlucky twenty hands later, she returns, asking how long he’s

going to make her wait now, his answer will be unchanged: “I’ll be done in

about twenty more hands!” It sounds like our indefatigable card shark has

suffered a short-term memory loss—but, in fact, his prediction is entirely

correct. Indeed, distributions that yield the same prediction, no matter their

history or current state, are known to statisticians as “memoryless.”

Different prior distributions and their prediction rules.

These three very different patterns of optimal prediction—the Multiplicative,

Average, and Additive Rules—all result directly from applying Bayes’s Rule to

the power-law, normal, and Erlang distributions, respectively. And given the

way those predictions come out, the three distributions offer us different

guidance, too, on how surprised we should be by certain events.

In a power-law distribution, the longer something has gone on, the longer we

expect it to continue going on. So a power-law event is more surprising the

longer we’ve been waiting for it—and maximally surprising right before it

happens. A nation, corporation, or institution only grows more venerable with

each passing year, so it’s always stunning when it collapses.

In a normal distribution, events are surprising when they’re early—since we

expected them to reach the average—but not when they’re late. Indeed, by that

point they seem overdue to happen, so the longer we wait, the more we expect

them.

And in an Erlang distribution, events by definition are never any more or less

surprising no matter when they occur. Any state of affairs is always equally

likely to end regardless of how long it’s lasted. No wonder politicians are always

thinking about their next election.

Gambling is characterized by a similar kind of steady-state expectancy. If

your wait for, say, a win at the roulette wheel were characterized by a normal

distribution, then the Average Rule would apply: after a run of bad luck, it’d tell

you that your number should be coming any second, probably followed by more

losing spins. (In that case, it’d make sense to press on to the next win and then

quit.) If, instead, the wait for a win obeyed a power-law distribution, then the

Multiplicative Rule would tell you that winning spins follow quickly after one

another, but the longer a drought had gone on the longer it would probably

continue. (In that scenario, you’d be right to keep playing for a while after any

win, but give up after a losing streak.) Up against a memoryless distribution,

however, you’re stuck. The Additive Rule tells you the chance of a win now is

the same as it was an hour ago, and the same as it will be an hour from now.

Nothing ever changes. You’re not rewarded for sticking it out and ending on a

high note; neither is there a tipping point when you should just cut your losses.

In “The Gambler,” Kenny Rogers famously advised that you’ve got to “Know

when to walk away / Know when to run”—but for a memoryless distribution,

there is no right time to quit. This may in part explain these games’

addictiveness.

Knowing what distribution you’re up against can make all the difference.

When the Harvard biologist and prolific popularizer of science Stephen Jay

Gould discovered that he had cancer, his immediate impulse was to read the

relevant medical literature. Then he found out why his doctors had discouraged

him from doing so: half of all patients with his form of cancer died within eight

months of discovery.

But that one statistic—eight months—didn’t tell him anything about the

distribution of survivors. If it were a normal distribution, then the Average Rule

would give a pretty clear forecast of how long he could expect to live: about

eight months. But if it were a power-law, with a tail that stretches far out to the

right, then the situation would be quite different: the Multiplicative Rule would

tell him that the longer he lived, the more evidence it would provide that he

would live longer. Reading further, Gould discovered that “the distribution was

indeed, strongly right skewed, with a long tail (however small) that extended for

several years above the eight month median. I saw no reason why I shouldn’t be

in that small tail, and I breathed a very long sigh of relief.” Gould would go on

to live for twenty more years after his diagnosis.

Small Data and the Mind

The three prediction rules—Multiplicative, Average, and Additive—are

applicable in a wide range of everyday situations. And in those situations, people

in general turn out to be remarkably good at using the right prediction rule.

When he was in graduate school, Tom, along with MIT’s Josh Tenenbaum, ran

an experiment asking people to make predictions for a variety of everyday

quantities—such as human life spans, the grosses of movies, and the time that

US representatives would spend in office—based on just one piece of

information in each case: current age, money earned so far, and years served to

date. Then they compared the predictions people made to the predictions given

by applying Bayes’s Rule to the actual real-world data across each of those

domains.

As it turned out, the predictions that people had made were extremely close to

those produced by Bayes’s Rule. Intuitively, people made different types of

predictions for quantities that followed different distributions—power-law,

normal, and Erlang—in the real world. In other words, while you might not

know or consciously remember which situation calls for the Multiplicative,

Average, or Additive Rule, the predictions you make every day tend to implicitly

reflect the different cases where these distributions appear in everyday life, and

the different ways they behave.

In light of what we know about Bayes’s Rule, this remarkably good human

performance suggests something critical that helps to understand how people

make predictions. Small data is big data in disguise. The reason we can often

make good predictions from a small number of observations—or just a single

one—is that our priors are so rich. Whether we know it or not, we appear to

carry around in our heads surprisingly accurate priors about movie grosses and

running times, poem lengths, and political terms of office, not to mention human

life spans. We don’t need to gather them explicitly; we absorb them from the

world.

The fact that, on the whole, people’s hunches seem to closely match the

predictions of Bayes’s Rule also makes it possible to reverse-engineer all kinds

of prior distributions, even ones about which it’s harder to get authoritative realworld data. For instance, being kept on hold by customer service is a lamentably

common facet of human experience, but there aren’t publicly available data sets

on hold times the way there are for Hollywood boxoffice grosses. But if people’s

predictions are informed by their experiences, we can use Bayes’s Rule to

conduct indirect reconnaissance about the world by mining people’s

expectations. When Tom and Josh asked people to predict hold times from a

single data point, the results suggested that their subjects were using the

Multiplicative Rule: the total wait people expect is one and a third times as long

as they’ve waited so far. This is consistent with having a power-law distribution

as a prior, where a wide range of scales is possible. Just hope you don’t end up

on the Titanic of hold times. Over the past decade, approaches like these have

enabled cognitive scientists to identify people’s prior distributions across a broad

swath of domains, from vision to language.

There’s a crucial caveat here, however. In cases where we don’t have good

priors, our predictions aren’t good. In Tom and Josh’s study, for instance, there

was one subject where people’s predictions systematically diverged from

Bayes’s Rule: predicting the length of the reign of Egyptian pharaohs. (As it

happens, pharaohs’ reigns follow an Erlang distribution.) People simply didn’t

have enough everyday exposure to have an intuitive feel for the range of those

values, so their predictions, of course, faltered. Good predictions require good

priors.

This has a number of important implications. Our judgments betray our

expectations, and our expectations betray our experience. What we project about

the future reveals a lot—about the world we live in, and about our own past.

What Our Predictions Tell Us About Ourselves

When Walter Mischel ran his famous “marshmallow test” in the early 1970s, he

was trying to understand how the ability to delay gratification develops with age.

At a nursery school on the Stanford campus, a series of three-, four-, and fiveyear-olds had their willpower tested. Each child would be shown a delicious

treat, such as a marshmallow, and told that the adult running the experiment was

about to leave the room for a while. If they wanted to, they could eat the treat

right away. But if they waited until the experimenter came back, they would get

two treats.

Unable to resist, some of the children ate the treat immediately. And some of

them stuck it out for the full fifteen minutes or so until the experimenter

returned, and got two treats as promised. But perhaps the most interesting group

comprised the ones in between—the ones who managed to wait a little while, but

then surrendered and ate the treat.

These cases, where children struggled mightily and suffered valiantly, only to

give in and lose the extra marshmallow anyway, have been interpreted as

suggesting a kind of irrationality. If you’re going to cave, why not just cave

immediately, and skip the torture? But it all depends on what kind of situation

the children think they are in. As the University of Pennsylvania’s Joe McGuire

and Joe Kable have pointed out, if the amount of time it takes for adults to come

back is governed by a power-law distribution—with long absences suggesting

even longer waits lie ahead—then cutting one’s losses at some point can make

perfect sense.

In other words, the ability to resist temptation may be, at least in part, a matter

of expectations rather than willpower. If you predict that adults tend to come

back after short delays—something like a normal distribution—you should be

able to hold out. The Average Rule suggests that after a painful wait, the thing to

do is hang in there: the experimenter should be returning any minute now. But if

you have no idea of the timescale of the disappearance—consistent with a

power-law distribution—then it’s an uphill battle. The Multiplicative Rule then

suggests that a protracted wait is just a small fraction of what’s to come.

Decades after the original marshmallow experiments, Walter Mischel and his

colleagues went back and looked at how the participants were faring in life.

Astonishingly, they found that children who had waited for two treats grew into

young adults who were more successful than the others, even measured by

quantitative metrics like their SAT scores. If the marshmallow test is about

willpower, this is a powerful testament to the impact that learning self-control

can have on one’s life. But if the test is less about will than about expectations,

then this tells a different, perhaps more poignant story.

A team of researchers at the University of Rochester recently explored how

prior experiences might affect behavior in the marshmallow test. Before

marshmallows were even mentioned, the kids in the experiment embarked on an

art project. The experimenter gave them some mediocre supplies, and promised

to be back with better options soon. But, unbeknownst to them, the children were

divided into two groups. In one group, the experimenter was reliable, and came

back with the better art supplies as promised. In the other, she was unreliable,

coming back with nothing but apologies.

The art project completed, the children went on to the standard marshmallow

test. And here, the children who had learned that the experimenter was unreliable

were more likely to eat the marshmallow before she came back, losing the

opportunity to earn a second treat.

Failing the marshmallow test—and being less successful in later life—may

not be about lacking willpower. It could be a result of believing that adults are

not dependable: that they can’t be trusted to keep their word, that they disappear

for intervals of arbitrary length. Learning self-control is important, but it’s

equally important to grow up in an environment where adults are consistently

present and trustworthy.

Priors in the Age of Mechanical Reproduction

As if someone were to buy several copies of the morning paper to assure

himself that what it said was true.

—LUDWIG WITTGENSTEIN

He is careful of what he reads, for that is what he will write. He is careful

of what he learns, for that is what he will know.

—ANNIE DILLARD

The best way to make good predictions, as Bayes’s Rule shows us, is to be

accurately informed about the things you’re predicting. That’s why we can do a

good job of projecting human life spans, but perform poorly when asked to

estimate the reigns of pharaohs.

Being a good Bayesian means representing the world in the correct

proportions—having good priors, appropriately calibrated. By and large, for

humans and other animals this happens naturally; as a rule, when something

surprises us, it ought to surprise us, and when it doesn’t, it ought not to. Even

when we accumulate biases that aren’t objectively correct, they still usually do a

reasonable job of reflecting the specific part of the world we live in. For

instance, someone living in a desert climate might overestimate the amount of

sand in the world, and someone living at the poles might overestimate the

amount of snow. Both are well tuned to their own ecological niche.

Everything starts to break down, however, when a species gains language.

What we talk about isn’t what we experience—we speak chiefly of interesting

things, and those tend to be things that are uncommon. More or less by

definition, events are always experienced at their proper frequencies, but this

isn’t at all true of language. Anyone who has experienced a snake bite or a

lightning strike will tend to retell those singular stories for the rest of their lives.

And those stories will be so salient that they will be picked up and retold by

others.

There’s a curious tension, then, between communicating with others and

maintaining accurate priors about the world. When people talk about what

interests them—and offer stories they think their listeners will find interesting—

it skews the statistics of our experience. That makes it hard to maintain

appropriate prior distributions. And the challenge has only increased with the

development of the printing press, the nightly news, and social media—

innovations that allow our species to spread language mechanically.

Consider how many times you’ve seen either a crashed plane or a crashed car.

It’s entirely possible you’ve seen roughly as many of each—yet many of those

cars were on the road next to you, whereas the planes were probably on another

continent, transmitted to you via the Internet or television. In the United States,

for instance, the total number of people who have lost their lives in commercial

plane crashes since the year 2000 would not be enough to fill Carnegie Hall even

half full. In contrast, the number of people in the United States killed in car

accidents over that same time is greater than the entire population of Wyoming.

Simply put, the representation of events in the media does not track their

frequency in the world. As sociologist Barry Glassner notes, the murder rate in

the United States declined by 20% over the course of the 1990s, yet during that

time period the presence of gun violence on American news increased by 600%.

If you want to be a good intuitive Bayesian—if you want to naturally make

good predictions, without having to think about what kind of prediction rule is

appropriate—you need to protect your priors. Counterintuitively, that might

mean turning off the news.

7 Overfitting

When to Think Less

When Charles Darwin was trying to decide whether he should propose to his

cousin Emma Wedgwood, he got out a pencil and paper and weighed every

possible consequence. In favor of marriage he listed children, companionship,

and the “charms of music & female chit-chat.” Against marriage he listed the

“terrible loss of time,” lack of freedom to go where he wished, the burden of

visiting relatives, the expense and anxiety provoked by children, the concern that

“perhaps my wife won’t like London,” and having less money to spend on

books. Weighing one column against the other produced a narrow margin of

victory, and at the bottom Darwin scrawled, “Marry—Marry—Marry Q.E.D.”

Quod erat demonstrandum, the mathematical sign-off that Darwin himself then

restated in English: “It being proved necessary to Marry.”

The pro-and-con list was already a time-honored algorithm by Darwin’s time,

being endorsed by Benjamin Franklin a century before. To get over “the

Uncertainty that perplexes us,” Franklin wrote,

my Way is, divide half a Sheet of Paper by a Line into two Columns, writing over the one Pro, and

over the other Con. Then during three or four Days Consideration I put down under the different

Heads short Hints of the different Motives that at different Times occur to me for or against the

Measure. When I have thus got them all together in one View, I endeavour to estimate their

respective Weights; and where I find two, one on each side, that seem equal, I strike them both out: If

I find a Reason pro equal to some two Reasons con, I strike out the three. If I judge some two

Reasons con equal to some three Reasons pro, I strike out the five; and thus proceeding I find at

length where the Ballance lies; and if after a Day or two of farther Consideration nothing new that is

of Importance occurs on either side, I come to a Determination accordingly.

Franklin even thought about this as something like a computation, saying, “I

have found great Advantage from this kind of Equation, in what may be called

Moral or Prudential Algebra.”

Darwin’s journal, July 1838. Reprinted with permission of Cambridge University Library.

When we think about thinking, it’s easy to assume that more is better: that you

will make a better decision the more pros and cons you list, make a better

prediction about the price of a stock the more relevant factors you identify, and

write a better report the more time you spend working on it. This is certainly the

premise behind Franklin’s system. In this sense, Darwin’s “algebraic” approach

to matrimony, despite its obvious eccentricity, seems remarkably and maybe

even laudably rational.

However, if Franklin or Darwin had lived into the era of machine-learning

research—the science of teaching computers to make good judgments from

experience—they’d have seen Moral Algebra shaken to its foundations. The

question of how hard to think, and how many factors to consider, is at the heart

of a knotty problem that statisticians and machine-learning researchers call

“overfitting.” And dealing with that problem reveals that there’s a wisdom to

deliberately thinking less. Being aware of overfitting changes how we should

approach the market, the dining table, the gym … and the altar.

The Case Against Complexity

Anything you can do I can do better; I can do anything better than you.

—ANNIE GET YOUR GUN

Every decision is a kind of prediction: about how much you’ll like something

you haven’t tried yet, about where a certain trend is heading, about how the road

less traveled (or more so) is likely to pan out. And every prediction, crucially,

involves thinking about two distinct things: what you know and what you don’t.

That is, it’s an attempt to formulate a theory that will account for the experiences

you’ve had to date and say something about the future ones you’re guessing at.

A good theory, of course, will do both. But the fact that every prediction must in

effect pull double duty creates a certain unavoidable tension.

Life satisfaction as a function of time since marriage.

As one illustration of this tension, let’s look at a data set that might have been

relevant to Darwin: people’s life satisfaction over their first ten years of

marriage, from a recent study conducted in Germany. Each point on that chart is

taken from the study itself; our job is to figure out the formula for a line that

would fit those points and extend into the future, allowing us to make predictions

past the ten-year mark.

One possible formula would use just a single factor to predict life satisfaction:

the time since marriage. This would create a straight line on the chart. Another

possibility is to use two factors, time and time squared; the resulting line would

have a parabolic U-shape, letting it capture a potentially more complex

relationship between time and happiness. And if we expand the formula to

include yet more factors (time cubed and so on), the line will acquire ever more

inflection points, getting more and more “bendy” and flexible. By the time we

get to a nine-factor formula, we can capture very complex relationships indeed.

Mathematically speaking, our two-factor model incorporates all the

information that goes into the one-factor model, and has another term it could

use as well. Likewise, the nine-factor model leverages all of the information at

the disposal of the two-factor model, plus potentially lots more. By this logic, it

seems like the nine-factor model ought to always give us the best predictions.

As it turns out, things are not quite so simple.

Predictions of life satisfaction using models with different numbers of factors.

The result of applying these models to the data is shown above. The onefactor model, unsurprisingly, misses a lot of the exact data points, though it

captures the basic trend—a comedown after the honeymoon bliss. However, its

straight-line prediction forecasts that this decrease will continue forever,

ultimately resulting in infinite misery. Something about that trajectory doesn’t

sound quite right. The two-factor model comes closer to fitting the survey data,

and its curved shape makes a different long-term prediction, suggesting that after

the initial decline life satisfaction more or less levels out over time. Finally, the

nine-factor model passes through each and every point on the chart; it is

essentially a perfect fit for all the data from the study.

In that sense it seems like the nine-factor formula is indeed our best model.

But if you look at the predictions it makes for the years not included in the study,

you might wonder about just how useful it really is: it predicts misery at the

altar, a giddily abrupt rise in satisfaction after several months of marriage, a

bumpy roller-coaster ride thereafter, and a sheer drop after year ten. By contrast,

the leveling off predicted by the two-factor model is the forecast most consistent

with what psychologists and economists say about marriage and happiness.

(They believe, incidentally, that it simply reflects a return to normalcy—to

people’s baseline level of satisfaction with their lives—rather than any

displeasure with marriage itself.)

The lesson is this: it is indeed true that including more factors in a model will

always, by definition, make it a better fit for the data we have already. But a

better fit for the available data does not necessarily mean a better prediction.

Adding small amounts of random “noise” to the data (simulating the effects of repeating the survey with a

different group of participants) produces wild undulations in the nine-factor model, while the one-and twofactor models in comparison are much more stable and consistent in their predictions.

Granted, a model that’s too simple—for instance, the straight line of the onefactor formula—can fail to capture the essential pattern in the data. If the truth

looks like a curve, no straight line can ever get it right. On the other hand, a

model that’s too complicated, such as our nine-factor model here, becomes

oversensitive to the particular data points that we happened to observe. As a

consequence, precisely because it is tuned so finely to that specific data set, the

solutions it produces are highly variable. If the study were repeated with

different people, producing slight variations on the same essential pattern, the

one-and two-factor models would remain more or less steady—but the ninefactor model would gyrate wildly from one instance of the study to the next.

This is what statisticians call overfitting.

So one of the deepest truths of machine learning is that, in fact, it’s not always

better to use a more complex model, one that takes a greater number of factors

into account. And the issue is not just that the extra factors might offer

diminishing returns—performing better than a simpler model, but not enough to

justify the added complexity. Rather, they might make our predictions

dramatically worse.

The Idolatry of Data

If we had copious data, drawn from a perfectly representative sample,

completely mistake-free, and representing exactly what we’re trying to evaluate,

then using the most complex model available would indeed be the best approach.

But if we try to perfectly fit our model to the data when any of these factors fails

to hold, we risk overfitting.

In other words, overfitting poses a danger any time we’re dealing with noise

or mismeasurement—and we almost always are. There can be errors in how the

data were collected, or in how they were reported. Sometimes the phenomena

being investigated, such as human happiness, are hard to even define, let alone

measure. Thanks to their flexibility, the most complex models available to us can

fit any patterns that appear in the data, but this means that they will also do so

even when those patterns are mere phantoms and mirages in the noise.

Throughout history, religious texts have warned their followers against

idolatry: the worshipping of statues, paintings, relics, and other tangible artifacts

in lieu of the intangible deities those artifacts represent. The First

Commandment, for instance, warns against bowing down to “any graven image,

or any likeness of any thing that is in heaven.” And in the Book of Kings, a

bronze snake made at God’s orders becomes an object of worship and incenseburning, instead of God himself. (God is not amused.) Fundamentally,

overfitting is a kind of idolatry of data, a consequence of focusing on what

we’ve been able to measure rather than what matters.

This gap between the data we have and the predictions we want is virtually

everywhere. When making a big decision, we can only guess at what will please

us later by thinking about the factors important to us right now. (As Harvard’s

Daniel Gilbert puts it, our future selves often “pay good money to remove the

tattoos that we paid good money to get.”) When making a financial forecast, we

can only look at what correlated with the price of a stock in the past, not what

might in the future. Even in our small daily acts this pattern holds: writing an

email, we use our own read-through of the text to predict that of the recipient.

No less than in public surveys, the data in our own lives are thus also always

noisy, at best a proxy metric for the things we really care about.

As a consequence, considering more and more factors and expending more

effort to model them can lead us into the error of optimizing for the wrong thing

—offering prayers to the bronze snake of data rather than the larger force behind

it.

Overfitting Everywhere

Once you know about overfitting, you see it everywhere.

Overfitting, for instance, explains the irony of our palates. How can it be that

the foods that taste best to us are broadly considered to be bad for our health,

when the entire function of taste buds, evolutionarily speaking, is to prevent us

from eating things that are bad?

The answer is that taste is our body’s proxy metric for health. Fat, sugar, and

salt are important nutrients, and for a couple hundred thousand years, being

drawn to foods containing them was a reasonable measure for a sustaining diet.

But being able to modify the foods available to us broke that relationship. We

can now add fat and sugar to foods beyond amounts that are good for us, and

then eat those foods exclusively rather than the mix of plants, grains, and meats

that historically made up the human diet. In other words, we can overfit taste.

And the more skillfully we can manipulate food (and the more our lifestyles

diverge from those of our ancestors), the more imperfect a metric taste becomes.

Our human agency thus turns into a curse, making us dangerously able to have

exactly what we want even when we don’t quite want exactly the right thing.

Beware: when you go to the gym to work off the extra weight from all that

sugar, you can also risk overfitting fitness. Certain visible signs of fitness—low

body fat and high muscle mass, for example—are easy to measure, and they are

related to, say, minimizing the risk of heart disease and other ailments. But they,

too, are an imperfect proxy measure. Overfitting the signals—adopting an

extreme diet to lower body fat and taking steroids to build muscle, perhaps—can

make you the picture of good health, but only the picture.

Overfitting also shows up in sports. For instance, Tom has been a fencer, on

and off, since he was a teenager. The original goal of fencing was to teach

people how to defend themselves in a duel, hence the name: “defencing.” And

the weapons used in modern fencing are similar to those that were used to train

for such encounters. (This is particularly true of the épée, which was still used in

formal duels less than fifty years ago.) But the introduction of electronic scoring

equipment—a button on the tip of the blade that registers a hit—has changed the

nature of the sport, and techniques that would serve you poorly in a serious duel

have become critical skills in competition. Modern fencers use flexible blades

that allow them to “flick” the button at their opponent, grazing just hard enough

to register and score. As a result, they can look more like they’re cracking thin

metal whips at each other than cutting or thrusting. It’s as exciting a sport as

ever, but as athletes overfit their tactics to the quirks of scorekeeping, it becomes

less useful in instilling the skills of real-world swordsmanship.

Perhaps nowhere, however, is overfitting as powerful and troublesome as in

the world of business. “Incentive structures work,” as Steve Jobs put it. “So you

have to be very careful of what you incent people to do, because various

incentive structures create all sorts of consequences that you can’t anticipate.”

Sam Altman, president of the startup incubator Y Combinator, echoes Jobs’s

words of caution: “It really is true that the company will build whatever the CEO

decides to measure.”

In fact, it’s incredibly difficult to come up with incentives or measurements

that do not have some kind of perverse effect. In the 1950s, Cornell management

professor V. F. Ridgway cataloged a host of such “Dysfunctional Consequences

of Performance Measurements.” At a job-placement firm, staffers were

evaluated on the number of interviews they conducted, which motivated them to

run through the meetings as quickly as possible, without spending much time

actually helping their clients find jobs. At a federal law enforcement agency,

investigators given monthly performance quotas were found to pick easy cases at

the end of the month rather than the most urgent ones. And at a factory, focusing

on production metrics led supervisors to neglect maintenance and repairs, setting

up future catastrophe. Such problems can’t simply be dismissed as a failure to

achieve management goals. Rather, they are the opposite: the ruthless and clever

optimization of the wrong thing.

The twenty-first-century shift into real-time analytics has only made the

danger of metrics more intense. Avinash Kaushik, digital marketing evangelist at

Google, warns that trying to get website users to see as many ads as possible

naturally devolves into trying to cram sites with ads: “When you are paid on a

[cost per thousand impressions] basis the incentive is to figure out how to show

the most possible ads on every page [and] ensure the visitor sees the most

possible pages on the site.… That incentive removes a focus from the important

entity, your customer, and places it on the secondary entity, your advertiser.”

The website might gain a little more money in the short term, but ad-crammed

articles, slow-loading multi-page slide shows, and sensationalist clickbait

headlines will drive away readers in the long run. Kaushik’s conclusion:

“Friends don’t let friends measure Page Views. Ever.”

In some cases, the difference between a model and the real world is literally a

matter of life and death. In the military and in law enforcement, for example,

repetitive, rote training is considered a key means for instilling line-of-fire skills.

The goal is to drill certain motions and tactics to the point that they become

totally automatic. But when overfitting creeps in, it can prove disastrous. There

are stories of police officers who find themselves, for instance, taking time out

during a gunfight to put their spent casings in their pockets—good etiquette on a

firing range. As former Army Ranger and West Point psychology professor

Dave Grossman writes, “After the smoke had settled in many real gunfights,

officers were shocked to discover empty brass in their pockets with no memory

of how it got there. On several occasions, dead cops were found with brass in

their hands, dying in the middle of an administrative procedure that had been

drilled into them.” Similarly, the FBI was forced to change its training after

agents were found reflexively firing two shots and then holstering their weapon

—a standard cadence in training—regardless of whether their shots had hit the

target and whether there was still a threat. Mistakes like these are known in law

enforcement and the military as “training scars,” and they reflect the fact that it’s

possible to overfit one’s own preparation. In one particularly dramatic case, an

officer instinctively grabbed the gun out of the hands of an assailant and then

instinctively handed it right back—just as he had done time and time again with

his trainers in practice.

Detecting Overfitting: Cross-Validation

Because overfitting presents itself initially as a theory that perfectly fits the

available data, it may seem insidiously hard to detect. How can we expect to tell

the difference between a genuinely good model and one that’s overfitting? In an

educational setting, how can we distinguish between a class of students excelling

at the subject matter and a class merely being “taught to the test”? In the

business world, how can we tell a genuine star performer from an employee who

has just cannily overfit their work to the company’s key performance indicators

—or to the boss’s perception?

Teasing apart those scenarios is indeed challenging, but it is not impossible.

Research in machine learning has yielded several concrete strategies for

detecting overfitting, and one of the most important is what’s known as CrossValidation.

Simply put, Cross-Validation means assessing not only how well a model fits

the data it’s given, but how well it generalizes to data it hasn’t seen.

Paradoxically, this may involve using less data. In the marriage example, we

might “hold back,” say, two points at random, and fit our models only to the

other eight. We’d then take those two test points and use them to gauge how well

our various functions generalize beyond the eight “training” points they’ve been

given. The two held-back points function as canaries in the coal mine: if a

complex model nails the eight training points but wildly misses the two test

points, it’s a good bet that overfitting is at work.

Aside from withholding some of the available data points, it is also useful to

consider testing the model with data derived from some other form of evaluation

entirely. As we have seen, the use of proxy metrics—taste as a proxy for

nutrition, number of cases solved as a proxy for investigator diligence—can also

lead to overfitting. In these cases, we’ll need to cross-validate the primary

performance measure we’re using against other possible measures.

In schools, for example, standardized tests offer a number of benefits,

including a distinct economy of scale: they can be graded cheaply and rapidly by

the thousands. Alongside such tests, however, schools could randomly assess

some small fraction of the students—one per class, say, or one in a hundred—

using a different evaluation method, perhaps something like an essay or an oral

exam. (Since only a few students would be tested this way, having this

secondary method scale well is not a big concern.) The standardized tests would

provide immediate feedback—you could have students take a short

computerized exam every week and chart the class’s progress almost in real

time, for instance—while the secondary data points would serve to crossvalidate: to make sure that the students were actually acquiring the knowledge

that the standardized test is meant to measure, and not simply getting better at

test-taking. If a school’s standardized scores rose while its “nonstandardized”

performance moved in the opposite direction, administrators would have a clear

warning sign that “teaching to the test” had set in, and the pupils’ skills were

beginning to overfit the mechanics of the test itself.

Cross-Validation also offers a suggestion for law enforcement and military

personnel looking to instill good reflexes without hammering in habits from the

training process itself. Just as essays and oral exams can cross-validate

standardized tests, so occasional unfamiliar “cross-training” assessments might

be used to measure whether reaction time and shooting accuracy are generalizing

to unfamiliar tasks. If they aren’t, then that’s a strong signal to change the

training regimen. While nothing may truly prepare one for actual combat,

exercises like this may at least warn in advance where “training scars” are likely

to have formed.

How to Combat Overfitting: Penalizing Complexity

If you can’t explain it simply, you don’t understand it well enough.

—ANONYMOUS

We’ve seen some of the ways that overfitting can rear its head, and we’ve

looked at some of the methods to detect and measure it. But what can we

actually do to alleviate it?

From a statistics viewpoint, overfitting is a symptom of being too sensitive to

the actual data we’ve seen. The solution, then, is straightforward: we must

balance our desire to find a good fit against the complexity of the models we use

to do so.

One way to choose among several competing models is the Occam’s razor

principle, which suggests that, all things being equal, the simplest possible

hypothesis is probably the correct one. Of course, things are rarely completely

equal, so it’s not immediately obvious how to apply something like Occam’s

razor in a mathematical context. Grappling with this challenge in the 1960s,

Russian mathematician Andrey Tikhonov proposed one answer: introduce an

additional term to your calculations that penalizes more complex solutions. If we

introduce a complexity penalty, then more complex models need to do not

merely a better job but a significantly better job of explaining the data to justify

their greater complexity. Computer scientists refer to this principle—using

constraints that penalize models for their complexity—as Regularization.

So what do these complexity penalties look like? One algorithm, discovered in

1996 by biostatistician Robert Tibshirani, is called the Lasso and uses as its

penalty the total weight of the different factors in the model.* By putting this

downward pressure on the weights of the factors, the Lasso drives as many of

them as possible completely to zero. Only the factors that have a big impact on

the results remain in the equation—thus potentially transforming, say, an

overfitted nine-factor model into a simpler, more robust formula with just a

couple of the most critical factors.

Techniques like the Lasso are now ubiquitous in machine learning, but the

same kind of principle—a penalty for complexity—also appears in nature.

Living organisms get a certain push toward simplicity almost automatically,

thanks to the constraints of time, memory, energy, and attention. The burden of

metabolism, for instance, acts as a brake on the complexity of organisms,

introducing a caloric penalty for overly elaborate machinery. The fact that the

human brain burns about a fifth of humans’ total daily caloric intake is a

testament to the evolutionary advantages that our intellectual abilities provide us

with: the brain’s contributions must somehow more than pay for that sizable fuel

bill. On the other hand, we can also infer that a substantially more complex brain

probably didn’t provide sufficient dividends, evolutionarily speaking. We’re as

brainy as we have needed to be, but not extravagantly more so.

The same kind of process is also believed to play a role at the neural level. In

computer science, software models based on the brain, known as “artificial

neural networks,” can learn arbitrarily complex functions—they’re even more

flexible than our nine-factor model above—but precisely because of this very

flexibility they are notoriously vulnerable to overfitting. Actual, biological

neural networks sidestep some of this problem because they need to trade off

their performance against the costs of maintaining it. Neuroscientists have

suggested, for instance, that brains try to minimize the number of neurons that

are firing at any given moment—implementing the same kind of downward

pressure on complexity as the Lasso.

Language forms yet another natural Lasso: complexity is punished by the

labor of speaking at greater length and the taxing of our listener’s attention span.

Business plans get compressed to an elevator pitch; life advice becomes

proverbial wisdom only if it is sufficiently concise and catchy. And anything that

needs to be remembered has to pass through the inherent Lasso of memory.

The Upside of Heuristics

The economist Harry Markowitz won the 1990 Nobel Prize in Economics for

developing modern portfolio theory: his groundbreaking “mean-variance

portfolio optimization” showed how an investor could make an optimal

allocation among various funds and assets to maximize returns at a given level

of risk. So when it came time to invest his own retirement savings, it seems like

Markowitz should have been the one person perfectly equipped for the job. What

did he decide to do?

I should have computed the historical covariances of the asset classes and drawn an efficient frontier.

Instead, I visualized my grief if the stock market went way up and I wasn’t in it—or if it went way

down and I was completely in it. My intention was to minimize my future regret. So I split my

contributions fifty-fifty between bonds and equities.

Why in the world would he do that? The story of the Nobel Prize winner and

his investment strategy could be presented as an example of human irrationality:

faced with the complexity of real life, he abandoned the rational model and

followed a simple heuristic. But it’s precisely because of the complexity of real

life that a simple heuristic might in fact be the rational solution.

When it comes to portfolio management, it turns out that unless you’re highly

confident in the information you have about the markets, you may actually be

better off ignoring that information altogether. Applying Markowitz’s optimal

portfolio allocation scheme requires having good estimates of the statistical

properties of different investments. An error in those estimates can result in very

different asset allocations, potentially increasing risk. In contrast, splitting your

money evenly across stocks and bonds is not affected at all by what data you’ve

observed. This strategy doesn’t even try to fit itself to the historical performance

of those investment types—so there’s no way it can overfit.

Of course, just using a fifty-fifty split is not necessarily the complexity sweet

spot, but there’s something to be said for it. If you happen to know the expected

mean and expected variance of a set of investments, then use mean-variance

portfolio optimization—the optimal algorithm is optimal for a reason. But when

the odds of estimating them all correctly are low, and the weight that the model

puts on those untrustworthy quantities is high, then an alarm should be going off

in the decision-making process: it’s time to regularize.

Inspired by examples like Markowitz’s retirement savings, psychologists Gerd

Gigerenzer and Henry Brighton have argued that the decision-making shortcuts

people use in the real world are in many cases exactly the kind of thinking that

makes for good decisions. “In contrast to the widely held view that less

processing reduces accuracy,” they write, “the study of heuristics shows that less

information, computation, and time can in fact improve accuracy.” A heuristic

that favors simpler answers—with fewer factors, or less computation—offers

precisely these “less is more” effects.

Imposing penalties on the ultimate complexity of a model is not the only way

to alleviate overfitting, however. You can also nudge a model toward simplicity

by controlling the speed with which you allow it to adapt to incoming data. This

makes the study of overfitting an illuminating guide to our history—both as a

society and as a species.

The Weight of History

Every food a living rat has eaten has, necessarily, not killed it.

—SAMUEL REVUSKY AND ERWIN BEDARF, “ASSOCIATION OF ILLNESS WITH PRIOR INGESTION OF NOVEL FOODS”

The soy milk market in the United States more than quadrupled from the mid1990s to 2013. But by the end of 2013, according to news headlines, it already

seemed to be a thing of the past, a distant second place to almond milk. As food

and beverage researcher Larry Finkel told Bloomberg Businessweek: “Nuts are

trendy now. Soy sounds more like old-fashioned health food.” The Silk

company, famous for popularizing soy milk (as the name implies), reported in

late 2013 that its almond milk products had grown by more than 50% in the

previous quarter alone. Meanwhile, in other beverage news, the leading coconut

water brand, Vita Coco, reported in 2014 that its sales had doubled since 2011—

and had increased an astounding three-hundred-fold since 2004. As the New

York Times put it, “coconut water seems to have jumped from invisible to

unavoidable without a pause in the realm of the vaguely familiar.” Meanwhile,

the kale market grew by 40% in 2013 alone. The biggest purchaser of kale the

year before had been Pizza Hut, which put it in their salad bars—as decoration.

Some of the most fundamental domains of human life, such as the question of

what we should put in our bodies, seem curiously to be the ones most dominated

by short-lived fads. Part of what enables these fads to take the world by storm is

how quickly our culture can change. Information now flows through society

faster than ever before, while global supply chains enable consumers to rapidly

change their buying habits en masse (and marketing encourages them to do so).

If some particular study happens to suggest a health benefit from, say, star anise,

it can be all over the blogosphere within the week, on television the week after

that, and in seemingly every supermarket in six months, with dedicated star anise

cookbooks soon rolling off the presses. This breathtaking speed is both a

blessing and a curse.

In contrast, if we look at the way organisms—including humans—evolve, we

notice something intriguing: change happens slowly. This means that the

properties of modern-day organisms are shaped not only by their present

environments, but also by their history. For example, the oddly cross-wired

arrangement of our nervous system (the left side of our body controlled by the

right side of our brain and vice versa) reflects the evolutionary history of

vertebrates. This phenomenon, called “decussation,” is theorized to have arisen

at a point in evolution when early vertebrates’ bodies twisted 180 degrees with

respect to their heads; whereas the nerve cords of invertebrates such as lobsters

and earthworms run on the “belly” side of the animal, vertebrates have their

nerve cords along the spine instead.

The human ear offers another example. Viewed from a functional perspective,

it is a system for translating sound waves into electrical signals by way of

amplification via three bones: the malleus, incus, and stapes. This amplification

system is impressive—but the specifics of how it works have a lot to do with

historical constraints. Reptiles, it turns out, have just a single bone in their ear,

but additional bones in the jaw that mammals lack. Those jawbones were

apparently repurposed in the mammalian ear. So the exact form and

configuration of our ear anatomy reflects our evolutionary history at least as

much as it does the auditory problem being solved.

The concept of overfitting gives us a way of seeing the virtue in such

evolutionary baggage. Though crossed-over nerve fibers and repurposed

jawbones may seem like suboptimal arrangements, we don’t necessarily want

evolution to fully optimize an organism to every shift in its environmental niche

—or, at least, we should recognize that doing so would make it extremely

sensitive to further environmental changes. Having to make use of existing

materials, on the other hand, imposes a kind of useful restraint. It makes it harder

to induce drastic changes in the structure of organisms, harder to overfit. As a

species, being constrained by the past makes us less perfectly adjusted to the

present we know but helps keep us robust for the future we don’t.

A similar insight might help us resist the quick-moving fads of human society.

When it comes to culture, tradition plays the role of the evolutionary constraints.

A bit of conservatism, a certain bias in favor of history, can buffer us against the

boom-and-bust cycle of fads. That doesn’t mean we ought to ignore the latest

data either, of course. Jump toward the bandwagon, by all means—but not

necessarily on it.

In machine learning, the advantages of moving slowly emerge most

concretely in a regularization technique known as Early Stopping. When we

looked at the German marriage survey data at the beginning of the chapter, we

went straight to examining the best-fitted one-, two-, and nine-factor models. In

many situations, however, tuning the parameters to find the best possible fit for

given data is a process in and of itself. What happens if we stop that process

early and simply don’t allow a model the time to become too complex? Again,

what might seem at first blush like being halfhearted or unthorough emerges,

instead, as an important strategy in its own right.

Many prediction algorithms, for instance, start out by searching for the single

most important factor rather than jumping to a multi-factor model. Only after

finding that first factor do they look for the next most important factor to add to

the model, then the next, and so on. Their models can therefore be kept from

becoming overly complex simply by stopping the process short, before

overfitting has had a chance to creep in. A related approach to calculating

predictions considers one data point at a time, with the model tweaked to

account for each new point before more points are added; there, too, the

complexity of the model increases gradually, so stopping the process short can

help keep it from overfitting.

This kind of setup—where more time means more complexity—characterizes

a lot of human endeavors. Giving yourself more time to decide about something

does not necessarily mean that you’ll make a better decision. But it does

guarantee that you’ll end up considering more factors, more hypotheticals, more

pros and cons, and thus risk overfitting.

Tom had exactly this experience when he became a professor. His first

semester, teaching his first class ever, he spent a huge amount of time perfecting

his lectures—more than ten hours of preparation for every hour of class. His

second semester, teaching a different class, he wasn’t able to put in as much

time, and worried that it would be a disaster. But a strange thing happened: the

students liked the second class. In fact, they liked it more than the first one.

Those extra hours, it turned out, had been spent nailing down nitty-gritty details

that only confused the students, and wound up getting cut from the lectures the

next time Tom taught the class. The underlying issue, Tom eventually realized,

was that he’d been using his own taste and judgment as a kind of proxy metric

for his students’. This proxy metric worked reasonably well as an

approximation, but it wasn’t worth overfitting—which explained why spending

extra hours painstakingly “perfecting” all the slides had been counterproductive.

The effectiveness of regularization in all kinds of machine-learning tasks

suggests that we can make better decisions by deliberately thinking and doing

less. If the factors we come up with first are likely to be the most important ones,

then beyond a certain point thinking more about a problem is not only going to

be a waste of time and effort—it will lead us to worse solutions. Early Stopping

provides the foundation for a reasoned argument against reasoning, the thinking

person’s case against thought. But turning this into practical advice requires

answering one more question: when should we stop thinking?

When to Think Less

As with all issues involving overfitting, how early to stop depends on the gap

between what you can measure and what really matters. If you have all the facts,

they’re free of all error and uncertainty, and you can directly assess whatever is

important to you, then don’t stop early. Think long and hard: the complexity and

effort are appropriate.

But that’s almost never the case. If you have high uncertainty and limited

data, then do stop early by all means. If you don’t have a clear read on how your

work will be evaluated, and by whom, then it’s not worth the extra time to make

it perfect with respect to your own (or anyone else’s) idiosyncratic guess at what

perfection might be. The greater the uncertainty, the bigger the gap between

what you can measure and what matters, the more you should watch out for

overfitting—that is, the more you should prefer simplicity, and the earlier you

should stop.

When you’re truly in the dark, the best-laid plans will be the simplest. When

our expectations are uncertain and the data are noisy, the best bet is to paint with

a broad brush, to think in broad strokes. Sometimes literally. As entrepreneurs

Jason Fried and David Heinemeier Hansson explain, the further ahead they need

to brainstorm, the thicker the pen they use—a clever form of simplification by

stroke size:

When we start designing something, we sketch out ideas with a big, thick Sharpie marker, instead of

a ball-point pen. Why? Pen points are too fine. They’re too high-resolution. They encourage you to

worry about things that you shouldn’t worry about yet, like perfecting the shading or whether to use a

dotted or dashed line. You end up focusing on things that should still be out of focus.

A Sharpie makes it impossible to drill down that deep. You can only draw shapes, lines, and

boxes. That’s good. The big picture is all you should be worrying about in the beginning.

As McGill’s Henry Mintzberg puts it, “What would happen if we started from

the premise that we can’t measure what matters and go from there? Then instead

of measurement we’d have to use something very scary: it’s called judgment.”

The upshot of Early Stopping is that sometimes it’s not a matter of choosing

between being rational and going with our first instinct. Going with our first

instinct can be the rational solution. The more complex, unstable, and uncertain

the decision, the more rational an approach that is.

To return to Darwin, his problem of deciding whether to propose could

probably have been resolved based on just the first few pros and cons he

identified, with the subsequent ones adding to the time and anxiety expended on

the decision without necessarily aiding its resolution (and in all likelihood

impeding it). What seemed to make up his mind was the thought that “it is

intolerable to think of spending one’s whole life like a neuter bee, working,

working, & nothing after all.” Children and companionship—the very first

points he mentioned—were precisely those that ultimately swayed him in favor

of marriage. His book budget was a distraction.

Before we get too critical of Darwin, however, painting him as an inveterate

overthinker, it’s worth taking a second look at this page from his diary. Seeing it

in facsimile shows something fascinating. Darwin was no Franklin, adding

assorted considerations for days. Despite the seriousness with which he

approached this life-changing choice, Darwin made up his mind exactly when

his notes reached the bottom of the diary sheet. He was regularizing to the page.

This is reminiscent of both Early Stopping and the Lasso: anything that doesn’t

make the page doesn’t make the decision.

His mind made up to marry, Darwin immediately went on to overthink the

timing. “When? Soon or Late,” he wrote above another list of pros and cons,

considering everything from happiness to expenses to “awkwardness” to his

long-standing desire to travel in a hot air balloon and/or to Wales. But by the end

of the page he resolved to “Never mind, trust to chance”—and the result, within

several months’ time, was a proposal to Emma Wedgwood, the start of a

fulfilling partnership and a happy family life.

8 Relaxation

Let It Slide

In 2010 Meghan Bellows was working on her PhD in chemical engineering at

Princeton by day and planning her wedding by night. Her research revolved

around finding the right places to put amino acids in a protein chain to yield a

molecule with particular characteristics. (“If you maximize the binding energy of

two proteins then you can successfully design a peptidic inhibitor of some

biological function so you can actually inhibit a disease’s progress.”) On the

nuptial front, she was stuck on the problem of seating.

There was a group of nine college friends, and Bellows agonized over who

else to throw into the midst of such a mini-reunion to make a table of ten. Even

worse, she’d counted up eleven close relatives. Who would get the boot from the

honored parents’ table, and how could she explain it to them? And what about

folks like her childhood neighbors and babysitter, or her parents’ work

colleagues, who didn’t really know anyone at the wedding at all?

The problem seemed every bit as hard as the protein problem she was working

on at the lab. Then it hit her. It was the problem she was working on at the lab.

One evening, as she stared at her seating charts, “I realized that there was

literally a one-to-one correlation between the amino acids and proteins in my

PhD thesis and people sitting at tables at my wedding.” Bellows called out to her

fiancé for a piece of paper and began scribbling equations. Amino acids became

guests, binding energies became relationships, and the molecules’ so-called

nearest-neighbor interactions became—well—nearest-neighbor interactions. She

could use the algorithms from her research to solve her own wedding.

Bellows worked out a way to numerically define the strength of the

relationships among all the guests. If a particular pair of people didn’t know one

another they got a 0, if they did they got a 1, and if they were a couple they got a

50. (The sister of the bride got to give a score of 10 to all the people she wanted

to sit with, as a special prerogative.) Bellows then specified a few constraints:

the maximum table capacity, and a minimum score necessary for each table, so

that no one table became the awkward “miscellaneous” group full of strangers.

She also codified the program’s goal: to maximize the relationship scores

between the guests and their tablemates.

There were 107 people at the wedding and 11 tables, which could

accommodate ten people each. This means there were about 11

107 possible

seating plans: that’s a 112-digit number, more than 200 billion googols, a figure

that dwarfs the (merely 80-digit) number of atoms in the observable universe.

Bellows submitted the job to her lab computer on Saturday evening and let it

churn. When she came in on Monday morning, it was still running; she had it

spit out the best assignment it had found so far and put it back onto protein

design.

Even with a high-powered lab computer cluster and a full thirty-six hours of

processing time, there was no way for the program to evaluate more than a tiny

fraction of the potential seating arrangements. The odds are that the truly optimal

solution, the one that would have earned the very highest score, never came up

in its permutations. Still, Bellows was pleased with the computer’s results. “It

identified relationships that we were forgetting about,” she says, offering

delightful, unconventional possibilities that the human planners hadn’t even

considered. For instance, it proposed removing her parents from the family

table, putting them instead with old friends they hadn’t seen for years. Its final

recommendation was an arrangement that all parties agreed was a hit—although

the mother of the bride couldn’t resist making just a few manual tweaks.

The fact that all the computing power of a lab at Princeton couldn’t find the

perfect seating plan might seem surprising. In most of the domains we’ve

discussed so far, straightforward algorithms could guarantee optimal solutions.

But as computer scientists have discovered over the past few decades, there are

entire classes of problems where a perfect solution is essentially unreachable, no

matter how fast we make our computers or how cleverly we program them. In

fact, no one understands as well as a computer scientist that in the face of a

seemingly unmanageable challenge, you should neither toil forever nor give up,

but—as we’ll see—try a third thing entirely.

The Difficulty of Optimization

Before leading the country through the American Civil War, before drafting the

Emancipation Proclamation or delivering the Gettysburg Address, Abraham

Lincoln worked as a “prairie lawyer” in Springfield, Illinois, traveling the Eighth

Judicial Circuit twice a year for sixteen years. Being a circuit lawyer meant

literally making a circuit—moving through towns in fourteen different counties

to try cases, riding hundreds of miles over many weeks. Planning these circuits

raised a natural challenge: how to visit all the necessary towns while covering as

few miles as possible and without going to any town twice.

This is an instance of what’s known to mathematicians and computer

scientists as a “constrained optimization” problem: how to find the single best

arrangement of a set of variables, given particular rules and a scorekeeping

measure. In fact, it’s the most famous optimization problem of them all. If it had

been studied in the nineteenth century it might have become forever known as

“the prairie lawyer problem,” and if it had first come up in the twenty-first

century it might have been nicknamed “the delivery drone problem.” But like the

secretary problem, it emerged in the mid-twentieth century, a period

unmistakably evoked by its canonical name: “the traveling salesman problem.”

The problem of route planning didn’t get the attention of the mathematics

community until the 1930s, but then it did so with a vengeance. Mathematician

Karl Menger spoke of “the postal messenger problem” in 1930, noting that no

easier solution was known than simply trying out every possibility in turn.

Hassler Whitney posed the problem in a 1934 talk at Princeton, where it lodged

firmly in the brain of fellow mathematician Merrill Flood (who, you might recall

from chapter 1, is also credited with circulating the first solution to the secretary

problem). When Flood moved to California in the 1940s he spread it in turn to

his colleagues at the RAND Institute, and the problem’s iconic name first

appeared in print in a 1949 paper by mathematician Julia Robinson. As the

problem swept through mathematical circles, it grew in notoriety. Many of the

greatest minds of the time obsessed over it, and no one seemed able to make real

headway.

In the traveling salesman problem, the question isn’t whether a computer (or a

mathematician) could find the shortest route: theoretically, one can simply crank

out a list of all the possibilities and measure each one. Rather, the issue is that as

the number of towns grows, the list of possible routes connecting them explodes.

A route is just an ordering of the towns, so trying them all by brute force is the

dreaded O(n!) “factorial time”—the computational equivalent of sorting a deck

of cards by throwing them in the air until they happen to land in order.

The question is: is there any hope of doing better?

Decades of work did little to tame the traveling salesman problem. Flood, for

instance, wrote in 1956, more than twenty years after first encountering it: “It

seems very likely that quite a different approach from any yet used may be

required for successful treatment of the problem. In fact, there may well be no

general method for treating the problem and impossibility results would also be

valuable.” Another decade later, the mood was only more grim. “I conjecture,”

wrote Jack Edmonds, “that there is no good algorithm for the traveling salesman

problem.”

These words would prove prophetic.

Defining Difficulty

In the mid-1960s, Edmonds, at the National Institute of Standards and

Technology, along with Alan Cobham of IBM, developed a working definition

for what makes a problem feasible to solve or not. They asserted what’s now

known as the Cobham–Edmonds thesis: an algorithm should be considered

“efficient” if it runs in what’s called “polynomial time”—that is, O(n

2

), O(n

3

), or

in fact n to the power of any number at all. A problem, in turn, is considered

“tractable” if we know how to solve it using an efficient algorithm. A problem

we don’t know how to solve in polynomial time, on the other hand, is considered

“intractable.” And at anything but the smallest scales, intractable problems are

beyond the reach of solution by computers, no matter how powerful.*

This amounts to what is arguably the central insight of computer science. It’s

possible to quantify the difficulty of a problem. And some problems are just …

hard.

Where does this leave the traveling salesman problem? Curiously enough, we

are still not quite sure. In 1972, Berkeley’s Richard Karp demonstrated that the

traveling salesman problem is linked to a controversially borderline class of

problems that have not yet been definitively proven to be either efficiently

solvable or not. But so far there have been no efficient solutions found for any of

those problems—making them effectively intractable—and most computer

scientists believe that there aren’t any to be found. So the “impossibility result”

for the traveling salesman problem that Flood imagined in the 1950s is likely to

be its ultimate fate. What’s more, many other optimization problems—with

implications for everything from political strategy to public health to fire safety

—are similarly intractable.

But for the computer scientists who wrestle with such problems, this verdict

isn’t the end of the story. Instead, it’s more like a call to arms. Having

determined a problem to be intractable, you can’t just throw up your hands. As

scheduling expert Jan Karel Lenstra told us, “When the problem is hard, it

doesn’t mean that you can forget about it, it means that it’s just in a different

status. It’s a serious enemy, but you still have to fight it.” And this is where the

field figured out something invaluable, something we can all learn from: how to

best approach problems whose optimal answers are out of reach. How to relax.

Just Relax

The perfect is the enemy of the good.

—VOLTAIRE

When somebody tells you to relax, it’s probably because you’re uptight—

making a bigger deal of things than you should. When computer scientists are up

against a formidable challenge, their minds also turn to relaxation, as they pass

around books like An Introduction to Relaxation Methods or Discrete Relaxation

Techniques. But they don’t relax themselves; they relax the problem.

One of the simplest forms of relaxation in computer science is known as

Constraint Relaxation. In this technique, researchers remove some of the

problem’s constraints and set about solving the problem they wish they had.

Then, after they’ve made a certain amount of headway, they try to add the

constraints back in. That is, they make the problem temporarily easier to handle

before bringing it back to reality.

For instance, you can relax the traveling salesman problem by letting the

salesman visit the same town more than once, and letting him retrace his steps

for free. Finding the shortest route under these looser rules produces what’s

called the “minimum spanning tree.” (If you prefer, you can also think of the

minimum spanning tree as the fewest miles of road needed to connect every

town to at least one other town. The shortest traveling salesman route and the

minimum spanning tree for Lincoln’s judicial circuit are shown below.) As it

turns out, solving this looser problem takes a computer essentially no time at all.

And while the minimum spanning tree doesn’t necessarily lead straight to the

solution of the real problem, it is quite useful all the same. For one thing, the

spanning tree, with its free backtracking, will never be any longer than the real

solution, which has to follow all the rules. Therefore, we can use the relaxed

problem—the fantasy—as a lower bound on the reality. If we calculate that the

spanning tree distance for a particular set of towns is 100 miles, we can be sure

the traveling salesman distance will be no less than that. And if we find, say, a

110-mile route, we can be certain it is at most 10% longer than the best solution.

Thus we can get a grasp of how close we are to the real answer even without

knowing what it is.

Figure 8.1 The shortest traveling salesman route (top) and minimum spanning tree (bottom) for Lincoln’s

1855 judicial circuit.

Even better, in the traveling salesman problem it turns out that the minimum

spanning tree is actually one of the best starting points from which to begin a

search for the real solution. Approaches like these have allowed even one of the

largest traveling salesman problems imaginable—finding the shortest route that

visits every single town on Earth—to be solved to within less than 0.05% of the

(unknown) optimal solution.

Though most of us haven’t encountered the formal algorithmic version of

Constraint Relaxation, its basic message is familiar to almost anyone who’s

dreamed big about life questions. What would you do if you weren’t afraid?

reads a mantra you might have seen in a guidance counselor’s office or heard at

a motivational seminar. What would you do if you could not fail? Similarly,

when considering questions of profession or career, we ask questions like What

would you do if you won the lottery? or, taking a different tack, What would you

do if all jobs paid the same? The idea behind such thought exercises is exactly

that of Constraint Relaxation: to make the intractable tractable, to make progress

in an idealized world that can be ported back to the real one. If you can’t solve

the problem in front of you, solve an easier version of it—and then see if that

solution offers you a starting point, or a beacon, in the full-blown problem.

Maybe it does.

What relaxation cannot do is offer you a guaranteed shortcut to the perfect

answer. But computer science can also quantify the tradeoff that relaxation

offers between time and solution quality. In many cases, the ratio is dramatic, a

no-brainer—for instance, an answer at least half as good as the perfect solution

in a quadrillionth of the time. The message is simple but profound: if we’re

willing to accept solutions that are close enough, then even some of the hairiest

problems around can be tamed with the right techniques.

Temporarily removing constraints, as in the minimum spanning tree and the

“what if you won the lottery?” examples, is the most straightforward form of

algorithmic relaxation. But there are also two other, subtler types of relaxation

that repeatedly show up in optimization research. They have proven instrumental

in solving some of the field’s most important intractable problems, with direct

real-world implications for everything from city planning and disease control to

the cultivation of athletic rivalries.

Uncountably Many Shades of Gray: Continuous

Relaxation

The traveling salesman problem, like Meghan Bellows’s search for the best

seating arrangement, is a particular kind of optimization problem known as

“discrete optimization”—that is, there’s no smooth continuum among its

solutions. The salesman goes either to this town or to that one; you’re either at

table five or at table six. There are no shades of gray in between.

Such discrete optimization problems are all around us. In cities, for example,

planners try to place fire trucks so that every house can be reached within a fixed

amount of time—say, five minutes. Mathematically, each fire truck “covers”

whatever houses can be reached within five minutes from its location. The

challenge is finding the minimal set of locations such that all houses are covered.

“The whole [fire and emergency] profession has just adopted this coverage

model, and it’s great,” says University of Wisconsin–Madison’s Laura Albert

McLay. “It’s a nice, clear thing we can model.” But since a fire truck either

exists at a location or it doesn’t, calculating that minimal set involves discrete

optimization. And as McLay notes, “that’s where a lot of problems become

computationally hard, when you can’t do half of this and half of that.”

The challenge of discrete optimization shows up in social settings, too.

Imagine you wanted to throw a party for all your friends and acquaintances, but

didn’t want to pay for all the envelopes and stamps that so many invitations

would entail. You could instead decide to mail invitations to a few wellconnected friends, and tell them to “bring everyone we know.” What you’d

ideally want to find, then, is the smallest subgroup of your friends that knows all

the rest of your social circle—which would let you lick the fewest envelopes and

still get everyone to attend. Granted, this might sound like a lot of work just to

save a few bucks on stamps, but it’s exactly the kind of problem that political

campaign managers and corporate marketers want to solve to spread their

message most effectively. And it’s also the problem that epidemiologists study

in thinking about, say, the minimum number of people in a population—and

which people—to vaccinate to protect a society from communicable diseases.

As we noted, discrete optimization’s commitment to whole numbers—a fire

department can have one engine in the garage, or two, or three, but not two and a

half fire trucks, or π of them—is what makes discrete optimization problems so

hard to solve. In fact, both the fire truck problem and the party invitation

problem are intractable: no general efficient solution for them exists. But, as it

turns out, there do exist a number of efficient strategies for solving the

continuous versions of these problems, where any fraction or decimal is a

possible solution. Researchers confronted with a discrete optimization problem

might gaze at those strategies enviously—but they also can do more than that.

They can try to relax their discrete problem into a continuous one and see what

happens.

In the case of the invitation problem, relaxing it from discrete to continuous

optimization means that a solution may tell us to send someone a quarter of an

invitation, and someone else two-thirds of one. What does that even mean? It

obviously can’t be the answer to the original question, but, like the minimum

spanning tree, it does give us a place to start. With the relaxed solution in hand,

we can decide how to translate those fractions back into reality. We could, for

example, choose to simply round them as necessary, sending invitations to

everyone who got “half an invitation” or more in the relaxed scenario. We could

also interpret these fractions as probabilities—for instance, flipping a coin for

every location where the relaxed solution tells us to put half a fire truck, and

actually placing a truck there only if it lands heads. In either case, with these

fractions turned back to whole numbers, we’ll have a solution that makes sense

in the context of our original, discrete problem.

The final step, as with any relaxation, is to ask how good this solution is

compared to the actual best solution we might have come up with by

exhaustively checking every single possible answer to the original problem. It

turns out that for the invitations problem, Continuous Relaxation with rounding

will give us an easily computed solution that’s not half bad: it’s mathematically

guaranteed to get everyone you want to the party while sending out at most twice

as many invitations as the best solution obtainable by brute force. Similarly, in

the fire truck problem, Continuous Relaxation with probabilities can quickly get

us within a comfortable bound of the optimal answer.

Continuous Relaxation is not a magic bullet: it still doesn’t give us an efficient

way to get to the truly optimal answers, only to their approximations. But

delivering twice as many mailings or inoculations as optimal is still far better

than the unoptimized alternatives.

Just a Speeding Ticket: Lagrangian Relaxation

Vizzini: Inconceivable!

Inigo Montoya: You keep using that word. I do not think it means what you

think it means.

—THE PRINCESS BRIDE

One day as a child, Brian was complaining to his mother about all the things he

had to do: his homework, his chores.… “Technically, you don’t have to do

anything,” his mother replied. “You don’t have to do what your teachers tell you.

You don’t have to do what I tell you. You don’t even have to obey the law.

There are consequences to everything, and you get to decide whether you want

to face those consequences.”

Brian’s kid-mind was blown. It was a powerful message, an awakening of a

sense of agency, responsibility, moral judgment. It was something else, too: a

powerful computational technique called Lagrangian Relaxation. The idea

behind Lagrangian Relaxation is simple. An optimization problem has two parts:

the rules and the scorekeeping. In Lagrangian Relaxation, we take some of the

problem’s constraints and bake them into the scoring system instead. That is, we

take the impossible and downgrade it to costly. (In a wedding seating

optimization, for instance, we might relax the constraint that tables each hold ten

people max, allowing overfull tables but with some kind of elbow-room

penalty.) When an optimization problem’s constraints say “Do it, or else!,”

Lagrangian Relaxation replies, “Or else what?” Once we can color outside the

lines—even just a little bit, and even at a steep cost—problems become tractable

that weren’t tractable before.

Lagrangian Relaxations are a huge part of the theoretical literature on the

traveling salesman problem and other hard problems in computer science.

They’re also a critical tool for a number of practical applications. For instance,

recall Carnegie Mellon’s Michael Trick, who, as we mentioned in chapter 3, is in

charge of scheduling for Major League Baseball and a number of NCAA

conferences. What we hadn’t mentioned is how he does it. The composition of

each year’s schedule is a giant discrete optimization problem, much too complex

for any computer to solve by brute force. So each year Trick and his colleagues

at the Sports Scheduling Group turn to Lagrangian Relaxation to get the job

done. Every time you turn on the television or take a seat in a stadium, know that

the meeting of those teams on that court on that particular night … well, it’s not

necessarily the optimum matchup. But it’s close. And for that we have not only

Michael Trick but eighteenth-century French mathematician Joseph-Louis

Lagrange to thank.

In scheduling a sports season, Trick finds that the Continuous Relaxation we

described above doesn’t necessarily make his life any easier. “If you end up with

fractional games, you just don’t get anything useful.” It’s one thing to end up

with fractional allocations of party invitations or fire trucks, where the numbers

can always be rounded up if necessary. But in sports, the integer constraints—on

how many teams play a game, how many games are played in sum, and how

many times each team plays every other team—are just too strong. “And so we

cannot relax in that direction. We really have got to keep the fundamental

[discrete] part of the model.”

Nonetheless, something has to be done to reckon with the sheer complexity of

the problem. So “we have to work with the leagues to relax some of the

constraints they might like to have,” Trick explains. The number of such

constraints that go into scheduling a sports season is immense, and it includes

not only the requirements arising from the league’s basic structure but also all

sorts of idiosyncratic requests and qualms. Some leagues are fine with the

second half of the season mirroring the first, just with home and away games

reversed; other leagues don’t want that structure, but nonetheless demand that no

teams meet for a second time until they’ve already met every other team once.

Some leagues insist on making the most famous rivalries happen in the final

game of the season. Some teams cannot play home games on certain dates due to

conflicting events at their arenas. In the case of NCAA basketball, Trick also has

to consider further constraints coming from the television networks that

broadcast the games. Television channels define a year in advance what they

anticipate “A games” and “B games” to be—the games that will attract the

biggest audience. (Duke vs. UNC is a perennial A game, for instance.) The

channels then expect one A game and one B game each week—but never two A

games at the same time, lest it split the viewership.

Unsurprisingly, given all these demands, Trick has found that computing a

sports schedule often becomes possible only by softening some of these hard

constraints.

Generally, when people first come to us with a sports schedule, they will claim … “We never do x

and we never do y.” Then we look at their schedules and we say, “Well, twice you did x and three

times you did y last year.” Then “Oh, yeah, well, okay. Then other than that we never do it.” And

then we go back the year before.… We generally realize that there are some things they think they

never do that people do do. People in baseball believe that the Yankees and the Mets are never at

home at the same time. And it’s not true. It’s never been true. They are at home perhaps three games,

perhaps six games in a year at the same day. But in the broad season, eighty-one games at home for

each of the teams, it’s relatively rare, people forget about them.

Occasionally it takes a bit of diplomatic finesse, but a Lagrangian Relaxation

—where some impossibilities are downgraded to penalties, the inconceivable to

the undesirable—enables progress to be made. As Trick says, rather than

spending eons searching for an unattainable perfect answer, using Lagrangian

Relaxation allows him to ask questions like, “How close can you get?” Close

enough, it turns out, to make everyone happy—the league, the schools, the

networks—and to stoke the flames of March Madness, year after year.

Learning to Relax

Of the various ways that computational questions present themselves to us,

optimization problems—one part goals, one part rules—are arguably the most

common. And discrete optimization problems, where our options are stark

either/or choices, with no middle ground, are the most typical of those. Here,

computer science hands down a disheartening verdict. Many discrete

optimization problems are truly hard. The field’s brightest minds have come up

empty in every attempt to find an easy path to perfect answers, and in fact are

increasingly more devoted to proving that such paths don’t exist than to

searching for them.

If nothing else, this should offer us some consolation. If we’re up against a

problem that seems gnarly, thorny, impassable—well, we might be right. And

having a computer won’t necessarily help.

At least, not unless we can learn to relax.

There are many ways to relax a problem, and we’ve seen three of the most

important. The first, Constraint Relaxation, simply removes some constraints

altogether and makes progress on a looser form of the problem before coming

back to reality. The second, Continuous Relaxation, turns discrete or binary

choices into continua: when deciding between iced tea and lemonade, first

imagine a 50–50 “Arnold Palmer” blend and then round it up or down. The third,

Lagrangian Relaxation, turns impossibilities into mere penalties, teaching the art

of bending the rules (or breaking them and accepting the consequences). A rock

band deciding which songs to cram into a limited set, for instance, is up against

what computer scientists call the “knapsack problem”—a puzzle that asks one to

decide which of a set of items of different bulk and importance to pack into a

confined volume. In its strict formulation the knapsack problem is famously

intractable, but that needn’t discourage our relaxed rock stars. As demonstrated

in several celebrated examples, sometimes it’s better to simply play a bit past the

city curfew and incur the related fines than to limit the show to the available slot.

In fact, even when you don’t commit the infraction, simply imagining it can be

illuminating.

The conservative British columnist Christopher Booker says that “when we

embark on a course of action which is unconsciously driven by wishful thinking,

all may seem to go well for a time”—but that because “this make-believe can

never be reconciled with reality,” it will inevitably lead to what he describes as a

multi-stage breakdown: “dream,” “frustration,” “nightmare,” “explosion.”

Computer science paints a dramatically rosier view. Then again, as an

optimization technique, relaxation is all about being consciously driven by

wishful thinking. Perhaps that’s partly what makes the difference.

Relaxations offer us a number of advantages. For one, they offer a bound on

the quality of the true solution. If we’re trying to pack our calendar, imagining

that we can magically teleport across town will instantaneously make it clear that

eight one-hour meetings is the most we could possibly expect to fit into a day;

such a bound might be useful in setting expectations before we face the full

problem. Second, relaxations are designed so that they can indeed be reconciled

with reality—and this gives us bounds on the solution from the other direction.

When Continuous Relaxation tells us to give out fractional vaccines, we can just

immunize everyone assigned to receive half a vaccine or more, and end up with

an easily calculated solution that requires at worst twice as many inoculations as

in a perfect world. Maybe we can live with that.

Unless we’re willing to spend eons striving for perfection every time we

encounter a hitch, hard problems demand that instead of spinning our tires we

imagine easier versions and tackle those first. When applied correctly, this is not

just wishful thinking, not fantasy or idle daydreaming. It’s one of our best ways

of making progress.

9 Randomness

When to Leave It to Chance

I must admit that after many years of work in this area, the efficacy of

randomness for so many algorithmic problems is absolutely mysterious to

me. It is efficient, it works; but why and how is absolutely mysterious.

—MICHAEL RABIN

Randomness seems like the opposite of reason—a form of giving up on a

problem, a last resort. Far from it. The surprising and increasingly important role

of randomness in computer science shows us that making use of chance can be a

deliberate and effective part of approaching the hardest sets of problems. In fact,

there are times when nothing else will do.

In contrast to the standard “deterministic” algorithms we typically imagine

computers using, where one step follows from another in exactly the same way

every time, a randomized algorithm uses randomly generated numbers to solve a

problem. Recent work in computer science has shown that there are cases where

randomized algorithms can produce good approximate answers to difficult

questions faster than all known deterministic algorithms. And while they do not

always guarantee the optimal solutions, randomized algorithms can get

surprisingly close to them in a fraction of the time, just by strategically flipping a

few coins while their deterministic cousins sweat it out.

There is a deep message in the fact that on certain problems, randomized

approaches can outperform even the best deterministic ones. Sometimes the best

solution to a problem is to turn to chance rather than trying to fully reason out an

answer.

But merely knowing that randomness can be helpful isn’t good enough. You

need to know when to rely on chance, in what way, and to what extent. The

recent history of computer science provides some answers—though the story

begins a couple of centuries earlier.

Sampling

In 1777, George-Louis Leclerc, Comte de Buffon, published the results of an

interesting probabilistic analysis. If we drop a needle onto a lined piece of paper,

he asked, how likely is it to cross one of the lines? Buffon’s work showed that if

the needle is shorter than the gap between the blines, the answer is

2

⁄π

times the

needle’s length divided by the length of the gap. For Buffon, deriving this

formula was enough. But in 1812, Pierre-Simon Laplace, one of the heroes of

chapter 6, pointed out that this result has another implication: one could estimate

the value of π simply by dropping needles onto paper.

Laplace’s proposal pointed to a profound general truth: when we want to

know something about a complex quantity, we can estimate its value by

sampling from it. This is exactly the kind of calculation that his work on Bayes’s

Rule helps us to perform. In fact, following Laplace’s suggestion, several people

have carried out exactly the experiment he suggested, confirming that it is

possible—although not particularly efficient—to estimate the value of π in this

hands-on way.*

Throwing thousands of needles onto lined paper makes for an interesting

pastime (for some), but it took the development of the computer to make

sampling into a practical method. Before, when mathematicians and physicists

tried using randomness to solve problems, their calculations had to be

laboriously worked out by hand, so it was hard to generate enough samples to

yield accurate results. Computers—in particular, the computer developed in Los

Alamos during World War II—made all the difference.

Stanislaw “Stan” Ulam was one of the mathematicians who helped develop

the atomic bomb. Having grown up in Poland, he moved to the United States in

1939, and joined the Manhattan Project in 1943. After a brief return to academia

he was back at Los Alamos in 1946, working on the design of thermonuclear

weapons. But he was also sick—he had contracted encephalitis, and had

emergency brain surgery. And as he recovered from his illness he worried about

whether he would regain his mathematical abilities.

While convalescing, Ulam played a lot of cards, particularly solitaire (a.k.a.

Klondike). As any solitaire player knows, some shuffles of the deck produce

games that just can’t be won. So as Ulam played, he asked himself a natural

question: what is the probability that a shuffled deck will yield a winnable

game?

In a game like solitaire, reasoning your way through the space of possibilities

gets almost instantly overwhelming. Flip over the first card, and there are fifty-

two possible games to keep track of; flip over the second, and there are fifty-one

possibilities for each first card. That means we’re already up into thousands of

possible games before we’ve even begun to play. F. Scott Fitzgerald once wrote

that “the test of a first-rate intelligence is the ability to hold two opposing ideas

in mind at the same time and still retain the ability to function.” That may be

true, but no first-rate intelligence, human or otherwise, can possibly hold the

eighty unvigintillion possible shuffled-deck orders in mind and have any hope of

functioning.

After trying some elaborate combinatorial calculations of this sort and giving

up, Ulam landed on a different approach, beautiful in its simplicity: just play the

game.

I noticed that it may be much more practical to [try] … laying down the cards, or experimenting with

the process and merely noticing what proportion comes out successfully, rather than to try to

compute all the combinatorial possibilities which are an exponentially increasing number so great

that, except in very elementary cases, there is no way to estimate it. This is intellectually surprising,

and if not exactly humiliating, it gives one a feeling of modesty about the limits of rational or

traditional thinking. In a sufficiently complicated problem, actual sampling is better than an

examination of all the chains of possibilities.

When he says “better,” note that he doesn’t necessarily mean that sampling will

offer you more precise answers than exhaustive analysis: there will always be

some error associated with a sampling process, though you can reduce it by

ensuring your samples are indeed random and by taking more and more of them.

What he means is that sampling is better because it gives you an answer at all, in

cases where nothing else will.

Ulam’s insight—that sampling can succeed where analysis fails—was also

crucial to solving some of the difficult nuclear physics problems that arose at

Los Alamos. A nuclear reaction is a branching process, where possibilities

multiply just as wildly as they do in cards: one particle splits in two, each of

which may go on to strike others, causing them to split in turn, and so on.

Exactly calculating the chances of some particular outcome of that process, with

many, many particles interacting, is hard to the point of impossibility. But

simulating it, with each interaction being like turning over a new card, provides

an alternative.

Ulam developed the idea further with John von Neumann, and worked with

Nicholas Metropolis, another of the physicists from the Manhattan Project, on

implementing the method on the Los Alamos computer. Metropolis named this

approach—replacing exhaustive probability calculations with sample

simulations—the Monte Carlo Method, after the Monte Carlo casino in

Monaco, a place equally dependent on the vagaries of chance. The Los Alamos

team was able to use it to solve key problems in nuclear physics. Today the

Monte Carlo Method is one of the cornerstones of scientific computing.

Many of these problems, like calculating the interactions of subatomic

particles or the chances of winning at solitaire, are themselves intrinsically

probabilistic, so solving them through a randomized approach like Monte Carlo

makes a fair bit of sense. But perhaps the most surprising realization about the

power of randomness is that it can be used in situations where chance seemingly

plays no role at all. Even if you want the answer to a question that is strictly yes

or no, true or false—no probabilities about it—rolling a few dice may still be

part of the solution.

Randomized Algorithms

The first person to demonstrate the surprisingly broad applications of

randomness in computer science was Michael Rabin. Born in 1931 in Breslau,

Germany (which became Wrocław, Poland, at the end of World War II), Rabin

was the descendant of a long line of rabbis. His family left Germany for

Palestine in 1935, and there he was diverted from the rabbinical path his father

had laid down for him by the beauty of mathematics—discovering Alan Turing’s

work early in his undergraduate career at the Hebrew University and

immigrating to the United States to begin a PhD at Princeton. Rabin would go on

to win the Turing Award—the computer science equivalent of a Nobel—for

extending theoretical computer science to accommodate “nondeterministic”

cases, where a machine isn’t forced to pursue a single option but has multiple

paths it might follow. On sabbatical in 1975, Rabin came to MIT, searching for a

new research direction to pursue.

He found it in one of the oldest problems of them all: how to identify prime

numbers.

Algorithms for finding prime numbers date back at least as far as ancient

Greece, where mathematicians used a straightforward approach known as the

Sieve of Erastothenes. The Sieve of Erastothenes works as follows: To find all

the primes less than n, begin by writing down all the numbers from 1 to n in

sequence. Then cross out all the numbers that are multiples of 2, besides itself

(4, 6, 8, 10, 12, and so on). Take the next smallest number that hasn’t been

crossed out (in this case, 3), and cross out all multiples of that number (6, 9, 12,

15). Keep going like this, and the numbers that remain at the end are the primes.

For millennia, the study of prime numbers was believed to be, as G. H. Hardy

put it, “one of the most obviously useless branches” of mathematics. But it

lurched into practicality in the twentieth century, becoming pivotal in

cryptography and online security. As it happens, it is much easier to multiply

primes together than to factor them back out. With big enough primes—say, a

thousand digits—the multiplication can be done in a fraction of a second while

the factoring could take literally millions of years; this makes for what is known

as a “one-way function.” In modern encryption, for instance, secret primes

known only to the sender and recipient get multiplied together to create huge

composite numbers that can be transmitted publicly without fear, since factoring

the product would take any eavesdropper way too long to be worth attempting.

Thus virtually all secure communication online—be it commerce, banking, or

email—begins with a hunt for prime numbers.

This cryptographic application suddenly made algorithms for finding and

checking primes incredibly important. And while the Sieve of Erastothenes is

effective, it is not efficient. If you want to check whether a particular number is

prime—known as testing its “primality”—then following the sieve strategy

requires trying to divide it by all the primes up to its square root.* Checking

whether a six-digit number is prime would require dividing by all of the 168

primes less than 1,000—not so bad. But checking a twelve-digit number

involves dividing by the 78,498 primes less than 1 million, and all that division

quickly starts to get out of control. The primes used in modern cryptography are

hundreds of digits long; forget about it.

At MIT, Rabin ran into Gary Miller, a recent graduate from the computer

science department at Berkeley. In his PhD thesis, Miller had developed an

intriguingly promising, much faster algorithm for testing primality—but there

was one small problem: it didn’t always work.

Miller had found a set of equations (expressed in terms of two numbers, n and

x) that are always true if n is prime, regardless of what values you plug in for x.

If they come out false even for a single value of x, then there’s no way n can be

prime—in these cases, x is called a “witness” against primality. The problem,

though, is false positives: even when n is not prime, the equations will still come

out true some of the time. This seemed to leave Miller’s approach hanging.

Rabin realized that this was a place where a step outside the usually

deterministic world of computer science might be valuable. If the number n is

actually nonprime, how many possible values of x would give a false positive

and declare it a prime number? The answer, Rabin showed, is no more than onequarter. So for a random value of x, if Miller’s equations come out true, there’s

only a one-in-four chance that n isn’t actually prime. And crucially, each time

we sample a new random x and Miller’s equations check out, the probability that

n only seems prime, but isn’t really, drops by another multiple of four. Repeat

the procedure ten times, and the probability of a false positive is one in four to

the tenth power—less than one in a million. Still not enough certainty? Check

another five times and you’re down to one in a billion.

Vaughan Pratt, another computer scientist at MIT, implemented Rabin’s

algorithm and started getting results late one winter night, while Rabin was at

home having friends over for a Hanukkah party. Rabin remembers getting a call

around midnight:

“Michael, this is Vaughan. I’m getting the output from these experiments. Take a pencil and paper

and write this down.” And so he had that 2

400−593 is prime. Denote the product of all primes p

smaller than 300 by k. The numbers k × 338 + 821 and k × 338 + 823 are twin primes.* These

constituted the largest twin primes known at the time. My hair stood on end. It was incredible. It was

just incredible.

The Miller-Rabin primality test, as it’s now known, provides a way to quickly

identify even gigantic prime numbers with an arbitrary degree of certainty.

Here we might ask a philosophical question—about what the meaning of “is”

is. We’re so used to mathematics being a realm of certainty that it’s jarring to

think that a number could be “probably prime” or “almost definitely prime.”

How certain is certain enough? In practice, modern cryptographic systems, the

ones that encrypt Internet connections and digital transactions, are tuned for a

false positive rate of less than one in a million billion billion. In other words,

that’s a decimal that begins with twenty-four zeros—less than one false prime

for the number of grains of sand on Earth. This standard comes after a mere forty

applications of the Miller-Rabin test. It’s true that you are never fully certain—

but you can get awfully close, awfully quick.

Though you may have never heard of the Miller-Rabin test, your laptop,

tablet, and phone know it well. Several decades after its discovery, it is still the

standard method used to find and check primes in many domains. It’s working

behind the scenes whenever you use your credit card online, and almost any time

secure communications are sent through the air or over wires.

For decades after Miller and Rabin’s work, it wasn’t known whether there

would ever be an efficient algorithm that allows testing primality in

deterministic fashion, with absolute certainty. In 2002, one such method did get

discovered by Manindra Agrawal, Neeraj Kayal, and Nitin Saxena at the Indian

Institute of Technology—but randomized algorithms like Miller-Rabin are much

faster and thus are still the ones used in practice today.

And for some other problems, randomness still provides the only known route

to efficient solutions. One curious example from mathematics is known as

“polynomial identity testing.” If you have two polynomial expressions, such as

2x

3 + 13x

2 + 22x + 8 and (2x + 1) × (x + 2) × (x + 4), working out whether those

expressions are in fact the same function—by doing all the multiplication, then

comparing the results—can be incredibly time-consuming, especially as the

number of variables increases.

Here again randomness offers a way forward: just generate some random xs

and plug them in. If the two expressions are not the same, it would be a big

coincidence if they gave the same answer for some randomly generated input.

And an even bigger coincidence if they also gave identical answers for a second

random input. And a bigger coincidence still if they did it for three random

inputs in a row. Since there is no known deterministic algorithm for efficiently

testing polynomial identity, this randomized method—with multiple

observations quickly giving rise to near-certainty—is the only practical one we

have.

In Praise of Sampling

The polynomial identity test shows that sometimes our effort is better spent

checking random values—sampling from the two expressions we want to know

about—than trying to untangle their inner workings. To some extent this seems

reasonably intuitive. Given a pair of nondescript gadgets and asked whether they

are two different devices or two copies of the same one, most of us would start

pushing random buttons rather than crack open the cases to examine the wiring.

And we aren’t particularly surprised when, say, a television drug lord knifes

open a few bundles at random to be reasonably certain about the quality of the

entire shipment.

There are cases, though, where we don’t turn to randomness—and maybe we

should.

Arguably the most important political philosopher of the twentieth century

was Harvard’s John Rawls, who set for himself the ambitious task of reconciling

two seemingly opposite key ideas in his field: liberty and equality. Is a society

more “just” when it’s more free, or more equal? And do the two really have to

be mutually exclusive? Rawls offered a way of approaching this set of questions

that he called the “veil of ignorance.” Imagine, he said, that you were about to be

born, but didn’t know as whom: male or female, rich or poor, urban or rural, sick

or healthy. And before learning your status, you had to choose what kind of

society you’d live in. What would you want? By evaluating various social

arrangements from behind the veil of ignorance, argued Rawls, we’d more

readily come to a consensus about what an ideal one would look like.

What Rawls’s thought experiment does not take into account, however, is the

computational cost of making sense of a society from behind such a veil. How

could we, in this hypothetical scenario, possibly hope to hold all of the relevant

information in our heads? Set aside grand questions of justice and fairness for a

moment and try to apply Rawls’s approach merely to, say, a proposed change in

health insurance regulations. Take the probability of being born, perhaps, as

someone who grows up to become a town clerk in the Midwest; multiply that by

the distribution of the different health care plans available to government

employees across various midwestern municipalities; multiply that by actuarial

data that offer the probability of, for instance, a fractured tibia; multiply that by

the average medical bill for the average procedure for a fractured tibia at a

midwestern hospital given the distribution of possible insurance plans.… Okay,

so would the proposed insurance revision be “good” or “bad” for the nation? We

can barely hope to evaluate a single injured shin this way, let alone the lives of

hundreds of millions.

Rawls’s philosophical critics have argued at length about how exactly we are

supposed to leverage the information obtained from the veil of ignorance.

Should we be trying, for instance, to maximize mean happiness, median

happiness, total happiness, or something else? Each of these approaches,

famously, leaves itself open to pernicious dystopias—such as the civilization of

Omelas imagined by writer Ursula K. Le Guin, in which prosperity and harmony

abound but a single child is forced to live in abject misery. These are worthy

critiques, and Rawls deliberately sidesteps them by leaving open the question of

what to do with the information we get from behind the veil. Perhaps the bigger

question, though, is how to gather that information in the first place.

The answer may well come from computer science. MIT’s Scott Aaronson

says he’s surprised that computer scientists haven’t yet had more influence on

philosophy. Part of the reason, he suspects, is just their “failure to communicate

what they can add to philosophy’s conceptual arsenal.” He elaborates:

One might think that, once we know something is computable, whether it takes 10 seconds or 20

seconds to compute is obviously the concern of engineers rather than philosophers. But that

conclusion would not be so obvious, if the question were one of 10 seconds versus 10

1010 seconds!

And indeed, in complexity theory, the quantitative gaps we care about are usually so vast that one has

to consider them qualitative gaps as well. Think, for example, of the difference between reading a

400-page book and reading every possible such book, or between writing down a thousand-digit

number and counting to that number.

Computer science gives us a way to articulate the complexity of evaluating all

possible social provisions for something like an injured shin. But fortunately, it

also provides tools for dealing with that complexity. And the sampling-based

Monte Carlo algorithms are some of the most useful approaches in that toolbox.

When we need to make sense of, say, national health care reform—a vast

apparatus too complex to be readily understood—our political leaders typically

offer us two things: cherry-picked personal anecdotes and aggregate summary

statistics. The anecdotes, of course, are rich and vivid, but they’re

unrepresentative. Almost any piece of legislation, no matter how enlightened or

misguided, will leave someone better off and someone worse off, so carefully

selected stories don’t offer any perspective on broader patterns. Aggregate

statistics, on the other hand, are the reverse: comprehensive but thin. We might

learn, for instance, whether average premiums fell nationwide, but not how that

change works out on a more granular level: they might go down for most but,

Omelas-style, leave some specific group—undergraduates, or Alaskans, or

pregnant women—in dire straits. A statistic can only tell us part of the story,

obscuring any underlying heterogeneity. And often we don’t even know which

statistic we need.

Since neither sweeping statistics nor politicians’ favorite stories can truly

guide us through thousands of pages of proposed legislation, a Monte Carlo–

informed computer scientist would propose a different approach: sampling. A