Why Is Recursion Hard to Comprehend? An Experiment with Experienced Programmers in Python

Recursion has the reputation of being hard to teach and understand. Our goal is to identify precisely what it is about recursion that makes it hard, and use this to devise a systematic teaching plan. We first make a distinction between regular recursion and tail recursion --- the special case where the recursive call is the last command that is executed by the function. Tail recursion is the preferred form used in functional programming, because it simplifies memory management, and we hypothesize that it is also easier to understand. We conducted a controlled experiment with 139 participants, in which they were asked to understand different recursive functions. This revealed that tail recursion, when it is natural to use, is indeed easier to understand than the more general form of recursion where significant processing is performed after the recursive call. But it also showed that using tail recursion may come with a price, as when achieving the tail form requires a transformation of the code that obfuscates the underlying recursive algorithm. We conclude that having significant processing after the recursive call, or distorting the code so as to remove such processing, are major factors that make recursion hard. We therefore suggest to start teaching recursion with a basic form of tail recursion, and then progress to more complicated cases with processing after the recursive call. Transformations to tail form should be taught last, if at all.