Predictive vs. passive animation learning tools

Reviewer: Andrew Brooks

Animation tools help students learn and solve problems. Are active interaction tools better than passive tools__?__ The paper reports on an experiment where students answer three questions on six different graph algorithms. The first question is answered without tool support, while the second and third questions are answered with the aid of a tool. Subjects are randomly assigned to receive either an active interaction tool or a passive tool for the second question; tool support is then swapped for the third question. Taylor et al. claim that their results show that active interaction tools are more effective. An experienced analyst would, however, declare that the conclusions are not supported by the data. Figure 1 represents the raw data for the three questions on Dijkstra's algorithm. It clearly shows that the two groups do not start from an equal baseline. One group has an average score of two out of six on the first question, while the other has an average score of four out of six. Table 1 suggests that the groupings for Kruskal's algorithm and Prim's algorithm are also strongly unbalanced. Without knowing the reasons for these imbalances, interpretation of the data lacks a proper foundation. Also, despite the fact that times were collected, timing data is neither presented nor analyzed. Understanding the role of the tradeoff between time and accuracy is extremely important in studies of human performance. Although this study is fatally flawed, I can recommend it to those who develop and evaluate algorithm animation tools, because such work is difficult and many lessons can be learned from Section 3.1, "Study Limitations." Online Computing Reviews Service