Article Critique #2 1

Article Critique #2:

Molly Frendo

CEP 900/932
Dr. Koehler and Dr. Roseth
July 17, 2010
 Article Critique #2 2

Article Critique #2:

In “Affordances of mobile technologies for experiential learning: the interplay of

technology and pedagogical practices,” the authors investigate the role of technology in

supporting experiential education. The purpose of the article is to better understand the

affordances of mobile technology and its ability to impact the learner’s transformative

experience. More specifically, the authors examine the role of personal digital assistants (PDAs)

within experiential science education of fifth graders. In developing their conceptual framework,

the authors build upon defining characteristics of experiential education in order to then better

explore the role of technology in enhancing it. By noting the criticisms of experiential learning,

the authors make explicit their goal of designing a system that eliminates the weaknesses of this

learning theory using mobile technology. Further, in examining PDAs as an exploratory tool, the

authors discuss two potential affordances. First, mobile technology give students access to real-

time information any time and place a learner needs it. Second, PDAs provide learners with easy

and quick tools for information retention in order to avoid cognitive overload. The authors seek

ways to understand how these affordances are perceived and implemented by learners in an

experiential learning context. Conceptually, the authors provide a framework that is strong: it

follows a logical progression and clearly defines how the major elements the argument is built

upon relate to one another. Their goals are transparent and fully developed; additionally, the

authors do not stray from the focus of their goals throughout the study.
 Article Critique #2 3

Significance and Need for Study

On a micro level, the questions examined in this study do not necessarily seem critical.

For instance, it is only specific to one technology and one learning theory. The nature of

technology is fleeting and there is a potential for educators to have moved on to something new

by the time this research is analyzed and published. On a macro level, I believe that this

particular study is important because it explores the role of a popular and easily accessible

technology in a commonly used teaching method, experiential education. The United States is a

country that embraces the notion of discovery learning and many educators rank it above other

teaching methodologies like lecture and memorization. Despite its popularity, however,

experiential education has the potential to be ineffective if the teacher does not carefully monitor

student progress and provide adequate and unique scaffolding to each student to ensure that

content is being learned and motivation is being maintained. Technology is frequently proffered

as a solution to eliminate pitfalls associated with certain educational methodologies, sometimes

without much thought about whether or not it will be effective. If not carefully and thoughtfully

integrated, technology can make an unstructured environment even more haphazard or

ineffective. This study is important because it examines the affordances of these mobile

technologies: that is, the elements of PDAs that could possibly enhance experiential education

and also minimize its negative characteristics. Additionally, the authors dissect the various

features of the personal digital assistant, detailing its photo capturing abilities, interactive

interfacing capabilities (prompts, quizzing, looking up additional information, etc.), sound

recording function, and authoring module options. In breaking down these elements, the authors

are able to encompass educational technology interventions beyond that of a PDA. Because the

researchers examine how each of these features impact a learner’s potential to experience all of
 Article Critique #2 4

the positive characteristics associated with experiential education, this study is able to provide

additional insight into technologies with more specific and limited functions. As new

technologies emerge, researchers need to understand their functionalities in order to recommend

them to educators who seek to utilize or repurpose them in the classroom. This study is necessary

because it critically examines a new and popular technology and provides the reader with a better

understanding of the benefits and pitfalls of this technology within the context of the widely

utilized method of experiential education.

Study’s Connection to Theory and Prior Research

The authors are effective in locating their argument within appropriate theoretical

frameworks. Several experiential education theories are discussed and contextualized, beginning

with Dewey’s (1938) ‘learning by doing’ theory and moving to Kolb’s experiential learning

theory (1984) as well as recent applications of his theory (Vince 1998; Oxendine et al. 2004;

Barker et al. 2002). Additionally, they briefly discuss how Christian (2003) applied Kolb’s

experiential learning theory in integrating computer-supported tools in a classroom setting as

well as how the procedure of experiential education could be improved (Trindale et al. 2002;

Whitelock et al. 2000; Jensen 2004). The authors could further discuss and examine other

research done around the integration of technology into experiential learning environments.

Their conversation seems to be limited to one study, and while I do not know the full extent of

the research base on this subject, I feel confident that there are other studies that could better

inform this study. Beyond discussing experiential learning theories, the authors also explore the

work done on affordances by Gibson (1977) and Kirschner (2002). In tying mobile technologies

to the research on affordances, the authors provide a scope or lens for understanding the

characteristics of PDAs and how the learner interacts with them in order to create and implement
 Article Critique #2 5

new knowledge. The conversation about affordances helps to concretely tie the goals of the study

to an existing research base and theoretical framework.

Clarity and Appropriateness of Hypothesis

The authors provide a clear picture of their research questions and goals. They seek to

better understand if learners are capable of recognizing and implementing the affordances of

PDAs as an educational technology within an experiential environment. By breaking down the

various components of the mobile technology, the authors can examine the affordances of each

in order to evaluate PDAs as a whole and how integrating PDAs can affect the learning flow.

However, the authors do not provide a definition for “learning flow” or fully explain it. For the

purpose of moving forward with the critique, my understanding of a learning flow is that it is the

process by which a learner comes to attain and create new knowledge. The authors articulate the

goals of their project as follows: consider how mobile technologies can be employed to support

experiential learning, design a curriculum in which use of PDAs can support student learning,

and evaluate if PDA-supported knowledge creation was superior to more traditional

methodologies. The first two of these goals speak more to the appropriateness of the questions;

the questions logically follow from the conceptual and theoretical frameworks. The final goal is

clear and appropriate. By comparing two demographically similar fifth grade classrooms, one of

whom receives the educational treatment and one who does not, the authors are able to measure,

observe, and compare the impacts of the intervention.

Adequacy of the Study Design

The authors conducted an experimental study consisting of several sections including a

pre-test, main activity, post-test, and questionnaire in order to test their hypothesis. Their goal
 Article Critique #2 6

was to justify the hypothesis that mobile technologies (in this instance, PDAs) can increase the

degree of knowledge created through the method of experimental learning beyond traditionally

utilized methods. Two fifth grade classrooms taught by the same teacher participated in the

study; the educational intervention being assessed was personal digital assistants. The content

was the same with the difference being the use of technology for learning guidance and note

taking purposes. The authors used identical multiple choice pre- and post-tests as well as

qualitative analysis of students’ final reports. Qualitative data were coded into two categories:

knowledge gained through the instructional prompts and knowledge created through the

experiential education exercise. Students were also given a questionnaire to better gauge their

attitude towards technology and the learning flow more generally. From my understanding of

study design, the choice in methodologies was both appropriate and adequate based on the

research question. The goal was to investigate how the learners perceived the technology as well

as how the technology impacted their learning; the study was designed to test how the PDA use

changed a student’s knowledge level following the intervention as well as to determine a

baseline comparison between the two groups. The qualitative data allow the students to share

their experience of using the PDA to assess their perception; when coupled with the

questionnaire, it provides a full picture of what happened from the learner’s point of view.

Adequacy of Sampling Methods

From the information given, it is unclear why the researchers selected this particular

group of participants. One may assume that this was due to convenience and that the authors had

easy access to this particular group of students. The choice of a convenience sample is

congruent with the study design and commonly accepted in the field. The treatment and control

groups are similar in size and demographics and both classes shared the same teacher. Therefore,
 Article Critique #2 7

it seems like an appropriate choice in sampling method because we can isolate the variable as the

use of the PDA. The authors do not provide a discussion of the study’s generalizability to larger

populations as a whole. However, they make explicit the fact that this is an exploratory study

about the use of PDAs in experiential education early on in their discussion; as a result, it does

not seem like being able to generalize their results to broader groups is their intention or an

appropriate goal.

Adequacy of Procedures and Materials

The authors utilize two different interventions; the conditions are identified as those using

PDAs and those not using them. These interventions related directly back to their hypothesis.

The authors provided a clear explanation of the study’s procedures: pre-test, main activity, post-

test, and questionnaire. They described in detail how the PDA was implemented into the learning

environment and flow as well as how those in the control group proceeded with their lesson.

Appendix I provides the reader with access to the questions asked on the pre- and post-tests;

however, the questions on the questionnaire are described but not all of them were provided.

Quality of Measures

Based on my limited understanding, the measures used in this study were both high

quality and appropriate. The same neutral teacher administered both interventions, though it is

not known if the students in the treatment and control group had any opportunity to discuss their

experiences and therefore introduce any diffusion or bias. The authors use a trick question on

their survey in order to increase validity; those students who failed to answer the trick question

were not included in the results. In terms of reliability, the authors primarily used a 5-point
 Article Critique #2 8

Likert scale in their four part questionnaire to gauge levels of agreement to the questions

answered. Likert scales are commonly understood as reliable.

Adequacy of Data Analyses

Upon completion of the experiment, the data were analyzed using a one-tailed t-test

because of the authors’ assumption that the students using the PDA would acquire more

knowledge. The choice of analysis seems appropriate here because they are testing whether or

not one intervention is superior over another and have assumed that the PDA will be the superior

choice. The t-test assesses whether the means of two groups are statistically different from one

another. In this instance, the p value is less than the alpha value of .05 (which is standard in

social science research); therefore, we reject the null hypothesis in favor of the alternate

hypothesis. Because their potential for a Type I error is low, the authors reached an appropriate

level for their statistical assumptions. Additionally, the authors used a Mann-Whitney U-test on

the survey portion of the experiment; Mann-Whitney is used in place of a t-test when the

distribution is not parametric. Based on my limited understanding of Mann-Whitney, it is

virtually identical in function to a t-test but is used in instances where there is a non-parametric

distribution.

Discussion of Methodological and/or Conceptual Limitations

From a methodological and conceptual perspective, the authors do not fully develop their

discussion on the limitations of their results. In terms of methodology, the authors do explain that

their use of trap-setting statements disrupted the pedagogical flow of the experiment and seemed

to muddy the results. Additionally, the authors do mention that a long-term experiment is

required to further determine if the students were truly motivated by the PDA itself rather than
 Article Critique #2 9

the novelty of a new technology. Finally, they pose the question about whether or not the use of

photos in the observation stage would not also be as successful if the students had used standard

library photos instead of taking pictures. Further research needs to be done to verify that it was

the act of physically taking the photos using the PDA that improved student learning and

knowledge creation rather than just the clarity provided by using a photographic image rather

than a crude sketch. There is no discussion provided by the authors about how these findings

may be limited to this particular demographic group. It is unclear if the same results would occur

with a different age or cultural group.

Consistency and Completeness of Conclusions

The authors’ conclusions seem to generally be in alignment with their reported results.

The authors did not overstate or oversell their results; they viewed their own results with a

critical eye. For instance, the authors caution the reader not to consider performance to be the

only factor when assessing the effectiveness level of experiential education. While the study

shows that students with a PDA were more successful than those without, the authors provide

further analysis in recommending a strategy based on the evidence suggesting that the order in

which one progresses through the learning flow is significant when using new technologies.

They are explicit in explaining that it is not simply the addition of technology that positively

impacts learning through experiential education; rather, the critical element is how the

technology interplays within the pedagogical practice given the content matter to be learned.

Relation of Results to Theoretical Base

The authors effectively related the results of the study to its theoretical base. The

discussion not only discussed the affordances of the various components of the PDA within
 Article Critique #2 10

experiential education, but also how the overall use of the PDA impacted learning motivation

and flow. The findings shared how the PDA allowed students to be in direct contact with their

subject matter more effectively in certain situations. Additionally, the authors acknowledge that

their attempts to design a learning system using mobile technologies specifically to eliminate the

weaknesses of experiential education were not fully successful based on waning motivation

following the photo-taking stage and the struggle many students faced in recalling and

organizing their observations results from the photo.

Significance of Study

In my opinion, this study is significant because it helps us to better understand the

affordances of technology for educational purposes. Though the specific scope of studying the

impact of using personal digital assistants within experiential education is narrow, there are

many lessons to take away on a broader level as we work to better understand appropriate

integration of technology to assist or improve learning. Because the PDA is multifaceted, the

researchers were able to break down the various components (camera, interactive interface, audio

recording, etc.) and better understand how their individual affordances affect the implementation

of the learning flow. Additionally, we are better able to understand how technology can motivate

students in new ways by reinvigorating them to a subject. Within their results section, the authors

explain how the PDA’s camera and audio recording function impressed the students and created

a belief that the learning process was more efficient. By taking the stress off of processes that

seem to inhibit learning (i.e., sketching the flower), space was freed up within the student’s

cognitive load to create knowledge about the flower in general. This study affirms the notion that

technology has a capacity to provide new sources of learning motivation for students while also

serving as an individualized scaffolding mechanism. Of equal significance is the point that

technology is not the ultimate answer for every learning scenario. The authors report that the

pedagogical methods employed by the teacher in using the technology are important. Educators

must continue to provide students with an opportunity to process and reflect upon what they have

learned in order to promote retention of the correct information. Additionally, teachers must

carefully consider how the technology interacts with their chosen pedagogy; it needs to be

properly designed and implemented in order to achieve maximum impact.

This study makes clear the need for further research on the intersections of technology,

pedagogy, and content. It is clear that there are critical moments where technology can positively

impact learning as well as those that inhibit it. Research is needed to better understand what

technologies merit certain affordances and when it is appropriate to employ them to increase a

learning outcome. Research drives practice; this study can help educators better understand ways

in which they can use technology to support learning methods and potential pitfalls to avoid

when doing so.

References

Lai, C.H., Yang, J.C., Chen, F.C., Ho, C.W., & Chan, T.W. (2007). Affordances of mobile

technologies for experiential learning: the interplay of technology and pedagogical

practices. Journal of Computer Assisted Learning 23. 326-337.