See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/356642039

Mathematical Modeling for Theory-Oriented Research in Educational

Technology

Article in Educational Technology Research and Development · November 2021

DOI: 10.1007/s11423-021-10069-6

CITATIONS READS
2 587

1 author:

Elena Novak
Kent State University
45 PUBLICATIONS 799 CITATIONS

SEE PROFILE

All content following this page was uploaded by Elena Novak on 30 November 2021.

The user has requested enhancement of the downloaded file.

 Mathematical Modeling for Theory-Oriented Research in Educational Technology

Elena Novak

Kent State University, School of Teaching, Learning and Curriculum Studies

150 Terrace Drive, P.O. Box 5190, Kent, OH 44242-0001, USA
Phone: +1-330-672-0536, Email: elannovak@gmail.com

Citation: Novak, E. (2021). Mathematical modeling for theory-oriented research in educational

technology. Educational Technology Research & Development. https://doi.org/10.1007/s11423-
021-10069-6

Abstract

Mathematical modeling describes how events, concepts, and systems of interest behave in the

world using mathematical concepts. This research approach can be applied to theory construction

and testing by using empirical data to evaluate whether the specific theory can explain the

empirical data or whether the theory fits the data available. Although extensively used in the

physical sciences and engineering, as well as some social and behavioral sciences to examine

theoretical claims and form predictions of future events and behaviors, theory-oriented

mathematical modeling is less common in educational technology research. This article explores

the potential of using theory-oriented mathematical modeling for theory construction and testing

in the field of educational technology. It presents examples of how this approach was used in

social, behavioral, and educational disciplines, and provides rationale for why educational

technology research can benefit from a theory-oriented model-testing approach.

Keywords: theory; model-testing; mathematical modeling; educational technology

1
 1. Introduction

Research in the field of educational technology and design has often been criticized for

having little impact on improving learning and contributing to basic theories that inform teaching

(Cilesiz & Spector, 2014; Reeves, 2006; Seel, 2009; Suppes, 1978), being methodologically

weak (Bulfin, Henderson, Johnson, & Selwyn, 2014) and under-theorized (Jones & Czerniewicz,

2011; Markauskaite & Reimann, 2014). For example, a recent analysis of educational technology

research of over 500 empirical articles found that few papers engaged or informed theory; most

papers either did not mention theory at all or provided vague references to theory (Hew, Lan,

Tang, Jia, & Lo, 2019). The authors argued about the importance of using existing theories to

frame empirical research, because they allow for findings to be generalized across different

contexts and learners, thus contributing to a better understanding of a specific phenomenon.

It is important to note that the term ‘theory’ means different things to different people and

there is no consensus about what theory is. For example, Simon and Newell (1956) proposed to

“define a theory simply as a set of statements or sentences” (p. 67). Other scholars viewed theory

as a set of statements and propositions that explain the relationship(s) of concepts or variables

(Hollander, 1967; Jaccard & Jacoby, 2009). Merton (1968) distinguished between middle-range

theories and unified theories: “[middle-range] theories that lie between the minor but necessary

working hypotheses that evolve in abundance during day-to-day research and the all-inclusive

systematic efforts to develop a unified theory that will explain all the observed uniformities of

social, behavior, social organization and social change" (p. 39). Opp (1970) refined Merton’s

definition of a middle-range theory as a theory that can "explain a certain class of particular

('sociological') events" (p. 245) and of a unified theory as "a theory from which at least all

existing theories of the middle range can be deduced in their original or in modified form" (p.

2
245). He argued that “the unified theory is empirically testable and has an extremely high

explanatory power. Otherwise it could not explain the particular events and theories which it is

designed to explain” (p. 245).

Constructing new theories and testing existing ones is considered among the most important

scientific efforts that advance science (Kuhn, 1996; Mintzberg, 2005; Morgan & Wildemuth,

2009). A growing number of researchers advocate the use of mathematical modeling (also

referred to as model-testing) in educational and behavioral research as a vehicle for expanding

theoretical foundations of the field and improving empirical research (Jaccard & Jacoby, 2009;

Rodgers, 2010). Mathematical modeling describes how events, concepts, and systems of interest

behave in the world using mathematical concepts. It is a powerful research tool that has been

used extensively in various disciplines, including engineering, science, criminology, health

science, information technology, psychology, and economics, to explain theoretical propositions

and form predictions and forecasts of future events and behaviors. However, development and

utilization of mathematical models is more common in the physical sciences and engineering

than in the social and behavioral sciences (Jaccard & Jacoby, 2009). Within the realm of the

social and behavioral sciences, some disciplines like economics, for example, are also known for

their extensive use of mathematical modeling as a research and policy tool. In education and

educational technology fields, however, mathematical modeling is less common, and its potential

is yet to be recognized.

2. What Is a Mathematical Model?

A mathematical model, or any model in general, represents reality in a simplified form.

According to Pearl (2000, p. 202), model is “an idealized representation of reality that highlights

3
some aspects and ignores others.” A mathematical model can be described using a mathematical

equation or a series of mathematical equations that represent relationships between quantitative

variables. The goal of mathematical model building is to develop a model that will provide an

accurate, yet relatively simple representation of reality (Figure 1). Social and behavioral

scientists usually do so by using regression modeling that can predict the impact of the

intervention using desired moderator variables. Of course, many would rightly argue that human

behaviors are complex and cannot be captured using simple linear relationships. In mathematical

model building, this tension is often addressed with the use of appropriate statistical methods.

The real world The conceptual world

Theory
Real-world problem Conceptual model

Model variables, parameters

and relationships between the
variables

Identify/construct
mathematical model (e.g.,
equations, graphs)

Real-world solution Interpret solutions/results with

empirical data

Fig1. An elementary view of mathematical modeling that shows the phases in building a
mathematical model and how models relate to real-world problems (inspired by Dym, 2004, and
Jaccard & Jacoby, 2009)

Although variables are critical to many theories, not all theoretical approaches use variable-

oriented approaches to develop theories. These theories use a process-oriented approach and/or

4
narratives to explain a specific phenomenon (Jaccard & Jacoby, 2009) and mathematical

modeling may not be an appropriate approach for these theoretical frameworks. For instance,

instead of defining gender as a variable with two levels, female and male, these theories focus on

the ways in which gender is experienced by different people. Nevertheless, theoretical

approaches that rely on variables to explain different relationships among specific concepts, such

as explanatory (“Y because of X”) or design theories (“to achieve Y, do X”) can use

mathematical modeling to construct and test the theories (Malone, 1985). Such approaches rely

on the concept of causality to identify systematic relationships between variables that can explain

our environment, organize constructs of interest, and predict future events. Typical model

construction involves the following steps (Boland, 2014; Jaccard & Jacoby, 2009):

1. Identify exogeneous and endogenous variables: The process begins with identifying

variables that explain the causal relationship(s) of interest as well as basic behavioral

relationships between the variables. In particular, the modeler needs to identify which of those

variables are exogeneous and which are endogenous. Variable X is exogeneous if a change in X

causes changes in Y. That makes variable Y endogenous, or the variable that can be explained by

the model. After identifying the variables, the modeler needs to consider how to measure the

variables and the values of the parameters.

2. Include model assumptions: Boland (2014) argues that during the second phase of model

construction specifying or simplifying assumptions should be added to the identified behavioral

relationships. Those can include behavioral assumptions about individuals or organizations, as

well as assumptions about the relationships between the identified variables and their parametric

specifications.

5
3. Identify possible functions for model construction: The modeler needs to identify possible

functions that can capture the relationship between the variables of interest. Toward this end, a

wide variety of functions can be employed including, but not limited to linear, exponential,

logarithmic, and polynomial functions. Although linear relationships are common in modeling,

they do not always explain all possible relationships between quantitative variables. For instance,

the relationship between levels of test anxiety and test performance takes a form of a nonlinear

inverted-U function (a U drawn upside down) where low levels of anxiety are beneficial for test

performance but high levels of anxiety are detrimental to a person’s success on the test, as

tension and worries consume critical cognitive resources that could otherwise be directed to the

task at hand (Hoffman, 2010).

4. Collect data to examine the model performance: Once the mathematical model is

constructed (e.g., equations, graphs), it is time to collect empirical data to examine the model

performance. Specifically, the modeler needs to explore how well the values of the endogenous

variable(s) fit the real-world values. If the degree of fit is unsatisfactory, the original model can

be adjusted, or a new function may need to be employed.

5. Revise the model and evaluate its performance with new data. After making the necessary

adjustments to the original model, a new set of empirical data should be collected to evaluate the

revised model’s performance. If the revised model is conceptually plausible and performs well,

then it is deemed to be the choice model.

The model construction steps, as described, are indubitably an oversimplification of the

process of developing and testing mathematical models. To discuss it in detail would be too

extensive for the purpose of this paper. Interested readers may find Jaccard and Jacoby’s (2009)

6
book “Theory Construction and Model-Building Skills” helpful for learning about the cognitive

heuristics, tools, and approaches for model construction. Moreover, structural equation modeling

can be used for mathematical modeling as well (Jaccard & Jacoby, 2009).

3. How Can Mathematical Modeling Be Used in Educational Technology Research and

Why Do We Need It?

The basic idea behind mathematical model building is to “translate the verbal problem into a

mathematical problem to solve” (Boland, 2014, p. 2) or to represent a descriptive theory using

mathematical concepts. But how are theories different from models? Theories explain certain

phenomena and critical explanatory variables or mechanism, but generally are “flexible about the

expected conclusions of empirical research” (Nelson & Winter, 1974, p. 886), while models

serve as “representations of theories constructed for the purpose of serving as tools or

instruments” (Boland, 2014, p. 22).

Roald Hoffman (2003), a Nobel Prize-winning scientist, in his commentary about theories

and models compared the process of a theory development to storytelling: “When things are

complex yet understandable, human beings weave stories” (p. 224), and “Many theories are

popular because they tell a rollicking good story, one that is sage in capturing the way the world

works, and could be stored away to deal with the next trouble” (p. 225). His view on the

importance of models is that “relatively uncomplicated models … play a special role in the

acceptance and popularity of theories among other theorists… The models become modules in a

theoretical Erector set, shuttled into any problem as a first (not last) recourse. … taking one piece

of experience over to another” (p. 225). That is to say that mathematical modeling is clearly

important to the construction of theories.

7
 Earlier attempts in the late 1930s to represent behavioral theories using mathematical

equations were done primarily for theoretical purposes only (see Boland (2014) for an extensive

overview of model building in economics). Today, models are designed with a specific goal in

mind, usually as a research or policy instrument. Such models are called empirical models.

Empirical models can be data-driven or theory-oriented. Data-driven empirical models are

designed to identify patterns in the available data in order to form short-term forecasts or

evaluate the impact of possible policy changes, without drawing its inferences from a specific

theory. Theory is used, but only as it fits the data of interest (Boland, 2014). Theory-oriented

empirical models are designed to test a theory or theoretical claims of interest. This type of

model building utilizes empirical data to evaluate whether the specific theory can explain the

empirical data or whether the theory fits the data available.

It is worth mentioning that producing empirical evidence of a mathematical model that

cannot support a theory in question does not imply that the theory is wrong, -- also known as the

Duhem-Quine thesis in the philosophy of science. It only suggests that the theoretical statement

is not confirmed (not to be confused with ‘disconfirmed’) or that the model builder made false

background assumptions in the model design, e.g., assuming a linear relationship between

variables instead of a quadratic one, or using tools/instruments that produce measurement errors.

Morgan and Wildemuth (2009) wrote: “A theoretical model is not defeated by the discovery of

exceptions to the theory, but only by the evolution of a better model, or at least one that better

answers the questions that scientists are asking at a given time” (p. 42). Kuhn (1996) notes that

theory testing and the discovery of anomalies is central to developing a scientific understanding

about our world, as “theory works only until a critical mass of anomalies and exceptions to it are

8
found, at which point, it becomes untenable and must be replaced by a new theoretical model”

(Morgan & Wildemuth, 2009, p. 42).

There are many challenges to implementing mathematical modeling in education, social, and

behavioral disciplines due to the complex, dynamic, unique, and mostly obscured nature of

reality (Jaccard & Jacoby, 2009), lack of appropriate methodological data analysis approaches,

and high levels of uncertainty (Dabbaghian & Mago, 2014). Nevertheless, Rodgers (2010)

argues that mathematical modeling should be used more extensively in behavioral research and

graduate training to build and evaluate various behavioral models: “Researchers should be

focusing on building a model, embedded within well-developed theory… we should be

developing mathematical models and evaluating them statistically” (p. 5). Yet, it has not been

used extensively in educational technology research.

Mathematical models are prominent in social and behavioral sciences for analyzing human

decision making. They are also used in theories of memory, learning, language and in the

analysis of social networks, such as institutions, communities, and trade networks. For instance,

mathematical modeling was used to study short-term and long-term memory (Atkinson &

Shiffrin, 1968), musical perception (Shepard, 1982), human reproduction describing childbearing

decision making (Rodgers & Doughty, 2001), onset of various social activities, like smoking and

drinking (Rodgers, 2003; Rowe & Rodgers, 1991) and sexual behavior (Rodgers & Rowe, 1993).

In political science, mathematical models are prominent in analyzing political behavior such as

voting and fairness (Jaccard & Jacoby, 2009).

Among other benefits of using mathematical modeling in research is its ability to address the

causal generalization problem in research that evaluates the causal impacts of educational

interventions in order to inform policy decisions, such as whether to integrate a new educational

9
technology into a curriculum (Rodgers, 2010; Scheerens & Blömeke, 2016; Shadish, Cook, &

Campbell, 2002; Tipton & Olsen, 2018). Oftentimes, the results from the study do not generalize

to the population that would be impacted by these decisions, because the impact of an

intervention varies across educational settings and learners. This is one of the reasons why

educational technology research had very little impact on improving instructional practices and

student learning, because we cannot adapt our instructional interventions to the myriad variables

that surround and interact with any given instance of instruction (Cronbach, 1975; Reeves, 2006;

Seel, 2009). Using a model-testing approach can address the causal generalization problem

(Tipton & Olsen, 2018).

Nevertheless, it is important to note that mathematical modeling is not the only approach for

improving the quality and impact of educational research. Other types of inquiry such as design-

based or qualitative research are used extensively to generate new knowledge, construct theories,

and devise solutions to complex educational problems. For instance, one of the important goals

of design-based research is to generate new knowledge that can inform theory development

(McKenney & Reeves, 2014; Puntambekar, 2018). Those developed theories can then be further

tested and refined using mathematical modeling.

Many books have been written about theory construction and modeling as these ideas are at

the core of the scientific process. However, few of those introduce the topic to students or

professionals in education, social and behavioral sciences, who are new to theory construction

and modeling. This shortage of literature on teaching theory construction and modeling reflects

the curricular trends of many graduate programs in these disciplines. Graduate students in

education, social and behavioral sciences usually take several courses in research methods and

data analysis, but seldom receive extensive training in model building and theory construction

10
(Jaccard & Jacoby, 2009; Rodgers, 2010). Unsurprisingly, we do not see many applications of

these foundational ideas in the field of education and educational technology.

This article presents examples of model-testing from social, behavioral, and education

disciplines that demonstrate the applications of theory-oriented mathematical modeling for

research and theory advancement in education and educational technology. In addition, it

highlights how mathematical modeling can contribute to expanding theoretical foundations of

the field of educational technology.

4. Theory-Driven Model-Testing Examples

4.1 Example 1: The Integrative Theory of Motivation, Volition, and Performance

The first example demonstrates the construction and testing of a unified theory in educational

technology. Keller (2008) developed the integrative theory of Motivation, Volition, and

Performance (MVP) by synthesizing multiple theoretical approaches and explanatory frames of

reference to take a broad view of the learning, motivational, volitional, psychological, and

environmental factors that affect human performance. This theory can be characterized as a

macro theory that incorporates various micro theories, including information-processing theories

(Atkinson & Schiffrin, 1971; Mayer, 2001), motivation (Keller, 1999), action control theory

(Kuhl, 1987), self-regulation (Zimmerman, 2001), and implementation intentions (Gollwitzer,

1999). Keller’s motivational theory (1999), also known as the ARCS model, served as one of the

foundation blocks that led to the development of the integrative theory of MVP. The ARCS

model describes motivational factors that influence learning and performance. Specifically, the

first three factors, i.e., attention, relevance, and confidence, relate to learning conditions. The

fourth ARCS factor, satisfaction, describes the product of the learner’s mental appraisal of their

11
ability to apply the newly acquired skills and knowledge to everyday life, which can be viewed

as a learning outcome.

Using a systems model approach, Keller developed a conceptual MVP model to

systematically represent relationships among explanatory components of the MVP theory (Figure

2). The conceptual MVP model describes inputs (i.e., considered to play an important role in

explaining the theoretical principle), interactions among them, and outputs (i.e., measurable

behaviors to be explained). This conceptual model was later statistically tested using four

different data sets from distinct digital learning environments (electronic textbooks, simulations,

video games), learning content (statistics, biology, economic theory), settings (classroom, lab),

and learners (undergraduate and graduate students from various majors; Huang et al., 2010;

Novak, 2014; Novak et al., 2018).

Fig2. The MVP model of motivation, volition, and performance according to Keller (2008a).
Reprinted with the permission of John Keller from “An Integrative Theory of Motivation,
Volition, and Performance,” by J. Keller, 2008, Technology, Instruction, Cognition and
Learning, 16, 79-104, p. 94.

12
 The first investigation in this area revealed a significant model that provided empirical

support for the ARCS model in an online instructional game with undergraduate students from

various majors (Huang et al., 2010). This mathematical model described the relationship between

the motivational (attention, relevance, and confidence) and outcome processing (satisfaction)

variables using the following equation:

Satisfaction (s) = A × Attention(s) + B × Relevance(s) + C × Confidence(s) (1)

(A, B, and C are the positive parameters; s = student)

Later, Novak (2014) used Huang et al.’s (2010) model (Equation 1) to develop an initial

mathematical MVP model and statistically evaluate it using motivational, volitional, and

performance data with graduate students in a digital simulation-based learning environment. The

following mathematical MVP model was revealed:

Satisfaction(s) = A × Attention(s) + B × Relevance(s) + C × Confidence(s) +

+ D × Volition(s) (2)

(A, B, C, and D are the positive parameters; s = student)

Recently, Novak, Daday, and McDaniel (2018) statistically evaluated the initial MVP model

(Equation 2) for (a) fitting two new data sets with undergraduate biology from two different

technology-rich learning environments, and (b) expanding the model by including additional

variables that affect student outcome processing in e-text learning environments. Their study

confirmed the generalizability of the initial MVP model across different digital learning

environments and identified four additional variables in the MVP model, i.e., intrinsic cognitive

load (IL), extraneous cognitive load (EL), number of academic hours earned in the degree

(Academic_Background), and ACT math score (Prior_Knowledge):

13
Satisfaction (s) = A × Attention(s) + B × Relevance(s) + C × Confidence(s) + D × IL(s) +

+ E × EL(s) + F × Academic_Background(s) + G × Prior_Knowledge(s) (3)

(A, B, C, and D are positive parameters; E, F, and G are negative parameters; s = student)

This expanded model fit both data sets well.

A comparison of MVP model’s variables across the studies that developed and tested

MVP models showed that the model produced consistent empirical support for the elements of

the MVP theory across various technology-rich learning environments and student populations

(Huang et al., 2010; Novak, 2014; Novak et al., 2018). The MVP model has research and

practical implications. For researchers, it can explain how well the MVP theory matches the

reality of the learning, motivational, volitional, psychological, and environmental factors that

influence learning and performance. For practitioners, it can serve as a tool for diagnosing and

improving learning environments and student experiences in these environments.

4.2 Example 2: The Unified Theory of Acceptance and Use of Technology

The second example illustrates the development and testing of a unified theory of user

acceptance of new technology. User acceptance of new technology is one of the most mature

research areas in the information systems field, and the literature offers myriad competing user

acceptance models and constructs. Oftentimes, researchers chose a “favored model” or had to

“pick and choose” related constructs (Venkatesh, Morris, Davis, & Davis, 2003), which did not

contribute to expanding theoretical foundations of the field. To address this challenge, Venkatesh

et al. (2003) developed the Unified Theory of Acceptance and Use of Technology (UTAUT) to

create a unified view of the information technology acceptance research that explores user

acceptance of new technology.

14
 Venkatesh and colleagues empirically compared various user acceptance models to identify

conceptual and empirical similarities across the models that were later used to create and

empirically validate a unified model. The authors empirically tested the UTAUT in four

different professional settings and cross-validated it with data from two additional organizations.

This empirical testing revealed variables that influenced usage intention and usage behavior as

well as several additional variables that mediated these relationships (Figure 3).

Performance
Expectancy

Effort
Expectancy
Behavioral
Use Behavior
Intention

Social Influence

Facilitating
Conditions

Gender Age Voluntariness of

Experience
Use

Fig3. UTAUT Research Model. Adapted from “User acceptance of information technology:
Toward a unified view,” by Venkatesh et al., 2003, MIS Quarterly, 27(3), p. 447.

To examine the impact of the UTAUT on the field, Venkatesh, Thong, and Xu (2016)

conducted a comprehensive review of the UTAUT literature eleven years after the initial

development of the UTAUT. Their review revealed that the UTAUT was used extensively across

15
a variety of settings and users, including schools, hospitals, manufacturing and service sectors,

digital learning contexts, e-government services, repeatedly validating the relationships found in

the original UTAUT model. A total of 37 different UTAUT extensions were developed based on

the original UTAUT model (Figure 4). These UTAUT extensions extended the baseline UTAUT

model with new mechanisms and variables predicting technology use and/or behavioral

intention, thus making a considerable contribution to the research on technology acceptance.

Performance
Expectancy

Effort New
Expectancy Behavioral Technology Outcome
Intention Use Mechanism
s
New
Social Influence
Exogenous
Mechanisms
Notes:
1. Boxes and arrows with
Facilitating
dashed lines represent
Conditions UTAUT extensions.

New Exogenous 2. The italic style for social

New Exogenous Mechanisms influence, facilitating
Mechanisms conditions, and technology
use represents new
conceptualization.
Fig4. Types of UTAUT Extensions. Adapted from “Unified theory of
acceptance and use of technology: A synthesis and the road ahead,” by Venkatesh et al., 2016,
Journal of the Association for Information Systems, 17(5), p. 335.

Example 3: Teacher Education and School Effectiveness

Examination of the Curriculum of Teacher Education Across 15 Countries

The effect of teacher education programs on future teachers’ knowledge and teaching

practices is one of the major research areas in the field of teacher education (Qian & Youngs,

2016). Research on teacher education clearly shows that teacher quality is the most important

school-related factor that affects student achievement (Goe, 2007; Qian & Youngs, 2016).

16
Blömeke and Kaiser (2011) used a model-testing approach to examine curricula of teacher

education programs in mathematics across 15 countries. Their research was motivated by a Tatto

et al. (2008) study that compared teacher’s mathematics content and pedagogical knowledge at

the end of their training from 15 countries in Africa, America, Asia, and Europe. Tatto et al.

(2008) found considerable differences of about one standard deviation in teachers’ knowledge

outcomes across the countries. These findings prompted Blömeke and Kaiser (2011) to examine

whether prospective teachers in different countries had similar opportunities to learn (OTL)

mathematics, mathematics pedagogy, and general pedagogy during their training. They grounded

their research in theories connecting educational opportunity and educational achievement

(McDonnell, 1995), noting that research on OTL in teacher education is weak and not supported

by robust empirical data. The model-testing approach allowed Blömeke and Kaiser (2011) to

identify similar and distinct features of the curriculum of teacher education programs (OTL

profiles) across the countries and examine the relationships of these OTL profiles to cultural

contexts. In addition to laying a foundation for the discussion about homogeneity in mathematics

pedagogy, this approach helped pinpoint culturally shaped visions of teacher knowledge and the

teaching profession across different countries and within the countries’ programs.

An Integrated Model of School Effectiveness and Teacher Education Effectiveness

Scheerens and Blömeke (2016) examined the relationships among teacher candidates’

learning in teacher education programs and PreK-12 student achievement using a modeling

approach. They created and tested an integrated theoretical model of school effectiveness and

teacher education effectiveness to better understand the processes mediating the transformation

of teacher knowledge into instructional quality that influences student achievement. Scheerens

17
and Blömeke (2016) argued that this “extended educational effectiveness model … enriches a

systemic interpretation of key levers of educational effectiveness and opens up black boxes at the

system and the classroom level” (p. 70) by allowing the systematical examination of the

relationships among different variables affecting teacher education and school effectiveness.

Example 4: A Computer Frustration Model

The topic of computing frustration – an emotional outcome of a negative experience with

using technology – has been studied primarily in the field of human-computer interface.

Bessière, Newhagen, Robinson, and Shneiderman (2006) developed a theoretical model for

computer frustration by synthesizing existing theories on frustration to empirically examine

frustration as an emotional or pre-emotional response to various obstacles in computer usage that

hamper goal achievement. They explored how computer frustration affects users’ experiences

associated with negative computing experiences focusing on two types of outcomes: (1)

situational factors that relate to specific events, e.g., the importance of the task, severity of the

interruption, and frequency of occurrence, and (2) dispositional factors, e.g., user self-efficacy,

experience, mood, demographic characteristics (Figure 5).

In the field of educational technology, computing frustration can be viewed as an

emotional or pre-emotional response to unexpected obstacles impeding learning with technology,

as students’ experiences with technology directly relate to their attitudes toward the technology

and intentions to continue using the technology as a learning tool. Novak, McDaniel, Daday, and

Soyturk (2021) examined the application of Bessière and colleagues’ (2006) model of computer

frustration to study students’ frustration with e-textbooks. Novak et al.’s (2021) study pinpointed

the relationships between students’ computer experiences, cognitive load, motivation,

background characteristics and e-textbook frustration, confirming several of the relationships

18
presented in the Bessière and colleagues’ (2006) original model but also expanding it by

identifying new cognitive and motivational variables that influence e-text frustration.

TASK

Goal Interruption

Level of Goal
Computer

DISPOSITIONAL MEDIATORS
Commitment/
SITUATIONAL MEDIATORS

Importance Experiences

Severity of Psychological
Interruption/ Time Factors/ Self Efficacy
Loss

Mood
Strength of Desire/
Anticipation
Expectations

Immediate Frustration Emotional Outcome

Fig5. The computer frustration model. Adapted from “A model for computer frustration: The
role of instrumental and dispositional factors on incident, session, and post-session frustration
and mood,” Bessière et al., 2006, Computers in Human Behavior, 22(6), p. 949.

5. Implications: Toward an Integrated Theory of Effective Online Learning

The field of online (also referred to as distance) education has seen a tremendous growth

in scholarship over the past several decades. Michael Graham Moore in the Preface to his and

William Diesl’s (2018) Handbook of Distance Education notes that this growth has contributed

to the “splitting of the field into component specialties” (Moore, 2018; p. xii), such as e-learning,

19
Internet learning, distributed learning, networked learning, blended learning, virtual learning,

Web-based learning, distance learning, and tele-learning. Each of the component parts of the

online/distanced education field has generated an enormous number of publications, new

journals, and unique conferences and interest groups. Although this “splitting” and the breadth of

the field is common to any maturing field, it creates a problem – a lack of unified foundational

knowledge and theory that is readily accessible to all specialties of online/distance education

(Moore, 2018). Research in this area utilizes a variety of recognized theoretical foundations such

as Community of Inquiry (Garrison, Anderson, & Archer, 2000), Connectivism (Siemens, 2004),

Theory of Interaction and Communication (Holmberg, 1985), Online Collaborative Learning

(Harasim, 2012), Transactional Distance (Moore, 1997), and Independent Study (Wedemeyer,

1981). In addition, various learning theories from behaviorist, constructivist, and cognitive

schools of learning can be applied to online learning (Anderson, 2008). To integrate the work of

many theorists, Terry Anderson developed a model of online learning as a “first step toward a

theory in which the two predominant forms of online learning – collaborative and independent

study – are considered” (Anderson, 2008, p. 7; Figure 6). Moreover, he emphasized the utility

aspect of online learning (i.e., effective online learning): "We need theories of online learning

that help us to invest our time and limited resources most effectively" (Anderson, 2008, p. 46).

Although the proposed theory was incomplete, the model identified many important variables

and relationships among them.

20
Fig6. Anderson’s (2011) model of online learning. Adapted from “The theory and practice of
online learning,” Anderson, T., 2011, Edmonton, AB: AU Press, p. 49.

Picciano (2017) continued Anderson’s work of constructing an integrated model for

online education (Figure 7). Picciano’s model was developed based on an analysis of some of the

aforementioned theories of online education as well as behavioral, social constructivist, and

cognitive theories and their derivatives, e.g., Wenger and Lave’s (1991) Communities of

Practice, Wenger’s (1998) Situated Learning, Atkinson and Shiffrin’s (1968) Information

21
Processing, neuroscience research (Willingham, 2008), and Knowles’ (1998) Adult Learning

Theory (Andragogy). Picciano’s model identifies several theoretical constructs of online

education but does not explain the relationships among them. Picciano (2017) concludes his

discussion of the proposed model by posing a critical question of whether it is possible to

develop an integrated theory of online education. Mathematical modeling can help answer this

question by empirically testing the proposed theory of online education. It can also expand the

theory by examining additional factors that are critical to online learning. Moreover, it can help

advance the much-needed research on educational globalization and cross-cultural online

learning (Moore, 2018; Zawacki-Richter, Backer, & Vogt, 2009).

Fig7. Picciano’s (2017) model for online education. Adapted from “Theories and frameworks for
online education: Seeking an integrated model,” Picciano, A. G., 2017, Online Learning, 21(3),
166-190, p. 182.

22
 Means, Bakia, and Murphy’s (2014) framework for research on online learning (Figure 8)

may prove to be a useful tool when applying mathematical modeling to testing and/or

constructing an integrated theory of online learning. After conducting a seminal meta-analysis on

the effectiveness of online and blended learning (Means at al., 2013), Means and colleagues

(2014) proposed a framework for research on online learning that offers four possible

dimensions that characterize online learning: (1) context, (2) instructional design, (3)

implementation, and (4) outcomes (Figure 8).

Fig8. Means and colleagues’ (2014) Four dimensions of online learning. Adapted from
“Learning online: What research tells us about whether, when and how,” Means et al., 2014,
New York: Routledge, p. 9.

They argued: “Our ability to accumulate knowledge about the kinds of online learning

experiences that produce desired effects for specific kinds of learners under a given set of

circumstances would be greatly enhanced if every research report used these dimensions and an

agreed set of more specific features within each dimension to generate comprehensive

23
descriptions of the interventions they studied” (Means et al., 2014, pp. 8-9). Clearly, this

recommendation suggests a unified or integrated approach to research on online learning and can

potentially contribute to the development of an integrated theory of online education.

In summary, this section provided one example of how mathematical modeling can

advance the field of online education. There are many other areas of educational technology that

can benefit from using mathematical modeling. For example, research on barriers to technology

integration in K-12 education (Ertmer, 1999; Hew & Brush, 2007) is continuing experiencing an

overwhelming growth in scholarship. But most studies in this field take a narrow approach when

examining possible barriers to technology integration focusing on teachers’ technical skills and

attitudes toward technology (Hew & Brush, 2007). This approach is dangerous, because it

presents a very limited view, without taking into consideration the environmental, curricular,

psychological, and content knowledge factors that affect teachers’ ability and willingness to use

technology in a classroom. A unified approach that relies on mathematical modeling to develop a

theory of technology integration in K-12 education can advance the theoretical foundations of

the field and contribute to more impactful research.

6. Conclusion

The field of educational technology, learning, and design offers a great number of

middle-range theories. In such fields, it is advisable to attempt the construction and statistical

testing of unified theories that synthesize multiple middle-range theories (Opp, 1970; Rodgers,

2010). Since a unified theory can explain a set of middle-range theories, then it can explain at

least the same events as all middle-range theories that informed the unified theory. Thus, unified

theories can facilitate and speed-up the process of inquiry as well as create opportunities for new

24
research directions that would not have otherwise happen. Moreover, a unified theory in the field

of educational technology can serve as a platform for research across different research

paradigms and digital learning environments.

At the same time, it is imperative to continue using middle-range theories to further

expand the theoretical foundations of the field as well as develop new theories that offer a new

perspective of looking at phenomena. Studies that use a theory to frame their research and find

refinements to the theory are especially valued due to their contribution to research and theory

development (Hew et al., 2019; Straub, 2009).

Although mathematical modeling is not the only approach to theory construction, it offers

a sophisticated way of thinking about relationships between variables that can benefit theory

construction and testing efforts in the field of educational technology. For example,

mathematical modeling can be used to expand theories examining relationships among different

variables that do not have extensive empirical support. Well-developed theories can be used to

develop and test mathematical models in culturally diverse settings. In addition, models can be

tested with different educational technologies and learners in different contexts and settings.

Models that produce consistent results are considered more robust (Johns, 2006; Venkatesh et al.,

2016). These theories and models are particularly suited for future research that aims to examine

possible theoretical extensions in new contexts and settings. Models that do not provide

consistent support for the theories, or do not generalize well to new contexts and learners,

present research opportunities that can expand theoretical foundations of the field.

25
 References

Anderson, T. (2008). The theory and practice of online learning (2nd Edition). Edmonton, AB:

AU Press.

Atkinson, R. C., & Shiffrin, R. M. (1968). Human memory: A proposed system and its control

processes. In Spence, K. W., & Spence, J. T. The psychology of learning and motivation

(Volume 2). 89–195. New York: Academic Press.

Atkinson, R. C., & Schiffrin, R. M. (1971). The control of short-term memory. Scientific

American, 225, 82-90.

Atkinson, R. C., & Shiffrin, R. M. (1968). Human memory: A proposed system and its control

processes. In K. W. Spence & J. T. Spence (Eds.), The psychology of learning and

motivation: Advances in research and theory (Vol. 2), 89-195. New York, NY: Academic

Press.

Bessière, K., Newhagen, J. E., Robinson, J. P., & Shneiderman, B. (2006). A model for computer

frustration: the role of instrumental and dispositional factors on incident, session, and

post-session frustration and mood. Computers in Human Behavior, 22(6), 941-961.

doi:https://doi.org/10.1016/j.chb.2004.03.015

Blömeke, S., & Kaiser, G. (2011). Homogeneity or heterogeneity? Profiles of opportunities to

learn in primary teacher education and their relationship to cultural context and

outcomes. ZDM, 44(3), 249-264. doi:10.1007/s11858-011-0378-6

Boland, L. A. (2014). Model building in economics: Its purposes and limitations. New York,

NY: Cambridge University Press.

26
Bulfin, S., Henderson, M., Johnson, N. F., & Selwyn, N. (2014). Methodological capacity within

the field of “educational technology” research: An initial investigation. British Journal of

Educational Technology, 45(3), 403–414.

Cilesiz, S., & Spector, J. M. (2014). The Philosophy of Science and Educational Technology

Research. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of

Research on Educational Communications and Technology, 875-884. New York, NY:

Springer New York.

Cronbach, L. J. (1975). Beyond the two disciplines of scientific psychology. American

Psychologist, 20, 116-117.

Dabbaghian, V., & Mago, V. K. (2014). Theories and Simulations of Complex Social Systems.

Berlin Heidelberg: Springer.

Dym, C. (2004). Principles of Mathematical Modeling: Academic Press.

Ertmer, P. A. (1999). Addressing first- and second-order barriers to change: Strategies for

technology integration. Educational Technology Research and Development, 47(4), 47–

61.

Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based

environment: Computer conferencing in higher education model. The Internet and

Higher Education, 2(2-3), 87-105.

Goe, L. (2007). The link between teacher quality and student outcomes: A research synthesis.

Washington, DC: National Comprehensive Center for Teacher Quality. Retrieved from

http://ncctq.learningpt.org/ publications/LinkBetweenTQandStudentOutcomes.pdf

Gollwitzer, P. M. (1999). Implementation Intentions. Strong effects of simple plans. American

Psychologist, 54(7), 493-503.

27
Harasim, L. (2012). Learning theory and online technologies. New York: Routledge/Taylor &

Francis.

Hew, K.F., & Brush, T. (2007). Integrating technology into K-12 teaching and learning: Current

knowledge gaps and recommendations for future research. Education Technology

Research and Development, 55, 223–252. https://doi.org/10.1007/s11423-006-9022-5

Hew, K. F., Lan, M., Tang, Y., Jia, C., & Lo, C. K. (2019). Where is the “theory” within the field

of educational technology research? British Journal of Educational Technology, Advance

online publication. doi:10.1111/bjet.12770

Hoffman, B. (2010). “I think I can, but I'm afraid to try”: The role of self-efficacy beliefs and

mathematics anxiety in mathematics problem-solving efficiency. Learning and Individual

Differences, 20(3), 276-283.

Hoffman, R. (2003). Why buy that theory? In O. Sacks (Ed.), The best American science writing:

2003, 222–227. New York: Harper-Collins.

Hollander, E. P. (1967). Principles and methods of social psychology. Oxford, UK: Oxford

University Press.

Holmberg (1985). The feasibility of theory of teaching for distance education and a proposed

theory (ZIFF Paiere 60). Hagen, West Germany: Fern Universitat, Zentrales Institute fur

Fernstudienforscgung Arbeitsbereich. (ERIC Document Reproduction Service No.

ED290013).

Huang, W.-H., Huang, W.-Y., & Tschopp, J. (2010). Sustaining iterative game playing processes

in DGBL: The relationship between motivational processing and outcome processing.

Computers & Education, 55(2), 789-797.

28
Jaccard, J., & Jacoby, J. (2009). Theory construction and model building skills: A practical guide

for social scientists. New York, NY: Guilford Press.

Johns, G. (2006). The essential impact of context on organizational behavior. Academy of

Management Review, 31(2), 386-408.

Jones, C., & Czerniewicz, L. (2011). Theory in learning technology. Research in Learning

Technology, 19(3), 173-177.

Keller, J. M. (1999). Using the ARCS motivational process in computer-based instruction and

distance education. New Directions for Teaching and Learning, 78, 39-48.

Keller, J. M. (2008). An integrative theory of motivation, volition, and performance. Technology,

Instruction, Cognition and Learning, 16, 79-104.

Knowles, M.S., Holton, E.F. & Swanson, R.A. (1998). The adult learner (5th Edition). Houston:

Butterworth-Heinemann Publishers.

Kuhl, J. (1987). Action control: The maintenance of motivational states. In F. Halisch & J. Kuhl

(Eds.), Motivation, Intention and Volition, 279-291. Berlin: Springer.

Kuhn, T. S. (1996). The Structure of Scientific Revolutions. Chicago, IL: University of Chicago

Press.

Malone, T. W. (1985). Designing organizational interfaces. In L. Borman & R. Smith (Eds.),

Proceedings of the CHI’85 Conference on Human Factors in Computing Systems, 66-71.

New York, NY: ACM Press.

Markauskaite, L., & Reimann, P. (2014). Editorial: e-Research for education: Applied,

methodological and critical perspectives. British Journal of Educational Technology,

45(3), 385–391.

Mayer, R. E. (2001). Multimedia Learning. Cambridge: Cambridge University Press.

29
Means, B., Bakia, M., & Murphy, R, (2014). Learning online: What research tells us about

whether, when and how. New York: Routledge.

Means, B., Toyama, Y., Murphy, R., & Baki, M. (2013). The Effectiveness of Online and

Blended Learning: A Meta-Analysis of the Empirical Literature. Teachers College

Record, 115.

McDonnell, L. M. (1995). Opportunity to learn as a research concept and a policy instrument.

Educational Evaluation and Policy Analysis, 17(3), 305–322.

McKenney, S. & Reeves, T.C. (2014). Educational design research. In J. M. Spector, M. D.

Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational

communications and technology, 131-140. New York, NY: Springer.

Mintzberg, H. (2005). Developing theory about the development of theory. In M. Hitt & K.

Smith (Eds.), Minds in Management: The Process of Theory Development, 355–372.

New York, NY: Oxford University Press.

Moore, M. G. & Diehl, W. C, (2018). Handbook of Distance Education. New York: Routledge

Moore, M. G. (1997). Theory of Transactional Distance. In Keegan, D. (Ed.). Theoretical

Principles of Distance Education. 22-38. New York: Routledge.

Moore, M. G. (1973). Towards a theory of independent learning and teaching. Journal of Higher

Education, 44, 661-679.

Morgan, C., & Wildemuth, B. M. (2009). Questions related to theory. Applications of social

research methods to questions in information and library science (pp. 40-50). Westport,

CT: Libraries Unlimited.

Nelson, R., & Winter, S. (1974). Neoclassical vs. Evolutionary Theories of Economic Growth:

Critique and Prospectus. Economic Journal, 84(336), 886-905.

30
Novak, E. (2014). Toward a mathematical model of motivation, volition, and performance.

Computers & Education, 74, 73-80. https://doi.org/10.1016/j.compedu.2014.01.009

Novak, E., Daday, J., & McDaniel, K. (2018). Using a mathematical model of Motivation,

Volition, and Performance to examine students’ e-text learning experiences. Educational

Technology Research & Development. 66(5), 1189-1209. https://doi.org/10.1007/s11423-

018-9599-5

Novak, E., McDaniel, K., Daday, J., & Soyturk, I. (2021). Understanding student frustration

with e-learning materials: Development and validation of an E-Text Frustration scale.

Featured Research Paper presented at the Association for Educational Communications

and Technology (AECT), Chicago, IL. November 2021.

Opp, K.-D. (1970). Theories of the middle range as a strategy for the construction of a general

sociological theory. Quality and Quantity, 4(2), 243-253. doi:10.1007/BF00199565

Pearl, J. (2000). Causality: Models, reasoning, and inference. Cambridge, England: Cambridge

University Press.

Picciano, A. G. (2017). Theories and frameworks for online education: Seeking an integrated

model. Online Learning, 21(3), 166-190. doi: 10.24059/olj.v21i3.1225

Puntambekar, S. (2018). Design-based research. In F. Fisher at al. (Eds.), International

Handbook of the Learning Sciences, 383-392. New York, NY: Routledge.

Qian, H., & Youngs, P. (2016). The effect of teacher education programs on future elementary

mathematics teachers’ knowledge: a five-country analysis using TEDS-M data. Journal

of Mathematics Teacher Education, 19(4), 371-396. doi:10.1007/s10857-014-9297-0

31
Reeves, T. C. (2006). Design research from the technology perspective. In J. V. Akker, K.

Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational design research, 86-109.

London: Routledge.

Rodgers, J. L. (2003) EMOSA sexuality models, memes, and the tipping point: Policy and

program implications. In D. Romer (Ed.), Reducing adolescent risk: Toward an

integrated approach, 185–192. Newbury Park, CA: Sage.

Rodgers, J. L. (2010). The epistemology of mathematical and statistical modeling: A quiet

methodological revolution. American Psychologist, 65(1), 1-12.

Rodgers, J. L., & Doughty, D. (2001). Does having boys or girls run in the family? . Chance, 14,

8-13.

Rodgers, J. L., & Rowe, D. C. (1993). Social contagion and adolescent sexual behavior: A

developmental EMOSA model. Psychological Review, 100, 479–510. doi:10.1037/0033–

295X.100.3.479

Rowe, D. C., & Rodgers, J. L. (1991). Adolescent smoking and drinking—are they epidemics?

Journal of Studies on Alcohol, 52, 110–117.

Scheerens, J., & Blömeke, S. (2016). Integrating teacher education effectiveness research into

educational effectiveness models. Educational Research Review, 18, 70-87.

doi:https://doi.org/10.1016/j.edurev.2016.03.002

Seel, N. M. (2009). Bonjour tristesse: Why don’t we research as we have been taught?

Methodological considerations on instructional technology research. Technology,

Instrustion, Cognition and Learning, 6, 151-176.

Shadish, W., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental

designs for generalized causal inference. Boston, MA: Houghton Mifflin.

32
Shepard, R. N. (1982). Geometrical approximations to the structure of musical pitch.

Psychological Review, 89, 305-333.

Siemens, G. (2004). Connectivism: A learning theory for the digital age. International Journal of

Instructional Technology and Distance Learning, 2, 1-8; Available from

http://www.itdl.org/Journal/Jan_05/article01.htm

Simon, H. A., & Newell, A. (1956). Models: Their uses and limitations. In D. White (Ed.), The

state of the social sciences, 61–83. Chicago: University of Chicago Press.

Straub, D. W. (2009). Editor’s comments: Why top journals accept your paper. MIS Quarterly,

33(3), iii–x.

Suppes, P. (1978). Impact of research on education: Some case studies. Washington, DC:

National Academy of Education.

Tatto, M. T., Schwille, J., Senk, S., Ingvarson, L., Peck, R., & Rowley, G. (2008). Teacher

Education and Development Study in Mathematics (TEDS-M): Policy, practice, and

readiness to teach primary and secondary mathematics. Conceptual framework. East

Lansing, MI: Teacher Education and Development International Study Center, College of

Education, Michigan State University.

Tipton, E., & Olsen, R. B. (2018). A Review of Statistical Methods for Generalizing From

Evaluations of Educational Interventions. Educational Researcher, 47(8), 516-524.

doi:10.3102/0013189x18781522

Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of

information technology: Toward a unified view. MIS Quarterly, 27(3), 425-478.

33
 Venkatesh, V., Thong, J. Y. L., & Xu, X. (2016). Unified theory of acceptance and use of

technology: A synthesis and the road ahead. Journal of the Association for Information

Systems, 17(5), 328-376.

Wedemeyer, C.A. (1981). Learning at the back door: Reflections on non-traditional learning in

the lifespan. Madison, WI: The University of Wisconsin Press.

Wenger, E. & Lave, J. (1991). Learning in doing: Social, cognitive and computational

Perspectives. Situated learning: Legitimate peripheral participation. Cambridge:

Cambridge University Press.

Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. Cambridge:

Cambridge University Press.

Willingham, D. (2008). What is developmentally appropriate? American Educator, 32, 2, 34-39.

Zawacki-Richter, O., Bäcker, E. & Vogt, S. (2009). Review of distance education research (2000

to 2008): Analysis of research areas, methods, and authorship patterns. International

Review of Research in Open and Distributed Learning, 10(6), 21–50.

https://doi.org/10.19173/irrodl.v10i6.741

Zimmerman, B. J. (2001). Theories of self-regulated learning and academic achievement: An

overview and analysis. In B. J. Zimmerman & D. H. Schunk (Eds.), Self-Regulated

Learning and Academic Achievement. Theoretical Perspectives. 1-38. Erlbaum, NJ:

Mahwah.

34

View publication stats