Article

An Experimental Study on the Effectiveness of Students’

Learning in Scientific Courses through Constructive
Alignment—A Case Study from an MIS Course
Hongfeng Zhang 1,*, Shaodan Su 1,*, Yumeng Zeng 2 and Johnny F. I. Lam 1

1 Faculty of Humanities and Social Sciences, Macao Polytechnic University, Macao 999078, China;
filam@ipm.edu.mo
2 Faculty of Education, The University of Hong Kong, Hong Kong 999077, China; u3584752@connect.hku.hk

* Correspondence: hfengzhang@ipm.edu.mo (H.Z.); fsussd@163.com (S.S.)

Abstract: This study aimed to explore students’ effectiveness in scientific courses that have adopted
the framework of constructive alignment. The researchers conducted an experimental study in the
education sector to compare two different teaching models—traditional and following constructive
alignment—and used statistical tools to analyse differences in students’ learning effectiveness. The
course “Management Information System” (MIS) was specifically chosen to investigate how con-
structive alignment initiatives used in the course influenced students’ learning effectiveness. Two
groups of students were selected as the control group and the experimental group, respectively. In
the experimental group, the intended learning outcomes, teaching and learning activities, and as-
sessment tasks were always aligned by the instructor. The learning effectiveness of the two groups
was evaluated by the Course Experience Questionnaire Survey and academic grades. After this
Citation: Zhang, H.; Su, S.; Zeng, Y.;
study, it was found that significant improvement—in terms of students’ learning experience scores
Lam, J. F. I. An Experimental Study
and academic grades—was seen in the experimental group compared with the control group. This
on the Effectiveness of Students’
study has further verified that implementing a constructive alignment template can significantly
Learning in Scientific Courses
through Constructive Alignment—A
improve students’ learning effectiveness in scientific courses, hence providing theoretical and prac-
Case Study from an MIS Course. tical references for teaching and learning in scientific courses.
Educ. Sci. 2022, 12, 338.
https://doi.org/10.3390/ Keywords: constructive alignment; intended learning outcomes; learning effectiveness; science
educsci12050338 course; teaching experiment; management information system

Academic Editors: Han Reichgelt

and Kelum A.A. Gamage

Received: 11 February 2022 1. Introduction

Accepted: 10 May 2022
With more countries and regions entering the era of massification of higher educa-
Published: 11 May 2022
tion, more attention is being paid to the teaching quality of institutions. Good teaching is
Publisher’s Note: MDPI stays neu- believed to help cultivate students’ high-level cognitive skills that elite students can spon-
tral with regard to jurisdictional taneously use [1]. However, students attending universities today are no longer only the
claims in published maps and institu-
academic elite; they share classrooms and study spaces with students who are of different
tional affiliations.
statuses (p. 192, [2]). Therefore, higher education institutions demand paradigmatic and
epistemic change brought about by transformative learning [3,4]. Previous studies have
also argued that in the process of teaching and learning, learning outcomes should be the
product of students’ activities and experiential processes rather than teachers’ instruction
Copyright: © 2022 by the authors. Li-
alone; students should construct knowledge and meaning initiatively [5,6]. Although the
censee MDPI, Basel, Switzerland.
This article is an open access article
university teaching model is gradually shifting focus from instructors’ teaching to stu-
distributed under the terms and con-
dents’ learning, there are still many obstacles. Much empirical research remains at the
ditions of the Creative Commons At- institutional level and lacks specific analysis of the characteristics of the curriculum [7,8].
tribution (CC BY) license (https://cre- In response to this shift, John Biggs proposed constructive alignment principles for
ativecommons.org/licenses/by/4.0/). higher education in 1996 [9]. Constructive alignment aims to provide the majority of stu-

Educ. Sci. 2022, 12, 338. https://doi.org/10.3390/educsci12050338 www.mdpi.com/journal/education

Educ. Sci. 2022, 12, 338 2 of 16

dents with a highly challenging and supportive academic environment [2]. The frame-
work involves two aspects of theoretical connotations. First, from the instructors’ perspec-
tive, alignment is manifested as an instructional design with intended learning outcomes,
teaching and learning processes, and assessment tasks. The teaching and learning activi-
ties, designed by instructors to support and guide students with the intended learning
outcomes, can be proven to be efficient through students’ participation in later assessment
tasks [10]. Second, from the students’ perspective, learners are expected to follow the con-
structive approach to learning; they are to actively construct knowledge and meaning by
changing their cognitive thinking modes to assimilate new knowledge, information, and
methods [11]. Therefore, constructive alignment principles focus on the following ques-
tions in the instructional design of the curriculum: what are the specific learning outcomes
that students need to achieve in their learning process? To achieve these intended out-
comes, what learning activities and learning experiences should students have? How do
teachers determine if their students have learned effectively?
Biggs focused on constructing a student-centred learning environment using the
framework of constructive alignment principles—including presage, process, and prod-
uct—without deliberately discriminating between humanities and social sciences in terms
of curriculum nature and models of teaching and learning. It is generally believed that in
humanities and social sciences courses, an autonomous learning atmosphere for students
is more likely to form with the involvement of learning outcomes. Additionally, the teach-
ing and learning methods can be more diversified [8,12]. Conversely, due to the nature of
their curriculum, scientific courses tend to place more emphasis on deterministic
knowledge results. Billett [13] points out that teaching and learning are commonly re-
graded as synonyms; the absence of teaching would impact the quality of the learning
experience. Based on the information above, this study intends to explore the effectiveness
of students’ learning in scientific courses using constructive alignment principles. This
study conducted an experimental study in the education sector to compare two different
teaching models—traditional and following constructive alignment—and used statistical
tools to analyse differences in students’ learning effectiveness. The results provide theo-
retical and practical references for teaching and learning in scientific courses.

2. The Framework of Constructive Alignment Principles

2.1. Intended Learning Outcomes
The first step of constructive alignment is to devise intended learning outcomes [14].
University-level dialogues have increasingly focused on students’ learning outcomes, in-
volving students’ skills, competencies, continuous improvement, retention, etc. [15]. Cen-
tral to the idea of intended learning outcomes is the belief that education should be
planned around the competence that students intend to develop rather than the content
that teachers intend to teach (p. 2295, [16]). Biggs and Tang proposed the Structure of Ob-
served Learning Outcomes Taxonomy (SOLO Taxonomy, 1982) to promote students’
learning from surface to deeper levels and clarify learning outcomes for students’ under-
standing levels. This taxonomy was based on Bloom’s taxonomy of educational objectives.
It divided students’ understanding levels into five categories from low to high: prestruc-
tural, unistructural, multistructural, relational, and extended abstract. Accordingly, it is
important to wisely select suitable verbs to describe students’ future competencies in a
subject related to developing the intended learning outcomes [17]. Corresponding verbs
reflect the gradation of setting learning outcomes from lower-order to higher-order think-
ing, demonstrating the increasingly challenging process of thinking.
With this system, previously randomly used verbs with relatively vague meanings—
such as comprehend, appreciate, know, understand, recognise, and others—are replaced
by verbs with more explicit meanings, such as identify, analyse, apply, interpret, compare,
critique, or evaluate. Intended learning outcomes composed using these verbs can better
Educ. Sci. 2022, 12, 338 3 of 16

reflect the specificity of intended learning activities and the richness of knowledge con-
tent. Furthermore, these learning outcomes could also help students to answer the ques-
tion of what they should be able to demonstrate at the end of learning experiences. These
student responses, in turn, provide enlightenment for educators, enabling them to con-
duct more student-centred teaching practices as encouraged by constructive alignment (p.
125, [18]).

2.2. Teaching and Learning Activities

When intended learning outcomes are established, teaching activities need to be con-
sistent with the intended learning outcomes to ensure that they are directly related to
learning. Constructivism implies students’ ability to actively construct knowledge and
meaning, form opinions and connections, and develop skills and abilities during learning.
Non-elite students in college classrooms show relatively passive coping mechanisms and
exhaustive manners of learning when compared to their elite counterparts’ learning initi-
atives. Therefore, teaching and learning activities in college should provide richer learn-
ing environments and more organised activities to enable students to stimulate their ac-
tive learning and constructive consciousness. According to Hailikari [11], formative teach-
ing assessments that require students’ active participation clearly encourage students to
adopt more sophisticated learning methods; meanwhile, the opposite is true for more tra-
ditional courses (p. 1).
Thus, students’ positive behaviours when making learning decisions are more im-
portant than teachers’ behaviours [19]. The term “teaching and learning activities” is
adopted by constructive alignment because it can better demonstrate the relationship be-
tween students and teachers in the classroom [1]. With constructive alignment as the basis,
teaching and learning activities have promoted the construction of teacher–student learn-
ing communities. In this learning environment, during the teacher–student interaction,
teachers give students timely feedback according to the different dimensions and stand-
ards of observation. They adjust students’ proficient learning content and intended learn-
ing outcomes to provide students with enlightening and challenging learning tasks, ena-
bling them to conduct their learning innovatively. Simultaneously, some view these learn-
ing interactions as formative assessment processes that help students improve their own
self-regulated learning through assessment [20].

2.3. Assessment Tasks

At the final stage, assessment tasks of a constructive alignment model must be con-
sistent with the teaching activities and intended learning outcomes. Sally Brown [21] has
pointed out that assessment tasks must be authentic to meet the corresponding intended
learning outcomes and should relate to one or several of the outcomes directly [22]. Ad-
ditionally, the assessment tasks during the learning process should be more engaging than
the post-instructional assessment. From the perspective of students, the assessment
should define the actual content of the curriculum and what teachers intend to teach
(p.187, [23]). This view corresponds with Elton’s belief that the assessment, rather than
curriculum objectives or curriculum content, determines students’ learning content and
method[24]. From the teachers’ perspective, the intended learning outcomes are seen as
the central pillars of the aligned teaching system, and the summative assessment is the
final stage of teaching–learning events. This viewpoint differs from the perspective of stu-
dents, who hold that the assessment tasks begin with the teaching activities [14].
Many college instructors are still accustomed to norm-referenced assessments when
choosing or designing a final assessment, signifying that education is selective. Students’
results are always compared to those of their peers and the normally distributed curve of
students’ grades. Alternatively, within the concept of constructive alignment, a criterion-
referenced assessment aligned with the intended learning outcomes is identified as the
most suitable assessment method. This style of assessment can also help students make
positive, meaningful choices that help them enhance their learning [25].
Educ. Sci. 2022, 12, 338 4 of 16

3. Literature Review
3.1. Constructive Alignment
The literature related to constructive alignment involves a range of professional dis-
ciplines [1,3,4,6,8,26,27], because the purpose of constructive alignment is to promote the
improvement in students’ learning effectiveness across different courses. Although the
importance of constructive alignment has been widely recognised, there are few studies
on the effectiveness of its implementation. Many of these studies are purely theoretical,
focusing on topics such as assessing learning outcomes in interactive learning spaces [28],
exploring the relationship between constructive alignment and different criteria levels
[29], discussing the international vision of intended learning outcomes in programmes
[30], or examining the different theoretic models of constructive alignment in different
contexts [1,31]. However, some of the other research studies directly explore the applica-
tion of constructive alignment in scientific courses [32–34].

3.2. Empirical Research on Constructive Matching

Many countries, such as Malaysia, have already begun implementing empirical re-
search on constructive alignment. However, most of the implementation in these coun-
tries remains at the institutional level, for example, studying the matching extent of the
curriculum systems and graduate capabilities of institutions [22,35]. From the micro-level,
there are other various types of research. The constructive alignment model was used in
Harvey and Kamvounias’s study [36] to investigate how the content of subject outlines
could be consistent with graduate attributes. Teachers’ personal experiences were exam-
ined to determine the achievement degree of teaching assessments and intended learning
outcomes. However, before implementing this operation, subject coordinators were asked
to “tick” some suitable learning outcomes from a list related to the graduate attributes.
This led to the similarity of many choices about subject learning outcomes and a lack of
analysis of subjects’ characteristics. This task also failed to distinguish humanities and
social sciences subjects from scientific courses. In Sumison and Goodfellow’s study [37], a
detailed constructive alignment template was adopted to map the relationship between
curriculum and generic skills required by students in the Bachelor of Education pro-
gramme. This study was in depth and meticulous, encouraging academic staff to reflect
on their teaching practice. However, the researchers did not consider students’ learning
outcomes. Additionally, only education courses were studied, while scientific courses
were ignored in the scope of research. Furthermore, Simper [29] used thematic analysis to
discover some troublesome aspects from the perspective of academic staff, including in-
structor expectations, lack of consistency, differentiation of performance, and more. Con-
tributing to this research, the present study focused on assessment thresholds of construc-
tive alignment and differentiation of standards. However, it still lacked a holistic explora-
tion and the perspective of students.

3.3. Research on Constructive Alignment from the Perspective of Students

Treleaven and Voola [38] studied the effectiveness of the implementation of construc-
tive alignment from the students’ perspective. The researchers integrated graduate attrib-
utes with a post-graduate course in marketing, using the matrix map to record the match-
ing degree of graduate attributes with intended learning outcomes, learning activities,
and assessment tasks. Students’ grades and feedback data (both quantitative and qualita-
tive) were used to evaluate the effectiveness of the study. However, this study still had
limitations, such as only having an experimental group without a control group, which
resulted in the unclear causation of “changes of students’ performance and satisfaction”
with constructive alignment implementation [38]. In another study, Higgins et al. [39]
used the online questionnaire to investigate the experience of level-5 students in Research-
informed Teaching (RiT) activities. Based on the analysis of the questionnaire, it sup-
ported the effectiveness of this RiT activity within the curriculum in terms of student
Educ. Sci. 2022, 12, 338 5 of 16

learning and research skill development. However, this study did not consider the adop-
tion of standard experimental methods.

3.4. The Application of Constructive Alignment in Science Courses

In contrast, there is a lack of empirical research on the impact of constructive align-
ment on students’ learning effectiveness in scientific education. Brabrand and Dahl [40]
used the method of comparison to investigate the proportion of learning outcomes with
different understanding levels in SOLO taxonomy when setting intended learning out-
comes for courses such as computer science, mathematics, and other natural sciences.
However, this study did not analyse the achievement effect of different learning out-
comes. Later, Maxworth [41] used an extended constructive alignment model in an engi-
neering electromagnetics course to establish three levels of learning outcomes and evalu-
ation methods. He elaborated on the implementation paths and advantages of this study
simultaneously in detail. However, this model lacked rigorous data verification. In a study
by Gynnild et al. [42], researchers matched the project exercises with the tasks of the final
examination in science and engineering courses, focusing on the specific problems of con-
cepts and algorithms. Results show that students’ ability to cope with conceptual and cal-
culative questions improved significantly in this study. Although this study gathered ob-
servations of specific empirical data, it lacked the discussion of the overall model of con-
structive alignment. A field study in the second year of a self-retained undergraduate
course in the Geology (BSc) programme at Royal Holloway University has proven con-
structive alignment’s desirability and typical achievement [43]. Obada et al. [34] continu-
ously investigated how science and engineering courses could be taught more effectively
in their universities. After the initial deep dive into a few courses taught by the science
and engineering faculties of selected Nigerian universities, the researchers adopted a re-
verse design approach in five case studies. However, two of these did not use an experi-
mental method.
This study, with the reference of the studies above and the acquisition of data and
research perspectives, tried to explore the effectiveness of the constructive alignment prin-
ciple through teaching experiments. The research hypothesis was as follows: compared to
the traditional model, implementing constructive alignment strategy in scientific courses
can significantly improve students’ learning effectiveness.

4. Methodology
This study adopted an experimental methodology. The course “Management Infor-
mation System” (MIS) at a tertiary institution in a southern city of mainland China was
selected as the implementation case based on the research objectives. The MIS course at
this university is a science-oriented, interdisciplinary course. It is offered in the third se-
mester of the undergraduate study, which involves different disciplines, such as computer
science management, database technology, operations research, communication, and net-
work technology. This study used the experimental group to explore the impact of imple-
menting a constructive alignment theoretical framework on students’ learning effective-
ness in the MIS course.

4.1. Experimental Procedure

Step 1: Select the experimental and control groups. The MIS course is a large course
consisting of several small classes from the same grade. This study chose two successive
MIS courses as the experimental groups. A total of 108 students (34 males and 74 females)
from the class of 2019 served as the control group (3 small classes were combined), and 62
students (18 males and 44 females) from the class of 2020 served as the experimental group
(2 small classes were combined).
Educ. Sci. 2022, 12, 338 6 of 16

Step 2: Control the irrelevant variables. To achieve better experimental results, this
study attempted to control all variables other than the independent variable (implemen-
tation of constructive alignment principle) and dependent variable (learning effective-
ness) as much as possible. First, researchers aimed to accomplish this by ensuring the age
and proportion of male and female students in the two courses were roughly the same.
Secondly, these students primarily originated from the same province in southern China,
and their college entrance examination scores fell mainly between the score requirements
of attending first-tier and second-tier universities. Thirdly, the courses were taught by the
same instructor—who used the same teaching topics, materials, and reference data—and
the total amount of instructional and assessment time was 18 weeks. Finally, the assess-
ment criteria of the course were reviewed and verified by the programme coordinator and
other academic staff, which helped counter the inconsistency of grading standards during
the performance evaluation process.
Step 3: Implement the experiment. In the control group, the teacher adopted the tra-
ditional teaching style based on the courseware and supplemented by classroom Q & A,
computer practice, and other supplementary instructional methods. There were only two
assessments throughout the semester: a mid-term computer experiment and the final test.
Students’ final grades were given as a combination of their scores on these two assess-
ments and their usual classroom performance.
Alternatively, the teacher in the experimental group adopted constructive alignment
teaching templates. The instructor started the course by introducing and explaining the
abstract of curriculum contents, the concrete intended learning outcomes, teaching and
learning activities, and all formative and summative assessment tasks; simultaneously,
they answered students’ questions.
Step 4: Set up the measurement methods. The objective of this study was to investi-
gate the difference in learning effectiveness between the control group and the experi-
mental group. The primary measuring methods were considered from the students’ per-
spective with two variables: students’ course learning experience scores and their aca-
demic performances. Using both students’ course learning experience scores and their ac-
ademic performances as learning outcomes variables, the validity of the constructive
alignment can be understood not only from students’ subjective experiences but also from
the quantified scores. Therefore, to improve the credibility of this research, the qualitative
and quantitative analyses are combined.
Step 5: Conduct statistical analysis. The experimental results were analysed and pro-
cessed using IBM SPSS Statistics 26 so final discussions and the formation of conclusions
could be carried out.

4.2. Description of the Experimental Group Variable Manipulation

The research team communicated with the instructor before the experiment. Re-
searchers informed the teacher about the principle and template of the constructive align-
ment but let the instructor set intended learning outcomes, organise instruction, and ar-
range assessments by himself.
The core feature of the constructive alignment template is to set and implement the
intended learning outcomes. According to Biggs’s SOLO taxonomy, the instructor used
“concrete” verbs to describe intended learning outcomes in the experimental group to
express cognitive categories and levels that students needed to achieve, such as identify,
describe, compare, interpret, generalise, classify, reflect, apply, evaluate, design, create,
and more. Here, learning outcomes were divided into three categories: technological, in-
formational, and managemental, which reflect the increasing degree of challenge faced by
students in terms of cognitive levels. For example, when the teaching objective in the con-
trol group was to “understand the theoretical basis of MIS and to appreciate the process
and methods of system analysis and design”, the intended learning outcomes were
changed to “elaborate the theoretical basis of MIS and apply the system analysis methods
and design into actual situations”. In addition, the instructor also adopted several levels
Educ. Sci. 2022, 12, 338 7 of 16

of intended learning outcomes for different stages of instruction. It is argued that, as teach-
ing projects advance from the primary to the advanced levels, the breadth of knowledge
used in the implementation of instructional plans would be adjusted accordingly, and the
integration of knowledge would also become more purposeful [44].
Compared to the control group, the teaching and learning activities of the experi-
mental group focused more on a student-centred approach, emphasising the formation of
a teacher–student learning community and insisting that activities align with intended
learning outcomes. In the learning process, the experimental group adopted a case-driven
learning method with the help of a flipped classroom. Students were asked to read case
materials before classes and formed cooperative learning groups during class time. With
the framework of “research skill development, RSD”, teachers guided students in devel-
oping research skills during the learning process of each case study [45]. The experimental
group also developed simulated projects to align learning with intended learning out-
comes. For example, for the intended outcome of analysing the system business process,
students were first asked to describe their phase of information processing. Then, the
teacher would generalise their language. With the help of the teacher’s adjustment and
supplemental elements, students formed the prototype of the business process and com-
pleted the flowchart. Because students’ enthusiasm was fully mobilised in this process,
they wanted to participate positively. The project achieved the intended learning out-
comes of interpreting, analysing, and demonstrating the business needs of enterprises
with the help of the integrated use of case analysis, problem guidance, situation design,
group project cooperation, and other comprehensive methods under the collaborative
community. Earlier studies also investigated the role of cooperative learning environ-
ments in improving students’ higher-order skills [46] and enhancing critical thinking and
reasoning abilities [47]. Case-based learning with cooperation has also long been proven
to facilitate student discussion and improve their advanced cognitive reasoning skills [48].
Teaching and learning activities in the experimental group were committed to providing
a learning atmosphere aligned with their intended learning outcomes.
Formative independent and cooperative learning activities were provided as assess-
ment tasks for the experimental group. While the formative assessment focused on teacher
feedback on different perspectives and students’ improvement, students’ learning process
was also viewed as belonging to the teachers’ assessment process. Additionally, since stu-
dents would achieve one or several learning outcomes during the learning process, stu-
dents’ continuous assessment grade was formed by two parts of usual performance (at-
tendance, motivation, etc.) and independent learning and cooperative practice (cases, pro-
jects, integrated application, etc.). Teachers focused on students’ achievement of academic
tasks rather than just simply grading [49]. The comprehensive case analysis was con-
ducted as the summative assessment. A criterion-referenced assessment was adopted, set-
ting five levels (excellent, good, medium, passed, and failed) according to different eval-
uative dimensions. The corresponding levels students could attain were determined by
their performance scores and were not established in advance based on the normal distri-
bution curve [50].

4.3. Measurement and Statistical Analysis

The MIS Course Experience Questions (online), with reference to the more mature
student rating scale, presented questions divided into two parts: basic information and
scale testing (5-point Likert scale, see item in Table 1).
Educ. Sci. 2022, 12, 338 8 of 16

Table 1. Basic information of participating students in the experimental and control groups.
Course of Scores of College Entrance Examina- Year of
Gender Age Place of Residence
Study tion Study
Between the
Above the Threshold Score
(19–20 Years Threshold Score of First-Tier Uni- The Second Guang Outside
Male Female MIS
Old) of First-Tier Uni- versities and Sec- Year Dong Guang Dong
versities ond-Tier Univer-
sities
Control group size 31 73 104 104 15 89 104 104 0
Experimental group
14 41 55 55 6 49 55 52 3
size

In the basic information section, the questions gathered information on students’

grades, gender, places they took the college entrance examination, and their scores. The
scale testing part of the questionnaire contained 16 questions about intended learning out-
comes, teaching and learning activities, and learning assessments for students to rate
based on the principle of constructive alignment. Each question item was treated inde-
pendently in the statistical analyses since each had addressed specific elements of the
three components of constructive alignment. Therefore, the results of the statistical anal-
ysis helped us determine whether the implementation of constructive alignment was ef-
fective or not in each specific section.
After obtaining consent from the instructor and class counsellors, the researcher
asked students from the control and experimental groups to complete these question-
naires in Wechat after finishing the course. The study ran from September 2020 to January
2022, lasting more than one year for two classes of students. In January of 2022, after the
experimental group completed the course, all the collected data were summarised and
statistically analysed to determine the difference between students’ experiences in the
control and experimental groups.
Additional evidence of the effectiveness of constructive alignment was found in the
final assessment score of the two groups of students. Since the overall score distribution
of the experimental group was unknown or skewed after implementing the constructive
alignment framework, it was an appropriate choice to convert the students’ specific scores
into ordinal variables with good robustness. Therefore, the academic performance was
assigned as follows: 59 or below = 1, failed; 60–69 = 2, passed; 70–79 = 3, medium; 80–89 =
4, good; 90 or above = 5, excellent. This study used the nonparametric statistical approach
of ordinal variables to investigate the differences between the experimental and control
groups and the changing trend of the proportion of students at all levels.

5. Results
After excluding five invalid questionnaires, the effective response rates of the two
groups that completed the students’ Course Experience Questions were 96.3% and 88.7%.
Irrelevant variables, such as age, gender percentage, place of residence, and college en-
trance examination scores, were controlled for with the participants’ characteristics in Ta-
ble 1. This study adopted exploratory descriptive statistics based on different groups and
tested for normality using the Shapiro–Wilk test (Table 2). The statistical results show that
the kurtosis and skewness of the scores of each question item in different groups followed
the normal trend; the p-value of the S-W statistics was mostly greater than 0.05, indicating
that the first and second sample groups met the assumption of normality, respectively.
The results in Table 3 show that the average scores of the 16 items improved. After
an independent sample t-test, when p < 0.05 except for “the teacher returns assignments
promptly” and “the teacher is enthusiastic about his/her subject”, the scores of other items
had a statistically significant improvement. The statistically significant Cohen’s d measure
ranged from 0.34 to 0.67, with a relatively high effect, indicating a larger difference be-
Educ. Sci. 2022, 12, 338 9 of 16

tween the experimental and control groups. The average scores of two items demonstrat-
ing no statistically significant improvement also showed a positive direction trend. In ad-
dition, the standard deviation of each item score of the experimental group was smaller
compared to the control group. This finding indicates that the statistical dispersion of stu-
dents’ scoring in the experimental group was smaller, and the consistency of course expe-
rience was higher.

Table 2. Exploratory descriptive statistics for different groups.

Question Items Group N M SD Mdn Skewness Kurtosis S.W. (p)
1. The teacher 1 104 3.770 0.895 4 −0.234 −0.347 0.287
proposes teach-
ing objectives or
learning out-
2 55 4.040 0.693 4 −0.214 −0.483 0.053
comes at the be-
ginning of the
course.
2. The learning 1 104 3.680 1.017 4 −0.222 −0.389 0.156
outcomes of
each class are
2 55 4.020 0.680 4 −0.357 −0.412 0.112
specific and
clear.
3. Students can 1 104 3.630 1.025 4 0.389 0.370 0.172
understand the
teacher’s teach-
ing intentions in 2 55 4.020 0.782 4 −0.059 −0.759 0.032
the learning
process.
4. The teacher 1 104 3.580 0.972 3 0.028 0.107 0.258
encourages
2 55 3.960 0.719 4 −0.055 −0.734 0.043
class discussion.
5. The teacher 1 104 3.400 1.010 3 0.389 0.370 0.172
invites
students to
share their 2 55 3.980 0.828 4 −0.059 −0.559 0.075
knowledge and
experiences.
6. The teacher 1 104 3.860 0.989 4 0.136 0.198 0.356
returns assign-
ments 2 55 4.020 0.652 4 −0.089 −0.625 0.033
promptly.
7. The teacher 1 104 3.560 1.003 3 −0.334 −0.265 0.254
invites com-
ments on
2 55 4.040 0.719 4 −0.251 −0.312 0.124
his/her own
ideas.
8. The teacher 1 104 3.760 0.930 4 0.278 0.262 0.122
has a genuine
interest in stu- 2 55 4.110 0.762 4 −0.074 −0.654 0.042
dents.
9. The teacher 1 104 3.520 0.924 3 −0.133 −0.343 0.223
relates to stu-
dents as indi- 2 55 3.890 0.832 4 −0.254 −0.385 0.087
viduals.
10. The teacher 1 104 3.910 0.936 4 0.112 0.232 0.168
is enthusiastic
about his/her 2 55 4.160 0.739 4 −0.037 −0.725 0.023
subject.
11. The learning 1 104 3.550 0.994 3 0.156 0.226 0.321
modules are
2 55 4.050 0.650 4 −0.178 −0.325 0.134
well organized.
12. The learning 1 104 3.560 1.032 3 0.088 0.207 0.245
modules are 2 55 4.040 0.693 4 −0.066 −0.456 0.088
Educ. Sci. 2022, 12, 338 10 of 16

useful in en-
hancing my un-
derstanding of
the subject/ana-
lytic abil-
ity/practical
skills.
13. The various 1 104 3.600 0.950 3.5 0.333 0.245 0.243
learning mod-
ules’ compo-
nents (e.g., lec-
tures, tutorials, 2 55 4.050 0.756 4 −0.025 −0.418 0.078
seminars, etc.)
are well inte-
grated.
14. The relative 1 104 3.530 1.070 4 −0.185 −0.456 0.093
weightings of
learning mod-
ules assessment
(e.g., assign- 2 55 4.110 0.685 4 −0.062 −0.215 0.154
ments, tests, ex-
ams, etc.) are
appropriate.
15. The course 1 104 3.630 0.996 4 0.142 0.332 0.068
has adopted an
assessment
method that
combines quan- 2 55 4.040 0.693 4 −0.055 −0.764 0.036
titative scoring
and qualitative
feedback.
16. The course 1 104 3.540 1.023 3 0.288 0.270 0.164
assessment can
reflect students’
2 55 4.110 0.658 4 −0.049 −0.325 0.066
learning pro-
cess.
Note: M = mean; SD = standard deviation; Mdn = median; S. W. = Shapiro–Wilk statistic.

Table 3. Comparison of students’ learning experience in MIS.

Experimental
Control Group t
Question Items Group df Cohen’s d
M(SD) (* p < 0.05)
M(SD)
1. The teacher proposes teaching objec-
tives or learning outcomes at the begin- 3.770 (0.895) 4.040 (0.693) 2.084 * 136 0.34
ning of the course.
2. The learning outcomes of each class are
3.680 (1.017) 4.020 (0.680) 2.476 * 148 0.39
specific and clear.
3. Students can understand the teacher’s
teaching intentions in the learning pro- 3.630 (1.025) 4.020 (0.782) 2.634 * 137 0.43
cess.
4. The teacher encourages class discussion. 3.580 (0.972) 3.960 (0.719) 2.844 * 140 0.44
5. The teacher invites students to share
3.400 (1.010) 3.980 (0.828) 3.874 * 130 0.63
their knowledge and experiences.
6. The teacher returns assignments
3.860 (0.989) 4.020 (0.652) 1.240 149 0.20
promptly.
7. The teacher invites comments on his/her
3.560 (1.003) 4.040 (0.719) 3.465 * 143 0.55
own ideas.
Educ. Sci. 2022, 12, 338 11 of 16

8. The teacher has a genuine interest in

3.760 (0.930) 4.110 (0.762) 2.545 * 130 0.41
students.
9. The teacher relates to students as indi-
3.520 (0.924) 3.890 (0.629) 2.495 * 157 0.47
viduals.
10. The teacher is enthusiastic about
3.910 (0.936) 4.160 (0.528) 1.718 157 0.33
his/her subject.
11. The learning module is well organized. 3.550 (0.994) 4.050 (0.650) 3.863 * 150 0.60
12. The learning module is useful in en-
hancing my understanding of the sub- 3.560 (1.032) 4.040 (0.693) 3.476 * 148 0.55
ject/analytic ability/practical skills.
13. The various learning module compo-
nents (e.g., lectures, tutorials, seminars, 3.600 (0.950) 4.050 (0.565) 3.320 * 133 0.58
etc.) are well integrated.
14. The relative weightings of learning
modules’ assessment (e.g., assignments, 3.530 (1.070) 4.110 (0.592) 4.151 * 151 0.67
tests, exams, etc.) are appropriate.
15. The course has adopted an assessment
method that combines quantitative scor- 3.630 (0.996) 4.040 (0.513) 2.973 * 145 0.52
ing and qualitative feedback.
16. The course assessment can reflect stu-
3.540 (1.023) 4.110 (0.658) 4.262 * 151 0.66
dents’ learning process.
Note: M = mean; SD = standard deviation.

When deciding how to measure the second learning effectiveness variable, research-
ers considered that the ordinal variable represented the student’s performance. Therefore,
the study adopted the Nonparametric Sum of Ranks Tests (Mann–Whitney Test) to study
the changes in the performances of the experimental and control groups.
The statistical results in Table 4 show the value of ASM. Sig. (two-tailed) was 0.010,
less than 0.05, proving that the academic performance of the experimental group had a
statistically significant improvement compared to that of the control group.
To further verify the specific differences between various groups of students in dif-
ferent grades, the study examined the differences between the experimental and control
groups in each grade; this was determined using Pearson chi-square statistics after
weighting the number of percentages for each grade (Table 5). The statistical results show
that the significant differences between the groups were mainly in the good grade (p =
0.043 < 0.05), while the differences in the other grades were not that significant.

Table 4. NPar Sum of Ranks Test Statistics.

Asymp. Sig.
Mann–Whitney U Wilcoxon W Z
(2-Tailed)
Value 2577.500 8463.500 −2.586 0.010

As shown by the sample data in Figure 1, compared to the control group, the exper-
imental group showed a decrease in the proportion of students in the low-scoring band
(failing, passing, and moderate) and an increase in the proportion of students in the high-
scoring band (good and excellent). Together with the statistically significant increase in
good grades, these findings ensure a significant increase in the experimental group com-
pared to the control group.
Educ. Sci. 2022, 12, 338 12 of 16

Table 5. Chi-square test for each grade between the two groups.

Comparison of Each
Chi-Square Test
Grade between the TOTAL N df Asymp. Sig. (2-Tailed)
Statistic
Two Groups
Excellent 10 3.600 1 0.058
Good 55 4.091 1 0.043
Medium 63 1.921 1 0.166
Passed 48 0.083 1 0.773
Failed 24 2.667 1 0.102

35.48% 37.00%

25.81% 25.00% 22.58%

20.37%
15.74%
1.85% 8% 8.06%

EXCELLENT GOOD MEDIUM PASSED FAILED

Control group Experimental group

Figure 1. Comparison of the proportion of students in each grade.

6. Discussion
The experiment was designed to observe the impact of the constructive alignment
initiative on student learning effectiveness. The connotation of constructive alignment is
composed of three intentionally aligned elements: intended learning outcomes, construc-
tive nature of teaching and learning processes, and learning assessments. For the specific
results of the experiment, this study was interpreted according to these three components.
The results show that after the implementation of constructive alignment, the students in
the experimental group showed significant improvement in their academic performance
and learning experience scores compared with the control group. After controlling for
other variables’ influence as much as possible, this result better reflected the effectiveness
of implementing constructive alignment and verified the earlier hypothesis proposed by
this study.
After examining the results of students’ intended learning outcomes (from item 1 to
item 3), the experimental group gave a clear, satisfied response towards learning out-
comes and the teaching intentions of teachers. The central element of constructive align-
ment is setting intended learning outcomes and using Bloom’s and SOLO taxonomies as
frames of reference [51]. Previous research has confirmed that using specific verbs for in-
tended learning outcomes can be closely related to learning activities and assessments
[15,16,18]. The statistically significant improvement in the scores of corresponding items
also showed that students could lead their own learning using specific verbs in their stud-
ies, “relating” and “extending” their knowledge and abilities to a higher level.
The curriculum design in the teaching and learning processes included forms of
learning, teaching methods, teacher–student relationships, course learning, structures,
and other aspects. The primary goal of constructive alignment is to keep teaching and
learning activities aligned with intended learning outcomes and form a learning commu-
nity. This study adopted case and project teaching methods used in the teaching and
learning process, hoping to stimulate students to take the initiative in knowledge con-
struction. Simultaneously, formative assessments involving feedback and improvement
were added to classroom teaching, strengthening the viewpoint that “assessment activi-
Educ. Sci. 2022, 12, 338 13 of 16

ties are the primary learning activities in higher education” [52] and emphasising the im-
portance of integrating formative assessment with teaching and learning activities. Some
studies have provided empirical examples of how formative assessment may influence an
individual student’s use of self-regulated learning skills or criteria [20,53]. These findings
coincide with the empirical results of this study. During the process of cooperating, ques-
tioning, commenting, and thinking in the case and project analyses, students expanded
their thinking, developed a great interest in learning, showed their unique ability, and
were often able to “have an epiphany”. Recent studies also confirmed that perceiving
teaching–learning activities as being aligned with intended learning outcomes was asso-
ciated with enjoyment of the course and usefulness ratings [54]. The significant improve-
ment in the scores of students in the experimental group from items 4 to 14 could further
indicate that the teaching and learning process in the experimental group was more con-
structive than the traditional teaching model of the control group.
Finally, the assessment model of the experimental group tends to be a criterion-ref-
erenced assessment, which could better reflect the “qualitative” characteristic of the stand-
ard. For example, in the comprehensive case analysis of the final assessment, students can
obtain specific grades when they meet the corresponding standards of each assessing di-
mension [14,25]. The students in the experimental group not only had a better learning
experience but also had a statistically significant improvement in their final scores com-
pared to the control group. In addition, the experimental group combined the formative
assessment in daily learning with the final summative assessment. Students in this group
showed significant improvement in their experience scores on the assessment. Further-
more, because the assessment was aligned with the intended learning outcomes and
teaching and learning activities, it also reflected the students’ learning content, which was
supported by the significant improvement in students’ scores on the last question item in
the experimental group.

7. Conclusions
This study adopted an experimental research methodology to investigate students’
perspectives on their learning effectiveness after the implementation of constructive align-
ment in an MIS course. Researchers obtained results with statistical significance. Accord-
ing to Bloxham and Boyd [52], the structures and characteristics of courses are the key to
supporting and reinforcing students’ ability to take an initiative and sophisticated ap-
proach to their learning, and form clear structures as well as the knowledge bases to pro-
mote students’ integration of content and expansion of new topics that relate to previous
knowledge. The experiment in this study illustrated the effectiveness of implementing
constructive alignment by verifying the changes in student scores on the course experi-
ences and final assessment. Through independent sample t-tests, students’ scores on the
course experiences in the experimental group showed statistically significant improve-
ments. A nonparametric Mann–Whitney test for ordinal variables and a chi-square test on
each grade also confirmed that students from the experimental group improved signifi-
cantly in their performances. The evidence from both tests verified our research hypothe-
sis, explaining that the implementation of constructive alignment could be productive for
students’ learning in scientific courses.
In today’s era of massification of higher education, many students from tertiary ed-
ucation find themselves unsuited to the university environment, lacking confidence in
their learning ability and feeling doubtful about whether they will be able to complete
their studies [55]. The constructive alignment theory can help improve students’ learning
effectiveness by providing an operational framework for science education through the
close interrelation of intended learning outcomes, teaching and learning activities, and
learning assessment. By verifying the effectiveness of the experimental group after imple-
menting the theoretical framework of constructive alignment, this study provided empir-
ical evidence and insight into the teaching and learning processes of scientific education.
Educ. Sci. 2022, 12, 338 14 of 16

Nevertheless, this study also has limitations. On the one hand, the present study did
not consider uncontrollable factors, such as the need to change specific teaching content
due to the changing teaching methods. On the other hand, the study only compared the
experimental and control groups; it did not consider whether the improvement in stu-
dents’ academic performance and experience scores was affected by other factors (such as
environmental change or professional interest). This limitation also shows that the con-
structive alignment principle tends to emphasise the establishment of theoretical models,
but the factors related to the external environment and emotional perspective have not
been considered. In future in-depth research, we can further study the impact of construc-
tive alignment on students’ academic performance in different disciplines to explore
whether the results of this study can be extended to other situations. In addition, to make
the research more scientific, researchers should control for the interference factors that
may be involved in the experiment. In short, there is still a broader and more in-depth
research space available for studying the impact of constructive alignment on students’
learning results. Additionally, we should pay more attention to the scientific design of the
experimental process and other influencing factors within the experimental group to im-
prove the theory and practice of the teaching process in scientific education.

Author Contributions: Conceptualization, H.Z.; Formal analysis, H.Z., S.S., and Y.Z.; Investigation,
H.Z., and S.S.; Methodology, H.Z.; Project administration, L.J.F.I.; Resources, H.Z.; Writing—origi-
nal draft, H.Z.; Writing—review and editing, Y.Z., H.Z., and S.S. All authors have read and agreed
to the published version of the manuscript.
Funding: This work was financially supported by the Macao Polytechnic Institute (RP/ESCHS-
04/2021).
Institutional Review Board Statement: Ethical review and approval were waived for this study,
due to being a study involving a small number of healthy adults, participating under informed con-
sent, with no sensitive data collection.
Informed Consent Statement: Written informed consent was obtained from all subjects involved in
the study.
Data Availability Statement: The data presented in this study are available on request from the
corresponding author. The data are not publicly available due to privacy issues.
Acknowledgments: The authors gratefully acknowledge the support of the Macao Polytechnic In-
stitute (RP/ESCHS-04/2021).
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Walsh, A. An exploration of Biggs’ constructive alignment in the context of work-based learning. Assess. Eval. High. Educ. 2007,
32, 79–87. https://doi.org/10.1080/02602930600848309.
2. Larkin, H.; Richardson, B. Creating high challenge/high support academic environments through constructive alignment: Stu-
dent outcomes. Teach. High. Educ. 2013, 18, 192–204. https://doi.org/10.1080/13562517.2012.696541.
3. Wilhelm, S.; Förster, R.; Zimmermann, A.B. Implementing competence orientation: Towards constructively aligned education
for sustainable development in university-level teaching-and-learning. Sustainability 2019, 11, 1891.
https://doi.org/10.3390/su11071891.
4. Noy, S.; Capetola, T.; Patrick, R. The wheel of fortune as a novel support for constructive alignment and transformative sustain-
ability learning in higher education. Int. J. Sustain. High. Educ. 2021, 22, 854–869.
5. Seery, N.; Buckley, J.; Delahunty, T.; Canty, D. Integrating learners into the assessment process using adaptive comparative
judgement with an ipsative approach to identifying competence based gains relative to student ability levels. Int. J. Technol. Des.
Educ. 2019, 29, 701–715.
6. Buckley, J.; Seery, N.; Gumaelius, L.; Canty, D.; Doyle, A.; Pears, A. Framing the constructive alignment of design within tech-
nology subjects in general education. Int. J. Technol. Des. Educ. 2020, 31, 867–883.
7. Martins, J.; Branco, F.; Gonçalves, R.; Au-Yong-Oliveira, M.; Oliveira, T.; Naranjo-Zolotov, M.; Cruz-Jesus, F. Assessing the
success behind the use of education management information systems in higher education. Telemat. Inform. 2019, 38, 182–193.
8. Nurmikko-Fuller, T.; Hart, I. Constructive alignment and authentic assessment in a media-rich undergraduate course. EMI
Educ. Media. Int. 2020, 57, 167–182.
Educ. Sci. 2022, 12, 338 15 of 16

9. Biggs, J. Enhancing teaching through constructive alignment. High. Educ. 1996, 32, 347–364. https://www.jstor.org/sta-
ble/3448076.
10. Paskevicius, M. Conceptualizing open educational practices through the lens of constructive alignment. Open Praxis 2017, 9,
125–140.
11. Hailikari, T.; Virtanen, V.; Vesalainen, M.; Postareff, L. Student perspectives on how different elements of constructive align-
ment support active learning. Active Learn. High. Educ. 2021. 1469787421989160. https://doi.org/10.1177%2F1469787421989160.
12. Weichselbraun, A.; Kuntschik, P.; Francolino, V.; Saner, M.; Dahinden, U.; Wyss, V. Adapting data-driven research to the fields
of social sciences and the humanities. Future Internet. 2021, 13, 59. https://doi.org/10.3390/fi13030059.
13. Billett, S. Critiquing workplace learning discourses: Participation and continuity at work. Stud. Educ. Adults. 2002, 34, 56–67.
https://doi.org/10.1080/02660830.2002.11661461.
14. Biggs, J.; Tang, C. Teaching for Quality Learning at University; Buckingham: SRHE & Open University Press: Milton Keynes, UK,
2007. Available online: https://cetl.ppu.edu/sites/default/files/Publications/-John_Biggs_and_Catherine_Tang-_Teach-
ing_for_QualiBookFiorg-.pdf (accessed on 1 September 2021).
15. Goss, H. Student Learning Outcomes Assessment in Higher Education and in Academic Libraries: A Review of the Literature.
J. Acad. Librariansh. 2022, 48, 102485. https://doi.org/10.1016/j.acalib.2021.102485.
16. Erikson, M.G.; Erikson, M. Learning outcomes and critical thinking–good intentions in conflict. Stud. High. Educ. 2019, 44, 2293–
2303. https://doi.org/10.1080/03075079.2018.1486813.
17. Havnes, A.; Prøitz, T.S. Why use learning outcomes in higher education? Exploring the grounds for academic resistance and
reclaiming the value of unexpected learning. Educ. Assess. Eval. Account. 2016, 28, 205–223.
18. Loughlin, C.; Lygo-Baker, S.; Lindberg-Sand, Å. Reclaiming constructive alignment. Eur. J. High. Educ. 2021, 11, 119–136.
19. Shuell, T. Cognitive Conceptions of Learning. Rev. Educ. Res. 1986, 56, 411–436. https://doi.org/10.2307/1170340.
20. Granberg, C.; Palm, T.; Palmberg, B. A case study of a formative assessment practice and the effects on students’ self-regulated
learning. Stud. Educ. Eval. 2021, 68, 100955. https://doi.org/10.1016/j.stueduc.2020.100955.
21. Brown, S. Assessment for learning. Learn. Teach. High. Educ. 2005, 81–89. https://eprints.glos.ac.uk/id/eprint/3607.
22. Rust, C. The impact of assessment on student learning: How can the research literature practically help to inform the develop-
ment of departmental assessment strategies and learner-centred assessment practices? Active Learn. High. Educ. 2002, 3, 145–158.
https://doi.org/10.1177%2F1469787402003002004.
23. Ramsden, P. Learning to Teach in Higher Education; Routledge: London, UK, 2003; p. 187. https://doi.org/10.4324/9780203507711.
24. Elton, L. Teaching in Higher Education: Appraisal and Training; Kogan: London, UK, 1987; p. 92.
25. Pui, P.; Yuen, B.; Goh, H. Using a criterion-referenced rubric to enhance student learning: A case study in a critical thinking and
writing module. High. Educ. Res. Dev. 2021, 40, 1056–1069.
26. Veldman, F.J.; De Wet, M.A.; Ike Mokhele, N.E.; Bouwer, W.A.J. Can engineering education in South Africa afford to avoid
problem-based learning as a didactic approach? Eur. J. Eng. Educ. 2008, 33, 551–559. https://doi.org/10.1080/03043790802564152.
27. Kalmpourtzis, G.; Romero, M. Constructive alignment of learning mechanics and game mechanics in Serious Game design in
Higher Education. Int. J. Serious Games 2020, 7, 75–88. https://doi.org/10.17083/ijsg.v7i4.361.
28. Jones, C. Constructive Alignment: A journey for new eyes. J. Enterprising Cult. 2006, 14, 291–306.
https://doi.org/10.1142/S0218495806000179.
29. Simper, N. Assessment thresholds for academic staff: Constructive alignment and differentiation of standards. Assess. Eval.
High. Educ. 2020, 1–15. https://doi.org/10.1080/02602938.2020.1718600.
30. Mcdonald, R.; Van Der Horst, H. Curriculum alignment, globalization, and quality assurance in South African higher education.
J. Curric. Stud. 2007, 39, 1–9. https://doi.org/10.1080/00220270500422715.
31. Ruge, G.; Tokede, O.; Tivendale, L. Implementing constructive alignment in higher education–cross-institutional perspectives
from Australia. High. Educ. Res. Dev. 2019, 38, 833–848. https://doi.org/10.1080/07294360.2019.1586842.
32. Israel, N.; Pitman, M.; Greyling, M. Engaging critical thinking: Lessons from the RDA tutorials and projects. S. Afr. J. Psychol.
2007, 37, 375–382. https://doi.org/10.1177%2F008124630703700214.
33. Chan, C.K.; Lee, K.K. Constructive alignment between holistic competency development and assessment in Hong Kong engi-
neering education. J. Eng. Educ. 2021, 110, 437–457. https://doi.org/10.1002/jee.20392.
34. Obada, D.O.; Bajeh, A.; Alli-Oke, R. Exploring the Constructive Alignment of Pedagogical Practices in Science and Engineering
Education in Sub-Saharan African Universities: A Nigerian Case Study. Int. J. Eng. Peda. 2021, 11.
https://doi.org/10.3991/ijep.v11i2.15539.
35. Thian, L.B.; Ng, F.P.; Ewe, J.A. Constructive alignment of graduate capabilities: Insights from implementation at a private uni-
versity in Malaysia. Malays. J. Learn. Instr. 2018, 15, 111–142. https://doi.org/10.32890/mjli2018.15.2.5.
36. Harvey, A.; Kamvounias, P. Bridging the implementation gap: A teacher-as-learner approach to teaching and learning policy.
High. Educ. Res. Dev. 2008, 27, 31–41. https://doi.org/10.1080/07294360701658716.
37. Sumison, J.; Goodfellow, J. Identifying generic skills through curriculum mapping: A critical evaluation. High. Educ. Res. Dev.
2004, 23, 329–346. https://doi.org/10.1080/0729436042000235436.
38. Treleaven, L.; Voola, R. Integrating the development of graduate attributes through constructive alignment. J. Mark. Educ. 2008,
30, 160–173. https://doi.org/10.1177%2F0273475308319352.
39. Higgins, R.; Hogg, P.; Robinson, L. Constructive alignment of a research-informed teaching activity within an undergraduate
diagnostic radiography curriculum: A reflection. Radiography 2017, 23, S30–S36. http://dx.doi.org/10.1016/j.radi.2016.11.004.
Educ. Sci. 2022, 12, 338 16 of 16

40. Brabrand, C.; Dahl, B. Constructive alignment and the SOLO taxonomy: A comparative study of university competences in
computer science vs. mathematics; Conferences in Research and Practice in Information Technology. Aust. Comput. Soc. 2008,
88, 3–17.
41. Maxworth, A. An Extended Constructive Alignment Model in Teaching Electromagnetism to Engineering Undergraduates.
Educ. Sci. 2019, 9, 199. https://doi.org/10.3390/educsci9030199.
42. Gynnild, V.; Leira, B.; Myrhaug, D.; Holmedal, L.; Mossige, J. Constructive Alignment in Science and Engineering: From Prin-
ciple to Practice. Nordic J. STEM Educ. 2019, 1, 33–37. https://doi.org/10.5324/njsteme.v3i1.2992.
43. Heron, D. Constructive Alignment: A Desirable and Achievable Aspiration in Geological Field Teaching; University of London, Surrey:
London, UK, 2011.
44. Lowe, D.B.; Goldfinch, T. Lessons from an analysis of the intended learning outcomes of integrative project units within engi-
neering programs. IEEE Trans. Educ. 2021, 64, 361–366. https://doi.org/10.1109/TE.2021.3057622.
45. Willison, J.; O’Regan, K. Commonly known, commonly not known, totally unknown: A framework for students becoming
researchers. High. Educ. Res. Dev. 2007, 26, 393–409. https://doi.org/10.1080/07294360701658609.
46. Collier, K.G. Peer-group learning in higher education: The development of higher order skills. Stud. High. Educ. 1980, 5, 55–62.
https://doi.org/10.1080/03075078012331377306.
47. Bennett, N.; Dunne, E. Talking and Learning in Groups; Routledge: London, UK, 2003; pp. 16–28.
48. Paul, R.; Elder, L. Critical Thinking: How to Prepare Students for a Rapidly Changing World; Foundation for Critical Thinking: Rohner
Park, CA, USA, 1995; pp. 33–48.
49. Voinea, L. Formative assessment as assessment for learning development. Rev. Pedagogie. 2018, 66, 7–23.
50. Royal, K.D.; Guskey, T.R. On the appropriateness of norm-and criterion-referenced assessments in medical education. Ear. Nose.
Throat. J. 2015, 94, 252–254. https://doi.org/10.1177%2F014556131509400701.
51. Krathwohl, D. A Revision of Bloom’s Taxonomy: An Overview. Theory. Pract. 2002, 41, 212–218.
https://doi.org/10.1207/s15430421tip4104_2.
52. Bloxham, S.; Boyd, P. Developing Effective Assessment: A Practical Guide; Open University Press, McGraw-Hill Education: Berk-
shire, UK, 2007; pp. 15–23.
53. Hawe, E.; Dixon, H. Assessment for learning: A catalyst for student self-regulation. Assess. Eval. High. Educ. 2017, 42, 1181–1192.
https://doi.org/10.1080/02602938.2016.1236360.
54. Stamov Roßnagel, C.; Fitzallen, N.; Lo Baido, K. Constructive alignment and the learning experience: Relationships with student
motivation and perceived learning demands. High. Educ. Res. Dev. 2021, 40, 838–851.
https://doi.org/10.1080/07294360.2020.1787956.
55. Crozier, G.; Reay, D.; Clayton, J.; Colliander, L.; Grinstead, J. Different strokes for different folks: Diverse students in diverse
institutions–experiences of higher education. Res. Pap. Educ. 2008, 23, 167–177.https://doi.org/10.1080/02671520802048703.