Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                

Project Grading System: University of Twente

Download as pdf or txt
Download as pdf or txt
You are on page 1of 95

University of Twente

Design Module Project Report

Project Grading System

Group 13
Yevhen Khavrona - s2093421
Triet Ngo - s2188414
Dylan Koldenhof - s2154854
Adrijus Prieskenis - s1971514
Robin Waterval - s1477196

supervised by

Marieke Huisman
Ömer Sakar

April 16, 2021


Contents

1 Introduction 5

2 Domain, Stakeholders and Requirements Specification 7

2.1 Stakeholders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.2 Mission Statement . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.3 Current Approach . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.4 Constraints, Assumptions, Definitions . . . . . . . . . . . . . . . 10

2.5 Functional Requirements . . . . . . . . . . . . . . . . . . . . . . . 11

2.5.1 Must . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.5.2 Should . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.5.3 Could . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.5.4 Won’t . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3 Coding Toolset and Canvas Integration 20

3.1 Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.1.1 Languages and Frameworks . . . . . . . . . . . . . . . . . 20

3.2 Canvas Integration . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4 Architecture and Structure 25

1
Contents -

4.1 Front End . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

4.2 Back End . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

4.3 Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

4.4 Combined . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

5 Design in Detail 28

5.1 Database Structure . . . . . . . . . . . . . . . . . . . . . . . . . . 28

5.2 Back End . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

5.2.1 User . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

5.2.2 Course, Project and Permissions . . . . . . . . . . . . . . 30

5.2.3 Rubric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

5.2.4 Graders and Submissions . . . . . . . . . . . . . . . . . . 32

5.2.5 Grading . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

5.2.6 Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

5.2.7 Transactions . . . . . . . . . . . . . . . . . . . . . . . . . 34

5.2.8 Locks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

5.3 Front End . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

5.3.1 Sign-in and import . . . . . . . . . . . . . . . . . . . . . . 37

5.3.2 Students and Submissions . . . . . . . . . . . . . . . . . . 39

5.3.3 Grading rubric . . . . . . . . . . . . . . . . . . . . . . . . 42

5.3.4 Assigning graders . . . . . . . . . . . . . . . . . . . . . . . 44

5.3.5 Grading . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

5.3.6 Sending feedback . . . . . . . . . . . . . . . . . . . . . . . 46

6 Testing 48

6.1 Automated Testing . . . . . . . . . . . . . . . . . . . . . . . . . . 49

2
Contents -

6.2 Usability Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

7 Development Process 52

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

7.2 Project timeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

7.2.1 Planning and Research . . . . . . . . . . . . . . . . . . . . 53

7.2.2 Canvas integration . . . . . . . . . . . . . . . . . . . . . . 53

7.2.3 Implementing ”Must”, ”Should” and ”Could” Requirements 54

7.3 Responsibility and Contribution . . . . . . . . . . . . . . . . . . . 55

8 Conclusion 56

8.1 Extensions and Upgrades . . . . . . . . . . . . . . . . . . . . . . 56

8.2 Final Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

Appendix A User Manual 58

A.1 Signing in and out . . . . . . . . . . . . . . . . . . . . . . . . . . 58

A.1.1 Signing In . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

A.1.2 Signing Out . . . . . . . . . . . . . . . . . . . . . . . . . . 60

A.2 Home . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

A.2.1 Import Courses . . . . . . . . . . . . . . . . . . . . . . . . 62

A.2.2 To-Do List . . . . . . . . . . . . . . . . . . . . . . . . . . 64

A.2.3 Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

A.3 Course . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

A.3.1 Import Projects . . . . . . . . . . . . . . . . . . . . . . . . 67

A.4 Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

A.5 Students . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

A.6 Graders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

3
Contents -

A.6.1 Edit Graders . . . . . . . . . . . . . . . . . . . . . . . . . 72

A.6.2 Assign Graders . . . . . . . . . . . . . . . . . . . . . . . . 73

A.7 Submissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

A.7.1 Submission Details . . . . . . . . . . . . . . . . . . . . . . 76

A.8 Grading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

A.8.1 Rubric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

A.8.2 Grade Editor and History . . . . . . . . . . . . . . . . . . 81

A.8.3 Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

A.9 Rubric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

A.9.1 Viewing a Rubric . . . . . . . . . . . . . . . . . . . . . . . 83

A.9.2 Editing a Rubric . . . . . . . . . . . . . . . . . . . . . . . 84

A.9.3 Importing and Exporting . . . . . . . . . . . . . . . . . . 87

A.9.4 Download PDF . . . . . . . . . . . . . . . . . . . . . . . . 87

A.10 Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

A.11 Upload and Export . . . . . . . . . . . . . . . . . . . . . . . . . . 90

A.12 Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

Appendix B Diagrams 91

B.1 Grading Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

B.1.1 Current Grading Flow . . . . . . . . . . . . . . . . . . . . 92

B.1.2 Our application Grading Flow . . . . . . . . . . . . . . . 93

B.2 Database Class Diagram . . . . . . . . . . . . . . . . . . . . . . . 94

4
Chapter 1

Introduction

Projects constitute an important element of formal education. They offer stu-


dents the chance to apply theoretical knowledge on practice and collaborate
with fellow students to solve interesting problems. That’s why most universities
include project work in their curriculum. It is especially true for the University
of Twente (UT) that strives to engage students in project work in every module
of each bachelor’s programme offered at the university.

Besides offering significant learning benefits, project work presents a lot of


challenges to the teaching staff at UT. Grading projects require considerable
amounts of human resources and coordination efforts in order to maintain high
quality of grading and provide helpful feedback. The ever-growing student body
of UT increases the pressure on teaching staff which, in turn, calls for improve-
ments to the current project grading process.

The current practices and tools used for grading projects at UT do not scale well
to the growing student population and become more and more inefficient and
resource-demanding. Teachers and teaching assistants have to cope with the
arduous management of project grading, perform many activities manually and
rely on tools not suitable for large-scale project grading. Our group is confident
that the project grading process can be streamlined and simplified with the
right set of tools of a project grading system and careful crafting of the project
grading workflow in it.

The development of a system for grading projects involves many complicating


factors, such as a large variety of project types and grading workflows. Thus,
a unified approach that relies on commonalities shared by projects and project
grading styles across multiple fields had to be developed.

Hence, the goal of our project group in the Design Module was to create a dedi-

5
Introduction -

cated system for handling project grading that has the potential to address the
most pressing issues graders face. In doing so, we strove not only to introduce
a new tool with a new set of features to graders, but also to keep and leverage
the positives that the current approach offers. The system we envisioned au-
tomatically prepares data required for grading, allows graders to create useful
grading rubrics, makes it easy to track graders’ responsibilities and aids with
sending concise and transparent feedback to students.

In this report we will present the results of our work, including the detailed
description of the application’s structure and innards. Chapter 2 will touch on
the preparatory steps that we took before starting the implementation work,
Chapter 3 will describe our toolset, Chapter 4 will tell about the structure and
architecture of our application, Chapter 5 will describe in detail the design of
the application’s front end, back end and database, Chapter 6 will go in depth
into the testing activities we performed and Chapter 7 will outline the way our
work was organised and Chapter 8 will deliver our final remarks and conclusions.

6
Chapter 2

Domain, Stakeholders and


Requirements Specification

2.1 Stakeholders

Grading projects (including the tools used for it) touches interests of multiple
parties inside UT. Some stakeholders are interested in the process itself while
others are interested only in the results of that process. Some parties are in-
volved in decision making related to project grading, thus also affecting the
development of a project grading system. Therefore, it is important to iden-
tify the stakeholders that might have influence on our project and decisions we
make.

Ultimately, we thought of the following list of stakeholders:

• Teachers
They organize and come up with ideas for projects, create grading rules and
rubrics. They are responsible for handling grading and assigning TAs to
grade. They might also review the grading process or grade some projects
themselves. Teachers are also the ones who are responsible for delivering
(or managing the delivery of ) feedback to students.
• Teaching Assistants (TAs)
TAs form the bulk of the users of our application. They are often recent
students of the module that help its teachers by among other things, grading
projects. They do not have much influence on the management part of the
application, but they will be most of the users doing the actual grading.

7
Domain, Stakeholders and Requirements Specification 2.2- Mission Statement

• Module Coordinators
They are responsible for organizing entire modules and often are teachers
of a particular topic themselves. Hence they overlap mostly with teachers
and generally their desires would correspond to those of any particular
teacher, at least for our application.

• Students
While not the users of the application, they are its primary subjects, as
their projects are being graded. Thus, students are the party with arguably
one of the greatest interests in the application.
• Library, IT Services & Archive (LISA)
LISA is responsible for the management of IT systems at UT. Hence,
they are responsible for integration of student-developed systems into UT
infrastructure and maintaining these systems in future. Also, our project
requires close cooperation with LISA stuff as our application is tightly in-
terconnected with the existing UT infrastructure.

• Us (Group 13) as the developers of the system


Of course, we as developers have a large (and broad) interest in the appli-
cation.
• Our project supervisors
The supervisors are responsible for overseeing our project and guiding us
with issues. There is some overlap between them and Teachers since they
the supervisors also interested in the grading aspect themselves.
• Module Coordinator of the Design Project Module
The Coordinator of the module is responsible for overseeing all design
projects, and so would want every group including ours to do well.

• Reflection Component Teachers


As the teachers whose main responsibility lies within helping the students
deal with ethical issues in their projects, they also constitute a party with
interest in that specific aspect of our application.
• University as an institution
Finally, most of these stakeholders can also be fused into one entity,
namely the university itself. Besides that, the university might interact
with our project and group via specific institutional forms, such as the
university or program boards.

2.2 Mission Statement

Keeping the stakeholders and their interests in mind, we set the goals for the
project.

8
Domain, Stakeholders and Requirements Specification 2.3- Current Approach

Our motivation: resolve the flaws and inefficiencies in the grading process
that graders face.

Our goal: to create a system for efficient project grading that can be used by
teaching staff to grade students’ projects and provide feedback to them.

Our approach: a web application (back end in Spring, front end in ReactJS,
Postgres database) tightly integrated with UT’s Canvas Learning management
system (LMS).

2.3 Current Approach

To understand the specific areas of improvement and define a set of require-


ments, we analysed the way projects are currently graded. As a result of our
analysis and research, we identified the tools that are used for grading, despite
often being not suited for large-scale grading:

• Google Sheets is used to manage and coordinate grading in spreadsheets.


The sheets contain grading rubrics, submission grades, grading progress
reports and project feedback.
• Short scripts and formulae in Google Sheets are used for grouping stu-
dents, calculating grades etc.
• Google Docs is often used in grading for collecting project feedback.
• IDEs and other software for working with code
• File Viewers
• email services are used for sending feedback to students and communica-
tion within the teaching staff.
• Canvas is used by students to submit projects and by the teaching staff
to download those submissions, provide grades and feedback.

A good (and generalisable) example of the grading process of the programming


project of the Software Systems module can be found in Figure B.1.

The current grading process is not efficient due to the following reasons:

• Google Sheets
– Besides having lots of benefits, Google Sheets is not flexible enough
to handle management of a large grading process. It has a sheet

9
Domain, Stakeholders and Requirements Specification2.4- Constraints, Assumptions, Definitions

and cell limit that is not always sufficient for grading projects in
large modules. Moreover, some browsers cannot cope with the load
of dozens or hundreds of sheets created and, consequently, might
occasionally crash under that load.
– It can be hard to present a readable and convenient grading rubric
in Google Sheets. Also, managing the rubric in Google Sheets might
be a rather tedious process because rubric version tracking, which is
required to keep every grader updated, is done completely manually.
– The grading interface is cluttered with lots of unnecessary pieces of
data in Google Sheets (e.g. being presented with grading results of
all submission when attempting to grade a single submission).
– Although Google Sheets supports automation to some extent, its
capabilities are pretty limited.
– There is no simple way to present useful and detailed grading-related
statistics (mostly due to the fact that the absence of a database
behind Google Sheets makes preparing statistics rather hard).
– Usually, the grading progress of each TA is only manually tracked
by making them fill in their progress in a separate sheet in Google
Sheets.
– As a brief conclusion, Google Sheets is the best and the worst tool
for grading projects at the same time.

• Feedback on projects are gathered in a separate document, making it


harder to keep a good overview.
• Documents and files of submissions that are being graded are not easily
visible on the screen next to the rubric and a grading sheet, so the grader
needs to constantly switch between the files, rubric and sheet.

• There are no notifications about changes in the rubric or the potential re-
quired re-grading activities, so it’s easy to miss acting upon these changes.
• Grading feedback is managed manually and usually sent via email with
not many details or explanations.

2.4 Constraints, Assumptions, Definitions

In solving the mentioned issue through development of a dedicated system, we


were constrained by two major factors:

• The system should have been prototyped with sufficient func-


tionality within 10 weeks.

10
Domain, Stakeholders and Requirements Specification2.5- Functional Requirements

• The system had to be highly interoperable with Canvas LMS -


the learning management system used by UT.

Within our project and the report, we operate the following set of definitions:

• Teaching assistant (TA) - a student who is hired by the university to help


teachers with supervising the study process, grading students’ submissions
and assisting students with learning.
• Teacher - a university employee who is responsible for teaching a specific
set of courses/modules; acts as a coordinator of teaching assistants.
• Module coordinator - a person who is responsible for organisation and
administration of a module, including defining its structure, timetable,
people involved etc.
• Grading rubric (schema) - a set of guidelines describing how a particu-
lar project should be graded; usually contains grading criteria that are
subdivided into groups (blocks).
• Submission report card (Assessment of Grading sheet) - a document with
feedback for a group of students containing the final project grade, expla-
nations, comments and possibly partial (component) grades.

• Grader - a person assigned to grade a project (usually, a teaching assistant


but sometimes a teacher can grade projects themselves).
• Student project - an assignment (usually, mandatory) that involves stu-
dents working on a particular problem for a fixed period of time; allows
students to apply their knowledge on practice and conduct research activ-
ities.
• Submission - a particular instance of a project created by a student project
group and provided to the teaching staff for review and grading.
• Student project group (group) - a group of students working on a project;
NB we consider a student working alone on a project to be a member of
a one-person group.

2.5 Functional Requirements

After we outlined the domain we work with, identified the stakeholders and
dissected the current approach to grading projects, we moved to defining a list
of requirements for our project. We split the list into 4 groups according to
the MoSCoW model: Must, Should, Could, Won’t. For ease of reading, we

11
Domain, Stakeholders and Requirements Specification2.5- Functional Requirements

further split each of these groups into more granular categories representing
parts of the application. As for implementation plans, we set a plan to carry
out the work needed to implement the first three groups of requirements - Musts,
Should s, Could s. In the end, we managed to successfully cover the original set
of requirements as well as most of the new requirements that we added in the
process.

Below the (updated) requirements are presented with explanations and motiva-
tion for some.

2.5.1 Must

Courses and Tasks Overview

• As a grader, I want to be able to see a list of courses I am taking part in.


• As a teacher, I want to be able to import courses from canvas I am taking
part in.

• As a grader, I want to view a list of submission I need to grade within


projects.
To help the user further organise their work, the system provides a to-do
list where each item shows a project with the number of open issues and
submissions that are not fully graded.

Project Overview

• As a grader, I want to see the list of projects in a course.

• As a grader, I want to view and filter the issues that I opened or addressed
to in my project view
• As a teacher, I want to be able to import projects from canvas I am taking
part in.

• As a teacher, I want to be able to assign deadlines to projects that I


supervise or administrate.
• As a teacher, I want to be able to sync a project with the information in
canvas.
After a project is imported, the system should take a snapshot from canvas
regarding people, submissions from projects, etc. Each time the teacher
decides to sync, the system will take another snapshot from canvas and add
new submissions, re-submissions, new students, new staffs to the project
without affecting the old data.

12
Domain, Stakeholders and Requirements Specification2.5- Functional Requirements

Submissions Overview

• As a teacher, I want to see the list of project submissions of groups in a


course.
• As a grader, I want to see the list of all submissions in a specific project
that I am assigned to grade.

Submission Details

• As a grader, I want to be able to create labels in a project.


Custom labels such as ”require-attention”, ”resit students”, ”suspicion pla-
giarism”, etc. can prove to be useful for graders to manage their grading.
Furthermore, these labels can be used by others in a project to ensure syn-
chronisation between graders.

• As a grader, I want to be able to apply/remove labels to/from a submis-


sion.
• As a grader, I want to be able to view a submission’s attachments, includ-
ing type, name, size and a way to download them.

• As a grader, I want to be able to view students’ comments from Canvas.


• As a grader, I want to be able to see to whom a submission belongs.
• As a grader, I want to be able to update to whom a submission belongs.
In some unforeseen circumstances, there are students that are in a group
but did not contribute to the final product, or there are students that are
not in a group at the moment of submission but participated in the final
product. It should be possible to add/remove these student accordingly.
• As a grader, I want to be able to grade certain group members differently
from the other group members.
It is possible that in some grading scheme, some students in a group can
have different results. It should therefore be possible for graders to apply
multiple grading sheets to one group and select which student is graded by
which grading sheet.
• As a grader, I want to be able to select which grading sheet to use for a
student.

TA Management

• As a teacher, I want to see which TAs are assigned to which submissions.

13
Domain, Stakeholders and Requirements Specification2.5- Functional Requirements

• As a teacher, I want to select which TAs are graders.


There is usually a considerable larger number of TAs in a course than the
number of TAs responsible for grading a specific project. Therefore, it is
efficient for teachers to manage graders.
• As a teacher, I want to be able to see which submissions are currently not
assigned to any graders.
• As a teacher, I want to be able to move projects that are currently assigned
to some grader to the “Not assigned” category for later (re-)assigning.
• As a teacher, I want to be able to (re-)assign TAs to submissions either
manually by selecting a TA or automatically by selecting a random TA.
It is usually the case that assigning submissions to TAs for grading is done
randomly initially, and then adjusted manually if it’s needed. We would
like to offer the users two ways of manual assigning: via a modal form
and with an easy to use drag-and-drop functionality.
• As a grader, I want to be able to remove myself as a grader from submis-
sions assigned to me.
Sometimes, due to some circumstances, graders are not able to finish grad-
ing in certain submissions. It should be feasible for them return the as-
signed submissions so they can be assigned to and graded by others.

Grading Interface

• As a grader, I want to enter grades for a specific grading criterion in a


group’s submission.
• As a grader, I want to be able to regrade a specific grading criterion in a
group’s submission.
Graders usually change grading in a criterion. The system should allow
the user to create multiple grading for a criterion and allow choosing the
current one.
• As a grader, I want to add a comment with motivation for grading ac-
cording to a specific grading criterion.
Graders usually provide additional explanations and comments to the grades
they give, so the system should support such functionality.
• As a grader, I want to manually enter the final grade for a submission.
Even though a teacher might have specified a formula for calculating the
final grade (e.g. summing up all the points given for criteria), the grader
should be able to manually override that calculation of the final grade.
• As a grader, I want to open an issue on a grading sheet to ask for help
from others.
It often happens that a TA encounters an unusual case that require helps

14
Domain, Stakeholders and Requirements Specification2.5- Functional Requirements

from a teacher. By allowing a TA to open an issues in the application


that points to a teacher/grader to ask for their help, it should be easier to
resolve these issues.
• As a grader, I want to resolve/close an open issue that is addressed to me
or opened by me.

• As a teacher, I want to be able to override grades that were given before.


It might happen that a submission requires regrading, so the teacher should
be able to overwrite an old grade with a new one by adding and setting a
new grade to the criterion.

Rubric and Rubric Builder

• As a teacher, I want to be able to create a rubric with grading criteria and


categorize them in blocks.

• As a teacher, I want to be able to provide details for a grading criterion.


Teachers usually add elaborate descriptions, grading scheme, max/min
grades of grading criteria where they mentioned the specifics of grading.
• As a teacher, I want to be able to manipulate the rubric with a Rubric
Builder.

Feedback and Exporting

• As a teacher, I want to be able to transfer project grades and comments


automatically to Canvas.
Currently, exporting grades is done by uploading an Excel file to Canvas.
We would like to make this process automatic by letting the teacher export
grades to Canvas with a single tap.

Authentication

• As a user, I want to log in to the system using a UT account and log out
when I finish my session.
We will use the university’s existing infrastructure for user authentication
and fetching necessary data about courses and users.

15
Domain, Stakeholders and Requirements Specification2.5- Functional Requirements

2.5.2 Should

Courses and Tasks Overview

• As a grader, I want to be able to see a list of projects that I recently


interacted with.
Instead of navigating through a list of all courses and projects, the user
should be able to quickly jump to the “recent” projects.
• As a grader, I want to be able to view tasks assigned to me.
The user should be able to quickly see what work they have to do and how
much of that work is left.
• As a grader, I want to be able to sort the courses I am taking part in by
name and academic year.
• As a grader, I want to be able to search for a specific course by name.

Project Overview

• As a teacher, I want to be able to see the percentage of passes and fails and
distribution of grades (including mean and median) for a specific project.
– We want to present useful statistics to the user of the system. We
will come up with more useful measures while working on the project.
• As a teacher, I want to be able to see the progress of the grading process
(the number of projects graded and the number of projects left to grade).
• As a teacher, I want to be able to lock grading of a project to prevent
others from making edits to current grades and feedback.
• As a TA, I want to see the list of submissions that require re-grading or
additional checking.
If something changes in the rubric or somebody tags a submission so that
the submissions assigned to the TA require additional actions, they should
clearly see such projects in their overview. Also teacher can create issue
to ask TAs to recheck the grading

Submissions Overview

• As a teacher, I want to be able to sort submissions by group number, final


grade and date of uploading the submission.
• As a grader, I want to be able to search for a submission in a list of
submissions by different criteria.
It should be possible for grader to search for a specific submission based on
student’s name, student’s number, labels of submissions, etc.

16
Domain, Stakeholders and Requirements Specification2.5- Functional Requirements

TA Management

• As a teacher, I want to be able to reverse my last TA (re-)assigning action.


• As a teacher, I want to be able to observe the grading progress per TA.
Teachers are usually interested in how far each TA is with grading and
how much work is left.
• As a teacher, I want to switch between letting only the TAs that are as-
signed to specific submissions see the grading of those submissions (done
by these TAs) and letting every TA freely view the grading of all submis-
sions done by other TAs.
We would like to offer two visibility settings for submissions: either all
TAs see all submissions with grades or only TAs that grade certain sub-
missions can see them. It might be important for TAs to observe how other
TAs grade their submissions to “set the standard”.

Grading Interface

• As a grader, I want to clearly see which parts of a submission require


re-grading when the rubric is changed.
In order to streamline the process of re-grading submissions, we would
like to point out those parts of submissions (grading) which might require
attention due to changes in rubric.

Rubric and Rubric Builder

• As a TA, I want to be able to leave comments and suggestions to the


current rubric.
– Although TAs will not be able to change the rubric directly, they will
have an opportunity to leave comments to the rubric parts for the
teacher to review and possibly apply.

2.5.3 Could

TA Management

• As a teacher, I want to be able to check out the (averaged) statistics of


grading of each TA.
TAs are different and have different grading standards. Teachers are in-
terested in seeing how a specific TA grades projects on average in order to
ensure fairness in grading.

17
Domain, Stakeholders and Requirements Specification2.5- Functional Requirements

Grading Interface

• As a grader, I want to clearly see which projects had formulae for auto-
matic final grade calculations overridden with a manually entered grade.
• As a grader, I want to be able to save the comments and explanations I
type as (personal) templates and reuse them later.
It happens often that graders need to explain the same points in the sub-
missions they grade, so we would like to offer the graders a possibility to
save those comments as templates for future use.
• As a grader, I want to open a file from the group’s submission directly in
the browser.
We would like to provide graders with a way to see what they are grading
without the need to constantly switch to other applications or tabs.

Rubric and Rubric Builder

• As a teacher, I want to export a rubric as a file to re-use it in future


projects.
• As a teacher, I want to import a rubric from a file.

• As a grader, I want to be able to export the rubric as a PDF file.


• As a grader, I want to see the history of edits of the rubric.
• As a grader, I want to see the list of projects that are affected by a change
in the rubric.

• As a grader, I want to be able to type a formula for computing the final


(or partial) grade for a submission.
Usually, teachers use formulae or weighted composition of partial (crite-
rion) grades in order to calculate the final grade.

Feedback and Exporting

• As a teacher, I want to be able to create/edit a feedback template for a


project.
Teachers usually want to send feedback to student with a predefined sub-
ject/body.
• As a teacher, I want to be able to export grading as Excel file.
It often happens that teachers want to export detailed grading to an Excel
file for further calculating/analysing.

18
Domain, Stakeholders and Requirements Specification2.5- Functional Requirements

• As a teacher, I want to be able to export the project results and feedback


as a PDF file or send it via email/canvas to the students.
Teachers might want to give students results and feedback with only specific
types of information included and often prefer not to send the full graded
rubric to students.

2.5.4 Won’t

Project Overview

• As a teacher, I want to be able to retrieve logs (history of actions) of a


project.
This was deemed to be quite complicated to implement while the usefulness
and desire for it was limited.

Grading Interface

• As a grader, I want the system to show me whether the submission adheres


to the packaging rules of the project.
We would like to automate some parts of the laborious process of checking
the formal submission requirements. However, it was deemed to to be
outside the scope of this project and too much work within the deadline.

19
Chapter 3

Coding Toolset and Canvas


Integration

After all the necessary preparatory work was done, we were ready to move on to
find the right implementation tools. Our choice was limited by one of the most
important constraints mentioned in the previous chapter - the application we
had to develop would have to be highly interoperable with Canvas LMS (and
other UT systems). This chapter will shed light on the process of choosing the
programming languages, frameworks and libraries to use and will provide the
motivation behind the choices we made.

3.1 Technologies

The application we developed relies on many proven web and software technolo-
gies to deliver the best experience and ease of use to end users as well as the
necessary convenience and flexibility to code maintainers.

3.1.1 Languages and Frameworks

The decision about the programming language that is going to be used in an


application is never a trivial one. Although all major programming languages
have existing solutions for web development, they usually have strong and weak
sides that have to be considered before settling on a specific language. Moreover,
the choice of the software stack often depends on external factors, as it was the
case in our project.

20
Coding Toolset and Canvas Integration 3.1- Technologies

After analysing the requirements, weighing multiple external and internal fac-
tors and assessing the best fit for the project, it was determined that Java
(Spring) for back end, ReactJS for front end and PostgreSQL database would
be used for the project. In the further subsections, the possibilities and choices
made are explained in more detail.

Front end

For front end, we had concluded that it was best to use a framework and not
settle for plain JavaScript, given the level of complexity and interactivity of the
planned application. The group had considered adopting one of the popular
JavaScript (JS) frameworks: Vue.js, ReactJS (React) or Angular, as all of them
could be beneficial for the project. The choice between a JavaScript framework
and the vanilla JavaScript affected not only the development process, but also
the whole philosophy of the application’s architecture. All frameworks men-
tioned use the ”Single-page application” approach (SPA) to developing front
ends that moves most of the responsibility of navigating between pages and
delivering content from back end to front end to make pages fast and reactive.
Such common downsides of SPA as poor search-engine optimisation were not
relevant to our application, so by having in mind the advantages of SPA, we
made a decision to employ that paradigm for developing our application.

ReactJS was chosen, as it seemed to be the best combination of extreme feature


versatility and a gradual learning curve. Additionally, several group members
had previous experience with React, which simplified the choice even further.
At the same time, this decision wasn’t taken without hesitation, as the standard
front end framework used at UT is Angular and not React. Considering Angu-
lar’s steep learning curve and lack of apparent benefits over React, we decided
to stick with the latter.

In developing the front end, we supplied React with multiple additional packages
and libraries to speed up development and rely on existing proven solutions to
common issues. To name a few of them, navigation in our application is handled
by React Router library, permissions and roles by CASL React, state changes
are tracked by React Redux, Lodash is used for working with JS objects, React
Quill aids us with using a rich text editor and ESlint helps us with finding
problems in our code.

Initially, React Bootstrap package was used for styling, with the aim of mini-
mizing the time needed for styling the application and simplifying visual design
works. However, after conducting several usability testing sessions and gath-
ering initial feedback on the design, we decided to make changes and replaced
Bootstrap with MaterialUI. MaterialUI gives a more unique feel to the applica-
tion and handles user interactions with the interface better, although it requires
more time spent on development. CSS Modules package is used for distributed

21
Coding Toolset and Canvas Integration 3.1- Technologies

and flexible styling of the application.

To deliver all the packages and our code to the user’s browser, a bundling
software called Webpack is used. It converts non-standard JS, JSX, SASS,
LESS and other types of files to pure JS and CSS and efficiently combines them
together into a single bundle that can be used by the browser for rendering and
running the application logic.

Back end

Since software infrastructure at UT is mostly built using Java and Java Spring
(most popular web framework written in Java), we were inclined towards using
Java for implementing our application’s back end functionality. The other option
that the group considered was using Python and its most popular web framework
Django. After holding several conversations with LISA staff and investigating
pros and cons of both approaches by creating two small testing applications
with their help, we chose to sacrifice Python’s simplicity for Java’s apparent
interoperability with UT infrastructure. While for front end we eventually opted
for React (even though Angular is the standard at UT), we decided that choosing
Java and Spring Boot for back end is a more sustainable approach, especially
considering a possible need to maintain the application by UT staff in the future.

As we anticipated, working with Java Spring turned out to be more difficult


than with Python and Django. However, eventually, the group got used to the
framework and understood how to work with its documentation which helped
us to better see and appreciate the strong sides of Java Spring.

Maven is used to manage dependencies for Spring and Java in our project.
Among the libraries we use are Spring Security for managing OAuth2 authen-
tication and permissions, Spring JPA for handling data, iText7 for generating
grading feedback and rubrics in PDF format, Apache POI for exporting grading
results into an Excel spreadsheet, Jsoup for parsing the HTML formatting of
parts of our application’s rubric representation and Java Jackson for working
with JSON instances.

For storing data our group resorted to PostgreSQL. It’s a powerful yet simple
database that integrates flawlessly with Spring JPA. At one point, we also used a
NoSQL database MondoDB to store and process rubrics. In our implementation,
the rubric is a flexible hierarchical document that fits well in the world of NoSQL
databases. However, due to the complexity of managing and synchronising two
databases, we chose to move rubrics into our main SQL database and work with
its JSON representation using SQL toolset.

22
Coding Toolset and Canvas Integration 3.2- Canvas Integration

3.2 Canvas Integration

The application we developed heavily relies on Canvas LMS. Canvas is the


centrepiece of educational software infrastructure at UT, so integrating our ap-
plication with it was one of the ”unwritten” requirements. In order to not
introduce any additional hurdles for students and not force them to adjust to
new ways of uploading project submissions, we decided to let Canvas handle re-
ceiving submission data from students and then use Canvas as a data provider
for our application. Moreover, while not explicitly storing students’ submissions
in our database makes our dependence on Canvas deeper, it also means that
we can worry less about the security aspects related to storing such data in our
database.

We interacted a lot with UT LISA support team to get access to UT’s devel-
opment version of Canvas and configure Canvas OAuth2 authentication. We
worked with a ”sandbox” course created for us by LISA that allowed us to
develop our application without affecting the production version of UT Canvas.

The data that our application fetches from Canvas can be split into four groups:

• authentication and credentials data that is used to authenticate users in


our application and obtain the access token to Canvas API
• user profile data, including user’s name, email address, avatar and student
number
• course and project (assignment) data, including the list of groups, users
and their roles in a specific course or project
• submission data, including submission comments, attachments (only links
to Canvas data storage) and users associated with the submission

This data is retrieved through Canvas API by using the access token obtained
after the user signs in via Canvas OAuth2 flow. Moreover, the application uses
Canvas not only to fetch data but also to upload data to Canvas or initiate
certain actions there. For instance, Canvas API is used for sending messages
with feedback to students.

While Canvas provides us with a complete and easy-to-use set of data necessary
for grading, it also affects our application’s design a lot, especially with regards
to the database design. The application has to reflect the specific structure of
Canvas’ courses, assignments, submissions, groups and grading results.

Reliance on Canvas also introduces several serious challenges with regards to


feature implementation. As an example, Canvas has a rather odd management
of the relation between groups and submissions, which makes working with the

23
Coding Toolset and Canvas Integration 3.2- Canvas Integration

relevant API particularly hard. In addition, managing grades on Canvas is also


not straightforward as there is not way to submit a grade via API for a specific
student - Canvas can only handle group grades. There are other important API
features that we would like to use but that are missing in Canvas API, such as
the ability to fetch user’s group participation in a course.

The ideal solution to these issues would be to incorporate the whole submission
uploading process into our app to create a closed grading ecosystem. However,
by taking the importance of Canvas for UT and the additional security concerns
into account, we decided not to proceed with development of a fully independent
system (at least not within the 10 weeks of the module).

24
Chapter 4

Architecture and Structure

After the choices for software and tools were made, the group started with
the actual implementation of the requirements. The first task was to outline
the structure that the application would have. This overall structure of the
application, split into front end, back end and database, is described in this
chapter.

4.1 Front End

The front end of the application is structured based on React components. A


component is a JavaScript class that represents a specific part of the function-
ality or interface of the app. These components can then be combined together
to form larger components that allows for creation of highly sophisticated and
efficient structures and designs. A modal (selection window) and the sidebar
are two examples of React components, but the whole grading interface screen
is a component in itself too. Furthermore, these components are responsible
for managing state information pertinent to them that affect their rendering.
Such component-based structure provides a modular and flexible design where
components can easily be changed or replaced without affecting the rest of the
front end.

Components are changing based on the user’s actions that are fed into the React
Router library. React Router is used to define a list of routes in the application
that are parsed and traversed by the library to render a correct component. All
of these activities take place solely within the front end, and back end is not
involved in routing and navigation. The front end interacts with the back end
only by performing web requests and acting upon them.

25
Architecture and Structure 4.3- Database

The React Redux library, that we use for managing complicated interactions in
our front end, consists of a list of actions that modify the state and a list of
so-called reducers that take the existing state, merge it with an update supplied
by an action and output the new updated state. Whenever the state is updated
in Redux and a component depends on that state, this component is automat-
ically re-rendered by React. The central global Redux state is combined with
internal states of components in our front end implementation to achieve the
state management our application requires.

The CSS Modules library we use for styling is based on the creation of multiple
styling sheets called “modules”. They are simple CSS style sheets but every
class name in a module is localized, i.e. it is hashed to a class name that is
globally unique even if the declared class names in different modules are reused.
The modules again act like the components and make the front end much more
modular and easy to work with. To apply styling, these modules are injected
into JS components.

4.2 Back End

The structure of the back end is based on the peculiarities of the Java Spring
framework. Spring follows the Model-View-Controller (MVC) pattern. The
model represents some structure in the back end (usually connected with the
database), such as an assessment, a rubric, etc. Meanwhile, the view is sup-
posed to represent what the user sees, but in this project this represents more
what the front end code interacts with, in the form of an API that can han-
dle request. Since the server only sends a single HTML web page back to the
client and the front end controls further interaction, the view part of MVC is
mostly covered by the front end. Finally, the controller is a part that handles
business logic, i.e. processing the requests the user (with the front end acting as
intermediary) makes. In Spring, the controller concept can be further split into
several subconcepts - the service, repository and controller classes, where the
service contains functions that perform actions like interacting with database
repositories, while the controller handles receiving the actual HTTP requests
and sending back responses.

4.3 Database

As was mentioned before, we use PostgreSQL as a database and Spring JPA


to interface connection and interaction with it. JPA allows using Java objects
as database entries and automatically generates the required data tables in
the database. JPA also greatly enhances interactions with the database by

26
Architecture and Structure 4.4- Combined

providing a simple yet powerful way of writing queries as method declarations in


repository interfaces which are then used to automatically generate SQL queries.
Furthermore, JPA provides a way to manually write either native SQL queries
or queries in JPQL (special JPA query language), and we use this functionality
in several places in the back end.

4.4 Combined

A visual representation of the overall architecture of the application can be seen


in Figure B.2. This intentionally corresponds well with the image found on UT’s
Canvas Development Guidelines page, allowing for possible future development
and maintenance of the app by UT.

27
Chapter 5

Design in Detail

This chapter discusses the design of the application’s components in detail. Each
of the three major parts - the front end, back end and database - are explored
in-depth to explain how they work, why certain choices were made and what
are the possible effects of such choices.

5.1 Database Structure

We dedicated a substantial amount of time to designing the database as the


database structure is a basis for the whole back end implementation. Despite
that we did not arrive at a perfect solution of database design at our first
attempt.

Initially, we chose to keep data stored in our database in constant sync with
Canvas. The SQL tables we defined back then closely reflected the structure of
Canvas API response formats in order to enable easier data syncing. However,
such an approach turned out to be quite error-prone and required a substantial
amount of work to make it functional (for little to no benefit). Furthermore,
it mandated implementing such convoluted yet essential features as handling
data that fell out of sync, which did not seem feasible in the 10-week time span
of the module. Therefore, we abandoned full data synchronisation in favour of
fetching data on demand from Canvas.

Our second approach was based on the idea of minimising syncing needed be-
tween our application’s and Canvas’ databases. Thus we converted our services
and controllers to fetch data from Canvas when needed and store only bare
minimum of data in our database. Such an approach eliminated the need to

28
Design in Detail 5.2- Back End

keep data in sync with Canvas but introduced even more serious issues. The
amount of data that was needed to be fetched from Canvas on each request
quickly grew out of hand and as a result the approach quickly proved to be un-
sustainable. It also required implementing sophisticated caching policies on the
server to reduce the amount of calls made to Canvas. After analysing the feed-
back on functionality that we received during our first testing session with one
of the module coordinators, we decided to make a switch to the third approach
– actively storing data from Canvas with syncing on demand.

The third approach, which is our current, in essence is a blend of the two ap-
proaches presented above. According to it, data fetched from Canvas is stored in
the application but no active synchronisation happens unless the user demands
it. Moreover, the data is synchronised in a non-destructive manner which means
that whenever the data is stored in our database, it’s never deleted from it when
performing a sync (even if such data was removed on Canvas). This mode of
work resembles capturing a ”screenshot” of Canvas state at the moment of syn-
chronisation. It turned to be the best approach to database design as it has a
very limited data footprint (data is fetched only occasionally during syncing in
comparison with the second approach) and does not need any complicated data
synchronisation mechanisms (in comparison with the first approach).

The model of our application’s database can be seen in B.3. Individual tables
presented in the diagram (i.e. Spring JPA models) and design choices made
regarding them are discussed in more details in the following section.

5.2 Back End

In this section the individual parts of the back end will be expanded on further
and some of the notable features of Spring that were used to handle data will
be explained.

5.2.1 User

The concept of user is linked to two models in the application - User and Activity.
The User model is used to represent a user (a student, teacher or TA) which is
either created when a grader signs in to the system or is imported within a batch
when a course is added to the database. The following user information retrieved
by Canvas is stored: name, email, student/employee number and avatar URL.
The Activity model is used to record grader’s recent activity that is then shown
in the front end such that the users can access their recently opened projects
quicker. The User and Activity repositories are used to give services access to
the database without the need to explicitly write SQL queries.

29
Design in Detail 5.2- Back End

The User service consists of two types of methods. The first type is methods
with so-called mandatory transactions that are used by other services (so they
can only be called within an already started database transaction in another
method) to interact with User entities in a variety of ways, such as by call-
ing query methods findUserById or saveUser. The second type is controller
transactions that are called by controllers to get data for specific endpoints.
For instance, the endpoint /api/user/to-do in UserController initiates a call
to getTodoForUser(that starts a transaction) to get a list of projects that con-
tain submissions the user is assigned to grade.

UserController contains endpoints that are responsible for getting user infor-
mation from the database, getting own profile information via /api/user/self
and retrieving the list of user’s recent activities and tasks.

5.2.2 Course, Project and Permissions

In our application, there are three models related to courses and projects:
Course, CourseParticipation and Project. The Course model represents a course
from Canvas that is created when a teacher imports it. On import, the course id,
name and starting date are saved to the database. Then, the course users, i.e.
student, teachers and TAs, are stored in the database (if not already there) and
are connected to the course by creating a CourseParticipation instance (which
implicitly stores a record in the respective table in the database).

Creating Project instances works in a similar way. When a teacher imports a


project from Canvas into the application, an instance of Project is created with
the project id, date of creation and name are stored to the database. Moreover,
the application also updates the list of students that are associated with that
project’s course and fetches the list of students’ submissions for that project from
Canvas. Additionally, for each user in the project a new instance of Settings is
created, that stores user’s preferences within the project. Currently, it’s used
to stored notification settings, such as whether the user wants to receive email
notifications when a new issue mentioning them is created.

As a result, by simply importing a course and a project within that course, the
teacher receives all the necessary grading information right in the application,
with no additional preparatory steps needed.

A privileges-based system is implemented in the back end to distinguish users


and validate their access rights. On the course level, each CourseParticipation
entry has a Role associated, which means that each course participant has a
specific role within that course, be it a STUDENT, TEACHER or TA. These roles are
used to determine the range of actions that the user can perform, since each
role corresponds to a set of privileges stored in Privilege entities. For instance,
the RUBRIC_READ privilege allows graders to read the rubric and RUBRIC_WRITE

30
Design in Detail 5.2- Back End

allows a teacher to edit the rubric since only the TEACHER role has that privilege.

The CourseController and ProjectController provide graders the full control of


accessing and managing courses and projects while ensuring proper security
for their methods. First of all, each method in the controllers is protected
by the introduced above privilege-based system. Privileges are retrieved from
the user’s identifier that is extracted from the session key, then depending on
what endpoints the grader wants to access, the method checks for the required
privileges needed to access that method’s functionality. If the user doesn’t have
the required privileges, then that request is rejected and an error is sent back to
the client. As an example, to synchronise a state of project in the application
with Canvas (i.e. update the submission and user lists), a request is sent to
the endpoint /api/courses/{courseId}/projects/{projectId}/sync. This
request will only be processed if the requesting user has the SUBMISSIONS_SYNC
privilege, and since only teachers have this privilege, no TA has access to project
synchronisation.

Another example is the PROJECT_READ privilege which is required to access


grading data of a project. Non-grading TAs will be prevented from accessing
such data as only the TEACHER and TA_GRADING roles have that privilege.

In general, the endpoints in the controllers provide full capabilities to teachers:


managing courses and projects, retrieving course and project information, get-
ting course users, managing project graders, synchronising a course or project
with Canvas etc.

5.2.3 Rubric

As one of the main pillars of our grading application, rubrics are carefully de-
signed to preserve ease of use while ensuring the flexibility of a hierarchical
rubric. The Rubric model consists of a list of Element instances, where each
Element is either a Criterion or a Block. In the case of a Criterion, title, descrip-
tion and grade details (such as minimum, maximum and the step) are saved. In
the case of a Block, only a title is saved along with a list of children Elements
to allow for the hierarchical structure.

SQL tables are not the most suitable way to store hierarchical recursive struc-
tures, so we initially decided to use a separate MongoDB database. However,
we eventually switched to storing the rubric in Postgres directly (more details
are provided in the chapter on used technologies). The RubricLinker model that
contains a Rubric is a simple bridge for connecting a project to its rubric.

Rubric updates are performed with JSON patches compliant with RFC 6902
standard. Such a way of updating the rubric is chosen to lay out the basis for
rubric history tracking as one of the possible extensions of the application. For

31
Design in Detail 5.2- Back End

these purposes RubricPatch and RubricHistory models are created. Instead of


sending an updated version of the complete rubric and saving it in the database,
the front end sends an array of JSON patches that instructs the back end about
how to update the rubric. Each patch consists of an operation to perform (edit,
add, delete, etc.), path to the relevant part of the target JSON and, for some
operations, value of the update. Each patch is additionally processed before
being applied so that the application can perform additional actions based on
the type of patch. Such a structure is used to allow for possible future extensions,
such as sending an email notification to graders if a new criterion is added to
the rubric. Since each patch is stored in the RubricHistory model, it makes it
not too hard to create a page where the history of rubric changes will be shown
in the future.

The RubricService also provides the functionality to download and upload a


rubric as a JSON structure in our custom data format. Last but not least,
using Itext7, the RubricService can generate a PDF version of the rubric with
rich text formatting and proper structure for graders to download.

5.2.4 Graders and Submissions

Since not all TAs in a course participate in grading, the application provides
a way to select which TAs will be involved in grading. This information is
stored in the GradingParticipation model, that links users (TAs and teachers)
with projects. Similarly to CourseParticipation, this model provides a set of
necessary permissions for users who acts as graders in a project. Two roles
exist within a project - (TEACHER and TA_GRADING).

As was mentioned above, submissions are fetched when a project is imported


into the application. Each Submission instance stores the Canvas group id (if
any), submitter’s identity and submission date. When the project is synchro-
nised with Canvas, previously fetched submissions are never removed or altered,
so the data integrity is preserved.

Furthermore, SubmissionAttachment and SubmissionComment models are used


to store submission comments and links to submission attachments. Thus, there
is no need to manually download and search through submissions on Canvas if
something needs to be found - all the information related to submissions is
stored in the application.

The Label model represents a label that can be assigned to a submission. For
instance, in case a submission is involved in a potential case of plagiarism, it
can be marked with a label “Plagiarism”. This label will be saved to the Label
instance and linked to the respective submission and project. Labels are reusable
within projects.

32
Design in Detail 5.2- Back End

ProjectController contains all the endpoints related to managing graders. It


allows requests to assign or reassign submissions to TAs and return an assigned
submissions back to the unassigned group. Only teachers have full access to
managing graders while the only action that can be performed by a TA is
returning an assigned submission.

Despite the limited submission-related functionality on Canvas, our application


provides a way to add students to submissions or remove them from submissions.

5.2.5 Grading

Grades can acquire a variety of different forms, from simple points to pass/fail
grades. However, to save time, we decided to focus on the simplest type of grades
– points with step-based increments. Each grade given is stored in the Grade
model, that contains information about the grade given, (optional) explanation,
time of grading, identity of the grader and grade’s status. Grades can either be
active or not. If there are multiple grades provided for a single criterion of some
submission, only one such grade can be active at a time.

Furthermore, it happens sometimes that students’ grades need to be differen-


tiated, even if these students submitted a single shared submission (e.g. there
might be bonus points given to one of the students). Thus, our application
allows creation of separate grading sheets for each student within a submission
that are stored in the Assessment model.

Since graders can change students associated with a submission and change
grading sheets for every submission, a very flexible database structure is re-
quired. To accommodate such functionality, a three-way association model As-
sessmentLink is created that connects students (of type User), grading sheets
(of type Assessment) and submissions (of type Submission). As students need
to have a single final grade in the end, only one grading sheet can be active at
a time, even though a student might have multiple submissions and multiple
grading sheets.

It’s common to run into grading issues, so the application provides a way to
create and track them. Issues are stored as Issue instances and linked to grading
sheets and users (both creators and addressees).

5.2.6 Feedback

Finally, when the grading process is completed, we offer a way for teachers
to send feedback to students. Currently, the default way is to send feedback
through Canvas messages. To send such feedback a new FeedbackTemplate

33
Design in Detail 5.2- Back End

instance for email message is created via a request to ProjectService which


stores name, subject and body. EmailUtils appends student’s feedback to such
a message and then it’s delivered via Canvas to student using Canvas API.

We also developed a way to send feedback through Gmail by asking users to


authorise our application to obtain their Gmail access key and allow sending
emails. Then, PdfUtils is used to generate a PDF file of student’s grading sheet
that will be included to the email body.

In both cases, regardless of which method of sending feedback is used, each sent
email is recorded in the FeedbackLog model and stored to keep track of which
students received feedback. When the teacher wants to send feedback only to
students who haven’t received feedback before, they can do it by choosing not
to send feedback to students who already have FeedbackLog entries stored for
them.

5.2.7 Transactions

By default in Spring, each database query is executed in its own transaction.


Therefore, this results in creation of multiple transactions that are running in
parallel and might cause data corruption. To preserve ACID characteristics of
data in the database (Atomicity, Consistency, Isolation and Durability), trans-
actions are used in the back end.

Database operations usually run in groups, not as a single operation. Running


all of these related operations in a single transaction ensures database atomicity,
since either all the operations in the transaction are executed successfully and
the data is committed back to the database or these operations are rolled back
in case of any errors or clashes.

Transactions ensure that each execution will not directly update the database
until the transaction ends and is committed to the database. This prevents
parallel queries from retrieving data that is in the middle of being processed, in
order to preserve isolation.

Normally, JDBC (Java library used for accessing databases) has its own multi-
stage implementation of transactions: retrieve the connection, set auto commit
to false (to turn off default one-query transactions), execute operations, commit
or roll back on errors. However, this way of handling transactions usually makes
scaling and dividing services into parts difficult.

Spring Boot provides transaction annotations (e.g. @Transactional annota-


tion) that help generate boilerplate code, proxies, callbacks and exceptions.
They allow easier management of logical transactions in Spring that are con-
verted to underlying physical transactions in JDBC.

34
Design in Detail 5.2- Back End

There are a few parameters for the @Transaction annotation that are crucial for
transaction management - propagation and rollbackFor. Propagation can be
one of the following types: required, supports, mandatory, require_new or
not_supported. In our project, we mostly use required and mandatory types.
required is the default option in Spring Boot. With this option, JDBC creates
a new physical transaction for this logical transaction if there is no transaction
opened or joins this logical transaction with the existing transaction. Therefore,
this option was used for methods that are called by controllers. On the other
hand, mandatory ensures that when called, the logical transaction needs an
existing physical transaction to be joined with. Therefore, this option is suited
for methods that exist as supplementary methods to other methods in services.

Then, the rollbackFor parameter is used in transactions that are designed


to modify the database. These transactions are called with an implicit excep-
tion throwing functionality enabled so that when this exception is thrown, the
method catches it and rolls back the transaction to preserve the database’s in-
tegrity. For example, if there are exceptions thrown while a project is being
imported into the application, the transaction will be rolled back. On the other
hand, if a project was imported without a transaction, then it might lead to
inconsistencies in the database if something wrong happens during the import.

5.2.8 Locks

As mentioned before, transactions help preserve the ACID properties. However,


besides ensuring ACIDity, the application also needs to ensure that concurrent
modifications or parallel queries will not result into wrong data operations and
states. This can be achieved by employing locks in Java.

There are two types of database locks – optimistic and pessimistic. The opti-
mistic lock is essentially akin to the version control. Data fetched by repositories
that are bound to optimistic locks will get the current version of the fetched
data. Before committing, the transaction compares the version of the fetched
data to the current version of the data in the database. If the versions are
different, then there are changes that were possibly made by others. Thus, an
exception is thrown and this transaction is rolled back. One of the advantages
of using optimistic locks is that transactions can execute freely which otherwise
would have to wait if pessimistic locks are used. On the other hand, in case of
complex transactions, it might be inefficient to retry them over and over again.
In such cases, pessimistic locks can be used. Before executing methods anno-
tated with the pessimistic lock annotation, the transaction will try to obtain
the lock and execute the query only when the lock is obtained. Otherwise, the
transaction will have to wait until the lock can be obtained. There are 3 type of
pessimistic locks in Spring JPA: PESSIMISTIC_READ, PESSIMISTIC_WRITE and
PESSIMISTIC_FORCE_INCREMENT.

35
Design in Detail 5.3- Front End

A PESSIMISTIC_READ lock allows transactions obtain a shared lock that prevents


the data from being updated or deleted. Therefore this lock is commonly used
when the transaction only require reading from these Entities and preventing
dirty reads. On the other hand, PESSIMISTIC_WRITE and PESSIMISTIC_FORCE_INCREMENT
will force the transaction to obtain an exclusive lock on the queried Entities, that
would prevent all of the others transaction from reading, updating or deleting
the data. In the case of PESSIMISTIC_FORCE_INCREMENT, each successful com-
mit will increment the version value that is fitting for version control while using
pessimistic locks.

In the back end, the PESSIMISTIC_READ lock is used when the transaction has
no intentions to write to database and PESSIMISTIC_WRITE is used when the
transaction is designed to write (update or delete) the queried entities. We use
PESSIMISTIC_FORCE_INCREMENT for rubric updates to ensure that the user is
updating the current version of rubric.

There are a few limitations in Postgres with regards to JPA locks. For example,
the timeout for attempting to obtain a lock cannot be set as a parameter in JPA
due to Postgres’ limitations. Postgres’ default timeouts are essentially infinite
and can only be adjusted by editing the configuration file of the database. While
pessimistic locks are useful, there are cases where deadlocks can happen. In such
cases, an exception should be caught and the transaction (or query) has to be
rerun. These cases are hard to catch and test though, as they are very rare.

5.3 Front End

This section will dive deeper into the part of our application that is in a direct
contact with our users – the front end. The front end will be described in terms
of interactions it offers and will reflect the functional requirements outlined
above.

A clean and minimalist look was aimed for when designing the front end. In
order to better separate data visually and present it in an intuitive way to the
user, cards, shadows, an abundance of white space and vibrant standing-out
colours that aim to bring the user’s attention are used.

To make the user aware of a page’s place within the context of our application,
most of the pages feature navigation breadcrumbs.Links are used for navigation
between different screens as they present a natural and intuitive way of navigat-
ing around the Web. Buttons, including full-text and icon buttons, are mostly
reserved for performing actions.

Contextual menus are used to provide group interactions to the user instead of
presenting the user with a swarm of action buttons. Standalone actions that

36
Design in Detail 5.3- Front End

stand apart from the normal interaction flow are taken out into modals. For
instance, modals are used to create issues in the grading screen and to select
graders in the grading screen.

As the application distinguishes between several types of users, the front end is
able to present the right bits of information and components to the requesting
user depending on the user’s role. As an example, only users with the Teacher
role can see the administration buttons in the project screen. As TAs do not
have access to most of this functionality, their front end does not show them
these options altogether (of course, the actions are also fully protected in the
back end).

5.3.1 Sign-in and import

The user’s interaction with the application starts when they initiate the OAuth2
sign-in process using their UT account. The user’s session cookie is then stored
in the browser while the server stores the Canvas access token and takes care of
interaction with the Canvas API when the user’s actions require it. The user’s
profile information that is retrieved from Canvas on sign in is then stored in
Redux to allow easy access to it from any place in the application.

Figure 5.1: The sign in screen.

After signing in, the first screen that the user sees is the home screen. This screen
is used to observe the list of user’s courses, see the to-do list (which contains
the list of projects in which this user has assigned submissions to grade) and
initiate importing new courses from Canvas. The to-do list presented on the

37
Design in Detail 5.3- Front End

right always stays in that position to keep being accessible to the user, even if
the user scroll down the page.

Figure 5.2: The Home page.

Figure 5.3: The Course page.

After the user has selected a course, they see the course overview page. Cur-
rently, the only purpose of this page is to present the user with the current list
of imported project in the course and enable importing new projects. We chose
not to import all projects in a course automatically as it happens often that
a single Canvas course contains dozens of assignments, not all of which might
actually be real project assignments that need to be graded. When a project is
imported, only teachers and TAs that are added as graders to the project can

38
Design in Detail 5.3- Front End

access it.

The course page also exposes one more button to teachers. Besides being im-
ported, course data can also be synchronised with Canvas to update the course
user and project (assignment) list.

Selecting a specific project leads the user to the project page. Here teachers can
see a list of buttons with interactions and links to other parts of the application.
The teacher can navigate to the project rubric, move to see the list of submis-
sions or students, open the feedback page, export grading data to an Excel file
or switch to the graders page. Moreover, a distribution of grades in the project
is displayed within that screen too.

Similarly to the sync functionality of a course, a project can also be synchronised


with Canvas to fetch new submissions.

Figure 5.4: The Project page.

5.3.2 Students and Submissions

The users can see the list of students participating in a project and the list of
submissions for the project in the students and submissions screens, respectively.
Both pages support searching and filtering items for more convenient browsing.
Searching is performed with normalisation (NFD) to account for possible ac-
cented characters (and not only) in user names. Data in the submissions page
is presented in the form of cards as each single entry contains too much infor-
mation to be fit in a table, in contrast with the list of students in the students
page for which the table-based representation is sufficient.

Each individual student’s information, including the list of student’s submissions

39
Design in Detail 5.3- Front End

Figure 5.5: The Students page.

for the project, can be viewed on the student page.

Similarly, the users can select a submission and navigate to that submission’s
page. There they can see more details of the submission, such as the time it was
submitted, the user who submitted it, the list of comments left on Canvas and
the list of attachments. Each attachment can be downloaded (from Canvas) by
clicking on the button next to its name. This eliminates the need to download
a bundle with all submissions on Canvas and search for the needed submission
withing that bundle. The application displays the size of each download. All
in all, the users will likely not want to initiate a download of several gigabytes
of data while being on a limited data plan, so displaying the size information is
vital.

In the application, each submission fetched from Canvas is associated with a set
of students. It might be a group of students or a single student only, depending
on whether the student who submitted the work on Canvas was in a group or
not at the time of submitting. Although it’s not possible to manually link new
students to submissions on Canvas, our application supports such functionality.
In a situation when a student, who is supposed to be part of a group, submits
the project individually, the teacher can manually link other group members to
this submission and grade these students together.

In addition, a similar flexibility is offered for grading a submission. If students


associated with one submission require separate grading, the app provides a way
to split their grading into multiple grading sheets. A new grading sheet can be
created by either copying the existing one or starting a new empty sheet.

40
Design in Detail 5.3- Front End

Figure 5.6: The Submissions page.

If a submission needs to be marked with a note or label, it can be done in the


submission page. A new project-wide label can be created in the labels modal
and might be reused by all graders across the project. A submission can have
multiple labels attached to it and they all will be visible on the submissions
screen. Such functionality will be useful when, for instance, the grader wants
to distinguish submissions of resit students from regular module students.

Figure 5.8: The label selection win-


Figure 5.7: The edit label window. dow.

41
Design in Detail 5.3- Front End

Figure 5.9: A submissions with a label added.

5.3.3 Grading rubric

The rubric is displayed in the rubric page. It’s presented as a hierarchical


structure of sections and criteria. Each criterion consists of a name, description
and grade information. When we were making the choice for representing rubrics
within our application, we identified that such hierarchy covers most of the
rubrics we encountered and should be flexible enough to express any grading
ideas.

The criterion’s body can be edited with a rich text editor that has support for
such editing options as making text bold, italicised, underlined or putting this
text in an unordered or ordered list.

Rubric can be edited in the edit mode by making changes to criteria and sections.
New sections and criteria can be added as children of the existing sections by
choosing the respective options in the contextual menu of each section. Criteria
are terminal and cannot contain subcriteria within them.

When the user is done with editing, they can press the save button to update
the rubric in the database.

Rubrics can be downloaded in the PDF format and can be thus be incorporated
into module guides. Also, rubrics can be downloaded and uploaded in our
custom JSON-based format to be reused across different project in, possibly,
different years, although we plan to support such rubric copying functionality
within the application directly.

42
Design in Detail 5.3- Front End

Figure 5.10: A view of a rubric.

Figure 5.11: The rubric editor.

43
Design in Detail 5.3- Front End

5.3.4 Assigning graders

Graders can be assigned to grade submissions in the graders screen. As the first
step, the teacher can select TAs who will be responsible for grading submissions
for the project. Usually, not all TAs participate in grading, so the application
provides a way to make such selection of graders. Only the TAs selected will
have access to project grading data. By default, all teachers of the course are
automatically selected as graders for a project.

Figure 5.12: Modal for selecting TAs to grade.

Then, a submission can be assigned to a grader by opening a contextual menu


with a modal offering a choice of all graders in the project. The application also
supports a balanced randomised assignment of graders to submissions, so there
is no need to assign each submission manually. If a grader wants to return the
assigned submission (i.e. move it back to the ’unassigned’ status), such action
can be performed by clicking on the respective button. Only teachers are able
to assign submissions while TAs are only allowed to return the submissions they
are assigned to grade.

44
Design in Detail 5.3- Front End

Figure 5.13: Modal for (re)assigning graders to submissions.

5.3.5 Grading

The grading page is the central piece of the application as it present a way to
actually grade submissions. The page consists of three parts - the rubric panel,
the grading panel and the issues panel.

The rubric panel display the project rubric so that the grader can directly browse
it when grading. Each criterion can be clicked to show its contents. To quickly
move back and forth through the criteria, the user can use the arrow buttons
at the top of the rubric panel. A criterion is marked as graded in the rubric if
a grade is entered for it in the grading panel. Such a criterion can be visually
distinguished from ungraded criteria by a check mark and the grade displayed
next to it.

The grading panel is split into two parts: the upper serves as the means to
enter a grade and optional explanation and the lower part presents the grading
history for the chose criterion. Even if a criterion is already graded, it can be
regraded by a teacher or by the submission’s assigned grader if needed. The
grade that applies to the criterion can be selected among the grades by clicking
on it.

The rightmost part - the issues panel - allows users to resolve issues related
to grading of submissions. For instance, if the teacher receives an email from
students with a question regarding grading, the teacher can create an issue
with that question and address the submission’s grader. The users can opt to

45
Design in Detail 5.3- Front End

Figure 5.14: Grading view

receive notifications about created issues, so if these notifications are enabled,


the addressee will receive an email telling that an issue has been opened in the
submission. After fixing the issue, the grader can mark the issue as solved and
close it.

At one point we also considered showing submission attachments directly in the


browser, but after holding several rounds of discussions about this feature we
decided to postpone it and concentrate on more important requirements.

5.3.6 Sending feedback

As the last step of the grading process, the application offers a way to send
feedback to students. Feedback consists of a filled in rubric, so the student will
be able to see not only a list of grades but also any additional comments or
explanations provided by graders.

Feedback can be sent directly via Canvas using Canvas’ built-in messaging sys-
tem. The generated feedback is in the form of simple textual representation by
default. Teachers can create email templates that consist of an email subject
and body and are used to send feedback to students.

46
Design in Detail 5.3- Front End

Figure 5.15: Feedback page

47
Chapter 6

Testing

In order to deliver a product that will successfully fulfill the demands of the
stakeholders, it is important that no unexpected behaviour occurs while it is in
use. Hence, thorough testing must be done in order to find as many potential
issues as possible before the product is rolled out to stakeholders. For this
reason, a plan for testing was made and executed, which will be described in
this chapter.

Testing of software can generally be split up into several categories, from low to
high level. Namely unit testing, integration/system testing and usability testing.
Although naturally the chronological order of these categories within the project
development cycle is to some extent the same, in the project workflow, this
chronological order is not so strict and all categories are to some extent tested
from the very start. This is done both in order to show visual progress in a
reasonable time to the stakeholders and to catch wider problems (for example,
a bad user interface) before it is too late to change significantly.

Firstly, unit testing is about testing individual components (units) in isolation,


often automated. This allows for the discovery of crucial bugs early in the
development process and also makes the next forms of testing easier, as it is
much harder to trace the causes of bugs in higher level testing. The downsides
of this form of testing is that bugs relating to a connection between components
will go undetected and writing them is rather time-consuming. Hence, due to
time constraints, comprehensive unit testing was somewhat restricted to only
the most complex components.

Next, integration and system testing is a step above unit testing, where the
behaviour of multiple components combined is tested. For instance, this could
be testing if the front end interacts with the back end correctly to fulfill some
query. These could be automated for better completeness, so in this project

48
Testing 6.1- Automated Testing

we decided to combine manual testing with automated testing of controllers,


services and models. Manual testing was mostly done in the process of develop-
ment where after one integrated functionality was implemented, it was tested
by going to its corresponding page in the front end and using it with as many
scenarios as could be thought of, while inspecting the database or the console
of the back end to see whether the appropriate result was achieved.

Finally, usability testing is unlike the previous two categories, in that it is not
focused on ensuring that the developed functionality works correctly, but instead
on ensuring that the developed functionality works in the best way suited to
the user. Hence, this seeks to test the user experience (UX), and with that the
interface being used for it. Some examples of this would be testing whether
certain buttons will be clear in their functionality to the user, if some process
is deemed to be easy to do, etc. To some extent this could be done individually
by the developers, (especially as many have TA experience) by just using the
application as a real user would from time to time, but developers will obviously
have a biased view of their own work and they have much more knowledge on
how the app works than any other user would. Hence it is important to let
stakeholders with no prior knowledge of the application test it, and for this
some meetings were held.

6.1 Automated Testing

With the back end, unit testing (and partially integration testing) was suitable
for service classes, which implement the business logic of the application. Most
of the dependencies that were there were mocked; what this means is that the
dependent object is not actually created, but instead a version where every
function returns null is used. Of course, testing would not be possible if every
dependent function just returned null, so a technique called stubbing is used.
This means manually giving return values to a mocked object’s functions given
certain parameters, allowing for simulating the function actually being called.
An example where this is useful is in database retrieval, where without actually
making use of the database the desired object is pretended to be retrieved from
it using stubbing.

For mocking the Mockito library was used. This, in conjuction with JUnit,
Java’s unit testing framework and AssertJ for more readable and helpful asser-
tions, provided everything needed for unit testing the application. Most tests
were similar, and can be summarized in four steps. First, the test models are
created (e.g. a rubric to test something rubric related), then any expected return
values are defined, next the mocked methods are stubbed and finally the return
value of the tested function is compared to the expected value. In some cases
there were other components that could be compared, for example a certain field
in a test model which should have been changed by the function. Furthermore

49
Testing 6.2- Usability Testing

there were some cases where an exception should be thrown, and this was then
asserted and it’s messages checked to correspond to the expected message. For
void methods sometimes the opposite was the most useful, checking instead that
no exception was thrown given intended input variables.

6.2 Usability Testing

We held several testing sessions with Tom van Dijk, the module coordinator
of Software Systems, and teaching assistants to test both functionality and
appearance of the application. These sessions were of immense use as they
let us see our project from a new perspective each time and led to substantial
improvements.

In the first meeting with Tom, we discussed the overall approach to handling
grading data, the range of supported features and the overall design of the
application. As a result of the meeting, we:

• completely overhauled the back end and switched to the third version
of our database design (described in more detail in the section on the
database structure)
• added support for issues, labels, exporting grading data as an Excel spread-
sheet, manually adding students to submissions and creating separate
grading sheets
• reworked the design of the front end to make it more clear and intuitive
• had topics for further discussion with the supervisors, such as showing
feedback in our application directly

The second session with Tom was mostly concentrated on finding implementa-
tion errors and testing the user interface (UI) of the application. No crashes
or implementation errors were registered during the session, however, multiple
suggestions for UI improvements were provided.

Among discussions with TAs, the most important one took place with an experi-
enced TA, Martijn Verkleij, who provided us with lots of invaluable suggestions
for improvements and fixing UI mistakes. The meeting was focused purely on
the UI. Among the suggestions we received were:

• add tooltips to icons or provided explanations to certain, possibly unclear,


actions
• improve the way rubric is displayed in the grading screen (move control
buttons to the top, grey out disabled buttons etc)

50
Testing 6.2- Usability Testing

• adjust default grading settings to start from the minimal grade of 1


• display the full timestamp of a grade
• rework the way students are added to submissions or moved around as-
sessments Martijn also provided us with several suggestions with regards
to secure hosting of our application, which were helpful in setting it up
later.
As a result of the second session with Tom and the session with Mar-
tijn, we started a complete rework of the front end design to incorporate
the feedback received. We dropped React Bootstrap, switched to React
MaterialUI and overhauled the front end from the ground up.
Overall, the sessions were of immense help for our work and let us see the
project from the actual user’s perspective.

51
Chapter 7

Development Process

7.1 Introduction

To get any sort of project done, there needs to be some sort of organization.
Given the current situation these all had to be done digitally, which was not
ideal but ultimately did not provide too much problems.

For communication within the group a Discord server was used, which also
became a repository of information such as diagrams, links to tutorials to work
with certain libraries, etc. Sometimes some simple questions or discussions
were held in chat, but for more complicated matters were discussed in a voice
call on the server. The time and frequency of these were mostly based on the
circumstances, and thus there was no strict organization of them.

Our meetings with our supervisors were scheduled every Wednesday via Mi-
crosoft Teams, most of the time at 17:00. Here the progress over the week
was outlined and some uncertainties cleared up. Since the supervisors have lots
of experience grading projects themselves, their insights proved very useful in
making certain design decisions.

As part of the module were the organized peer reviews. Here a short presentation
outlining the progress, a demo, and future plans to other groups was held and
after that a short discussion with one other group. Since other groups are
working in different domains, there were limited insights to be gained from
them about this project specifically. However, a discussion and comparison of
the structure of the deliverables all groups share, such as this report, the manual
and the poster, was useful.

Finally, there were also meetings with others relevant to our project (as men-

52
Development Process 7.2- Project timeline

tioned before), either for testing, feedback or integration with Canvas.

The details of the project timeline are described in the next section.

7.2 Project timeline

We split the project work into several phases:

• planning and research


• Canvas integration
• implementing ”Must”, ”Should” and ”Could” requirements
• testing

• wrapping up

7.2.1 Planning and Research

In the first week, we spent time on designing and setting plans/milestones for our
application and its development process. Then, after the introductory meeting
with the supervisors, we worked on specifying and refining the requirements for
our application as well as prepared mockups for the design.

In the second week, we conducted an extensive research about possible frame-


works, languages and tools that could be used for project development (de-
scribed more in the section on technologies we used), improved our mockups
and discussed all the materials with our supervisors. Lastly, we prepared and
submitted the project proposal to our supervisors for review.

7.2.2 Canvas integration

Canvas integration is a major part of our application, so by following the advice


from our supervisors, we started working on it early. We had multiple meetings
with Ard Kosters in order to gather more information on the tools that are used
at UT, usage of Canvas Api, the inners of OAuth2 flow in Canvas and other
development-related topics. These meetings gave us a clear picture of what we
had to do and helped us with making the integration with Canvas work.

53
Development Process 7.2- Project timeline

7.2.3 Implementing ”Must”, ”Should” and ”Could” Re-


quirements

The largest phase of the project work was dedicated to actual work on imple-
menting the requirements.

Setting up Back end, Front end and Database

We started with setting up most of the boilerplate code for the back end and
making our rudimentary front end communicate with the back end. As Canvas
integration already worked at that time, we could already experiment with
retrieving data from Canvas and displaying it in the front end. Designing and
implementing the database in Spring proved to be very challenging, so it took
as a fair amount of time and effort to make it work correctly. Further research
and testing made us change the database design several times before arriving
at the current solution.

Implementing features

After having a skeleton for our project ready, we started implementing our
requirements in batches. The first part revolved around fetching user data,
submissions and the creation of rubric. Initially we used MongoDB for the
rubric, but later merged it with our main PostgreSQL database.

After having most of the initial parts of the application ready, we moved to
working on the grading screen. It took us several iterations to arrive at the
current design of it.

Testing

After having the first prototype ready, we held several testing sessions with
potential users of our application. These sessions turned out to be very useful
and provided lots of invaluable insights. As a result, we merged this phase with
the implementation phase and started reworking parts of the application that
required changes.

Wrapping Up

The last stage of the project work took place in weeks 9 and 10 of the project
when we started the work on the project report, poster, presentation slides,

54
Development Process 7.3- Responsibility and Contribution

documentation and user manual. Also, we handed in the reflection report which
was a substantial and time-consuming part of weeks 8 and 9.

7.3 Responsibility and Contribution


• Yevhen – managing group, presenting, front end design and implementa-
tion, database design and back end, report, reflection essay, poster

• Triet – database design and back end, report


• Dylan – parts of front end, automated testing, preparing slides, report,
reflection essay
• Robin – peer review presentation, report, reflection essay, poster

• Adrijus – parts of front end

55
Chapter 8

Conclusion

8.1 Extensions and Upgrades

We started the development of our application with extendability and modu-


larity in mind. This flexible structure allows us to relatively easily extend the
application and fit it with new features in future.

Among the features that can be considered for future updates are:

• enable rubric history and add notifications about rubric changes


• add more statistical measures calculated (mean and median grade, percent
of students passed)
• add more grade types (pass/fail, bonuses, penalties etc.)
• add grade formulae that can be used to calculate final scores
• upload (individual, not group) grades to Canvas
• enable logging
• simplify grading sheets and allow them to contain only the final grade
• integrate Outlook OAuth2 for sending feedback

8.2 Final Remarks

The project we worked on was a great opportunity not only to hone our coding
and working skills but also to put them for good use - make the grading process

56
Conclusion 8.2- Final Remarks

simpler and less stressful for teachers and TAs.

All in all, our group is proud of the work we did and the system we developed.
We learnt a lot by working on a real-life project with real-life implications and
within a real-life software development process.

Of course, the pandemic made it harder to efficiently manage the process and
imposed some limitations on the project activities and process. However, we
believe that despite these serious obstacles we managed to successfully conduct
all the necessary activities, go through all development phases and deliver a
working product of good quality.

57
Appendix A

User Manual

This manual is structured mostly along the pages of the application. Each
section is centered around a page and what the functionalities are on that page.

A.1 Signing in and out

A.1.1 Signing In

The first thing you are greeted with when loading the application is the sign in
screen (Figure A.1). This will redirect you to a screen where you can sign in
with your UT account. After this, you will receive a prompt (Figure A.2) where
you need to authorize your Canvas account with the application, so that all the
necessary data from Canvas can be retrieved. Accepting will lead into the home
screen (see section A.2).

58
User Manual A.1- Signing in and out

Figure A.1: The sign in screen.

Figure A.2: Authorization with Canvas

59
User Manual A.1- Signing in and out

A.1.2 Signing Out

To sign out, simply open the sidebar on the left (Figure A.3) and click the
button on the bottom-right of the sidebar. This will automatically redirect you
to the sign in page.

60
User Manual A.2- Home

Figure A.3: Opened side bar.

A.2 Home

The Home page is the first thing you see after logging into the application. Here
you can see your currently active Courses and import them. On the right you
can see the to-do list and a list of grading issues addressed to you. Throughout
the application, the sidebar on the left can be used for navigation. Another
commonality throughout the application for teachers is a Sync button, which
synchronizes data with Canvas.

61
User Manual A.2- Home

Figure A.4: The Home page.

A.2.1 Import Courses

The import button allows you to select any course where you have a TA or
teacher role. Once courses are selected, they will show up on the home page.
Clicking on any one leads you to its page.

62
User Manual A.2- Home

Figure A.5: The Home page with no courses imported.

Figure A.6: The import course window.

63
User Manual A.2- Home

Figure A.7: The Home page with a course imported.

A.2.2 To-Do List

The To-Do list gives an overview of the grading tasks that are assigned to a
grader, and provides a quick link to them.

64
User Manual A.2- Home

Figure A.8: The to do list.

65
User Manual A.3- Course

A.2.3 Issues

Issues addressed to you are also quickly available from the home page. Click
on them leads to the assessment where the issue was found. For more details
on issues see section A.8.3. Using the button on the top right, you can filter
between all, resolved or open issues.

Figure A.9: The pending issue list.

A.3 Course

The Course page exists mainly as a hub for projects within the course. From
here you can import projects into the application or synchronize them with
Canvas (as a teacher).

66
User Manual A.3- Course

A.3.1 Import Projects

Importing projects functions the same as courses. Clicking on the import button
presents a list of projects that can be imported. After a project is imported, it
will be shown on the Course page. Clicking on one will lead you to the project
page for that project (see section A.4).

Figure A.10: The Course page with no projects imported.

67
User Manual A.3- Course

Figure A.11: The import project window.

68
User Manual A.3- Course

Figure A.12: The Course page with projects imported.

69
User Manual A.5- Students

A.4 Project

This is the part of the app that provides links to almost all the functionality, via
the buttons in the administration card. Along with that there is a statistic view
for the selected project, at the time of writing this only shows score distribution.

Figure A.13: The Project page.

A.5 Students

This page shows a list of students and their student number, email, final grade
and amount of submissions. Using the search bar at the top the list can be
filtered.

70
User Manual A.6- Graders

Figure A.14: The student overview.

A.6 Graders

The Graders page serves as the place where teachers can assign TAs as graders
for submissions.

71
User Manual A.6- Graders

Figure A.15: The Graders page.

A.6.1 Edit Graders

The ”Edit Graders” button on the top right allows you to select TAs to assign
as graders. After this is done, the graders can be assigned to a submission. How
this is done follows in section A.6.2.

72
User Manual A.6- Graders

Figure A.16: The edit graders window.

A.6.2 Assign Graders

Down in the unassigned section the submissions that are not assigned are visible.
Clicking on the button on the right of any such submission opens the assign-
ment window, where a grader can be selected to be assigned to this submission.
While this is not necessary for teachers, TAs cannot grade the submission other-
wise. After a grader is assigned the submission will be moved into the assigned
grader’s card. From there the same button as before can be used to reassign
the submission, except there is an option to ”Return Submission”. This makes
the submission unassigned again.

73
User Manual A.6- Graders

Figure A.17: The assign graders window.

74
User Manual A.6- Graders

Figure A.18: The reassign window.

75
User Manual A.7- Submissions

A.7 Submissions

On this page an overview of all the submissions of a project is given, and from
here you can select one to go to its detailed page for grading. Also visible is
whether or not a submission is assigned.

Figure A.19: The submissions page.

A.7.1 Submission Details

Clicking on a submission from within the Submissions view will lead you to its
details. From here you can see and edit labels, add or remove students manually
to a submission, view comments, download attachments and manage and go to
its grading sheet.

76
User Manual A.7- Submissions

Figure A.20: The submission details page.

Labels

Labels provide a way for a grader to mark something about a submission, for
instance the submission is suspected to be fraudulent. Pressing the Edit Labels
button allows you to create a label, by giving it a title and a colour. After that
it can be enabled for the submission. Labels are saved across the project, so
other submissions can easily be given the same label. After the label is selected,
the title and colour will show up above the details page and also in the list of
submissions.

Figure A.22: The label selection


Figure A.21: The edit label window. window.

77
User Manual A.7- Submissions

Figure A.23: A submissions with a label added.

Manage students

In the students card, students within the project can be added or removed
from the submission using the plus and trashcan buttons respectively. Adding
a student provides a window where you can select the student and to which
grading sheet he will be added. (See more on grading sheets in section A.7.1)

78
User Manual A.7- Submissions

Figure A.24: The add student window.

Download attachments

The attachments of the submission can be downloaded using the download but-
tons next to them. The submission is downloaded directly from Canvas and the
button just provides a link to it.

Manage grading sheets

A grading sheet is how the application decouples submissions from graders. A


grading sheet contains the actual grading of the criteria in a rubric. This allows
for different grades for different persons; for instance when one member in a
group has contributed very little to the project. Using the button on the right
of a student, a new empty sheet can be made for that student, or the existing
one can be copied. When there are multiple sheets for a submission, a student
can also be moved from one to another using the move student window (see
Figure A.25).

79
User Manual A.8- Grading

Figure A.25: The move student window.

A.8 Grading

The grading view can be opened from within a submission detail page as de-
scribed in section A.7.1. TAs can only open this view if they are assigned to the
submission. This view shows the rubric for the project, the grade editor, grade
history and issues.

80
User Manual A.8- Grading

Figure A.26: The grading page.

A.8.1 Rubric

The Rubric as built using the Rubric Builder (see section A.9) can be seen here.
Here a criterion can be selected to be graded. If the criterion is already graded,
it will show the grade on the right.

A.8.2 Grade Editor and History

Once a criterion is selected from the rubric, the grade editor will open automat-
ically if the criterion is ungraded. If it was already graded, you need to press
the pencil button to regrade it. Any grade that is saved will be added to the
history below, and a teacher can select any grade from it to be the active one
by clicking on it.

A.8.3 Issues

On the far right of the screen is the issues panel. Issues are a way of contacting
other graders about what is graded, in case they believe something is not right
with the grading.

81
User Manual A.8- Grading

Creating an Issue

Creating an issue can be done using the plus button where a window will open.
Here an addressee for the issue can be chosen, who will receive an email noti-
fication if they have that enabled in the settings (see section A.12), along with
the subject and description. Finally there is the reference issue, which can be
used in case an issue relates to another issue. Once the issue is created, it will
remain as an open issue visible here and on the home page, until anyone presses
the check-mark button to resolve it.

Figure A.27: The initiate issue window.

Resolving an Issue

Pressing the check-mark button on an open issue leads to a window where the
issue can be resolved. A solution needs to be given and then saved, which will
mark the issue as resolved. The issue will still remain in the system, and remains
visible if not filtering on resolved issues.

82
User Manual A.9- Rubric

Figure A.28: The resolve issue window.

A.9 Rubric

On the Rubric page the project rubric can be viewed, and teachers can build
and manage a rubric for their project. The page also allows for importing and
exporting of rubrics. Rubrics are entirely composed of criteria and sections, with
a section being able to contain more sections and criteria. Criteria are ultimately
the elements that grades can be applied to, while sections just provide structure.

A.9.1 Viewing a Rubric

If there exists a rubric for the project, the rubric viewing page is shown. This
allows both teachers and TAs to navigate through the rubric via the left panel.
Upon selection of a criterion or section its details are shown on the right.

83
User Manual A.9- Rubric

Figure A.29: The rubric viewing screen.

A.9.2 Editing a Rubric

Creating a new rubric

When the rubric is empty, you are presented a button to edit a rubric. After
this a criterion or section can be selected, which will form the first element of
the rubric. No matter what option is selected, you will be led to the rubric
editing view (Figure A.31), which is described below.

84
User Manual A.9- Rubric

Figure A.30: The rubric window when there is no rubric made.

Editing an existing rubric

When there is already an existing rubric, the rubric page will first just be a
viewer for the rubric. To edit an existing rubric, the pencil button on the top
right accesses the same rubric editor view as creating a rubric does, which is
described below.

85
User Manual A.9- Rubric

Figure A.31: The rubric editor view.

Managing sections and criteria

Here the menu buttons on the right of a section or criterion can be used to
manage that element. For criterion that just means deletion, but for sections
more sections or criteria can be added as elements of the section. Furthermore
at the top there is the field labeled “Outline” which is the root section, the title
of which cannot be changed and cannot be deleted, but otherwise functions the
same as a regular section.

Editing sections and criteria

On the right the criterion or section selected can be edited. Sections only have a
title, while criteria contain alongside a title, a description and the grade fields.
The description of a criterion can be made using a rich text editor, similar
to ones found on other programs. This allows for features such as lists, bold
text, italic text, etc. Finally, for a criterion grade a minimum, maximum, step
size and weight can be specified. The weight is a value between 0 and 1 that
determines how much impact the grade has on the final grade. Once the rubric
is complete, the save button on the top right needs to be pressed to save it.

86
User Manual A.10- Feedback

A.9.3 Importing and Exporting

The Rubric view provides a way for teachers to save a rubric as a text file in a
JSON format, using the download button on the top-right. This file can then
be re-uploaded using the button next to it in a future instance of the module
so that the rubric only needs to be updated instead of having to make it from
scratch. If the rubric was general, it could also be shared with other teachers
to prevent having to reinvent the wheel.

A.9.4 Download PDF

The button on the far right allows any user to download the rubric as a PDF,
to be shared with students.

A.10 Feedback

When the submissions are graded, feedback for each submission can be sent to
each group member via a Canvas conversation. To set this up, first a template
needs to be created. The template contains the following things:

• The sender’s name.


• Subject of the message.

• Body - information that is sent to all groups equally. It is generally an


introductory information to the results.

After a template has been created and selected, there are two options to send
grades:

• To all students that are a part of a submission.


• To fully graded submissions that have not received their results yet.

Once either button is clicked, the relevant students will receive their grade where
the filled out rubric is put below the body of the template. Each criterion will
be listed with their result along with any note that may have been written with
it.

87
User Manual A.10- Feedback

Figure A.32: The Feedback page.

88
User Manual A.10- Feedback

Figure A.33: The feedback template creation window.

89
User Manual A.12- Settings

A.11 Upload and Export

From the project view there is an option to upload the grades for each submis-
sion directly to Canvas, which can be done with the “Upload Grades” button
and the option to export results in an excel file with the “Export Results” but-
ton.
Uploading grades currently only works when every group is graded with the
same grading sheet (some group members do not get a higher grade due to a
bonus or a lower grade due to a penalty). There are some limitations with can-
vas that an entire group has to receive the same grade.
Exporting grades will give generate an Excel file with a list of entries of stu-
dents, with their student id, submission id, submission name and assessment id,
followed by the results for each criterion.

A.12 Settings

The settings screen currently only has one setting, which is about issue notifi-
cations. This can be toggled on or off to receive or not receive emails on opened
issues addressed to the user.

Figure A.34: The Settings page.

90
Appendix B

Diagrams

91
Diagrams B.1- Grading Flow

B.1 Grading Flow

B.1.1 Current Grading Flow

Figure B.1: Flow chart of the current approach to grading projects

92
Diagrams B.1- Grading Flow

B.1.2 Our application Grading Flow

Figure B.2: Overview of the architecture of the application

93
Diagrams B.2- Database Class Diagram

B.2 Database Class Diagram

Figure B.3: Class diagram

94

You might also like