Project Grading System: University of Twente
Project Grading System: University of Twente
Project Grading System: University of Twente
Group 13
Yevhen Khavrona - s2093421
Triet Ngo - s2188414
Dylan Koldenhof - s2154854
Adrijus Prieskenis - s1971514
Robin Waterval - s1477196
supervised by
Marieke Huisman
Ömer Sakar
1 Introduction 5
2.1 Stakeholders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.5.1 Must . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.5.2 Should . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.5.3 Could . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.5.4 Won’t . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.1 Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
1
Contents -
4.3 Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.4 Combined . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
5 Design in Detail 28
5.2.1 User . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
5.2.3 Rubric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.2.5 Grading . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.2.6 Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.2.7 Transactions . . . . . . . . . . . . . . . . . . . . . . . . . 34
5.2.8 Locks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
5.3.5 Grading . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
6 Testing 48
2
Contents -
7 Development Process 52
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
8 Conclusion 56
A.1.1 Signing In . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
A.2 Home . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
A.2.3 Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
A.3 Course . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
A.4 Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
A.5 Students . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
A.6 Graders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3
Contents -
A.7 Submissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
A.8 Grading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
A.8.1 Rubric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
A.8.3 Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
A.9 Rubric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
A.10 Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
A.12 Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
Appendix B Diagrams 91
4
Chapter 1
Introduction
The current practices and tools used for grading projects at UT do not scale well
to the growing student population and become more and more inefficient and
resource-demanding. Teachers and teaching assistants have to cope with the
arduous management of project grading, perform many activities manually and
rely on tools not suitable for large-scale project grading. Our group is confident
that the project grading process can be streamlined and simplified with the
right set of tools of a project grading system and careful crafting of the project
grading workflow in it.
Hence, the goal of our project group in the Design Module was to create a dedi-
5
Introduction -
cated system for handling project grading that has the potential to address the
most pressing issues graders face. In doing so, we strove not only to introduce
a new tool with a new set of features to graders, but also to keep and leverage
the positives that the current approach offers. The system we envisioned au-
tomatically prepares data required for grading, allows graders to create useful
grading rubrics, makes it easy to track graders’ responsibilities and aids with
sending concise and transparent feedback to students.
In this report we will present the results of our work, including the detailed
description of the application’s structure and innards. Chapter 2 will touch on
the preparatory steps that we took before starting the implementation work,
Chapter 3 will describe our toolset, Chapter 4 will tell about the structure and
architecture of our application, Chapter 5 will describe in detail the design of
the application’s front end, back end and database, Chapter 6 will go in depth
into the testing activities we performed and Chapter 7 will outline the way our
work was organised and Chapter 8 will deliver our final remarks and conclusions.
6
Chapter 2
2.1 Stakeholders
Grading projects (including the tools used for it) touches interests of multiple
parties inside UT. Some stakeholders are interested in the process itself while
others are interested only in the results of that process. Some parties are in-
volved in decision making related to project grading, thus also affecting the
development of a project grading system. Therefore, it is important to iden-
tify the stakeholders that might have influence on our project and decisions we
make.
• Teachers
They organize and come up with ideas for projects, create grading rules and
rubrics. They are responsible for handling grading and assigning TAs to
grade. They might also review the grading process or grade some projects
themselves. Teachers are also the ones who are responsible for delivering
(or managing the delivery of ) feedback to students.
• Teaching Assistants (TAs)
TAs form the bulk of the users of our application. They are often recent
students of the module that help its teachers by among other things, grading
projects. They do not have much influence on the management part of the
application, but they will be most of the users doing the actual grading.
7
Domain, Stakeholders and Requirements Specification 2.2- Mission Statement
• Module Coordinators
They are responsible for organizing entire modules and often are teachers
of a particular topic themselves. Hence they overlap mostly with teachers
and generally their desires would correspond to those of any particular
teacher, at least for our application.
• Students
While not the users of the application, they are its primary subjects, as
their projects are being graded. Thus, students are the party with arguably
one of the greatest interests in the application.
• Library, IT Services & Archive (LISA)
LISA is responsible for the management of IT systems at UT. Hence,
they are responsible for integration of student-developed systems into UT
infrastructure and maintaining these systems in future. Also, our project
requires close cooperation with LISA stuff as our application is tightly in-
terconnected with the existing UT infrastructure.
Keeping the stakeholders and their interests in mind, we set the goals for the
project.
8
Domain, Stakeholders and Requirements Specification 2.3- Current Approach
Our motivation: resolve the flaws and inefficiencies in the grading process
that graders face.
Our goal: to create a system for efficient project grading that can be used by
teaching staff to grade students’ projects and provide feedback to them.
Our approach: a web application (back end in Spring, front end in ReactJS,
Postgres database) tightly integrated with UT’s Canvas Learning management
system (LMS).
The current grading process is not efficient due to the following reasons:
• Google Sheets
– Besides having lots of benefits, Google Sheets is not flexible enough
to handle management of a large grading process. It has a sheet
9
Domain, Stakeholders and Requirements Specification2.4- Constraints, Assumptions, Definitions
and cell limit that is not always sufficient for grading projects in
large modules. Moreover, some browsers cannot cope with the load
of dozens or hundreds of sheets created and, consequently, might
occasionally crash under that load.
– It can be hard to present a readable and convenient grading rubric
in Google Sheets. Also, managing the rubric in Google Sheets might
be a rather tedious process because rubric version tracking, which is
required to keep every grader updated, is done completely manually.
– The grading interface is cluttered with lots of unnecessary pieces of
data in Google Sheets (e.g. being presented with grading results of
all submission when attempting to grade a single submission).
– Although Google Sheets supports automation to some extent, its
capabilities are pretty limited.
– There is no simple way to present useful and detailed grading-related
statistics (mostly due to the fact that the absence of a database
behind Google Sheets makes preparing statistics rather hard).
– Usually, the grading progress of each TA is only manually tracked
by making them fill in their progress in a separate sheet in Google
Sheets.
– As a brief conclusion, Google Sheets is the best and the worst tool
for grading projects at the same time.
• There are no notifications about changes in the rubric or the potential re-
quired re-grading activities, so it’s easy to miss acting upon these changes.
• Grading feedback is managed manually and usually sent via email with
not many details or explanations.
10
Domain, Stakeholders and Requirements Specification2.5- Functional Requirements
Within our project and the report, we operate the following set of definitions:
After we outlined the domain we work with, identified the stakeholders and
dissected the current approach to grading projects, we moved to defining a list
of requirements for our project. We split the list into 4 groups according to
the MoSCoW model: Must, Should, Could, Won’t. For ease of reading, we
11
Domain, Stakeholders and Requirements Specification2.5- Functional Requirements
further split each of these groups into more granular categories representing
parts of the application. As for implementation plans, we set a plan to carry
out the work needed to implement the first three groups of requirements - Musts,
Should s, Could s. In the end, we managed to successfully cover the original set
of requirements as well as most of the new requirements that we added in the
process.
Below the (updated) requirements are presented with explanations and motiva-
tion for some.
2.5.1 Must
Project Overview
• As a grader, I want to view and filter the issues that I opened or addressed
to in my project view
• As a teacher, I want to be able to import projects from canvas I am taking
part in.
12
Domain, Stakeholders and Requirements Specification2.5- Functional Requirements
Submissions Overview
Submission Details
TA Management
13
Domain, Stakeholders and Requirements Specification2.5- Functional Requirements
Grading Interface
14
Domain, Stakeholders and Requirements Specification2.5- Functional Requirements
Authentication
• As a user, I want to log in to the system using a UT account and log out
when I finish my session.
We will use the university’s existing infrastructure for user authentication
and fetching necessary data about courses and users.
15
Domain, Stakeholders and Requirements Specification2.5- Functional Requirements
2.5.2 Should
Project Overview
• As a teacher, I want to be able to see the percentage of passes and fails and
distribution of grades (including mean and median) for a specific project.
– We want to present useful statistics to the user of the system. We
will come up with more useful measures while working on the project.
• As a teacher, I want to be able to see the progress of the grading process
(the number of projects graded and the number of projects left to grade).
• As a teacher, I want to be able to lock grading of a project to prevent
others from making edits to current grades and feedback.
• As a TA, I want to see the list of submissions that require re-grading or
additional checking.
If something changes in the rubric or somebody tags a submission so that
the submissions assigned to the TA require additional actions, they should
clearly see such projects in their overview. Also teacher can create issue
to ask TAs to recheck the grading
Submissions Overview
16
Domain, Stakeholders and Requirements Specification2.5- Functional Requirements
TA Management
Grading Interface
2.5.3 Could
TA Management
17
Domain, Stakeholders and Requirements Specification2.5- Functional Requirements
Grading Interface
• As a grader, I want to clearly see which projects had formulae for auto-
matic final grade calculations overridden with a manually entered grade.
• As a grader, I want to be able to save the comments and explanations I
type as (personal) templates and reuse them later.
It happens often that graders need to explain the same points in the sub-
missions they grade, so we would like to offer the graders a possibility to
save those comments as templates for future use.
• As a grader, I want to open a file from the group’s submission directly in
the browser.
We would like to provide graders with a way to see what they are grading
without the need to constantly switch to other applications or tabs.
18
Domain, Stakeholders and Requirements Specification2.5- Functional Requirements
2.5.4 Won’t
Project Overview
Grading Interface
19
Chapter 3
After all the necessary preparatory work was done, we were ready to move on to
find the right implementation tools. Our choice was limited by one of the most
important constraints mentioned in the previous chapter - the application we
had to develop would have to be highly interoperable with Canvas LMS (and
other UT systems). This chapter will shed light on the process of choosing the
programming languages, frameworks and libraries to use and will provide the
motivation behind the choices we made.
3.1 Technologies
The application we developed relies on many proven web and software technolo-
gies to deliver the best experience and ease of use to end users as well as the
necessary convenience and flexibility to code maintainers.
20
Coding Toolset and Canvas Integration 3.1- Technologies
After analysing the requirements, weighing multiple external and internal fac-
tors and assessing the best fit for the project, it was determined that Java
(Spring) for back end, ReactJS for front end and PostgreSQL database would
be used for the project. In the further subsections, the possibilities and choices
made are explained in more detail.
Front end
For front end, we had concluded that it was best to use a framework and not
settle for plain JavaScript, given the level of complexity and interactivity of the
planned application. The group had considered adopting one of the popular
JavaScript (JS) frameworks: Vue.js, ReactJS (React) or Angular, as all of them
could be beneficial for the project. The choice between a JavaScript framework
and the vanilla JavaScript affected not only the development process, but also
the whole philosophy of the application’s architecture. All frameworks men-
tioned use the ”Single-page application” approach (SPA) to developing front
ends that moves most of the responsibility of navigating between pages and
delivering content from back end to front end to make pages fast and reactive.
Such common downsides of SPA as poor search-engine optimisation were not
relevant to our application, so by having in mind the advantages of SPA, we
made a decision to employ that paradigm for developing our application.
In developing the front end, we supplied React with multiple additional packages
and libraries to speed up development and rely on existing proven solutions to
common issues. To name a few of them, navigation in our application is handled
by React Router library, permissions and roles by CASL React, state changes
are tracked by React Redux, Lodash is used for working with JS objects, React
Quill aids us with using a rich text editor and ESlint helps us with finding
problems in our code.
Initially, React Bootstrap package was used for styling, with the aim of mini-
mizing the time needed for styling the application and simplifying visual design
works. However, after conducting several usability testing sessions and gath-
ering initial feedback on the design, we decided to make changes and replaced
Bootstrap with MaterialUI. MaterialUI gives a more unique feel to the applica-
tion and handles user interactions with the interface better, although it requires
more time spent on development. CSS Modules package is used for distributed
21
Coding Toolset and Canvas Integration 3.1- Technologies
To deliver all the packages and our code to the user’s browser, a bundling
software called Webpack is used. It converts non-standard JS, JSX, SASS,
LESS and other types of files to pure JS and CSS and efficiently combines them
together into a single bundle that can be used by the browser for rendering and
running the application logic.
Back end
Since software infrastructure at UT is mostly built using Java and Java Spring
(most popular web framework written in Java), we were inclined towards using
Java for implementing our application’s back end functionality. The other option
that the group considered was using Python and its most popular web framework
Django. After holding several conversations with LISA staff and investigating
pros and cons of both approaches by creating two small testing applications
with their help, we chose to sacrifice Python’s simplicity for Java’s apparent
interoperability with UT infrastructure. While for front end we eventually opted
for React (even though Angular is the standard at UT), we decided that choosing
Java and Spring Boot for back end is a more sustainable approach, especially
considering a possible need to maintain the application by UT staff in the future.
Maven is used to manage dependencies for Spring and Java in our project.
Among the libraries we use are Spring Security for managing OAuth2 authen-
tication and permissions, Spring JPA for handling data, iText7 for generating
grading feedback and rubrics in PDF format, Apache POI for exporting grading
results into an Excel spreadsheet, Jsoup for parsing the HTML formatting of
parts of our application’s rubric representation and Java Jackson for working
with JSON instances.
For storing data our group resorted to PostgreSQL. It’s a powerful yet simple
database that integrates flawlessly with Spring JPA. At one point, we also used a
NoSQL database MondoDB to store and process rubrics. In our implementation,
the rubric is a flexible hierarchical document that fits well in the world of NoSQL
databases. However, due to the complexity of managing and synchronising two
databases, we chose to move rubrics into our main SQL database and work with
its JSON representation using SQL toolset.
22
Coding Toolset and Canvas Integration 3.2- Canvas Integration
We interacted a lot with UT LISA support team to get access to UT’s devel-
opment version of Canvas and configure Canvas OAuth2 authentication. We
worked with a ”sandbox” course created for us by LISA that allowed us to
develop our application without affecting the production version of UT Canvas.
The data that our application fetches from Canvas can be split into four groups:
This data is retrieved through Canvas API by using the access token obtained
after the user signs in via Canvas OAuth2 flow. Moreover, the application uses
Canvas not only to fetch data but also to upload data to Canvas or initiate
certain actions there. For instance, Canvas API is used for sending messages
with feedback to students.
While Canvas provides us with a complete and easy-to-use set of data necessary
for grading, it also affects our application’s design a lot, especially with regards
to the database design. The application has to reflect the specific structure of
Canvas’ courses, assignments, submissions, groups and grading results.
23
Coding Toolset and Canvas Integration 3.2- Canvas Integration
The ideal solution to these issues would be to incorporate the whole submission
uploading process into our app to create a closed grading ecosystem. However,
by taking the importance of Canvas for UT and the additional security concerns
into account, we decided not to proceed with development of a fully independent
system (at least not within the 10 weeks of the module).
24
Chapter 4
After the choices for software and tools were made, the group started with
the actual implementation of the requirements. The first task was to outline
the structure that the application would have. This overall structure of the
application, split into front end, back end and database, is described in this
chapter.
Components are changing based on the user’s actions that are fed into the React
Router library. React Router is used to define a list of routes in the application
that are parsed and traversed by the library to render a correct component. All
of these activities take place solely within the front end, and back end is not
involved in routing and navigation. The front end interacts with the back end
only by performing web requests and acting upon them.
25
Architecture and Structure 4.3- Database
The React Redux library, that we use for managing complicated interactions in
our front end, consists of a list of actions that modify the state and a list of
so-called reducers that take the existing state, merge it with an update supplied
by an action and output the new updated state. Whenever the state is updated
in Redux and a component depends on that state, this component is automat-
ically re-rendered by React. The central global Redux state is combined with
internal states of components in our front end implementation to achieve the
state management our application requires.
The CSS Modules library we use for styling is based on the creation of multiple
styling sheets called “modules”. They are simple CSS style sheets but every
class name in a module is localized, i.e. it is hashed to a class name that is
globally unique even if the declared class names in different modules are reused.
The modules again act like the components and make the front end much more
modular and easy to work with. To apply styling, these modules are injected
into JS components.
The structure of the back end is based on the peculiarities of the Java Spring
framework. Spring follows the Model-View-Controller (MVC) pattern. The
model represents some structure in the back end (usually connected with the
database), such as an assessment, a rubric, etc. Meanwhile, the view is sup-
posed to represent what the user sees, but in this project this represents more
what the front end code interacts with, in the form of an API that can han-
dle request. Since the server only sends a single HTML web page back to the
client and the front end controls further interaction, the view part of MVC is
mostly covered by the front end. Finally, the controller is a part that handles
business logic, i.e. processing the requests the user (with the front end acting as
intermediary) makes. In Spring, the controller concept can be further split into
several subconcepts - the service, repository and controller classes, where the
service contains functions that perform actions like interacting with database
repositories, while the controller handles receiving the actual HTTP requests
and sending back responses.
4.3 Database
26
Architecture and Structure 4.4- Combined
4.4 Combined
27
Chapter 5
Design in Detail
This chapter discusses the design of the application’s components in detail. Each
of the three major parts - the front end, back end and database - are explored
in-depth to explain how they work, why certain choices were made and what
are the possible effects of such choices.
Initially, we chose to keep data stored in our database in constant sync with
Canvas. The SQL tables we defined back then closely reflected the structure of
Canvas API response formats in order to enable easier data syncing. However,
such an approach turned out to be quite error-prone and required a substantial
amount of work to make it functional (for little to no benefit). Furthermore,
it mandated implementing such convoluted yet essential features as handling
data that fell out of sync, which did not seem feasible in the 10-week time span
of the module. Therefore, we abandoned full data synchronisation in favour of
fetching data on demand from Canvas.
Our second approach was based on the idea of minimising syncing needed be-
tween our application’s and Canvas’ databases. Thus we converted our services
and controllers to fetch data from Canvas when needed and store only bare
minimum of data in our database. Such an approach eliminated the need to
28
Design in Detail 5.2- Back End
keep data in sync with Canvas but introduced even more serious issues. The
amount of data that was needed to be fetched from Canvas on each request
quickly grew out of hand and as a result the approach quickly proved to be un-
sustainable. It also required implementing sophisticated caching policies on the
server to reduce the amount of calls made to Canvas. After analysing the feed-
back on functionality that we received during our first testing session with one
of the module coordinators, we decided to make a switch to the third approach
– actively storing data from Canvas with syncing on demand.
The third approach, which is our current, in essence is a blend of the two ap-
proaches presented above. According to it, data fetched from Canvas is stored in
the application but no active synchronisation happens unless the user demands
it. Moreover, the data is synchronised in a non-destructive manner which means
that whenever the data is stored in our database, it’s never deleted from it when
performing a sync (even if such data was removed on Canvas). This mode of
work resembles capturing a ”screenshot” of Canvas state at the moment of syn-
chronisation. It turned to be the best approach to database design as it has a
very limited data footprint (data is fetched only occasionally during syncing in
comparison with the second approach) and does not need any complicated data
synchronisation mechanisms (in comparison with the first approach).
The model of our application’s database can be seen in B.3. Individual tables
presented in the diagram (i.e. Spring JPA models) and design choices made
regarding them are discussed in more details in the following section.
In this section the individual parts of the back end will be expanded on further
and some of the notable features of Spring that were used to handle data will
be explained.
5.2.1 User
The concept of user is linked to two models in the application - User and Activity.
The User model is used to represent a user (a student, teacher or TA) which is
either created when a grader signs in to the system or is imported within a batch
when a course is added to the database. The following user information retrieved
by Canvas is stored: name, email, student/employee number and avatar URL.
The Activity model is used to record grader’s recent activity that is then shown
in the front end such that the users can access their recently opened projects
quicker. The User and Activity repositories are used to give services access to
the database without the need to explicitly write SQL queries.
29
Design in Detail 5.2- Back End
The User service consists of two types of methods. The first type is methods
with so-called mandatory transactions that are used by other services (so they
can only be called within an already started database transaction in another
method) to interact with User entities in a variety of ways, such as by call-
ing query methods findUserById or saveUser. The second type is controller
transactions that are called by controllers to get data for specific endpoints.
For instance, the endpoint /api/user/to-do in UserController initiates a call
to getTodoForUser(that starts a transaction) to get a list of projects that con-
tain submissions the user is assigned to grade.
UserController contains endpoints that are responsible for getting user infor-
mation from the database, getting own profile information via /api/user/self
and retrieving the list of user’s recent activities and tasks.
In our application, there are three models related to courses and projects:
Course, CourseParticipation and Project. The Course model represents a course
from Canvas that is created when a teacher imports it. On import, the course id,
name and starting date are saved to the database. Then, the course users, i.e.
student, teachers and TAs, are stored in the database (if not already there) and
are connected to the course by creating a CourseParticipation instance (which
implicitly stores a record in the respective table in the database).
As a result, by simply importing a course and a project within that course, the
teacher receives all the necessary grading information right in the application,
with no additional preparatory steps needed.
30
Design in Detail 5.2- Back End
allows a teacher to edit the rubric since only the TEACHER role has that privilege.
5.2.3 Rubric
As one of the main pillars of our grading application, rubrics are carefully de-
signed to preserve ease of use while ensuring the flexibility of a hierarchical
rubric. The Rubric model consists of a list of Element instances, where each
Element is either a Criterion or a Block. In the case of a Criterion, title, descrip-
tion and grade details (such as minimum, maximum and the step) are saved. In
the case of a Block, only a title is saved along with a list of children Elements
to allow for the hierarchical structure.
SQL tables are not the most suitable way to store hierarchical recursive struc-
tures, so we initially decided to use a separate MongoDB database. However,
we eventually switched to storing the rubric in Postgres directly (more details
are provided in the chapter on used technologies). The RubricLinker model that
contains a Rubric is a simple bridge for connecting a project to its rubric.
Rubric updates are performed with JSON patches compliant with RFC 6902
standard. Such a way of updating the rubric is chosen to lay out the basis for
rubric history tracking as one of the possible extensions of the application. For
31
Design in Detail 5.2- Back End
Since not all TAs in a course participate in grading, the application provides
a way to select which TAs will be involved in grading. This information is
stored in the GradingParticipation model, that links users (TAs and teachers)
with projects. Similarly to CourseParticipation, this model provides a set of
necessary permissions for users who acts as graders in a project. Two roles
exist within a project - (TEACHER and TA_GRADING).
The Label model represents a label that can be assigned to a submission. For
instance, in case a submission is involved in a potential case of plagiarism, it
can be marked with a label “Plagiarism”. This label will be saved to the Label
instance and linked to the respective submission and project. Labels are reusable
within projects.
32
Design in Detail 5.2- Back End
5.2.5 Grading
Grades can acquire a variety of different forms, from simple points to pass/fail
grades. However, to save time, we decided to focus on the simplest type of grades
– points with step-based increments. Each grade given is stored in the Grade
model, that contains information about the grade given, (optional) explanation,
time of grading, identity of the grader and grade’s status. Grades can either be
active or not. If there are multiple grades provided for a single criterion of some
submission, only one such grade can be active at a time.
Since graders can change students associated with a submission and change
grading sheets for every submission, a very flexible database structure is re-
quired. To accommodate such functionality, a three-way association model As-
sessmentLink is created that connects students (of type User), grading sheets
(of type Assessment) and submissions (of type Submission). As students need
to have a single final grade in the end, only one grading sheet can be active at
a time, even though a student might have multiple submissions and multiple
grading sheets.
It’s common to run into grading issues, so the application provides a way to
create and track them. Issues are stored as Issue instances and linked to grading
sheets and users (both creators and addressees).
5.2.6 Feedback
Finally, when the grading process is completed, we offer a way for teachers
to send feedback to students. Currently, the default way is to send feedback
through Canvas messages. To send such feedback a new FeedbackTemplate
33
Design in Detail 5.2- Back End
In both cases, regardless of which method of sending feedback is used, each sent
email is recorded in the FeedbackLog model and stored to keep track of which
students received feedback. When the teacher wants to send feedback only to
students who haven’t received feedback before, they can do it by choosing not
to send feedback to students who already have FeedbackLog entries stored for
them.
5.2.7 Transactions
Transactions ensure that each execution will not directly update the database
until the transaction ends and is committed to the database. This prevents
parallel queries from retrieving data that is in the middle of being processed, in
order to preserve isolation.
Normally, JDBC (Java library used for accessing databases) has its own multi-
stage implementation of transactions: retrieve the connection, set auto commit
to false (to turn off default one-query transactions), execute operations, commit
or roll back on errors. However, this way of handling transactions usually makes
scaling and dividing services into parts difficult.
34
Design in Detail 5.2- Back End
There are a few parameters for the @Transaction annotation that are crucial for
transaction management - propagation and rollbackFor. Propagation can be
one of the following types: required, supports, mandatory, require_new or
not_supported. In our project, we mostly use required and mandatory types.
required is the default option in Spring Boot. With this option, JDBC creates
a new physical transaction for this logical transaction if there is no transaction
opened or joins this logical transaction with the existing transaction. Therefore,
this option was used for methods that are called by controllers. On the other
hand, mandatory ensures that when called, the logical transaction needs an
existing physical transaction to be joined with. Therefore, this option is suited
for methods that exist as supplementary methods to other methods in services.
5.2.8 Locks
There are two types of database locks – optimistic and pessimistic. The opti-
mistic lock is essentially akin to the version control. Data fetched by repositories
that are bound to optimistic locks will get the current version of the fetched
data. Before committing, the transaction compares the version of the fetched
data to the current version of the data in the database. If the versions are
different, then there are changes that were possibly made by others. Thus, an
exception is thrown and this transaction is rolled back. One of the advantages
of using optimistic locks is that transactions can execute freely which otherwise
would have to wait if pessimistic locks are used. On the other hand, in case of
complex transactions, it might be inefficient to retry them over and over again.
In such cases, pessimistic locks can be used. Before executing methods anno-
tated with the pessimistic lock annotation, the transaction will try to obtain
the lock and execute the query only when the lock is obtained. Otherwise, the
transaction will have to wait until the lock can be obtained. There are 3 type of
pessimistic locks in Spring JPA: PESSIMISTIC_READ, PESSIMISTIC_WRITE and
PESSIMISTIC_FORCE_INCREMENT.
35
Design in Detail 5.3- Front End
In the back end, the PESSIMISTIC_READ lock is used when the transaction has
no intentions to write to database and PESSIMISTIC_WRITE is used when the
transaction is designed to write (update or delete) the queried entities. We use
PESSIMISTIC_FORCE_INCREMENT for rubric updates to ensure that the user is
updating the current version of rubric.
There are a few limitations in Postgres with regards to JPA locks. For example,
the timeout for attempting to obtain a lock cannot be set as a parameter in JPA
due to Postgres’ limitations. Postgres’ default timeouts are essentially infinite
and can only be adjusted by editing the configuration file of the database. While
pessimistic locks are useful, there are cases where deadlocks can happen. In such
cases, an exception should be caught and the transaction (or query) has to be
rerun. These cases are hard to catch and test though, as they are very rare.
This section will dive deeper into the part of our application that is in a direct
contact with our users – the front end. The front end will be described in terms
of interactions it offers and will reflect the functional requirements outlined
above.
A clean and minimalist look was aimed for when designing the front end. In
order to better separate data visually and present it in an intuitive way to the
user, cards, shadows, an abundance of white space and vibrant standing-out
colours that aim to bring the user’s attention are used.
To make the user aware of a page’s place within the context of our application,
most of the pages feature navigation breadcrumbs.Links are used for navigation
between different screens as they present a natural and intuitive way of navigat-
ing around the Web. Buttons, including full-text and icon buttons, are mostly
reserved for performing actions.
Contextual menus are used to provide group interactions to the user instead of
presenting the user with a swarm of action buttons. Standalone actions that
36
Design in Detail 5.3- Front End
stand apart from the normal interaction flow are taken out into modals. For
instance, modals are used to create issues in the grading screen and to select
graders in the grading screen.
As the application distinguishes between several types of users, the front end is
able to present the right bits of information and components to the requesting
user depending on the user’s role. As an example, only users with the Teacher
role can see the administration buttons in the project screen. As TAs do not
have access to most of this functionality, their front end does not show them
these options altogether (of course, the actions are also fully protected in the
back end).
The user’s interaction with the application starts when they initiate the OAuth2
sign-in process using their UT account. The user’s session cookie is then stored
in the browser while the server stores the Canvas access token and takes care of
interaction with the Canvas API when the user’s actions require it. The user’s
profile information that is retrieved from Canvas on sign in is then stored in
Redux to allow easy access to it from any place in the application.
After signing in, the first screen that the user sees is the home screen. This screen
is used to observe the list of user’s courses, see the to-do list (which contains
the list of projects in which this user has assigned submissions to grade) and
initiate importing new courses from Canvas. The to-do list presented on the
37
Design in Detail 5.3- Front End
right always stays in that position to keep being accessible to the user, even if
the user scroll down the page.
After the user has selected a course, they see the course overview page. Cur-
rently, the only purpose of this page is to present the user with the current list
of imported project in the course and enable importing new projects. We chose
not to import all projects in a course automatically as it happens often that
a single Canvas course contains dozens of assignments, not all of which might
actually be real project assignments that need to be graded. When a project is
imported, only teachers and TAs that are added as graders to the project can
38
Design in Detail 5.3- Front End
access it.
The course page also exposes one more button to teachers. Besides being im-
ported, course data can also be synchronised with Canvas to update the course
user and project (assignment) list.
Selecting a specific project leads the user to the project page. Here teachers can
see a list of buttons with interactions and links to other parts of the application.
The teacher can navigate to the project rubric, move to see the list of submis-
sions or students, open the feedback page, export grading data to an Excel file
or switch to the graders page. Moreover, a distribution of grades in the project
is displayed within that screen too.
The users can see the list of students participating in a project and the list of
submissions for the project in the students and submissions screens, respectively.
Both pages support searching and filtering items for more convenient browsing.
Searching is performed with normalisation (NFD) to account for possible ac-
cented characters (and not only) in user names. Data in the submissions page
is presented in the form of cards as each single entry contains too much infor-
mation to be fit in a table, in contrast with the list of students in the students
page for which the table-based representation is sufficient.
39
Design in Detail 5.3- Front End
Similarly, the users can select a submission and navigate to that submission’s
page. There they can see more details of the submission, such as the time it was
submitted, the user who submitted it, the list of comments left on Canvas and
the list of attachments. Each attachment can be downloaded (from Canvas) by
clicking on the button next to its name. This eliminates the need to download
a bundle with all submissions on Canvas and search for the needed submission
withing that bundle. The application displays the size of each download. All
in all, the users will likely not want to initiate a download of several gigabytes
of data while being on a limited data plan, so displaying the size information is
vital.
In the application, each submission fetched from Canvas is associated with a set
of students. It might be a group of students or a single student only, depending
on whether the student who submitted the work on Canvas was in a group or
not at the time of submitting. Although it’s not possible to manually link new
students to submissions on Canvas, our application supports such functionality.
In a situation when a student, who is supposed to be part of a group, submits
the project individually, the teacher can manually link other group members to
this submission and grade these students together.
40
Design in Detail 5.3- Front End
41
Design in Detail 5.3- Front End
The criterion’s body can be edited with a rich text editor that has support for
such editing options as making text bold, italicised, underlined or putting this
text in an unordered or ordered list.
Rubric can be edited in the edit mode by making changes to criteria and sections.
New sections and criteria can be added as children of the existing sections by
choosing the respective options in the contextual menu of each section. Criteria
are terminal and cannot contain subcriteria within them.
When the user is done with editing, they can press the save button to update
the rubric in the database.
Rubrics can be downloaded in the PDF format and can be thus be incorporated
into module guides. Also, rubrics can be downloaded and uploaded in our
custom JSON-based format to be reused across different project in, possibly,
different years, although we plan to support such rubric copying functionality
within the application directly.
42
Design in Detail 5.3- Front End
43
Design in Detail 5.3- Front End
Graders can be assigned to grade submissions in the graders screen. As the first
step, the teacher can select TAs who will be responsible for grading submissions
for the project. Usually, not all TAs participate in grading, so the application
provides a way to make such selection of graders. Only the TAs selected will
have access to project grading data. By default, all teachers of the course are
automatically selected as graders for a project.
44
Design in Detail 5.3- Front End
5.3.5 Grading
The grading page is the central piece of the application as it present a way to
actually grade submissions. The page consists of three parts - the rubric panel,
the grading panel and the issues panel.
The rubric panel display the project rubric so that the grader can directly browse
it when grading. Each criterion can be clicked to show its contents. To quickly
move back and forth through the criteria, the user can use the arrow buttons
at the top of the rubric panel. A criterion is marked as graded in the rubric if
a grade is entered for it in the grading panel. Such a criterion can be visually
distinguished from ungraded criteria by a check mark and the grade displayed
next to it.
The grading panel is split into two parts: the upper serves as the means to
enter a grade and optional explanation and the lower part presents the grading
history for the chose criterion. Even if a criterion is already graded, it can be
regraded by a teacher or by the submission’s assigned grader if needed. The
grade that applies to the criterion can be selected among the grades by clicking
on it.
The rightmost part - the issues panel - allows users to resolve issues related
to grading of submissions. For instance, if the teacher receives an email from
students with a question regarding grading, the teacher can create an issue
with that question and address the submission’s grader. The users can opt to
45
Design in Detail 5.3- Front End
As the last step of the grading process, the application offers a way to send
feedback to students. Feedback consists of a filled in rubric, so the student will
be able to see not only a list of grades but also any additional comments or
explanations provided by graders.
Feedback can be sent directly via Canvas using Canvas’ built-in messaging sys-
tem. The generated feedback is in the form of simple textual representation by
default. Teachers can create email templates that consist of an email subject
and body and are used to send feedback to students.
46
Design in Detail 5.3- Front End
47
Chapter 6
Testing
In order to deliver a product that will successfully fulfill the demands of the
stakeholders, it is important that no unexpected behaviour occurs while it is in
use. Hence, thorough testing must be done in order to find as many potential
issues as possible before the product is rolled out to stakeholders. For this
reason, a plan for testing was made and executed, which will be described in
this chapter.
Testing of software can generally be split up into several categories, from low to
high level. Namely unit testing, integration/system testing and usability testing.
Although naturally the chronological order of these categories within the project
development cycle is to some extent the same, in the project workflow, this
chronological order is not so strict and all categories are to some extent tested
from the very start. This is done both in order to show visual progress in a
reasonable time to the stakeholders and to catch wider problems (for example,
a bad user interface) before it is too late to change significantly.
Next, integration and system testing is a step above unit testing, where the
behaviour of multiple components combined is tested. For instance, this could
be testing if the front end interacts with the back end correctly to fulfill some
query. These could be automated for better completeness, so in this project
48
Testing 6.1- Automated Testing
Finally, usability testing is unlike the previous two categories, in that it is not
focused on ensuring that the developed functionality works correctly, but instead
on ensuring that the developed functionality works in the best way suited to
the user. Hence, this seeks to test the user experience (UX), and with that the
interface being used for it. Some examples of this would be testing whether
certain buttons will be clear in their functionality to the user, if some process
is deemed to be easy to do, etc. To some extent this could be done individually
by the developers, (especially as many have TA experience) by just using the
application as a real user would from time to time, but developers will obviously
have a biased view of their own work and they have much more knowledge on
how the app works than any other user would. Hence it is important to let
stakeholders with no prior knowledge of the application test it, and for this
some meetings were held.
With the back end, unit testing (and partially integration testing) was suitable
for service classes, which implement the business logic of the application. Most
of the dependencies that were there were mocked; what this means is that the
dependent object is not actually created, but instead a version where every
function returns null is used. Of course, testing would not be possible if every
dependent function just returned null, so a technique called stubbing is used.
This means manually giving return values to a mocked object’s functions given
certain parameters, allowing for simulating the function actually being called.
An example where this is useful is in database retrieval, where without actually
making use of the database the desired object is pretended to be retrieved from
it using stubbing.
For mocking the Mockito library was used. This, in conjuction with JUnit,
Java’s unit testing framework and AssertJ for more readable and helpful asser-
tions, provided everything needed for unit testing the application. Most tests
were similar, and can be summarized in four steps. First, the test models are
created (e.g. a rubric to test something rubric related), then any expected return
values are defined, next the mocked methods are stubbed and finally the return
value of the tested function is compared to the expected value. In some cases
there were other components that could be compared, for example a certain field
in a test model which should have been changed by the function. Furthermore
49
Testing 6.2- Usability Testing
there were some cases where an exception should be thrown, and this was then
asserted and it’s messages checked to correspond to the expected message. For
void methods sometimes the opposite was the most useful, checking instead that
no exception was thrown given intended input variables.
We held several testing sessions with Tom van Dijk, the module coordinator
of Software Systems, and teaching assistants to test both functionality and
appearance of the application. These sessions were of immense use as they
let us see our project from a new perspective each time and led to substantial
improvements.
In the first meeting with Tom, we discussed the overall approach to handling
grading data, the range of supported features and the overall design of the
application. As a result of the meeting, we:
• completely overhauled the back end and switched to the third version
of our database design (described in more detail in the section on the
database structure)
• added support for issues, labels, exporting grading data as an Excel spread-
sheet, manually adding students to submissions and creating separate
grading sheets
• reworked the design of the front end to make it more clear and intuitive
• had topics for further discussion with the supervisors, such as showing
feedback in our application directly
The second session with Tom was mostly concentrated on finding implementa-
tion errors and testing the user interface (UI) of the application. No crashes
or implementation errors were registered during the session, however, multiple
suggestions for UI improvements were provided.
Among discussions with TAs, the most important one took place with an experi-
enced TA, Martijn Verkleij, who provided us with lots of invaluable suggestions
for improvements and fixing UI mistakes. The meeting was focused purely on
the UI. Among the suggestions we received were:
50
Testing 6.2- Usability Testing
51
Chapter 7
Development Process
7.1 Introduction
To get any sort of project done, there needs to be some sort of organization.
Given the current situation these all had to be done digitally, which was not
ideal but ultimately did not provide too much problems.
For communication within the group a Discord server was used, which also
became a repository of information such as diagrams, links to tutorials to work
with certain libraries, etc. Sometimes some simple questions or discussions
were held in chat, but for more complicated matters were discussed in a voice
call on the server. The time and frequency of these were mostly based on the
circumstances, and thus there was no strict organization of them.
Our meetings with our supervisors were scheduled every Wednesday via Mi-
crosoft Teams, most of the time at 17:00. Here the progress over the week
was outlined and some uncertainties cleared up. Since the supervisors have lots
of experience grading projects themselves, their insights proved very useful in
making certain design decisions.
As part of the module were the organized peer reviews. Here a short presentation
outlining the progress, a demo, and future plans to other groups was held and
after that a short discussion with one other group. Since other groups are
working in different domains, there were limited insights to be gained from
them about this project specifically. However, a discussion and comparison of
the structure of the deliverables all groups share, such as this report, the manual
and the poster, was useful.
Finally, there were also meetings with others relevant to our project (as men-
52
Development Process 7.2- Project timeline
The details of the project timeline are described in the next section.
• wrapping up
In the first week, we spent time on designing and setting plans/milestones for our
application and its development process. Then, after the introductory meeting
with the supervisors, we worked on specifying and refining the requirements for
our application as well as prepared mockups for the design.
53
Development Process 7.2- Project timeline
The largest phase of the project work was dedicated to actual work on imple-
menting the requirements.
We started with setting up most of the boilerplate code for the back end and
making our rudimentary front end communicate with the back end. As Canvas
integration already worked at that time, we could already experiment with
retrieving data from Canvas and displaying it in the front end. Designing and
implementing the database in Spring proved to be very challenging, so it took
as a fair amount of time and effort to make it work correctly. Further research
and testing made us change the database design several times before arriving
at the current solution.
Implementing features
After having a skeleton for our project ready, we started implementing our
requirements in batches. The first part revolved around fetching user data,
submissions and the creation of rubric. Initially we used MongoDB for the
rubric, but later merged it with our main PostgreSQL database.
After having most of the initial parts of the application ready, we moved to
working on the grading screen. It took us several iterations to arrive at the
current design of it.
Testing
After having the first prototype ready, we held several testing sessions with
potential users of our application. These sessions turned out to be very useful
and provided lots of invaluable insights. As a result, we merged this phase with
the implementation phase and started reworking parts of the application that
required changes.
Wrapping Up
The last stage of the project work took place in weeks 9 and 10 of the project
when we started the work on the project report, poster, presentation slides,
54
Development Process 7.3- Responsibility and Contribution
documentation and user manual. Also, we handed in the reflection report which
was a substantial and time-consuming part of weeks 8 and 9.
55
Chapter 8
Conclusion
Among the features that can be considered for future updates are:
The project we worked on was a great opportunity not only to hone our coding
and working skills but also to put them for good use - make the grading process
56
Conclusion 8.2- Final Remarks
All in all, our group is proud of the work we did and the system we developed.
We learnt a lot by working on a real-life project with real-life implications and
within a real-life software development process.
Of course, the pandemic made it harder to efficiently manage the process and
imposed some limitations on the project activities and process. However, we
believe that despite these serious obstacles we managed to successfully conduct
all the necessary activities, go through all development phases and deliver a
working product of good quality.
57
Appendix A
User Manual
This manual is structured mostly along the pages of the application. Each
section is centered around a page and what the functionalities are on that page.
A.1.1 Signing In
The first thing you are greeted with when loading the application is the sign in
screen (Figure A.1). This will redirect you to a screen where you can sign in
with your UT account. After this, you will receive a prompt (Figure A.2) where
you need to authorize your Canvas account with the application, so that all the
necessary data from Canvas can be retrieved. Accepting will lead into the home
screen (see section A.2).
58
User Manual A.1- Signing in and out
59
User Manual A.1- Signing in and out
To sign out, simply open the sidebar on the left (Figure A.3) and click the
button on the bottom-right of the sidebar. This will automatically redirect you
to the sign in page.
60
User Manual A.2- Home
A.2 Home
The Home page is the first thing you see after logging into the application. Here
you can see your currently active Courses and import them. On the right you
can see the to-do list and a list of grading issues addressed to you. Throughout
the application, the sidebar on the left can be used for navigation. Another
commonality throughout the application for teachers is a Sync button, which
synchronizes data with Canvas.
61
User Manual A.2- Home
The import button allows you to select any course where you have a TA or
teacher role. Once courses are selected, they will show up on the home page.
Clicking on any one leads you to its page.
62
User Manual A.2- Home
63
User Manual A.2- Home
The To-Do list gives an overview of the grading tasks that are assigned to a
grader, and provides a quick link to them.
64
User Manual A.2- Home
65
User Manual A.3- Course
A.2.3 Issues
Issues addressed to you are also quickly available from the home page. Click
on them leads to the assessment where the issue was found. For more details
on issues see section A.8.3. Using the button on the top right, you can filter
between all, resolved or open issues.
A.3 Course
The Course page exists mainly as a hub for projects within the course. From
here you can import projects into the application or synchronize them with
Canvas (as a teacher).
66
User Manual A.3- Course
Importing projects functions the same as courses. Clicking on the import button
presents a list of projects that can be imported. After a project is imported, it
will be shown on the Course page. Clicking on one will lead you to the project
page for that project (see section A.4).
67
User Manual A.3- Course
68
User Manual A.3- Course
69
User Manual A.5- Students
A.4 Project
This is the part of the app that provides links to almost all the functionality, via
the buttons in the administration card. Along with that there is a statistic view
for the selected project, at the time of writing this only shows score distribution.
A.5 Students
This page shows a list of students and their student number, email, final grade
and amount of submissions. Using the search bar at the top the list can be
filtered.
70
User Manual A.6- Graders
A.6 Graders
The Graders page serves as the place where teachers can assign TAs as graders
for submissions.
71
User Manual A.6- Graders
The ”Edit Graders” button on the top right allows you to select TAs to assign
as graders. After this is done, the graders can be assigned to a submission. How
this is done follows in section A.6.2.
72
User Manual A.6- Graders
Down in the unassigned section the submissions that are not assigned are visible.
Clicking on the button on the right of any such submission opens the assign-
ment window, where a grader can be selected to be assigned to this submission.
While this is not necessary for teachers, TAs cannot grade the submission other-
wise. After a grader is assigned the submission will be moved into the assigned
grader’s card. From there the same button as before can be used to reassign
the submission, except there is an option to ”Return Submission”. This makes
the submission unassigned again.
73
User Manual A.6- Graders
74
User Manual A.6- Graders
75
User Manual A.7- Submissions
A.7 Submissions
On this page an overview of all the submissions of a project is given, and from
here you can select one to go to its detailed page for grading. Also visible is
whether or not a submission is assigned.
Clicking on a submission from within the Submissions view will lead you to its
details. From here you can see and edit labels, add or remove students manually
to a submission, view comments, download attachments and manage and go to
its grading sheet.
76
User Manual A.7- Submissions
Labels
Labels provide a way for a grader to mark something about a submission, for
instance the submission is suspected to be fraudulent. Pressing the Edit Labels
button allows you to create a label, by giving it a title and a colour. After that
it can be enabled for the submission. Labels are saved across the project, so
other submissions can easily be given the same label. After the label is selected,
the title and colour will show up above the details page and also in the list of
submissions.
77
User Manual A.7- Submissions
Manage students
In the students card, students within the project can be added or removed
from the submission using the plus and trashcan buttons respectively. Adding
a student provides a window where you can select the student and to which
grading sheet he will be added. (See more on grading sheets in section A.7.1)
78
User Manual A.7- Submissions
Download attachments
The attachments of the submission can be downloaded using the download but-
tons next to them. The submission is downloaded directly from Canvas and the
button just provides a link to it.
79
User Manual A.8- Grading
A.8 Grading
The grading view can be opened from within a submission detail page as de-
scribed in section A.7.1. TAs can only open this view if they are assigned to the
submission. This view shows the rubric for the project, the grade editor, grade
history and issues.
80
User Manual A.8- Grading
A.8.1 Rubric
The Rubric as built using the Rubric Builder (see section A.9) can be seen here.
Here a criterion can be selected to be graded. If the criterion is already graded,
it will show the grade on the right.
Once a criterion is selected from the rubric, the grade editor will open automat-
ically if the criterion is ungraded. If it was already graded, you need to press
the pencil button to regrade it. Any grade that is saved will be added to the
history below, and a teacher can select any grade from it to be the active one
by clicking on it.
A.8.3 Issues
On the far right of the screen is the issues panel. Issues are a way of contacting
other graders about what is graded, in case they believe something is not right
with the grading.
81
User Manual A.8- Grading
Creating an Issue
Creating an issue can be done using the plus button where a window will open.
Here an addressee for the issue can be chosen, who will receive an email noti-
fication if they have that enabled in the settings (see section A.12), along with
the subject and description. Finally there is the reference issue, which can be
used in case an issue relates to another issue. Once the issue is created, it will
remain as an open issue visible here and on the home page, until anyone presses
the check-mark button to resolve it.
Resolving an Issue
Pressing the check-mark button on an open issue leads to a window where the
issue can be resolved. A solution needs to be given and then saved, which will
mark the issue as resolved. The issue will still remain in the system, and remains
visible if not filtering on resolved issues.
82
User Manual A.9- Rubric
A.9 Rubric
On the Rubric page the project rubric can be viewed, and teachers can build
and manage a rubric for their project. The page also allows for importing and
exporting of rubrics. Rubrics are entirely composed of criteria and sections, with
a section being able to contain more sections and criteria. Criteria are ultimately
the elements that grades can be applied to, while sections just provide structure.
If there exists a rubric for the project, the rubric viewing page is shown. This
allows both teachers and TAs to navigate through the rubric via the left panel.
Upon selection of a criterion or section its details are shown on the right.
83
User Manual A.9- Rubric
When the rubric is empty, you are presented a button to edit a rubric. After
this a criterion or section can be selected, which will form the first element of
the rubric. No matter what option is selected, you will be led to the rubric
editing view (Figure A.31), which is described below.
84
User Manual A.9- Rubric
When there is already an existing rubric, the rubric page will first just be a
viewer for the rubric. To edit an existing rubric, the pencil button on the top
right accesses the same rubric editor view as creating a rubric does, which is
described below.
85
User Manual A.9- Rubric
Here the menu buttons on the right of a section or criterion can be used to
manage that element. For criterion that just means deletion, but for sections
more sections or criteria can be added as elements of the section. Furthermore
at the top there is the field labeled “Outline” which is the root section, the title
of which cannot be changed and cannot be deleted, but otherwise functions the
same as a regular section.
On the right the criterion or section selected can be edited. Sections only have a
title, while criteria contain alongside a title, a description and the grade fields.
The description of a criterion can be made using a rich text editor, similar
to ones found on other programs. This allows for features such as lists, bold
text, italic text, etc. Finally, for a criterion grade a minimum, maximum, step
size and weight can be specified. The weight is a value between 0 and 1 that
determines how much impact the grade has on the final grade. Once the rubric
is complete, the save button on the top right needs to be pressed to save it.
86
User Manual A.10- Feedback
The Rubric view provides a way for teachers to save a rubric as a text file in a
JSON format, using the download button on the top-right. This file can then
be re-uploaded using the button next to it in a future instance of the module
so that the rubric only needs to be updated instead of having to make it from
scratch. If the rubric was general, it could also be shared with other teachers
to prevent having to reinvent the wheel.
The button on the far right allows any user to download the rubric as a PDF,
to be shared with students.
A.10 Feedback
When the submissions are graded, feedback for each submission can be sent to
each group member via a Canvas conversation. To set this up, first a template
needs to be created. The template contains the following things:
After a template has been created and selected, there are two options to send
grades:
Once either button is clicked, the relevant students will receive their grade where
the filled out rubric is put below the body of the template. Each criterion will
be listed with their result along with any note that may have been written with
it.
87
User Manual A.10- Feedback
88
User Manual A.10- Feedback
89
User Manual A.12- Settings
From the project view there is an option to upload the grades for each submis-
sion directly to Canvas, which can be done with the “Upload Grades” button
and the option to export results in an excel file with the “Export Results” but-
ton.
Uploading grades currently only works when every group is graded with the
same grading sheet (some group members do not get a higher grade due to a
bonus or a lower grade due to a penalty). There are some limitations with can-
vas that an entire group has to receive the same grade.
Exporting grades will give generate an Excel file with a list of entries of stu-
dents, with their student id, submission id, submission name and assessment id,
followed by the results for each criterion.
A.12 Settings
The settings screen currently only has one setting, which is about issue notifi-
cations. This can be toggled on or off to receive or not receive emails on opened
issues addressed to the user.
90
Appendix B
Diagrams
91
Diagrams B.1- Grading Flow
92
Diagrams B.1- Grading Flow
93
Diagrams B.2- Database Class Diagram
94