1

ASSIGNMENT #2: LMS RUBRIC

Assignment #2: LMS Rubric

Alexis Handford, Andrew Yeung, Faeyza Mufti,

Mimi Ross, and Paige McClelland

ETEC 565A - 65A

University of British Columbia

January 28, 2018

 2
ASSIGNMENT #2: LMS RUBRIC
Case Background

Conseil Scolaire Francophone (CSF) is interested in partnering with

LearnNowBC to offer a new online program for adult Francophone learners who are

fluent in French, but need to learn and improve English to complete high school. The

majority of distance learners are living in Vancouver, but are spread across the entire

province of British Columbia. They are working professionals who may require mobile

access and may not always have fast Internet connection. They will be remotely signing

in to access their courses using a student number or login information.

The interested parties include:

● Conseil Scolaire Francophone (Conseil scolaire francophone de la Colombie-

Britannique, 2018), which has an online portal to support students

● LearnNowBC (Ray, 2017), which has its own portal with online program services

These two parties wish to launch a small, co-funded online program. The current users

of the LMS for this program are:

● Students

● Teachers

● Administrators

However, the future user base could expand; the online programs could be

offered to K-12 students (ages between 4 to 19 years) and the number of learners could

top 4000. Additional schools under CSF may also be offered access incorporating new

users like parents, each requiring their own customizations. Thus, planning to use an

LMS that will support long term growth is a major priority.

 3
ASSIGNMENT #2: LMS RUBRIC

An LMS is a big investment (Bates, 2014) needing much consideration. It is not

just the target audience that must be identified, but their individual needs and

requirements that also need to be considered. However, we do not have sufficient

information about this in the given scenario. Without particular information on course

design and media choices, the target LMS needs to be flexible, providing options for

educators to design courses in the format of their choice. In view of Malcolm Knowles’

Principles of Andragogy, programs designed for adult students are likely to “include a

wide range of instructional design models and theories to appeal to varied experience

levels and backgrounds,” and “utilize social media and online collaboration tools to tie

learning to social development” (Pappas, 2014). Additionally, Chickering and Ehrmann

(1996) suggest that learning improves through a variety of mediums that promote

student-faculty interactions and spontaneous peer collaboration. Immediate feedback

also contributes, so the LMS must be responsive to different assessment needs. Finally,

without knowing how much budget has been set aside for the CSF/LearnNowBC

initiative, the requirements for institution-wide, cloud-based or open-source LMS cannot

be suggested (Coates, James & Baldwin, 2005; M, 2017).

 4
ASSIGNMENT #2: LMS RUBRIC
Rubric Design

The LMS requirements are broken down as follows:

● High Priority: Mandatory, essential, non-negotiable (for majority of users)

● Medium Priority: Important to have (offers customization)

● Low Priority: Extras nice to have (for long term planning)

We reviewed several rubrics available on the Internet, considering features

included by popular LMS vendors. In particular, we found the Longsight (2018) rubric to

be quite useful. As we searched through possible features, we approached the LMS via

personas of students, teachers and administrators to help us identify the needs of

specific users. Our target was to create descriptors that align best with all three roles,

distinguishing high priority needs from low ones. Refer to Appendix A for our draft

template. Additionally, we used Bates’ (2014) SECTIONS model as a guide to organize

and categorize elements, and work by Porto (2015) and Spiro (2014) helped us identify

features that institutions should consider when forward planning as opposed to

traditional and rigid LMS designs. We incorporated LMS exemplars that facilitate the

learning experience rather than dictate, enhanced by fluid and flexible features.

Below, we have created three rubrics arranged by priority. The first table

incorporates essential, non-negotiable features that must be afforded by the LMS. Once

these features are met, then the second chart includes functionality possibly good to

have. The third chart helps users forecast future needs that may be important for the

ongoing success of this program. For example, Lorenzo and Ittelson (2005) describe

ePortfolios as a valuable learning and assessment tool, so we included this as a future

consideration for an LMS equipped to support student growth and reflection (p. 1).
 5
ASSIGNMENT #2: LMS RUBRIC

This overall format enables flexibility by moving elements between priorities

given a better understanding of requirements that emerge upon consultations with

interested parties within and outside institutions. Hence, we have designed an open-

ended rubric where actual reviewers can expand identified areas, making detailed

checklists based on context rating each criteria, possibly explaining what was missing.

The rubrics have been designed for readability and organized in alphabetical order for

quick access to enable ultimate customization. As well, we organized each rubric into

three categories of assessment: Functional, Technical, and Network to help further

identify areas of strength and areas of weakness.

 6
ASSIGNMENT #2: LMS RUBRIC

High Priority

DOES NOT MEET

CATEGORY CRITERIA MEETS (1) EXCEEDS (2) SCORE/NOTES
(0)

Accessibility Responsive to Responsive to mobile, Responsive to mobile,

(ex. Multiple mobile, laptop or laptop or tablet laptop or tablet
devices) tablet but not at the simultaneously. Users simultaneously. Users
same time. able to resume learning able to resume
on any device. learning on any
device.

Optimal for viewing

and interaction
experiences.

Functional

Assessment Simple test creator More advanced test Advanced test creator
& Feedback tool with options for creator tools that allow tools that allow for the
limited assessments for the creation of creation of
(ex. true/false differentiated differentiated
& multiple choice). assessments including assessments via
video/ audio files. multimedia tools (ex.
game-based,
 7
ASSIGNMENT #2: LMS RUBRIC

simulations, wiki
Limited feedback Provides immediate, design).
channels (ex. customizable feedback
email). (ex. text within LMS). Immediate, two-way
customizable and
Raw gradebook Some options for storing personalized
scores only. and tracking feedback.
assessments.
Customizable options
. for storing and
tracking assessments.

Collaboration No customization Some customization of Full customization of

(ex. for (a)synchronous (a)synchronous (a)synchronous
Discussion exchanges; limited experiences with social experiences with
boards, social media media integration. social media
virtual integration. integration.
classrooms)
Share files through
drag-and-drop to
Functional promote student-
(Continued) teacher- content
interactions.

LMS enables video

conferencing for social
learning from home.

Content Provides a basic Provides easy authoring Provides a suite of

Authoring means for uploading resources for importing tools for authoring and
and storing courses and managing content. importing media-rich
and content in a content, as well as
hierarchical manner uploading rich content
to support teaching types such as
 8
ASSIGNMENT #2: LMS RUBRIC

and learning podcasts, video clips,

(Longsight, 2018). etc. (Longsight, 2018).

Supports SCORM and

xAPI standards.

Course Permits course to Permits course to be Permits course to be

Management be exported from exported and reimported exported and
the LMS itself. into the LMS itself. reimported into the
LMS with flexibility.
Allows for manual
archival of completed Allows for automatic
courses, with student archival of student
Functional submissions and submissions and
(Continued)
discussions intact. discussions intact,
with full instructor
control.

Integration Allows individual Allows individual login Allows single sign-on

login options to options to existing by integrating many
existing portals. portals, integrating third party software
some third party applications, reflecting
software applications. anytime, on-demand
learning.
 9
ASSIGNMENT #2: LMS RUBRIC

Multilingual Language selection Bilingual language Multilingual language

User between English selection with embedded selection with
Interface and French. text translation embedded text
features. translation features
and voice interactivity
like text-to-speech
applications.

Functional Navigation Platform is Platform is somewhat Platform is intuitive;

(Continued)
(Ease of unintuitive, with intuitive, supporting searchability enables
Use) limited search search functionality and quick information
functionality and user engagement. access and
engagement. encourages
independent
engagement.

Icon-based dashboard
is user-friendly and
reflective of previous
media experience.
 10
ASSIGNMENT #2: LMS RUBRIC

Training Modules provided to Modules provided to System requires

Materials users are difficult to users are easy to minimal training,
understand. understand. supporting all users
(students, teachers,
and administrators).

Relevant manuals
available both in
person and online
(Porto, 2015).

Costs Unavoidable Some recurrent Fully transparent costs

recurrent expenses expenses with options within budget,
(ex. annual for customization. maximizing available
subscriptions). resources (ex. open-
Some maintenance source). Customizable
Maintenance issues issues are covered. plans available.
are not covered.
Primary maintenance
issues are covered.
Technical
Technical Bilingual customer Bilingual customer Multilingual customer
Support support via email or support available 24/7 support available 24/7
phone during via email, IM chat or via email, IM chat, and
business hours with phone number with phone with immediate
unknown response reasonable response response time.
time. time.

Reliability Negative reputation Positive standing; Reputable standing;

(ex. System for staying current committed to remaining committed to
speed) and creative. current and innovative. innovating while
remaining stable.
 11
ASSIGNMENT #2: LMS RUBRIC

Poor reliable Fair and reliable Efficient and reliable

connection connection interaction. connection interaction.
interaction.

Server Insecure data Secure data storage and Secure North

Capacities storage and privacy security with active American data storage
(ex. Cloud with incompatible backups compatible with with active, real-time
storage) backups with current backups compatible
current network network technologies. with current network
technologies. technologies. Regular
Technical server site audits.

(Continued) System Rigid design for Accommodates K-12 Widely implemented

Design adult learning along with adult learning for adult learning and
educational educational contexts. K-12 educational
settings. contexts.

Available tools varied

for different
environments within
LMS.

Universal Does not meet UDL Meets UDL requirements Meets UDL
Design requirements for for inclusive learning. requirements for
inclusive learning inclusive learning and
and WCAG. Provides built-in tools special needs.
Network (ex. adjustable fonts),
but access to external Provides adaptable
servers are limited. plugins (ex. screen
readers) to access
resources within LMS
or external server.
 12
ASSIGNMENT #2: LMS RUBRIC

Security & Single sign on Single sign on Single sign on

Privacy authentication not authentication used with authentication used
used. encrypted connection. with encrypted
connection.
Access through un- Access through secured
encrypted connection. (TLS/SSL & Access through
connection. HTTPS). secured connection.
(TLS/SSL & HTTPS).
Basic firewall Layered firewalls &
protection used. network segmentation High-availability
Network used. firewalls, advanced
FIPPA regulations network design and
(Continued) are not considered. FIPPA regulations are network segmentation
considered. used.

FIPPA regulations are

considered and
supported by LMS
server system audits.

TOTAL SCORE - Degree Towards Exceeding ( / 30 )

 13
ASSIGNMENT #2: LMS RUBRIC

Medium Priority

DOES NOT MEET

CATEGORY CRITERIA MEETS (1) EXCEEDS (2) SCORE/NOTES
(0)

Pedagogical Provides basic Provides basic Provides access to

Design access to access to content content that integrates
organized materials as well as tools for well with interactive
Functional but few engaging students tools and new
opportunities for through interactive pedagogical tools for
interaction, learning. interactive learning.
constructivist, or Routinely added to the
engaging methods. system.

Learning Basic learning Average learning Advanced and very

Analytics analytics available analytics with some detailed learning
(ex. demographics, room for analytics.
usage, and customization.
Technical achievement data). Plugins allow users to
Compiles and add new forms of
accommodates analytics.
user feedback and
demonstration. Used as a diagnostic
for adaptive, flexible,
and personalized
learning (Porto, 2015).

Communication Emphasis on a A few multimedia Multiple communication

particular communication modes (ex. text, visual,
communication modes available. audio) available that
mode (ex. text). reflect the needs of
multilingual learners.
 14
ASSIGNMENT #2: LMS RUBRIC

Customization Limited ability to Some ability to Complete ability to

and Notification customize customize customize scheduling
scheduling and scheduling and and notifications for
Network notifications for notifications for upcoming assignments,
upcoming upcoming tasks, and learning
assignments, assignments, tasks, objectives.
tasks, and learning and learning
objectives. objectives. Ability to customize
course creation and
meet
instructor/teaching
needs.

TOTAL SCORE - Degree Towards Exceeding ( / 10 )

 15
ASSIGNMENT #2: LMS RUBRIC

Low Priority / Future Predictions

CATEGORY CRITERIA DOES NOT MEET (0) MEETS (1) EXCEEDS (2) SCORE/NOTES

ePortfolio Limited sharing Simple sharing Convenient sharing

capabilities between capabilities between capabilities between
LMS and ePortfolio LMS and ePortfolio LMS and ePortfolio
platform. platform. platform.
Functional
Data is removed each Data is removed after Data is not removed,
semester, allowing for one year, allowing for allowing for
short term continued permanent
accessibility to accessibility to accessibility to
individual e- individual e- individual e-
portfolios. portfolios. portfolios.

Adaptability System records System tracks usage, System tracks usage,

usage, but cannot adapting to individual adapting to individual
adapt to individual needs over time. needs over time.
needs over time.
Systems like Chatbot
guide users through
changing
Technical environments.

LMS Growth Unsustainable Platform can grow to Platform can grow to

(Scalability platform cannot grow accommodate efficiently
for users) to meet growth in increasing user accommodate
user population and population and needs. increasing user
needs. population and
needs.
 16
ASSIGNMENT #2: LMS RUBRIC

Social Media No embedded social Limited social media Enables embedding

Network Connections media options. options for from a wide variety of
embedding content. social media
platforms directly
within the LMS.

TOTAL SCORE - Degree Towards Exceeding ( /8)

 17
ASSIGNMENT #2: LMS RUBRIC

References

Bates, T. (2014). Choosing and using media in education: The SECTIONS model. In

Teaching in a digital age. Retrieved from

https://opentextbc.ca/teachinginadigitalage/part/9-pedagogical-differences-between-

media/

Bates, T. (2018). What is the cost in money and time?. In Teaching in a digital age.

Retrieved from https://opentextbc.ca/teachinginadigitalage/chapter/c-what-is-the-cost-in-

money-and-time/

Cangelosi, V. V. (2009). Getting Control of Course Management Systems:

Recommendations for Selection and Maintenance of E-Learning Products. Journal Of

Instruction Delivery Systems, 23(1), 13-17. Retrieved from

http://salt.org/salt.asp?pn=jids

Chickering, A. W., & Ehrmann, S., C. (1996). Implementing the seven principles:

Technology as lever. American Association for Higher Education Bulletin, 49(2), 3-6.

Retrieved from http://www.aahea.org/articles/sevenprinciples.htm

Coates, H., James, R., & Baldwin, G. (2005). A critical examination of the effects of

learning management systems on university teaching and learning. Tertiary Education &

Management, 11(1), 19-36. Retrieved from http://www.tandfonline.com/loi/rtem20

Conseil scolaire francophone de la Colombie-Britannique [CSF]. (2018). Conseil

scolaire francophone de la Colombie-Britannique (CSF) [Web page]. Retrieved 23

January 2018 from https://www.csf.bc.ca/en/

Longsight. (2018). Open source software & support for Higher Education [Web page].
 18
ASSIGNMENT #2: LMS RUBRIC

Retrieved from https://www.longsight.com/

M, L. (2017). What type of LMS is best for your school: proprietary, open-source, or

cloud-based? [Web page]. Retrieved from http://blog.neolms.com/type-lms-best-school-

proprietary-open-source-cloud-based/

Lorenzo, G., & Ittelson, J. (2005, July). An overview of E-Portfolios [Article]. Retrieved

from https://connect.ubc.ca/bbcswebdav/pid-4622322-dt-content-rid-

24520522_1/courses/SIS.UBC.ETEC.590.65A.2017W2.93526/ETEC590/lessons/lesson

02/docs/ELI3001.pdf

Pappas, C. (2014). 9 Tips To Apply Adult Learning Theory to eLearning. Retrieved from

https://elearningindustry.com/9-tips-apply-adult-learning-theory-to-elearning

Porto, S. (2013). The Uncertain Future of Learning Management Systems. Retrieved from

http://www.evolllution.com/opinions/uncertain-future-learning-management

-systems/

Ray, R. (2017). Localizing apps for multilingual conversations. Multilingual, 28(9), 44-46.

(2018). Retrieved from LearnNowBC website: https://www.learnnowbc.ca/

Zanjani, N. N., Edwards, S. L., Nykvist, S., & Geva, S. (2017). The important elements

of LMS design that affect user engagement with e-learning tools within LMSs in the

higher education sector. Australasian Journal Of Educational Technology, 33(1), 19-31.

doi:10.14742/ajet.2938
 19
ASSIGNMENT #2: LMS RUBRIC

Appendix A - Rubric in Process

Figure 1: Initial draft rubric template evaluating LMS on four criteria (Functional, Platform
Requirement, Organizational Consideration, Networking) from three personas (Student,
Teacher, Administrator)