Keeley faith

Project Report
EDIT677-D1-D2-D3 Assessment Tech & Learn an SP21

Introduction

I have taught Kindergarten for ten years and watched online academic programs become
frequently more available for students in the classroom. Our district has adopted several
programs for math enrichment and promoted their use in the classroom. Recently educators in
our school system have been instructed to use Freckle, an online math program for young
learners, as a tool for practicing math skills. I often wonder if these same skills used to practice
math in an online game will carry over to our math benchmark assessment. The specific
research question I have targeted and answered through my studies is: Will having students
complete 60 minutes of the Freckle math program increase student math benchmark
percentages? Historically, Kindergarten students complete Freckle math as an independent
learning station in the classroom. This year my students are virtual learners and Freckle is not
part of the virtual HCS program. I have noticed that student math benchmark percentages are
slightly lower in this class than past classes and it may be due to a lack of practice time. To
complete my research question, I used a test group of 14 virtual kindergarten students. Through
research assessments, student surveys, and scholarly literature reviews on online math programs
I have been able conclude about how effective using Freckle is at raising math benchmark
scores.

Review of Literature

When reading “A Summary Report of the Instructional Effectiveness of the “Harcourt

Math Program.” Technical Report Number 104” I noticed the scores for the posttest went up in
each grade level and no test scores went down for individual students. The research questions
are “Is the Harcourt Math Program instructionally effective?” and “Do selected chapters
significantly increase students' understanding of key math skills, concepts, and strategies as
measured by the program's chapter tests and by standardized achievement tests (i.e., Stanford)?”
Students from a 2nd, 5th, and 7th grade class were given the Stanford pre- and post-test as well as a
National pre- and post-test for math. During that 6-week period teachers picked a unit that was
most beneficial for their students to use with Harcourt Math program and students completed it
with instruction from their teacher (who had been previously trained) (Lloyd, 2000).

1
Keeley faith
Project Report
EDIT677-D1-D2-D3 Assessment Tech & Learn an SP21

“Raising Early Achievement in Math with Interactive Apps: A Randomized Control

Trial“ is a very recent study about a randomized control group for 4-5 year old children that will
prove math apps along with small group instruction will promote higher achievement on tests.
There were three groups: a. used the math app and received small group instruction. B. used only
the app and c. using online small group instruction. The PTM5 was used to assess all students
before and after being in the group. Students in group a. were 3-4 months ahead of their peers
and students in groups b. and c. were about equal (Outhwaite, 2019). Essentially, math apps
could be used as a form of learning math and are about as effective as small group
instruction. Ideally, both apps and instruction in correlation with one another work the best.

In the article “Math Shelf: A Randomized Trial of a Prekindergarten Tablet Number

Sense Curriculum.” 100 PK Students played the math shelf game for 6 weeks on the iPad and
comparison students played another math game on the iPad. The pretests were the same for all
the students but the students who completed math shelf did significantly better on the post
test. This app was tested to help students in low-income area at local head start schools. Low-
income students typically start out behind their middle-income peers and this app can benefit
head start students (Schacter, 2016). This study is very similar to the one I conducted but I
decided to track the specific amount of time students were on the math app because I am a virtual
teacher.

The question for the article “Understanding the Relationships between Interest in Online
Math Games and Academic Performance.” is “do students spending time on math games
correlate with better math scores”. Web analytics was used to determine how many students
were getting on coolmathsite.com and its sibling sites. The information was broken down by
demographics, time spent on the site, and state. Then the numbers were compared to the fourth-
grade national math test NAEP. Correlations were found that students who spent time on this
website had lower test scores and students with low-income backgrounds that used this site had
lower scores (Zhang, 2015). This study monitored how a student was using the app and if they
were practicing or just logging in to the app. This made me take a deeper look at the analytics
for each area students were practicing, and the number of questions answered.

2
Keeley faith
Project Report
EDIT677-D1-D2-D3 Assessment Tech & Learn an SP21

This is another article written by one of the same authors where students have an increase
in scores from a pre- test to a post test. I found it interesting that in “Using Math Apps for
Improving Student Learning: An Exploratory Study in an Inclusive Fourth Grade
Classroom.” The post test scores were significantly better than post test scores in an inclusive
room (Zheng, 2015). I would like to know more about the individualized learning on the math
app being used.

Methodology

After reviewing a variety of academic literature, I was able to identify the most important
resources and create a strategy to collect the data need to answer my question. I plan to use my
virtual kindergarten class to create a randomized control trial. Group A has 14 students and
Group 13 students. Along with small group instruction sessions, control group A will have 20
minutes of Freckle math to the weekly agenda. Students will receive passwords and be
instructed to complete fact practice on Monday, number facts on Tuesday, and number basics on
Wednesday. I will be logged in to our google meet link and available to help trouble shoot issues
and answer questions. Group B will not participate in Freckle math but will complete small
group math sessions.

For the quantitative data I used the data from the third quarter benchmark and compare it
to the fourth quarter benchmark. I can track student progress in each section: fact practice,
number facts, and number basics. I will be able to see how much time students are dedicating to
Freckle and their success rate in the app. Then I can compare the data from Freckle Math and
see if there is any correlation between time spent in Freckle and increase in math Benchmark
percentages. The benchmark is separated by standards and has a rubric that scores students for
each section. Students are labeled as intensive, strategic, or benchmark with an overall
percentage.

For the qualitative data I used a google survey linked here that was added the LMS page. The
following questions were used:

1. How many days a week did you get on freckle?

3
Keeley faith
Project Report
EDIT677-D1-D2-D3 Assessment Tech & Learn an SP21

2. What was the average time you spend on Freckle each day?
3. Do you think Freckle math helps you learn better?
4. How much did you enjoy using Freckle math?

I asked parents to read the questions to the students and help them answer the questions. The
survey helped analyze the percentage of student interest, time spent on the program vs questions
completed, and how students felt about using the program.

Analysis

Once I implemented the benchmark assessments, monitored, and observed activity on the
Freckle dashboard, and received feedback from the student survey I began to analyze all my data
using several different methods.

First, I look at my third and fourth quarter benchmark data and created bar graphs based on how
the class progressed overall from being intensive, strategic, and benchmark.

Here is the 3rd and 4th quarter benchmark data and the grading scale the I used for my
quantitative data. Group A is above the row of blue cells and Group B is below the blue cells.

4
Keeley faith
Project Report
EDIT677-D1-D2-D3 Assessment Tech & Learn an SP21

The bar graphs that show group A overall progress vs group B overall progress:

43% more students met the benchmark goal on the fourth quarter assessment in group A (chart
1). There are 30% less students in the strategic group for the fourth quarter in group A (chart 2).
Both group A and group B only had one intensive student four the fourth quarter (chart 3).

I created a survey that parents and students completed together about Freckle usage and
satisfaction for my qualitive data.

5
Keeley faith
Project Report
EDIT677-D1-D2-D3 Assessment Tech & Learn an SP21

9 out of 14 students responded to the survey. 66% of the class gave Freckle a 4-5 on the
enjoyment scale (chart 1). 77% of the 9 students who answered completed all or more of the
required 3 days a week on the Freckle application (chart 2). 77.8% of students think freckle
helps them learn better (chart 3). 77% of the 9 students who answered were using the app the
required amount of time that was directed (chart 4).

Findings

After analyzing the data from the 3rd and 4th quarter benchmark assessments and
interpreting the student and parent questionnaires that were completed, I have been able compile
the results to my original question with the evidence I need. I wanted to know Will having
students complete 60 minutes of the Freckle math program increase student math benchmark
percentages. My data shows that student percentages increased overall from the 3rd to the 4th
quarter, but even more students were at the benchmark level when using Freckle math program
within the directions I presented. Based on the survey I was able to identify the general time and
number of days were week students spent on Freckle and it was close to what I originally asked

6
Keeley faith
Project Report
EDIT677-D1-D2-D3 Assessment Tech & Learn an SP21

students to complete. I also polled student interest and most students gave Freckle high ratings of
4-5 for enjoyment.

Now that I know when students complete Freckle math practice 3 or more days a week
and for 15-20 minutes per day their benchmark scores can increase. Since Freckle is a valuable
to for math practice, I plan to spend more time learning about the dashboard and how to
personalize activities for each student in conjunction with small group math lessons. Currently
freckle is not part of the virtual HCS kindergarten curriculum, but it should be added as
enrichment for math practice.

I also noticed that the biggest increase from group A to group B was in the number of
benchmark students based on the quantitative data. Using Freckle practice with students who are
strategic but close to benchmark has the biggest impact. I am wondering if adding more time or
using specific practice activities for small group extensions in the “strategic group” helps these
students get over the hump and into “benchmark”.

Freckle is fun and students felt like they were learning and enjoyed using the app based
on the survey given for qualitative data. I can incorporate Freckle into free time and as an
incentive while boosting math scores.

7
Keeley faith
Project Report
EDIT677-D1-D2-D3 Assessment Tech & Learn an SP21

References
Lloyd, P., & Educational Research Inst. of America, B. I. (2000). A Summary Report of the
Instructional Effectiveness of the “Harcourt Math Program.” Technical Report Number 104.
http://search.ebscohost.com.login.library.coastal.edu:2048/login.aspx?direct=true&db=eric&AN
=ED454090&site=ehost-live
Outhwaite, L. A., Faulder, M., Gulliford, A., & Pitchford, N. J. (2019). Raising Early
Achievement in Math with Interactive Apps: A Randomized Control Trial. Journal of
Educational Psychology, 111(2), 284–298.
http://search.ebscohost.com.login.library.coastal.edu:2048/login.aspx?direct=true&db=eric&AN
=EJ1205220&site=ehost-live
Schacter, J., Shih, J., Allen, C. M., DeVaul, L., Adkins, A. B., Ito, T., & Jo, B. (2016). Math
Shelf: A Randomized Trial of a Prekindergarten Tablet Number Sense Curriculum. Early
Education and Development, 27(1), 74–88
http://search.ebscohost.com.login.library.coastal.edu:2048/login.aspx?direct=true&db=eric&AN
=EJ1084762&site=ehost-live
Zhang, M. (2015). Understanding the Relationships between Interest in Online Math Games and
Academic Performance. Journal of Computer Assisted Learning, 31(3), 254–267.
http://search.ebscohost.com.login.library.coastal.edu:2048/login.aspx?direct=true&db=eric&AN
=EJ1058862&site=ehost-live
Zhang, M., Trussell, R. P., Gallegos, B., & Asam, R. R. (2015). Using Math Apps for Improving
Student Learning: An Exploratory Study in an Inclusive Fourth Grade Classroom. TechTrends:
Linking Research and Practice to Improve Learning, 59(2), 32–39
http://search.ebscohost.com.login.library.coastal.edu:2048/login.aspx?direct=true&db=eric&AN
=EJ1050818&site=ehost-live