Promising Practices Lessons Learned from a Pilot Study of the NES Essential Academic Skills Mathematics Subtest Nicholas D. Hartlep with George Omboga • Metropolitan State University, Saint Paul, Minnesota Promising Practices: Lessons Learned from a Pilot Study of the NES Essential Academic Skills Mathematics Subtest1 Nicholas D. Hartlep with George Omboga Metropolitan State University, Saint Paul, Minnesota This white paper shares how the School of Urban Education (UED) at Metropolitan State University implemented a pilot study of a test preparation tutoring program in order to help teacher candidates pass the National Evaluation Series (NES) Academic Basic Skills, a requirement to becoming a Tier 4 licensed teacher in the state of Minnesota. For interested readers, Metropolitan State University is a Minority Serving Institution (MSI); specifically, it is federally “designated” as an Asian American and Native American Pacific Islander Serving Institution (AANAPISI).2 The paper describes the activities that took place over a three semester period: Fall 2017 to Summer 2018. The report also shares lessons learned from offering NES mathematics test prep sessions, as well as lessons learned from the literature on test preparation programs at MSI’s. 1 Direct questions and correspondence to Nicholas D. Hartlep, Ph.D., Associate Professor of Urban Education, Chair of Early Childhood and Elementary Education Department, Graduate Program Coordinator, Metropolitan State University, School of Urban Education, 1450 Energy Park Drive, Suite 100-B1, St. Paul, MN 55108, Office Phone: (651) 999-5936 Study Background Teachers who seek initial licensure in Minnesota are required to demonstrate proficiency in teaching by passing a series of tests in reading, writing, math, pedagogy, and content knowledge in the specific licensure field. The Professional Educator Licensing and Standards Board (PELSB) is mandated by the state to oversee and implement teacher licensing. In addition to developing the teacher code of ethics and issuing of teacher licenses, PELSB is charged with the responsibility of approving teacher preparation programs, including implementation of the teacher testing requirements set forth in Minnesota statute. exams for pre-service teachers? Do the majority of pre-service teachers (test takers) perform at par across the different races? According to Petchauer (2016), “Praxis ultimately disadvantages test takers from cultural, linguistic, and economic minoritized groups and works to stratify the teaching profession along racial lines” (p. 173). Similarly, commenting on how helpful teacher licensure tests are, Hartlep and Antrop-González (2018) note the following: “Weighing a pig won’t make it fatter, and racist exams will not increase the number of teachers of color and American Indian teachers in the nation’s classrooms” (para. 1). Hartlep and Antrop-González (2018) further note that “the norms upon Pre-service teachers are required to which the NES basic skills tests were take three Minnesota NES Essential created do not reflect the lived expeAcademic Skills subtests: (1) Readriences of the bulk of Metropolitan ing, (2) Writing, and (3) Mathematics. State University pre-service teachers Indeed, the NES is a very common who are aspiring to become [Tier 4] test that states across the United licensed teachers. Moreover, these States use for teacher licensing. In tests do not measure what students fact, according to Mitchell, Robinson, know; instead they measure what Plake, and Knowles (2001), the “most students ‘have.’ Test takers who are of the commonly used teacher licenadvantaged materially, linguistically sure tests come from the Educational and culturally will pass the tests at Testing Service (ETS) or National Evaldisproportionate rates compared to uation Systems (NES)” (p. 83, italics diverse and poorer students” (para. added). Another common exam that 9). Despite the NES being racist and teachers take in the United States is culturally insensitive, would not it also the Praxis. But how helpful are these 2 be racist to inadequately prepare diverse pre-service teachers to pass the MTLE, because it would prevent them from becoming Tier 4 licensed teachers in the state of Minnesota? This “situation”—of passing or failing the NES—returns us to the present overview: What was done over three semesters to address NES Basic Skills Exams pass rates among students of color and American Indian students and what lessons were learned? See William Casey Boland (2018) “The Higher Education Act and Minority Serving Institutions: Towards a Typology of Title III and V Funded Programs.” Education Sciences, 8, “Designated’ is different than “grant-funded.” In order to be “designated,” and eligible to apply for grant funding, 10% of students must be Asian American and Native American Pacific Islander students and 50% low income. 2 tensive mathematics test preparation sessions also offered semi-personalized learning to help build on the students’ strengths and to also shore Metropolitan State University’s School up their weaknesses across several of Urban Education (UED) was inmath skill areas. The ultimate goal of terested in offering intensive NES this “intervention” was to ensure that mathematics test preparation sessions students felt confident that they were for its students and other interested well-prepared for and could do their participants from outside of its cambest on the test. It also was expected pus. Metropolitan State University that the students would take the NES partnered with Augsburg University, mathematics subtest immediately a nearby private liberal arts college following their attendance at all of the in Minneapolis, Minnesota, and lever- intensive sessions. Participants who aged its resources to offer an 8-week attended all the sessions were preintensive test preparation program sented with certificates (see Appendix during the Spring 2018 semester. A). can American students, friends, and family members; (c) social persuasion about their abilities from professors; and (d) emotional or physiological states such as nervousness and anxiety” (pp. 174–175). Building from the aforementioned work, and other work around stereotype threat (see Arbuthnot, 2009), this white paper aims to synthesize the lessons learned from the data collected while deploying a cross-institutional collaborative NES mathematics test preparation and tutoring program at Metropolitan State University (Midway Center site), as well as conceptualize actions it can do to extend the lessons learned based upon the literature over the The free NES mathematics test prep course of three semesters: Fall 2017, sessions were offered at one locawhich was a planning phase; Spring Promising Practices tion on Friday evenings during the 2018, when mathematics sessions Research has examined test preparaspring and on Saturday mornings were offered and data were collected; tion interventions for basic skills licenduring the summer. The foci and aim and Summer 2018 when mathematics sure examinations, such as ETS’ Praxis of the sessions were enhancing parsessions were offered and data were (see Tyler, 2011). For instance, in his ticipants’ knowledge of the mathecollected. The goal of carrying out study of African American pre-service matics concepts that are covered in this pilot study was to document the the NES mathematics subtest, and of teachers who attended a Historically lessons learned. These lessons would Black College/University (HBCU) and equal importance, to acclimate the be used to develop and institutionalwho took the Praxis, Petchauer (2016) participants to the NES mathematics ize promising practices regarding the found that students drew from “four subtest itself. For instance, students NES mathematics subtest for PK–12 were interested in knowing how many information sources to formulate their teacher candidates in Minnesota. beliefs about Praxis capabilities in questions would be asked, how long they had to complete the NES math- the following hierarchy: (a) previous ematics subtest, and whether or not it mastery or failure experiences with was done with pencil and paper or on other standardized tests and relevant a computer. Necessarily, then, the in- classes; (b) vicarious experiences from the symbolic models of other Afri- Establishing Actionable, Collaborative, Justice-Oriented Partnerships 3 Box 1: Passing Scores of Minnesota Licensure Exams 1. MTLE Pedagogy and Content Exams: a. Passing Scores are > 240 for all subtests. 2. Basic Skills (Reading, Writing, and Math): Options for demonstration of proficiency and passing scores are shown below: a) Multi-state NES Essential Academic Skills • Reading: 205 • Writing: 214 • Math: 213 b) Minnesota NES Essential Academic Skills • Reading: > 520 • Writing: > 520 • Math: > 520 c) MTLE Basic Skills Test (this test was discontinued June 8, 2016) • Reading: 240 • Writing: 240 • Math: 240 d) ACT Plus Writing (Taken February 2005 to present) • Combined Score: 22 • Combined English/Writing or ELA: 21 e) 1600 SAT (New Version) (Taken March 1, 2016 to present) • Reading + Writing: 550 • Math: 570 f) 2400 SAT (Old version) (Taken March 12, 2005 – February 28, 2016) • Reading: 510 • Writing: 510 • Math: 520 g) GRE (Taken 2001 to August 11, 2011) • Verbal Reasoning: 450 • Analytical Writing: 3 • Quantitative Reasoning: 540 h) GRE (Taken 2001 to August 11, 2011) • Verbal Reasoning: 150 • Analytical Writing: 3 • Quantitative Reasoning: 145 i) Praxis I (Taken prior to September 1, 2014) • Reading: 173 • Writing: 172 • Math: 171 j) Praxis CORE (Taken September 1, 2014 to present) • Reading: 156 • Writing: 162 • Math: 150 Licensure Exams in Minnesota because it was believed, at the time of on to write, “A less successful strategy Teachers in the state of Minnesota are required to demonstrate competency on a series of tests in reading, writing, math, pedagogy, and licensure field specific content knowledge.3 A listing of all exams and the required pass scores for the different set of exams is provided in Box 1. Prior to June 8, 2016, teacher candidates must have had taken and passed Minnesota Teacher Licensure Examinations (MTLE) consisting of Pedagogy, Content, and Basic Skills in order to become licensed teachers. Effective June 8, 2016, the MTLE Basic Skills Test was discontinued and replaced by a National Evaluation Series™ (NES®) assessment: Essential Academic Skills, effective August 30, 2016. The NES, in particular, the Mathematics subtest, is the focus of the current report. its inception, that other EPPs required students to pass the real MTLE before they could be admitted. Whether or not that was accurate does not matter. What does matter is that UED faculty and staff believed they were doing what was best for its diverse students. Historically, a 1 credit elective course in the mid-2000s EDU 150 Preparing for the PPST was structured very similarly to the current workshops with Friday evening sessions. It was taught several times by community faculty but always struggled to have sufficient enrollment and was eventually discontinued. Current Practice Metropolitan State University teacher candidates take the MTLE after completing their student teaching. However, a systematic review and analysis of candidate performance data showed Context: What Was Done? that many of the UTP students were Since 2010, UED has had a policy that not passing the MTLE Basic Skills required students to take the MTLE Exam. Table 1 provides the overall, inipractice test in order to be admitted tial, and retake pass-rate information, into the Urban Teacher Program (UTP). as well as pass-rates disaggregated The original idea had been that stuby race. Drawing from these data, a dents can take the test, and learn how significant number of students of color they would fare if they had taken the and American Indian students do not real test. This admission practice—of pass the MTLE compared to the White only having had tried the practice students. test—had been approved, because it was not burdensome for UTP stuIt was also noted that Rikard and Nordents, because the practice tests are den (2006) state, “Generally, first-time free and can be conveniently taken test takers who study four to six weeks from the comfort of one’s own home. in advance, understand the nature of This official UTP admission policy has the test, and prepare for testing condialso been thought to be more “demtions tend to have better results than ocratic” when compared to other those who do not” (p. 45). As they go educator preparation programs (EPPs), 3 https://mn.gov/pelsb/assets/Teacher%20Licensure%20Testing%20Information_tcm1113-321360.pdf is to take Praxis I with no preparation just to ‘see what it’s like’” (p. 45). The UTP has been for a long time simply requiring its students to take the practice MTLE as part of the admission process with limited direct assistance— something that is not advisable based on the literature. As a result, it was during the Fall 2017 semester that Metropolitan State University began talks with Augsburg University on how to help diverse teacher candidates pass the MTLE, namely the NES. A total of three planning meetings were held in November and December of 2017. During this period of time, peer-reviewed research was read and reviewed. Particular focus was given to research that had implications for how to be the most effective in planning test preparation sessions for diverse test takers who were hoping to pass the NES mathematics subtest. Interestingly, though, a short survey delivered over the summer of 2018 found that few of the 33 EPPs in the state of Minnesota currently require passing the NES as part of their admission process. Nearly three quarters of the EPPs (20 of 28) who responded to the survey do not require students to pass the NES to be admitted. Of the eight EPPs whose responses were different, 3 EPPs require the tests to be passed before full admittance, while the other five require teacher candidates to take the NES for admission; but they do not need to pass. 5 Table 1. Metropolitan State University Student Basic Skills Test Results 2016–17 Program Year (Best Attempt) OVERALL SKILLS PASS RATE Mean Total Scaled Score Subtest # Takers # Pass # Not Pass % Pass % Not Pass UED State Mathematics 79 37 42 47% 53% 226.8 247.8 Reading 75 38 37 51% 49% 238.7 247 Skills Writing 76 37 39 49% 51% 234.7 244.6 BASIC SKILLS PASS RATE BY ATTEMPT Subtest Cumulative Attempt 1 Attempt 2 Attempt 3 Attempt 4 Attempt 5+ # Takers Pass Rate # Takers Pass Rate # Takers Pass Rate # Takers Pass Rate # Takers Pass Rate # Takers Pass Rate Mathematics 79 47% 56 54% 23 9% 10 20% 6 33% 13 8% Reading 75 51% 53 55% 17 35% 8 0% 7 14% 30 7% Writing 76 49% 54 52% 23 17% 10 20% 5 20% 6 33% STUDENTS OF COLOR AND AMERICAN INDIAN STUDENTS Mean Total Scaled Score Subtest # Takers # Pass # Not Pass % Pass % Not Pass UED State Mathematics 41 13 28 32% 68% 211.5 233.2 Reading 40 13 27 33% 68% 224.0 235.0 Skills Writing 40 9 31 23% 78% 220.8 232.5 WHITE Mean Total Scaled Score Subtest # Takers # Pass # Not Pass % Pass % Not Pass UED State Mathematics 38 24 14 63% 37% 243.4 250.4 Reading 35 25 10 71% 29% 255.6 249.1 Skills Writing 36 28 8 78% 22% 250.2 246.7 Math NES Test Prep Program Program Description In partnership with Augsburg University, the School of Urban Education offered intensive mathematics tutoring sessions to help students prepare and pass the NES mathematics subtest, which is needed to receive a Tier 4 teaching license. The program provided insight into all aspects of mathematics with specific emphasis on exam taking skills. Emphasis was placed on highlighting issues and challenges that candidates face in taking the exam and exam taking strategies. The original curriculum that was used in the “intervention” was divided into four modules and was designed to be covered over a period of eight weeks. The sessions were also designed to ensure practical application of the learning and to enable the participants take the NES exam upon completion of the intensive tutoring. During the eight prep sessions, the tutor reviewed mathematical concepts with participants, led guided practice with participants, and assigned a variety of additional “at-home” practice work as a means to reinforce math skills. Each session begun with participant questions and all participants reviewing the practice problems assigned during the previous session. The program operated for one day a week (Fridays) for eight weeks in the spring and one day a week (Saturdays) for 4 weeks during the summer. Prep sessions lasted for two hours during the spring and 5 hours during the summer. During the summer sessions there were student panels and “Skype-A-Scholar” sessions. Students who successfully passed the NES mathematics subtest came and spoke to the test preparation attendees about their experience taking the test during the student panel sessions. “Skype-A-Scholar” sessions were sessions where experts on test taking were skyped into the classroom to talk about research and diverse teacher candidates’ test-taking experiences. The scholars discussed how to keep networks of people around you that are positive and encouraging. The panels and “Skype-AScholar” sessions occurred over lunch and lasted thirty minutes. Part of the intervention involved registering the test-takers for the real NES mathematics subtest via Pearson’s website. The process to register for the NES mathematics test is confusing and non-intuitive. The Pearson Vue website sometimes was not functioning, and students had many questions. This left many students in the proverbial dark when it came to testing accommodations they were entitled to, and how to get answers to their important questions. Many UED teacher candidates are entitled Figure 1. Students Taking Practice Test As Part of Summer 2018 Sessions—Error Analysis During Summer July 14, 2018 Session to receive a testing accommodation given that English is their second language. During the NES mathematics test prep sessions, participants had an opportunity to ask questions about how to register for the NES test, and were provided with letters which they could upload to the NES website, in order to request an accommodation for extra time (Figure 2). In the event this was not completed on site, the prep coordinator used a variety of synchronous technologies (iPhone’s FaceTime, Skype, and Google Hangout) to walk the teacher candidates through the registration process (Figures 3 and 4). Figure 2. Accommodation Letter 7 Figure 3. Using FaceTime to Assist Metropolitan State University Students 8 Figure 4. Using Google Hangout to Register a Metropolitan State University Student Target Group The NES mathematics test prep sessions were a localized program for students from Metropolitan State University and Augsburg University who were pursuing teaching license in the state of Minnesota. However, given that the tutoring sessions were free and open to the public, pre-service and in-service teachers who hadn’t passed the NES mathematics subtest frequently attended the sessions. Learning Objectives Upon completion of the NES Test preparation sessions, participants were able to do the following: (a) Mathematics Knowledge • Brush up content knowledge across several math skill areas. (b) Test Performance • Master the test-taking skills and strategies to maximize confidence and competence to take and pass the NES mathematics test. Program Modules The 8-week curriculum was divided into four modules designed to ensure practical application of the learning and to enable the participants take the NES exam upon completion of the tutoring. The modules were: 1. 2. 3. 4. Measurement and Geometry Problem Solving and Reasoning Pre-Algebra and Algebra Probability and Statistics Total duplicated number of participants who successfully completed the trainings over the Spring 2018 and Summer 2018 sessions, combined was 240+. Delivery Methodologies A variety of learning delivery and assessment methodologies were employed in an attempt to enable the prep sessions to be participatory and interactive and to allow participants to effectively assimilate the exam taking skills. Importantly, emphasis was placed on adult learning methodologies and over the duration of the program there were a combination of: o Lectures to explain the nature and format of the exam and teach the skills needed to master it; o Analysis of exam sample questions and class discussions of illustrative sample questions; o Group discussions on proven test-taking strategies to maximize participant performance in each exam area; o Tutorials on score-raising strategies for every tested topic delivered by former students who had passed the exams on panels; o Skype-A-Scholar sessions whereby expert scholars who research test preparation share their thoughts with session participants; o Individual and group practice tests to measure candidate performance and progress throughout the tutoring sessions, plus extensive materials for individual study; and o Individual readings outside of the classroom (in preparation for upcoming modules) Because the Spring 2018 sessions occurred later in the evening, snacks and beverages such as coffee and Somali tea were provided. The original plan was to offer dinner. Pizza Hut was provided for the first session, but it was The Spring 2018 intensive mathematics prep sessions were determined that it would be too costly, which is why snacks delivered over the period of 8 weeks, January 12, 2018 to were subsequently offered instead of pizza. The summer 2018 session participants, on the other hand, were proMarch 2, 2018. vided Little Caesars Pizza and bottled water for lunch. It was during this time that the student panels and Skype-AThe Summer 2018 intensive mathematics prep sessions Scholar sessions were provided. The aspect of food/snacks were delivered over the period of 4 weeks, June 23, 2018 is an important one and something that should be built to July 14, 2018. into an operating budget. 9 could have missed signing in (e.g., coming into the room late, not signAll participants were required to take ing in despite arriving to the room a practice test before attending the on time, etc.). Also, the numbers do tutoring session to determine how not include the participants who were much the participants knew before preparing to be a teacher at another the tutoring program. The practice institution of higher education and/or test scores gave us a sense of how who may have already graduated, but well a student performed on various had not passed the NES mathematics sections. Based on their performance, subtest but who attended the free we were able to determine whethsessions. er the participants should pursue a broad- or focused-style of prep The majority of the students who course so as to build their capacity. participated in the NES mathematics After each training session, particitest prep sessions during Spring 2018 pants were provided with a feedback were women (69.70%), and more than form to complete (see Appendix B). two thirds (78.80%) were students of The feedback forms were collected color and/or American Indian stuand collated and used when planning dents. A total of twenty students took for the subsequent week’s session. the NES practice test4, which contains Assessment Methodologies 36 questions distributed across five domains (competencies). Figure 6 shows the participant average scores A total of 240+ people benefited in the practice test on the five dofrom the tutoring sessions. Tables mains. Considering the students who 2 and 3 below show the number of did take the NES practice test, the participants who attended the 2018 practice test generally shows their NES mathematics test prep sessions individual strengths and weakness by each week. Though the target across the five domains. Overall, group was students from Metropolstudents need help on their mastery itan State University and Augsburg University, the test prep sessions drew on knowledge of the subject and their ability to apply that knowledge participants from a wide variety of to teaching a wide range of children PK–12 and other higher education with diverse needs. Most students institutions (see Figure 5). The numneed help in measurement and geber of participants who benefitted ometry, probability and statistics, as from all the training sessions ranged from 16 to 50. These numbers actual- well as mathematical problem solving. The results make us ask the question: ly could be larger, as some students Participants 10 4 The NES practice test scores refer to the NES Practice Tests that Metropolitan State University purchased as part of the pilot study. What do we do when we know the diagnostic tests predict that a student will not pass? What should Metropolitan State University or Augsburg University do for these students? How can mathematics remediation be offered for students who need it? When should students take the real exam? How can we avoid making their test performance on the practice test being a negative testing experience for them? Table 2. Duplicated Head Counts of NES Mathematics Test Prep Spring 2018 Participants Duplicated Head Counts W eek M etro State Augsburg Total 1 39 11 50 2 21 11 32 3 14 9 23 4 19 4 23 5 13 5 18 6 10 6 16 7 12 5 18 8 13 5 17 Total 141 56 197 Table 3. Duplicated Head Counts of NES Mathematics Test Prep Summer 2018 Participants Duplicated Head Counts W eek M etro State Augsburg Total 1 7 10 17 2 6 6 12 3 * * * 4 4 10 14 Total 17 26 43 * Sign-in sheet missing 11 Figure 5. Institutional Attendees Represented During the Three Semesters Figure 6. Average Practice Test Scores Number properties and operations 73.50% Principles of algebra 70.30% Mathematical problem solving 66.00% Probability and statistics 64.20% Measurement and geometry 52.00% 0% 10% 20% 30% 40% 50% Percent Correct 60% 70% 80% Overall Performance: NES Mathematics Test Results A systematic analysis of students MTLE scaled scores was carried to evaluate the performance of the students who took the NES test between January 2017 and June 2018. Over this period of time, 114 students took the NES mathematics test. Of this, 15 had attended the NES mathematics test preparation and tutoring sessions. Figure 7 shows that a majority (80%) of the participants who attended the NES mathematics test prep sessions passed the NES exams during their first attempt as opposed to Figure 8, which shows that 41% of those students who did not attend the exam prep sessions. The average scaled score for those participants who attended the NES mathematics test prep sessions was higher compared to those who did not attended the tutoring sessions. While the highest scaled score attained by either student was 594, those participants who attended the NES mathematics tutoring sessions failed to make the pass score of 520 by only 25 scores compared to 120 scores by those who did not attended the tutoring sessions (see Figure 9). Figure 7. Attended NES Mathematics Exam Preparation Sessions FAIL (n = 3) 20% PASS (n = 12) 80% Pass Fail Figure 8. Did Not Attend NES Mathematics Exam Preparation Sessions NES Essential Skills Exam Score 700 600 594 594 500 495 536 512 400 400 300 200 100 Pass (n = 41) 41% Fail (n = 58) 59% 0 Average Score Minimum Score Did Not Attend Exam Preparation Sessions Attended Exam Preparation Sessions Figure 9. Comparative NES Mathematics Test Score (First Attempt) Pass Fail Maximum Score Lessons Learned from the Pilot and the Literature 1. The NES mathematics practice tests are accurate gauges to whether or not a teacher candidate will pass or fail the actual real NES mathematics test. 2. Many of the UED students have not taken the ACT. This means, PELSB’s transfer rules are not consequential for UTP students. In this pilot study, only two students—both white and both female—had ACT scores. Both students’ ACT scores were high enough that they did not have to take and pass the NES. 3. The majority of the participants who attended the eight sessions were students of color. Some students had already graduated from college/university, but nevertheless had not passed the NES/MTLE Basic Skills Examination(s). 14 7. The School of Urban Education “should celebrate and systematically share students’ successful experiences on [NES] exams. This is less to reward successful students and more to provide vicarious models of success for students who have not yet taken th[e] exam” (Petchauer, 2016, p. 186). Twitter was used to share student success. Students who passed the NES mathematics test were asked if they would take a photo (see Appendix C) and a photo-release form was signed by the test taker, allowing Metropolitan State University and Augsburg University to share their success with others. The hashtags #iPassED and #SchoolofUrbanEd were used and shared with @ AugsburgU and @Choose_Metro (see Appendix C). 8. The School of Urban Education should host panels of students each semester who have passed the NES mathematics 4. Expecting students to take the NES mathematics exam upon test so they can share their positive testing experience with othcompletion of the tutoring, which is considered “best practice” ers (Petchauer, 2016). These panels should take place as a part (see Rikard & Norden, 2006, p. 46) in the literature and suggest- of the intensive test preparation sessions. “Skype-A-Scholar” sessions are beneficial as well. Scholars who conduct research ed by Skype-A-Scholar, it does cost more for teacher candidates. UED ought to solicit grants or scholarships that can fund on teacher licensure exams should share their knowledge with NES mathematics test prep attendees. students to take the tests. Vouchers are provided by Pearson, but only eighteen $25 vouchers are provided, which are not enough. This amounts to being able to pay for nine $50.00 NES 9. The School of Urban Education should create “short video profiles of students who have passed [the NES and give] advice, mathematics subtests. tips, and strategies for future test takers” (Petchauer, 2016, p. 187). 5. A dedicated staff/faculty person ought to assist UED students register for the NES Exams. The website is confusing to 10. The School of Urban Education should continue offering students and the dedicated person could also help students apply for testing accommodations if they have a disability or are certificates of perfect attendance because it was an incentive that participants pointed out was nice (Appendix A). The certifa non-native English speaker. icates were something that teachers could add to their professional portfolios. 6. Test prep sessions put on by Metropolitan State University and/or Augsburg University facilitators should make “explicit how preparation activities are aligned to [NES] and reproduc[e] 11. Research indicates that when stereotype is activated, cognitive processing is altered when taking standardized mathematactual testing circumstances” (Petchauer, 2016, p. 186). The ics tests. According to research by Arbuthnot (2009), stereotype curriculum used during the Spring 2018 session did this, but threat can lead to students of color to change the way they apneeded to be rewritten for clarity. Students offered feedback proaching solving mathematics problems in testing situations. after the sessions that the printouts were difficult to read beMath courses at Metropolitan State University and also NES cause they were blurry. We purchased a one year site license mathematics intensive test preparation sessions should inform from Longsdale Publishing which will expire on 05/14/2019 for students that this phenomenon exists and that they should be $750.00. The Longsdale curriculum that was used in the Sumaware of it when they go to take the math test. Test takers are mer 2018 session was much more improved. best served when they deploy “non-traditional” ways of solving mathematics problems on tests such as the NES. Pilot Study Limitations The attendance could be inaccurate. Some students were marked as having attended. That number may in fact be higher. Students signed in before the sessions began, and some may have in fact attended, but missed the sign-in sheet, such as was the case during week 3 (see Table 3). Also, the number of test-takers who have passed might also be suppressed, and not truly accurate because we do not have access to the scores of the students whose scores are not reported to Metropolitan State University. For example, Karen Valero, emailed indicating that she had passed the NES mathematics. This is what she had to say: This Saturday I took all three NES Essential Academic Skills tests (all three for the first time), and I passed the Reading and Mathematics tests (no immediate results were available for the Writing test because of the essay component, but I feel confident that I did well). So I want to take the opportunity to thank you for all that you did to put together the study sessions to prepare students for these tests at Metro and at Augsburg. I attended the second and fourth math sessions, as well as the writing session. I appreciated and took advantage of all of the materials provided during these sessions as well as those for those that I missed. It all made a huge difference for me. Not only did the sessions and materials helped me refresh concepts I had not reviewed in over 15 years (particularly for math), but they helped build the confidence I needed to pass those tests. In short, this is my testimony that these sessions do work and make a huge difference for students, and I thought you should know. Thank you for making these sessions possible, for your support in recommending the allowed extra time for those like me for whom English is our second language, and, overall, for caring. Gwyn Burnett, a Metropolitan State University student emailed this: I want to thank you and all other stakeholders involved for offering Basic Skills Math Prep Sessions this summer. Having taken those courses, even only being able to attend for two of the sessions, I was able to pass my Basic Skills Math test today. Not only was the instructor great, but so too was the video guest speaker. I learned both math and how to manage my study time, as well as how to keep the best circles of people around me, so as to ascertain my success. Thank you again, Gwyn Burnett References ACT. (2016, June 27). An ACT White Paper: 5 ways to compare 2015–2016 and 2016–2017 ACT writing scores. Retrieved from https://www.act.org/content/dam/ act/unsecured/documents/WhitePaper_5_Ways_to_Compare_ Writing_Scores.pdf Arbuthnot, K. (2009). The effects of stereotype threat on standardized mathematics test performance and cognitive processing. Harvard Educational Review, 79(3), 448–472. Holmes, V. (2015). Powerful beyond measure: African American assessments of the Minnesota Teacher Licensure Exam experience. Unpublished master’s thesis. Retrieved from https://file. taskstream.com/file/b3nvf3Uai2h0Hy3shvMquj11MxqohxcWnd3judYwx128Mge8twR6q4ggdN4yn1ldOvd2h2cFplq8scWt6l2scDgi0s4L1dfu3cAg2yyiYt4wj2cRegljgbZ90mayZ00iu4U1dc/ holmes_thesis_dec_2015.pdf Petchauer, E. (2015). Passing as white: Race, shame, and success in teacher licensure testing events for black preservice teachers. Race Ethnicity and Education, 18(6), 834–857. Petchauer, E. (2016). Shall we overcome? Self-efficacy, teacher licensure exams, and African American preservice teachers. The New Educator, 12(2), 171–190. Petchauer, E., & Baker-Doyle, K. J. (2016). Longwell-Grice, R., R., McIlheran, J., ‘Next thing you know, her hair turned Schroeder, M., & Scheele, S. (2013). green’: Absurdity and uncertainty The effect of tutoring on math scores Baker-Doyle, K. J., & Petchauer, E. (2015). in high-stakes teacher test space. for the Praxis I exam. The Learning Rumor has it: Investigating teachCritical Studies in Education. Advance Assistance Review, 18(1), 46–56. er licensure exam advice networks. Online publication. 1–18. Teacher Education Quarterly, 42(3), 3–32. Retrieved from https://files.eric. Mitchell, K. J., Robinson, D. Z., Plake, B. Rikard, L., & Norden, L. (2006). ed.gov/fulltext/EJ1090424.pdf S., & Knowles, K. T. (Eds.). (2001). Winning strategies for passing Praxis Testing teacher candidates: The role I and II. Journal of Physical Education, of licensure tests in improving teacher Recreation & Dance, 77(3), 44–47. Boland, W. C. (2018). quality. Washington, DC: National The Higher Education Act and MiAcademy Press. nority Serving Institutions: Towards Shuls, J. V. (2017). a typology of Title III and V funded Raising the bar on teacher qualPetchauer, E. (2012). programs. Education Sciences, 8(1), ity: Assessing the impact of Teacher licensure exams and Black 33–52. Retrieved from https://files. increasing licensure exam cutteacher candidates: Toward a new eric.ed.gov/fulltext/EJ1175006.pdf scores. Educational Policy. Adtheory and promising practice. The vance publication. https://dx.doi. Journal of Negro Education, 81(3), org/10.1177/0895904816682315 Goldhaber, D., & Hansen, M. (2010). 252–267. Race, gender, and teacher testing: How informative a tool is teacher liTyler, L. (2011). censure testing? American Education- Petchauer, E. (2014). Toward increasing teacher diversity: al Research Journal, 47(1), 218–251. “Slaying ghosts in the room”: IdenTargeting support and intervention tity contingencies, teacher licensure for teacher licensure candidates. testing events, and African American Hartlep, N. D., & Antrop-González, R. Retrieved from https://www.ets. preservice teachers. Teachers College (2018, February 27). org/Media/Research/pdf/ETSRecord, 116(7), 1–40. Cultural biases persist in national NEA-2011-01.pdf licensure exam for teachers. Diverse: Issues in Higher Education. Retrieved from http://diverseeducation.com/ article/111104/ 16 Appendix A Certificate of Perfect Attendance 17 Appendix B Sample Feedback Form Session 2 Feedback Form Please provide feedback on the second session, which was held on January 19, 2018 at Metropolitan State University (Midway Campus). This information will help us better serve you and better prepare for Session 3, which will be held on January 26, 2018 at 6:30p.m.-8:30p.m. at Metropolitan State University’s Midway Campus in Room 107. * Required Email address* What is your first and last name? Where do you attend?* Mark only one oval. Augsburg University Metropolitan State University Concordia University St.Paul Hamline University St. Catherine University University of St.Thomas Bethel University Macalaster College Saint Paul College Normandale Community College Minneapolis Community & Technical College I don’t attend any of these What teacher licensure area are you seeking?* What is your gender?* Mark only one oval. Female Male Non-binary/third gender Is English your native language?* Prefer not to say Prefer to self-describe Yes No Mark only one oval. What is your race/ethnicity? Mark ALL that apply.* Check all that apply. American Indian or Aslaskan Native Asian Black or African American Hispanic or Latino Native Hawaiian and other Pacific Islander White non-Hispanic Other: Why did you attend the session on January 19, 2018 at Metropolitan State University? (Check all that apply)* Check all that apply. 18 Because I have registered for the essential skills exam in math Because I plan to register for the essential skills exam in math Because I need clarification on the math content Other: Continued on page 21 Appendix B (cont.) Sample Feedback Form Continued from 20 Please select the appropriate answer for each question below:* Check all that apply. Disagree Strongly Disagree Undecided Agree Strongly Agree Dr. Brooks was knowledgeable of the subject material Dr. Brooks demonstrated effective communication Dr. Brooks was sensitive to different learning styles and cultural differences Dr. Brooks demonstrated good time management and organizational skills Dr. Brooks gave resource tips Dr. Brooks aided me in understanding math concepts Dr. Brooks explained charts, graphs, and calculations Dr. Brooks promoted group discussion Dr. Brooks explained the material in ways that helped me understand it Dr. Brooks was prepared for the session Session 2 helped me prepare for the essential skills exam in math* Mark only one oval. 1 2 3 4 5 6 7 I plan to attend the third session on January 26, 2018 from 6:30p.m.-8:30p.m* Mark only one oval. Yes No Undecided 19 Appendix C Sharing Success Via Social Media 20