Test Review: TOEFL, IELTS, ACT Compass ESL Test Kathleen Hamel Colorado State University
Test Review: TOEFL, IELTS, ACT Compass ESL Test Kathleen Hamel Colorado State University
Test Review: TOEFL, IELTS, ACT Compass ESL Test Kathleen Hamel Colorado State University
TEST REVIEW
Introduction
More and more students from around the world want to study at international institutions,
many of which require knowledge and application of the English language. These institutions
then need to be able to assess students abilities to perform at the level necessary to excel in their
programs. Assessments, like tests or other alternative types, provide this information to
administration, teachers and students of what has been learned and what needs to be learned
(Miller, Linn, & Gronlund, 2009). In order to be admitted into a university, the most appropriate
assessment is through various English proficiency exams. The purpose of placement exams is to
determine whether students possess the necessary language to succeed at a university (Stoynoff
& Chapelle, 2005).
The university setting is of particular interest to me as a future ESL teacher. My shortterm goals are to be able to teach students who are directly coming from various international
countries who need to improve their English skills before moving onto their field-specific study.
Therefore, for this paper, I will be reviewing the Test of English as a Foreign Language Internet
Based Test, the International English Language Testing System and the ACT Compass ESL
Placement test. The reason why I decided on these three tests is because they all claim to have a
similar purpose, that is, to be used to make decisions about non-native speakers of English
(NNSE) who want to be admitted to English-medium universities. Through this review, I hope to
gain an understanding of how these tests are structured, the reliability and validity of these tests
and to understand the concerns, apprehensions, and questions that students might have about
these exams.
TEST REVIEW
The purpose of the TOEFL iBT test is to assess NNSE who plan
on using English in an academic setting. According to Jamieson,
Jones, Kirsch, Mosenthal and Taylor, 2000, The test will measure
examinees English-language proficiency in situations and tasks
reflective of university life (as cited in TOEFL iBT test
framework, 2010, p. 2).
Test Structure
TEST REVIEW
The TOEFL iBT is scaled on a range from 0-120, with 120 being
the highest. Each testing section (Reading, Listening, Speaking and
Writing) is scaled from 0-30.
For the Reading section, there are two types of multiple-choice
questions: one that has a single correction answer and one that has
more than one answer, which lends itself to partial-credit scoring.
The Writing section is the same, with having more single criterion
questions.
The Speaking and Writing sections are scored a bit differently
with rubrics. Both rubrics are holistic and a 4-point rubric is used for
the Speaking section, while a 5-point rubric is used for the Writing
section. In order to maintain reliability, raters must have appropriate
qualifications and must pass a calibration test before they begin
rating.
(TOEFL iBT test framework, 2010)
Statistical Distribution
of the Scores for
Normed Group
TEST REVIEW
Reading
Listening
Speaking
Writing
Total
20.4
15.8
22.4
23.4
82.0
0.89
1.6
1.5
2.3
4.3
(Results of standard, 2005)
Standard Error of
Measurement
Evidence for
Reliability
For the TOEFL iBT test, the validation of their test has been
ongoing since the creation of their test in the 1970s. The ways in
which ETS claims the TOEFL is valid is that: the content is relevant
to the types of tasks and tests that test takers will encounter in a
university setting, the performance on the test is related to one's
academic language proficiency and the test results are used and
interpreted appropriately. (Validity evidence, 2011)
This is confirmed by Chapelle, Enright and Jamieson (2008) in
their book Building a Validity Argument for the Test of English as a
Foreign Language. Chapelle demonstrates that the evaluation of the
tasks reflect the respective targeted language abilities and that the
TEST REVIEW
The IELTS Academic test is for test takers who want to study at a
university where English is primarily used or for those who want to
join a professional organization. According to the British Council, the
IELTS Academic test measures English language proficiency
needed for an academic, higher learning environment (Understand
the IELTS).
TEST REVIEW
Test Structure
Scoring of Test
TEST REVIEW
Mean
6.0
6.1
-
Std. Dev.
1.2
1.3
-
For both Speaking and Writing, the mean and standard deviation
were not reported.
(Test performance, 2014)
Standard Error of
Measurement
Evidence for
Reliability
TEST REVIEW
One way that IELTS ensured validity of their tests was through
research comparing students relationship with their GPA and the
IELTS sections of the test. Overall there was mixed results to the
relationships of each. For example, there was a strong correlation
between Listening and Reading scores and students GPAs, while
there was not a correlation with their Speaking and Writing scores
(Humphreys, Haugh, Fenton-Smith, Lobo, Michael & Walkinshaw,
2012). This means that only students Listening and Reading scores
might predict their grades in those subsequent areas.
An additional study, which looked at students IELTS scores and
their acceptance and progress in postgraduate universities, noted that
opinions about the extent to which English proficiency test scores
could be relied upon to indicate that an individual student was
capable of successful study ... were contentious (Lloyd-Jones,
Neame & Medaney, 2011, p. 158). This once again shows that the
relationship between students scoring on the IELTS and their success
in academic settings are not significantly related.
TEST REVIEW
The ACT ESL Compass test (ACT ESL) contains three untimed
English proficiency tests: Grammar/Usage, Reading and Listening,
with an optional e-Write (Essay) option. The purpose of the ACT
ESL test is to place students in the appropriate IEP classes or
university courses according to their skill level. This test assesses a
students ability to understand Standard American English. It is a
computer adaptive college placement test; it is adaptive because
questions are tailored to test takers depending on their responses to
previous questions.
(About ACT Compass)
Test Structure
TEST REVIEW
10
hardest, meaning that each tasks increases with the rate of speech,
vocabulary, dictation and use of idiomatic and metaphorical
language. The purpose of this is to allow lower-proficiency level
students to be able understand dialogue that students might encounter
in face-to-face situations because a native speaker is likely to modify
their speech. Then as the tasks get more difficult, test takers are
assessed on their ability in the likelihood of encountering dialogues
that are to mimic real-life conversations.
The Reading section of the test is also structured from easy to hard
tasks. The easier tasks include common knowledge topics without the
use of idioms or metaphors, while the more difficult tasks include
academic or, possibly, unfamiliar contexts. Students read passages
about these varying topics which are then assessed by multiple-choice
items. Background knowledge should not be necessary in order to
understand the passages, and they are authentic materials appropriate
for ESL learners. Test takers are assessed on the ability to refer and
reason.
The Grammar/Usage section of the text assesses a students ability
to recognize and manipulate English in two main categories: sentence
elements and sentence structure and syntax. All items are in a
multiple-choice format and assess their English grammar and usage
within context. Examples of sentence elements include different verb
tenses and aspects, using subjects/objects correctly and writing
conventions. Examples of sentence structure and syntax include: verb
agreement, word order and use of various types of clauses.
The Essay section of the test is the only component that is timed
and test takers have 30 minutes to complete this section. The writing
sample asks task takers to write about a specific issue, which this
must take a position on, support with examples in an organized
manner with correct grammar and mechanics. This issue is a common
aspect of everyday life where test takers do not need background or
specialized knowledge in order to successfully respond to the prompt.
(About ACT Compass)
Scoring of Test
Each of the four areas of this test is scored into five different
levels, which are Pre-Level 1, Level 1, Level 2, Level 3 and Level 4.
Each students report includes both a proficiency descriptor and the
Listening, Reading and Grammar/Usage tests are given a numeric
score, on a scale from 1-99.
The Essay section of the test is scored on a 6-point analytic
scoring rubric. The purpose of using an analytic rubric is to provide
more specific and, potentially, diagnostic information regarding
specific writing strengths and weaknesses (Answers to
frequently). The analytic scores focus on: development, focus,
organization, language use and mechanics and are weighted 35%,
10%, 15%, 35% and 5%, respectively. The analytic scores are then
TEST REVIEW
11
Std. Dev.
16.7
18.3
16.7
(Internet Version)
Standard Error of
Measurement
The conditional SEM given by ACT was provided for the three
lengths of the test, standard, extended and maximum, given in 5-point
intervals for Reading, Listening and Grammar. The results below are
the averaged SEM for the maximum length of the test.
Reading
Listening
Grammar/Usage
Scale
25-90
25-90
25-90
SEM
6.2
6.9
5.4
(Internet Version)
Evidence for
Reliability
TEST REVIEW
12
TEST REVIEW
13
being assessed with materials that they could encounter in the real-world. Despite this, IELTS
does not provide researchers with all data related to their standard error of measurement which
makes it difficult to provide an accurate overview of the test itself.
Finally, the ACT ESL test does not provide an overview of the students capabilities for
universities. What I mean by this is they dont require assessment of students speaking abilities,
and the writing section is even optional. Without these two critical skills, which are mainly
production based, how would an admissions committee truly be aware of the students ability to
complete tasks in the classroom. Another shortcoming of the ACT ESL test is that its only
multiple choice, so once again it does not require much of the student in terms of production.
However, the ACT ESL test is the cheapest and least time consuming of the three.
My advice for students who plan on taking these tests is to look at the university in which
you wish to attend to find out what tests they accept. If these universities do not have a
preference then I recommend the TOEFL iBT test because it has the most integrative skills and is
highly reliable and valid.
TEST REVIEW
14
References
ACT Educational Services. (n.d.). Internet version reference manual. Retrieved from
https://www.act.org/content/dam/act/unsecured/documents/CompassReferenceManual.pdf
ACT Educational Services. (n.d.). Answers to frequently asked questions about COMPASS
e-Write & ESL e-Write. Retrieved from
http://www.act.org/content/dam/act/unsecured/documents/Compass-ewritefaq.pdf
Bachman, L., and Palmer, A. (2010). Language assessment in practice. New York, NY: Oxford
University Press.
Chapelle, C. A., Enright, M. K., and Jamieson, J. M. (2008). Building a validity argument of the
test of English as a foreign language. New York, NY: Routledge.
Educational Testing Service. (n.d.) About the TOEFL iBT test. Retrieved from
https://www.ets.org/toefl/ibt/about
Humphreys, P., Haugh, M.m Fenton-Smith, B., Lobo, A., Michael, R. and Walkinshaw, I.
(2012).
Tracking international students English proficiency over the first semester of
undergraduate study. IELTS Research Reports Online Series (1). Retrieved from
http://gallery.mailchimp.com/d0fe9bcdc8ba233b66e1e0b95/files/Humphreys2012_ORR.
pdf
IELTS Partners. (n.d.). Understand how to calculate your IELTS scores. Retrieved from
http://takeielts.britishcouncil.org/find-out-about-results/understand-your-ielts-scores
IELTS Partners. (n.d.) Understand the IELTS test format. Retrieved from
http://takeielts.britishcouncil.org/prepare-test/understand-test-format
IELTS Partners. (n.d.). Understand the writing test. Retrieved from
http://takeielts.britishcouncil.org/prepare-test/understand-test-format/writing-test
IELTS Partners. (2013). IELTS assessment criteria. Retrieved from
http://takeielts.britishcouncil.org/find-out-about-results/ielts-assessment-criteria
IELTS Partners. (2014). Test performance 2014. Retrieved from
https://www.ielts.org/teaching-and-research/test-performance-2014
Lloyd-Jones, G., Neame, C. and Medaney, S. (2011). A multiple case study of the relationship
between the indicators of students English language competence on entry and students
TEST REVIEW
15