A Critical Look at the TOEFL

Medical Education, 2001

Asian-Pacific Journal of Second and Foreign Language Education, 2017

Intensive English programs (IEPs) strive to make certain that international students have sufficient levels of speaking ability, which is typically assessed through a combination of different tasks. One drawback of including multiple tasks is that the development, administration, and scoring might not be practical. Therefore, it is important to investigate how well the tasks account for examinees' speaking ability, as using fewer tasks could help in minimizing resources. Using quantitative methods of analysis, this study evaluates how well four types of speaking tasks on proficiency and achievement tests account for students' speaking ability in an IEP. The findings indicate that several tasks uniquely contribute to the speaking construct. This study has implications for the importance of balancing practicality with construct representativeness and presents a model of how IEPs might approach this issue.

Journal of English for Academic Purposes, 2006

As the number of non-native speakers of English studying at universities in the United Kingdom (UK) has grown, so has the need to provide pre-sessional English for Academic Purposes (EAP) courses. One of the challenges facing the organisers of such courses is deciding how to determine whether students have completed them successfully and whether they are ready to enter their academic departments. This paper reviews research into assessment on pre-sessional and other EAP support courses, and summarises the results of a survey of practice in a number of universities in the UK and other countries. It describes how assessment on pre-sessional courses has evolved at one British institution (Lancaster University), and outlines some of the problems that have led to changes in recent years. Explanations will be given of the methods used to pilot and refine a new assessment procedure and of the attempts that have been made to investigate its validity.

This dissertation study aimed, first and foremost, to address the current scarcity of assessments of productive and contextualized academic vocabulary in English, and thus contribute to the Applied Linguistics and Language Assessment field, by developing what seems to be the first IRT-based computer-adaptive test of productive and contextualized breadth of academic vocabulary in English (henceforth, CAT-PAV), which will be made freely available online to any researchers, ESL instructors, or other interested parties who believe the test can potentially be useful for their specific scenarios. A second aim of the present study was to validate the interpretation of scores from the CAT-PAV and their use for two specific scenarios at Iowa State University (ISU): as a diagnostic tool, or as an ESL placement aid for Iowa State’s English Placement Test (EPT). The development of the CAT-PAV was informed by evidence-centered design (Mislevy & Riconscente, 2006), a test development framework that allows for a close alignment of test development decisions with interpretations of precisely what is being measured. The validation of the intended interpretation and specific uses of the CAT-PAV presented here, in turn, was informed by an argument-based approach to validation (Chapelle, Enright, Jamieson, 2008; Kane, 2013), which specifies through an interpretive argument a chain of inferences for the interpretation and uses of test scores, along with a detailed and explicit definition of what kind of support is required for each of those inferences to be warranted in a subsequent validity argument. Five inferences were employed in the validation of the interpretation and uses of CAT-PAV scores: Domain Description, Evaluation, Generalization, Explanation, and finally, Utilization. Analysis of the extent to which each of the five inferences could be supported was based primarily on the responses of over 900 test takers (which included ESL instructors) to test items and pre- and post-test questionnaires, collected over a period of four months. Results indicated that the first four inferences in the validity argument for CAT-PAV score interpretation and use were fully warranted, whereas the Utilization inference could only be partially warranted at this time. The task type utilized in the CAT-PAV was shown to be present in materials employed in ESL classes at ISU, and the great majority of ESL instructors polled indicated that the knowledge and abilities required to achieve a good score in the CAT-PAV are also necessary when using academic vocabulary in ESL classes at ISU (support for Domain Description inference). CAT-PAV items were shown to be increasingly monotonic, essentially unidimensional, and all versions of the test showed reliabilities above 0.91 (support for Evaluation inference), while alternate-form reliability for the test was also high (support for Generalization inference). Finally, correlations between CAT-PAV scores and scores on other tests requiring substantial knowledge of academic vocabulary, such as the TOEFL iBT or Laufer and Nation’s (1999) Vocabulary Levels Test Academic were positive (support for Explanation inference), and the majority of ESL test takers indicated that taking the test had a positive effect on their academic English, while the majority of ESL instructors believed that use of the CAT-PAV as a diagnostic or placement-aid tool at Iowa State could positively impact ESL learners and their development of academic English (partial support for the Utilization inference). Limitations of the present study, pertaining to item development, data collection, and test administration will be discussed. Validation issues that require further investigation will also be discussed, and suggestions for future research into the CAT-PAV will be provided, with a focus on possible ways to quickly expand the current item bank for the test.

ETS Research Report Series, 2004

Language in India , 2019

This study attempts to investigate the relationship between students' weakness in listening skill and its effect on their performance in the TOEFL IBT integrated task of writing. It also investigates the balance among the students' four language skills. The participants of the study are senior students of Rada'a Education College of Albaydaa University, Yemen. Thirty learners are systematically chosen. They are given an integrated task of writing taken from TOEFL IBT and an independent writing task. The analysis of the data revealed that the scores obtained in the writing task based on listening were very low, if compared with the scores obtained in the independent writing task. It showed that the lower performance of the students in the integrated task is not also attributed to students' reading or grammatical and lexical weakness, rather, it is due to the students' listening problems since most of the words given in the listening passage are familiar to the students. The findings revealed that the students' poor listening comprehension is merely a reflection of what they were taught. They have almost received no training in listening if compared with other courses of reading, writing and grammar.

Journal of Language and Translation, 2010

2006

KOTESOL Proceedings 2005, 2006

This paper is based on action research aimed at solving the problem of engineering students' insufficient vocabulary size for academic reading. In the study, vocabulary learning was integrated into an existing reading course, and classroom concordancing was applied with the objective of comparing its learning effects on vocabulary learning with those of a conventional method. In addition, the study aimed at exploring students' learning processes and attitudes while dealing with classroom concordancing. The study was conducted with two intact groups of engineering students in one academic semester. In the preparatory stage, a purpose-built corpus was compiled from academic texts in engineering fields. Then target words were selected from high frequency words in the corpus, which were also words in the GSL or AWL lists. This target word list was used to design all lessons, materials, activities, and tests. During the study, the experimental group was trained through paper-based and hands-on activities to deal with concordance information in the corpus, whereas the comparison group was taught through reading and vocabulary exercises. This paper describes the background of the study, general aspects of “classroom concordancing,” research methodology, and findings from the study.