Academia.edu no longer supports Internet Explorer.
To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to upgrade your browser.
2008 •
This study aims (a) to find out the students’ and the instructors’ perceptions of the Compulsory English Language Course exams used to assess language performance at Çanakkale Onsekiz Mart University (COMU). It further aims (b) to determine what other objective test techniques can also be used in these exams in addition to the multiple-choice test technique by taking all the students’ and the instructors’ opinions into consideration. Quantitative research methodology was used in this descriptive study. In the light of the literature; in order to achieve the aims stated above, two questionnaires were designed by the researcher and administered to 367 students and 33 instructors. After analyzing the internal consistency of the items in the questionnaires, the researcher found acceptable Alpha reliability values both for the students’ questionnaire and for the instructors’ questionnaire. Data from the students and instructors were collected by using these questionnaires. Instructors’ questionnaire was administered to the instructors who had worked or were still working as the instructors of ‘Compulsory English Language Course’ at COMU. The students who involved in the study were all in their second years at the university and they all had the “Compulsory English Language Course” the year before the study was conducted. The data obtained through the questionnaires were analyzed via Descriptive Statistics, One-way ANOVA, Independent Samples T-Test, Cronbach Alpha Reliability Test and Nonparametric Kruskal-Wallis Test by using SPSS (Statistical Package for Social Sciences) 13.0 for Windows. The findings of the descriptive statistics showed that students expect the instructors to attach more importance to the activities improving their speaking, listening and writing skills. Furthermore, the results displayed that nearly 73 percent of the instructors prefer the exams to be prepared by a testing office while more than half of the students expect them to be prepared by the instructor of the course. The results also revealed that both the students and the instructors believed it was necessary to use other test techniques in addition to the multiple-choice test technique commonly used in the exams. According to the results of the One-way ANOVA, the more successful the students are, the more satisfied they are with the exams’ different characteristics. As for the instructors, Nonparametric Kruskal-Wallis Test results indicated that there occurred no significant differences between instructors’ educational background and the objective test techniques that they use in their classrooms. Additionally, it was found out there were no significant differences between instructors’ educational background and their ideas on the objective test techniques that can be used in the exams. However, the more experienced the instructors are, the more efficient they find the exams prepared by the testing office. Another important finding was that although their order of preferring objective test techniques slightly differs, the first eight test techniques that the students and instructors preferred in the exams were completely same. The study concludes that both the students and the instructors have some doubts about the efficiency of the testing office’s current practices. Therefore, for more efficient exams, test constructors can include the eight objective test techniques [(1) multiple-choice questions, (2) matching, (3) ordering tasks, (4) completion, (5) true-false questions, (6) short-answer questions, (7) error correction and (8) word changing], which were commonly preferred by the instructors and the students, into the Compulsory English Language Course Exams. In addition to the centrally administered achievement tests of this course, instructors should use teacher-made achievement tests and take the scores that students get from these tests into consideration while assessing their learners’ language performance. Moreover, having a testing office with test constructors specialized just at testing will be a good idea for preparing better and more efficient tests.
The International English Language Test System (IELTS) is one of the most reputable English tests that is used to assess the language proficiency of those who intend to study or work in an English speaking context. It is one of the most large-scale proficiency tests which affects the lives of many students, as well as immigrants as the results of the test, are used for making critical decisions about the test takers. Moreover, the process of designing a good test requires a clear understanding of both validity and reliability of the test format. Therefore, in the current paper, we try to offer a descriptive review of the IELTS test by concentrating on various issues such as reliability, validity and washback.
Owing to their scope, and decisiveness, Ph. D. program entrance exams (PPEE) ought to demonstrate acceptable reliability and validity. The current study aims to examine the reliability and validity of the new Teaching English as a Foreign Language (TEFL) PPEE from the perspective of both university professors and Ph. D. students. To this end, in-depth unstructured interviews were conducted with ten experienced TEFL university professors from four different Iranian state universities along with ten Ph. D. students who sat both the new and old PPEEs. A detailed content analysis of the data suggested that the new exam was assumed to establish acceptable reliability through standardization and consistency in administration and scoring procedures. Conversely and more importantly, the new exam was perceived to demonstrate defective face, content, predictive, and construct validities. This study further discusses the significance and implications of the findings in the context of Iranian TEFL higher education.
In Central Europe, education has undergone considerable upheaval since the change of political systems at the end of the 1980s. One of the most radical is the revision and reform of school-leaving examination systems. From school- and teacher-based subjective assessments conducted in a largely ritualistic fashion, with no explicit criteria for evaluation, standards to refer to, or quality control procedures of any sort, systems are now being developed that introduce modern ideas of testing and quality assurance, external quality control procedures and alignment, in theory, to common European standards. In this context, it is well known that teachers' grades in final examinations vary enormously, from school to school, region to region and even within the same school. Indeed, a growing realisation that this variation is unacceptable in the New Europe is one of the driving forces for examination reform. Several governments have decreed that school-leaving examinations should be de...
During the meeting, various experts in the field of foreign language assessment presented state of the art research on the computer based assessment of speaking skills. The different scientific and technical issues discussed provide a good overview of current developments in the assessment of speaking proficiency in a foreign language in the context of large scale assessments. General practical benefits of CBA include the possibility of testing many examinees within a short time span and the assurance that standardized testing procedures are followed. Technical advances have made it possible to develop fully automated assessment instruments and we are now able to deliver computerized tests of oral proficiency that begin to emulate authentic interactions between human beings. The first oral language proficiency assessments relied solely on the interaction between humans established during the Oral Proficiency Interview (OPI). The SOPI, (Simulated Oral Proficiency Interview), introduced a simulated format whereby speech is elicited by a tape recorder, but rated by experts. Computer-assisted foreign language assessments, on the other hand, are of two sorts. One version requires no human intervention for test delivery, but the candidate’s performance is assessed by expert raters. On the other hand, fully automated versions are both delivered and scored by machine and require no human intervention. The Computerized Oral Proficiency Instrument (COPI), like the SOPI also produced by the Centre for Applied Linguistics (CAL), represents the former development by reliably capturing oral language ability, to be assessed by humans, whereas the latter, fully automated version, is represented by Versant (from Pearson Knowledge Technologies), which is scored entirely by computer. The testing of foreign language speaking ability has evolved over time according to specific guidelines that define levels of language proficiency. In Europe, the Common European Framework (CEFR) sets the guidelines against which proficiency levels are rated and reported, while in the US the American Council on the Teaching of Foreign Languages (ACTFL) provides proficiency level definitions. Importantly, despite differences in the two speaking proficiency frameworks, the work developed by CAL according to the ACTFL guidelines has been influential in the development of current assessment instruments in Europe. Thus, the work carried out at CAL on the development of OPIs and SOPIs is frequently referred to by other authors of this report. From the paper by Dorry Kenyon and Meg Malone the reader can follow the developments in test design from the OPI to the COPI and get a sense of the challenges involved in capturing oral language ability using computer-based approaches. Next, the paper by Evelina Galaczi argues that a “fitness for purpose” criterion should guide the adoption of test formats. Whereas computerized formats may not render the full range of speaking abilities in interactional contexts, they may offer a valid snapshot of linguistic ability. The paper by Ildikó Csépes describes how a paired exam mode following a SOPI format can be implemented without compromising test validity. Similarly, Thomas Eckes describes the design of a test of German as a Foreign Language as a particular implementation of the SOPI and discusses how scoring validity can be accomplished. Alistair Van Moere gives an account of how the fully automated Versant tests provide a set of performance-based measures linked to the CEFR. Finally, Carsten Roever challenges our thinking about how to capture pragmatic language ability using automated computer-based assessment and offers some examples for the contextualization of test items that eelicit pragmatic responses. As the papers included in this report clearly show, the field of foreign language assessment, and in particular the assessment of speaking ability, has come a long way and it is now possible to implement different assessment modes using computer-based technologies. Furthermore, regardless of whether a semi-automated or a fully automated system is implemented, we can be confident that reliable tests are now available that can offer a valid measure of a learner’s oral proficiency level. However, it is also clear that capturing the full range of the features of human spoken interaction is no easy task and future research efforts will likely focus on capturing complex linguistic interactions played out in a virtual world. Fortunately, it is already possible to create assessments that are fully delivered and scored by computers. In the context of large scale surveys, CBA clearly has the potential to facilitate implementation by reducing testing time and costs without compromising validity.
The chapter in the book 'Teaching and Learning Language Current Trend and Practices' is aimed to propose the much needed rating scale to assess English speaking proficiency of students graduating in 124 universities in Pakistan, keeping in mind its increasing utility in the domains of education and profession in the Outer Circle and the singular emphasis laid down recently by the Higher Education Commission of Pakistan to improve learner’s communicative skills, which in turn has encouraged the universities to introduce various compulsory English language proficiency courses. In fact, this study is motivated by the fact that the absence of any standard scale to rate the extended -response speaking skill in Pakistan seems to have created both difficulties and discrepancies in assessment. This preliminary work is divided into four phases: studying available rating scales and their practices in and outside Pakistan, examining relevant levels of analytic marking categories, drafting a rating scale with expert advice of eight qualified ESL teachers, and running the scale on forty-four students with eight raters. To achieve fairness, Multifaceted Rasch measurement, using FACETS (Linacre, 2006), was employed to analyze three facets: examinees, raters, and rating categories. The results suggest that the rating scale can be a proper indicator of student’s English speaking abilities with high reliability, however further investigations are needed to validate and improve the scale.
Language Teaching
Language testing and assessment (Part 2)2002 •
Educational Assessment
Voices From Test-Takers: Further Evidence for Language Assessment Validation and Use2011 •
Language Teaching
Language testing and assessment (Part I)2002 •
2012 •
Dictionary Contribution
An A to Z of second language assessment: How language teachers understand assessment2018 •
Applying the socio-cognitive framework to the BioMedical Admissions Test (BMAT) Insights from language assessment
Conclusions and Recommendations2017 •
Sollsintec Proceeding 2013
Juxtaposing the School Teachers’ Concept of Testing with Their Test Construction Practices2013 •
Journal for Language Teaching
[2014] Writing as construct in the Grade 12 Home Language curriculum and examination2014 •
efdergi.hacettepe.edu.tr
A STUDY ON THE ENGLISH LANGUAGE TEACHERS'PREPARATION OF TESTSLanguage Assessment Quarterly, 6, 151-167
Development of a practical speaking test with a positive impact on learning using a story retelling technique2009 •
2013 •
Vision: Journal for Language and Foreign Language Learning
Designing a Midterm Reading Test for Junior High School Students in Semarang: A Practical Experience of a Master Student in TESOL