Vocabulary Assessment

Within the communicative approach, often the assumption has been that with the right exposure, students will simply “pick up” the vocabulary required for learning and using English, and thus there is no need to focus on or teach it. Yet, as many teachers can attest, this is frequently not the case, and there have been recent efforts to reemphasize vocabulary learning and teaching in both research and practice. This article surveys the literature on word lists for vocabulary teaching in English as a second/foreign language (ESL/EFL), especially for adults, briefly summarizing their potential for learners and teachers in learning and teaching English vocabulary. After discussing general and academic word lists, it introduces contributions from recent corpus research resulting in 2 lists of English formulaic expressions and 8 subject-specific English word lists, in fields varying from agriculture, business, and engineering to medicine and theology. Finally, it offers suggestions for their potential in vocabulary teaching.

In this replication of a study by Barcroft (2009), Chinese University Learners were divided into four groups. All groups read the same short reading passage in English. Two of the four groups were instructed to focus their attention on ten L1 glossed words in the text. Two of the four groups were also instructed to provide synonyms for the target words. The study therefore investigated two pairs of conditions: incidental vs intentional learning and semantic elaboration vs no semantic elaboration. Results ran counter to those observed by Barcroft (2009) and showed slight significant positive effects for the intentional condition but no effect for semantic elaboration. Barcoft's proposed Type of Processing Resource Allocation (TOPRA) model of cognitive processing is not supported. Critical appraisal of the experimental approach in the light of previous studies involving Chinese learners reveals some shortcomings in the design of the experiment for this particular group of subjects.

This systematic review provides a comprehensive analysis of the empirical literature on the effectiveness of vocabulary instructional methods for English language learners (ELLs) in prekindergarten through 6th grade settings. Thus, a central goal of this study is to appraise the available research on vocabulary learning and assess intervention studies’ levels of evidence for effectiveness in order to make a critical assessment of the variables that are currently represented in the field. Specifically, the research questions guiding this systematic review are: (1) How do intervention studies developing L2 word-meaning acquisition with ELLs in pre-kindergarten through 6th grade settings vary in their methodological characteristics (i.e., research design, learners, instructional design and outcome)? (a) What are the studies’ levels of evidence for effectiveness based on their design and outcome characteristics? and (2) What is the impact of studies that meet requisite evidence standards on learners’ L2 word-meaning acquisition?
A total of 18 studies meeting the inclusion criteria were identified. Independent coding of 20% of the articles (n=4) was completed, the results were cross-checked against the original codes, and further discrepancies in coding were resolved through discussion between the two coders. The final criteria were subsequently reapplied by the author to all studies in the review. Coding categories for each study were organized by four main characteristics: (1) design characteristics (i.e., random assignment, baseline equivalence, comparison group type); (2) learner characteristics (i.e., L1/L2 proficiency, SES, age and grade level); (3) treatment characteristics (i.e., focus of instruction, level of L1 support, word tier targeted, total instructional time, training, explicitness of instruction, contextual/definitional support, group size, and level of word exposure) and (4) outcome characteristics (i.e., outcome objective, type of measure, type of ability measured, modality, word tier assessed, and type of knowledge). Based on the studies’ design and outcome characteristics, their level of evidence for effectiveness was also examined. Subsequent analyses revealed that a large proportion of studies underreported information and did not meet evidence standards for effectiveness. Additionally, studies with available effect sizes for measures of breadth and depth of vocabulary knowledge revealed inconclusive evidence about the best instructional practices for ELLs due to the large variations among a limited number of studies in their instructional approach, learners, and outcome characteristics.
An analysis of the merits and limitations of the reviewed studies reveals a number of methodological problems that threaten the validity and credibility of vocabulary research. These findings highlight the need for increased research in this field of study, particularly in revisiting existing practices to add to current understandings relating to vocabulary practices with ELLs. Additionally, this study points to specific issues in current research and provides recommendations for researchers seeking to address the significant knowledge gap in research that is centered on ELL vocabulary learning.

The current study investigated the extent to which L2 learners’ productive vocabulary knowledge could predict multiple dimensions of spontaneous speech production. A total of 39 EFL participants with varying L2 proficiency levels first completed a productive vocabulary knowledge task (Lex30). Their spontaneous speech, elicited via a series of picture description task, was then assessed for comprehensibility (i.e., ease of understanding), accentedness (i.e., linguistic nativelikeness), and fluency (i.e., speech rate). The findings showed that the productive vocabulary scores significantly correlated with L2 fluency, but not with comprehensibility or accentedness. Such results might indicate that more proficient L2 learners, as indicated by their productive vocabulary scores, might be able to speak spontaneously without too many pauses and repetitions, and at a faster tempo. Finally, future research directions will be discussed with a focus on the relationships between vocabulary knowledge and speaking.

The current study investigates the extent to which receptive vocabulary size test scores can predict second language (L2) speaking ability. Forty-six international students with an advanced level of L2 proficiency completed a receptive vocabulary task (Yes/No test; Meara & Miralpeix, 2017) and a spontaneous speaking task (oral picture narrative). Elicited speech samples were submitted to expert rating based on speakers' vocabulary features as well as lexical sophistication measures. Results indicate that vocabulary size was significantly associated with vocabulary rating. However, learners with large vocabulary sizes did not necessarily produce lexically sophisticated L2 words during speech. A closer examination of the data reveals complexities regarding the relationship between vocabulary knowledge and speaking. Based on these findings, we explore implications for L2 vocabulary assessment in classroom teaching contexts and provide important suggestions for future research on the vocabulary-and-speaking link.

Word Associates Format (WAF) tests are often used to measure second language learners' vocabulary depth with a focus on their network knowledge. Yet, there were often many variations in the specific forms of the tests and the ways they were used, which tended to have an impact on learners' response behaviors and, more importantly, the psychometric properties of the tests. This paper reviews the general practices, key issues, and research findings that pertain to WAF tests in four major areas, including the design features of WAF tests, conditions for test administration, scoring methods, and test-taker characteristics. In each area, a set of variables is identified and described with relevant research findings also presented and discussed. Around eight topics, the General Discussion section provides some suggestions and directions for the development of WAF tests and the use of them as research tools in the future. This paper is hoped to help researchers become better aware that the results generated by a WAF test may vary depending on what specific design the test has, how it is administered and scored, and who the learners are, and consequently, make better decisions in their research that involves a WAF test.

Vocabulary learning is an incremental process. Vocabulary knowledge, especially for second-language learners, may develop across a lifetime. For example, an advanced student in Uzbekistan told us that he recently learned that popular and famous have positive connotations, while notorious refers to being famous for something negative. An international graduate student in the United States complained that although she recognizes many words in English, especially when she reads books and journal articles, watches videos, or listens to audio materials, she cannot easily retrieve words effectively from her memory while speaking or writing. Because of this challenge, it takes her a lot of time to prepare for academic presentations or write academic papers for her graduate-level courses. Learning the meaning alone of new words does not guarantee word mastery-although it is an important first step.

http://hdl.handle.net/11380/1174890

The FIRB project “Adaptive Message-Learning” (2009-2014) has had the primary scientific goal of developing a didactic model of individualization which is the adaptation of the «learning message» to the «verbal skills» of the recipient (Vertecchi, 2010). According to Chris Whetton (2011, 32) the general construct underlying the project can be formulated in a very simple way. Each student has a personal vocabulary, which can be measured. This information can be used «to construct individualized learning materials, such that the learning for that person is supported and enhanced». For the group as a whole, in terms of measurement, «this should lead to a reduction in variance of scores and an overall increase in mean of their scores». Within the am-learning model, assessment activities have different times, contents and objectives, which Whetton (2011, 31) leads back to two basic elements: «the initial measurement of learners’ ability to understand, and the final measurement of the outcomes of the learning, measured at the conclusion of the course or module». For the first part, the project has developed a measure of the personal lexicon that uses the cloze test to assess the availability of relevant vocabulary. The initial and continuous assessment of the lexical competence of students, entrusted to the software LexMeter (Agrusti, 2010), is a core element in the am-learning model. It requires on the one hand the lexical analysis of the input to which students are exposed in a given context of formal learning, on the other hand the collection of measures on the reader in relation to the vocabulary of the texts and to the specific language of a subject area. In more detail, software-generated multiple-choice rational-deletion cloze-tests are used to measure the extent to which learners can presently handle relevant vocabulary in discipline specific texts, in order to estimate how effectively they could cope with standard course materials and then use this information for adapting the wording of the study text to the individual student. The validation of the diagnostic assessment of am-learning required specific attention. In am-learning, quantitative analysis of words in a corpus of LSP texts is used for identifying the relevant vocabulary, that is to define by statistical criteria the vocabulary list to take into exam (which words may be deleted from a given text). The assessment of the individual vocabulary, then the automated construction and evaluation of the tests, are based on the frequency of occurrences of the content words in a specific language. In a nutshell, the software: • produces a specific frequency word list through the lexical-statistical analysis of a representative sample of the language of the subject area; • given a text (analogue to the materials of the course), constructs, administers and corrects rational-deletion cloze with multiple choice answers. The function words are excluded from consideration. Once you determine which grammatical classes of words are eligible for deletion (in the case of experiments conducted: noun and adjective), the hidden words must fall within a range of frequencies determined by the author of the test. It provides examinee a text that presents a certain number of gaps, accompanied by a list of alternatives from which he must choose one to reintegrate. The list includes all the hidden words of the original text plus other that might be added by the author of the test. Surveys conducted by national research group has allowed the testing and development of this model within the OrbisDictus environment. In addition, it is possible to establish a system of automatic assessment on different aspects of vocabulary, provided that they can be described with reference to explicit and quantifiable data of the language. In one of the exploratory studies conducted in Reggio Emilia, assessment is related to the stratification of the vocabulary of a specific corpus not in terms of frequency, but peculiarity of use.

Local item dependence (LID) is one of the most critical assumption in the Rasch model when it comes to the validity of a test. As the field of vocabulary assessment is calling for more clarity and validity for vocabulary tests, such assumption becomes more important than ever. The article offers a Rasch-based investigation into the issue of LID with the focus on the two popular formats of Vocabulary Levels Tests (VLT): multiple-choice and matching. A Listening Vocabulary Levels Test (LVLT) and an Updated Vocabulary Levels Test (UVLT) were given to a single cohort of 311 university students in an English as a Foreign Language (EFL) context. The analyses of raw score and standardized residuals correlations were conducted. The findings found no relationship between the 4-option, multiple-choice format of the LVLT and item local dependence. However, results from score and standardized residuals correlations analyses showed a strong link between the 3-item-per-cluster, matching format and item local dependence. The study calls for greater attention to the format of future vocabulary tests and support the use of meaning-recall formats in vocabulary testing.

http://hdl.handle.net/11380/1174890

Il progetto Firb “Adaptive-Message Learning” (2009-2014) si è posto l’obiettivo scientifico primario di sviluppare un modello didattico di individualizzazione che consiste nell'adattamento del «messaggio di istruzione» alla «competenza verbale» di chi lo riceve (Vertecchi, 2010). Secondo Chris Whetton (2011, 33) il costrutto generale alla base del progetto può essere formulato in modo molto semplice. Ogni studente ha un suo personale lessico, che può essere misurato. Questa informazione può essere utilizzata per costruire materiali di apprendimento individualizzati, tali che l’apprendimento per quella persona sia sostenuto e potenziato. Per il gruppo nel suo insieme, in termini di misurazione «questo dovrebbe comportare una riduzione della varianza dei punteggi e un incremento generale delle medie dei punteggi». All'interno del modello am-learning la valutazione ha tempi, contenuti ed obiettivi diversi che Whetton (2011, 32) riconduce a due elementi fondamentali: «la misurazione iniziale delle abilità del discente di comprendere, e la misurazione finale dei risultati dell'apprendimento, misurati alla fine del corso o del modulo didattico. Per la prima parte, il progetto ha concepito una misura del lessico personale che utilizza i cloze test per valutare la disponibilità di vocabolario rilevante». La valutazione iniziale e continua della competenza lessicale degli studenti, affidata al software LexMeter (Agrusti, 2010), è un elemento centrale nel modello “am-learning”. Essa richiede da un lato l'analisi lessicale dell'input al quale gli studenti sono esposti in un determinato contesto di apprendimento formale, dall'altro la rilevazione di misure sul lettore in relazione al vocabolario dei testi e al linguaggio specifico di un settore disciplinare. Più in dettaglio, prove di cloze di tipo “mirato” con risposte a scelta multipla generate dal software sono utilizzate per misurare fino a che punto gli studenti sappiano attualmente maneggiare il vocabolario rilevante in testi di una specifica disciplina, al fine di stimare quanto efficacemente essi potrebbero affrontare i materiali standard del corso e quindi utilizzare questa informazione per adattare la formulazione del testo di studio per il singolo studente. La validazione della valutazione diagnostica di am-learning ha richiesto particolare attenzione. In am-learning, l'analisi quantitativa delle parole in un corpus di testi di un linguaggio specialistico viene utilizzata per identificare il vocabolario rilevante, cioè per definire con criteri statistici l'elenco delle parole da prendere in esame (quali parole possono essere cancellate da un dato testo). La verifica del vocabolario individuale, quindi la costruzione e la valutazione informatizzata delle prove, sono fondate sulla frequenza di occorrenze delle parole lessicali in un linguaggio specifico. In estrema sintesi, il programma: • produce un lessico di frequenza specifico attraverso l’analisi lessico-statistica di un campione rappresentativo del linguaggio del settore disciplinare; • dato un testo (analogo alle letture curricolari in programma), costruisce, somministra e corregge prove di cloze di tipo mirato con risposte a scelta multipla. Le parole grammaticali sono escluse dall’esame. Una volta stabilito quali classi grammaticali di parole siano eleggibili per la cancellazione (nel caso delle sperimentazioni condotte: nome e aggettivo), le parole nascoste devono ricadere all’interno di un intervallo di frequenze determinato dall’autore della prova. Si fornisce all’esaminato un testo che presenta un determinato numero di lacune, accompagnato da una lista di alternative tra le quali egli deve scegliere quella da reintegrare. La lista comprende tutte le parole del testo originale nascoste più altre eventualmente aggiunte dall’autore della prova. Le indagini condotte dal gruppo di ricerca nazionale hanno consentito la sperimentazione e la messa a punto di questo modello all'interno dell'ambiente OrbisDictus. Inoltre, è possibile fondare un sistema di valutazione automatico su aspetti diversi del vocabolario, purché possano essere descritti con riferimento a dati espliciti e quantificabili del linguaggio. In uno degli studi esplorativi condotti a Reggio Emilia la valutazione è riferita alla stratificazione del vocabolario di un corpus specifico non in termini di frequenza ma di peculiarità dell'uso.

The article presents an empirical study that investigates the single-and crossmodality relationships between different dimensions of receptive vocabulary knowledge and language skills, as well as the importance of academic vocabulary knowledge in academic listening and reading comprehension. An Updated Vocabulary Levels Test (UVLT), a Vietnamese version of the Listening Vocabulary Levels Test (LVLT), an International English Language Testing System (IELTS) listening test and an academic IELTS reading test were administered to 234 tertiary level Vietnamese learners of English as a foreign language (EFL). Research findings showed that (1) orthographic and aural vocabulary knowledge were strongly correlated (r = .88) and of equal significance to L2 listening and reading comprehension, (2) receptive vocabulary knowledge was a very powerful and reliable predictor of learners' receptive language proficiency, (3) knowledge of academic vocabulary strongly correlated with academic listening (r = .65) and reading (r = .60) comprehension and the mastery of the Academic Word List (AWL) could suggest a band score 6.0 in both the IELTS listening and academic reading tests.