Test Development

This paper is part of wider doctoral research exploring the use of audiovisual comprehension questions embedded in the image as subtitles and synchronized with the relevant fragments. A framework describing this technique (Casañ Núñez, 2015b) and an example in a pedagogical sequence (Casañ Núñez, 2015a) have already been published.

This present work details the planning, design and trialling of an audiovisual comprehension test with two variants to be administered together with other instruments in quasi-experimental studies with control and treatment groups. Basically, the aim is to investigate whether subtitled questions facilitate comprehension, whether students´ time looking at the screen increases, and what the treatment group thinks about the technique.

In the trialling phase, six studies were conducted. Forty-one SFL students participated in the last pilot study (twenty-two in the control and nineteen in experimental group). The observations of the informants during the test administration and the subsequent correction suggested that the directions on the test structure, the presentation of the input texts, the explanation of how the subtitled questions work for the experimental group and the wording of the items were all understood. The data of each version of the instrument was subjected to analysis of facility, discrimination, reliability and descriptives. Correlations between the tests and two tasks from a listening exam were also calculated. Results showed that the two versions of the test were ready to be administered.

RESUMEN
Este artículo forma parte de una investigación doctoral que estudia el uso de preguntas de comprensión audiovisual integradas en la imagen del vídeo como subtítulos y sincronizadas con los fragmentos de vídeo relevantes. Anteriormente se han publicado un marco teórico que describe esta técnica (Casañ Núñez, 2015b) y un ejemplo en una secuencia didáctica (Casañ Núñez, 2015a).

El presente trabajo detalla el proceso de planificación, diseño y experimentación de una prueba de comprensión audiovisual con dos variantes que será administrada junto con otros instrumentos en estudios cuasiexperimentales con grupos de control y tratamiento. Fundamentalmente, se pretende averiguar si la subtitulación de las preguntas facilita la comprensión, si aumenta el tiempo que los estudiantes miran en dirección a la pantalla y conocer la opinión del grupo de tratamiento sobre esta técnica.

En la fase de experimentación se efectuaron seis estudios. En el último estudio piloto participaron cuarenta y un estudiantes de ELE (veintidós en el grupo de control y diecinueve en el de tratamiento. Las observaciones de los informantes durante la administración de la prueba y su posterior corrección sugirieron que las indicaciones sobre la estructura del test, las presentaciones de los textos de entrada, la explicación sobre el funcionamiento de las preguntas subtituladas para el grupo experimental y la redacción de los ítems resultaron comprensibles. Los datos de las dos variantes del instrumento se sometieron a sendos análisis de facilidad, discriminación, fiabilidad y descriptivos. También se calcularon las correlaciones entre los test y dos tareas de un examen de comprensión auditiva. Los resultados mostraron que las dos versiones de la prueba estaban preparadas para ser administradas.

Öz Bu çalışmanın amacı, 2013-2014 Eğitim-Öğretim yılı, 2. döneminde bulunan ilkokul 2. sınıf öğrenci-lerinin Türkçe dersindeki okuduğunu anlama başarılarının cinsiyet ve yaşa göre incelenmesidir. Araştırmanın çalışma grubunu, İstanbul ili Avrupa yakasında bulunan Cemal Artüz ve Günebakan İlkokullarından 2. Sınıf düzeyinde 156 öğrenci oluşturmuştur. Bu çalışma tekil modeli ile yapılmış betimsel bir çalışmadır. Araştırmada veri toplama aracı olarak Ceran, Oğuzgiray ve Özdemir(2014) tarafından geliştirilen Türkçe dersi 2. Sınıf Okuduğunu Anlama Başarı Testi kullanılmıştır. Topla-nan verilerin analizinde birbirinden bağımsız yaş gruplarının Türkçe dersi 2. Sınıf Okuduğunu An-lama Başarı Testi başarı ortalamaları arasında fark olup olmadığını test etmek amacıyla Kruskal-Wallis analizi, başarının cinsiyete göre değişip değişmediğini anlamak amacıyla ise Mann-Whitney U testi yapılmıştır. Analiz sonuçları, başarı testi uygulanan çocukların yaş gruplarına göre incelen-diğinde, 86-92 ay olan çocukların lehine anlamlı bir fark olduğu tespit edilmiştir. Cinsiyet değişke-nine göre yapılan analizlerde ise kızların lehine anlamlı bir fark olduğu tespit edilmiştir. Abstract The aim of this study is to analyze reading comprehension skills of 2. grade students in Turkish lesson according to sex and age mobility. The data was collected during the spring semester of 2013-2014 of Cemal Artüz Primary School and Günebakan Primary School in İstanbul from European side. The participants were 156 students in second grade. This study was conducted with qualitative research methods. Research data was collected through questionnaires developed by Oguzgiray, Özdemir and Ceran (2014). In data analysis, Kruskal-Wallis and Mann-Whitney U tests were employed. According to data analysis there is an important significance among age groups, in favor of 86-92 months child. Apart from this, comparisons of gender variable showed that girls had higher scores in comprehension level.

Introduction: Teaching profession is involved in tensions and
pressures arising from the complexity of the actual job conditions. For this reason, it is necessary to examine their ways of coping with school stressors. In particular, attention is focused on the strategies used by teachers in order to manage the stressful demands encountered in work settings. The present study describes the subsequent steps of the development and the validation of a questionnaire designed toassess teachers’ coping strategies. Methods: a) In a first phase a structured interview was submitted to 55 high school teachers in order to identify the principal dimensions of coping and to genrate items; b) in a second study item reduction and validation analyses were conducted on a sample of 260 teachers. Results: According to the results of this
study, it appears that the instrument has promising preliminary psychometric properties. The questionnaire is both a reliable and a valid instrument. Conclusions: The instrument could be used in different training programs developed to promote well-being conditions in work setting.

The quantification of moisture susceptibility has been a major concern for researchers as it adversely affects the performance of asphalt pavements. Several methods have been developed to assess bond strength using asphalt mixtures in loose and compacted state. These tests lack in their ability to study fundamental properties that affect the bond between bitumen and aggregate. In this context, more fundamental techniques have been developed such as pull-off stub tests and direct tension tests. The first group only measures the maximum pull-off strength and second group has problems related to use of consistent binder film thickness and operational difficulties in test itself.

This paper presents a new test to evaluate bond strength, in an attempt, to solve problems associated with traditional direct tension tests. The aim is achieved through a review of existing techniques, development of a gap assembly, fabrication of direct tension test moulds, development of Compression Pull-Off Test (CPOT) and evaluation of its results. The key parameters for bitumen and mastics were evaluated. The CPOT shows promising results for use of this technique to study cohesive as well as adhesive bond strength of binder.

Very few attempts have been made in the past to develop instruments to measure pragmatic knowledge of second language (L2) learners. The absence of such instruments in the literature of English language teaching (ELT) underscores the need for the researchers to develop new tests that are specifically designed to assess this crucial but less explored aspect of language learners' (LLs) knowledge. In line with this objective, the present study was conducted to develop and validate four tests of pragmatic knowledge that measured LLs' knowledge of speech acts. The following steps were taken in this study to develop the written discourse completion tests (WDCTs) and the multiple-choice discourse completion tests (MCDCTs) that respectively measured the test takers' ability to produce and comprehend request speech act. During the "prototype step" the researchers identified the content and the number of items for each designated test battery. At the "test construction step" the sociolinguistic variables of power (P), social distance (D), and absolute rank of imposition (R) were inserted into the content of the test items. Finally, at the "validation step" the reliability of the tests was examined. The finding of the study showed that the constructed test batteries were sufficiently reliable and valid for measuring pragmatic knowledge of L2 learners.

Abstract: Student assessment provides teachers with information that is important for decision-making in the classroom. Assessment information helps teachers to understand their students’ performance better as well as improve suitability and effectiveness of classroom instruction. The purpose of the study was to compare the classroom assessment preferences of Japanese language teachers in the Philippines (n=61) and English language teachers in Japan (n=55) on the purposes of assessment as measured by the Classroom Assessment Preferences Survey Questionnaire for Language Teachers (CAPSQ-LT). Results revealed that overall, language teachers from both countries most preferred assessment practices that are focused towards assessment as learning and least preferred assessment practices that refer to the communicative function of assessment (assessing to inform). Comparatively, Japanese language teachers in the Philippines preferred assessment for learning, that is, they assessed to improve learning process and effectiveness of instruction, while the English language teachers in Japan are more concerned with the assessment of learning and the communicative and administrative function of assessment. The two groups did not significantly differ in their preference for assessment of learning and assessment as learning.

Resumen: Las tareas estudiantiles proveen de información relevante a los profesores sobre que es importante para la toma de decisiones en las aulas. La información de las tareas ayuda a los enseñantes a comprender mejor la evolución de estos alumnos, así como mejorar la efectividad y pertinencia de la instrucción en las aulas. El propósito de este estudio fue comparar las preferencias en la distribución de tareas de clase entre profesores de lengua Japonesa en Filipinas (n=61) y de enseñantes de lengua inglesa en Japón (n=55), mediante el uso de la Encuesta de tipo cuestionario para profesores de idiomas sobre tareas de clase (CAPSQ-LT). En general, los resultados revelaron que los enseñantes de idiomas de ambos países mayormente prefirieron tareas prácticas que estaban enfocadas hacia tareas como aprendizaje y menos sobre aquellas prácticas referidas a la función comunicativa de las tareas (tareas para informar). Comparativamente, los profesores de lengua Japonesa en Filipinas prefirieron tareas de aprendizaje mientras que en Filipinas prefirieron tareas para aprender, ergo, estos enseñantes se enfocaron en mejorar los procesos de aprendizaje y la efectividad de la instrucción, mientras que los profesores de lengua inglesa en Japón estaban más enfocados respecto a las tareas de aprendizaje y las funciones administrativas y comunicativas de las tareas. Los dos grupos no difirieron significativamente en sus preferencias entre las áreas de y para aprender.

Keywords: Student assessment, assessment information, assessment practices, classroom assessment preferences

Pragmatic competence is among the explicitly acknowledged sub-competences that make the communicative competence in any language (Bachman & Palmer, 1996; Council of Europe, 2001). Within the notion of pragmatic competence itself, “implicature (implied meanings)” comes to the fore as one of the five main areas there (Levinson, 1983). In this regard, the present study developed an online multiple-choice discourse completion test (MDCT) to specifically measure the interpretation of formulaic, thus teachable implied meanings in English. After the initial version produced with the help of a native speaker colleague, the test was piloted rigorously with different comparison groups. At the end of the process, 112 EFL Teacher Trainees, 33 native speakers of English, 37 EFL learners at a university School of Foreign Languages and 11 high school EFL students had taken the test as the comparison subgroups with varying proficiency levels. Moreover, seven ELT professional native speakers had bee...

This paper presents research and provides a method to ensure that parallel assessments, that are generated from a large test-item database, maintain equitable difficulty and content coverage each time the assessment is presented. To maintain fairness and validity it is important that all instances of an assessment, that is intended to test the same subject content, are presented to each test-taker without bias. The method described demonstrates how each test-item in an item bank (i) is assigned a difficulty rating using a recognized test-centered legally defensible cut-score determination method, (ii) assigned to either hard, moderate, or easy categories, and (iii) then selected in a stratified random selection fashion, by content and difficulty, to ensure equal content coverage and difficulty level within an acceptable range of the calculated cut score.

O afeto pode ser entendido como um estado de prazer ou desprazer vivenciado com algum grau de ativação frente a algum estímulo. Dessa forma, a investigação do afeto no campo esportivo contribui no entendimento sobre como determinados estímulos podem afetar o desempenho do atleta. O objetivo desse estudo foi identificar evidências de validade semântica de um instrumento de medida sobre afeto no contexto esportivo. O estudo contou com uma amostra de 12 atletas, sendo 1 atleta com ensino fundamental, 6 atletas com ensino médio e 5 atletas com ensino superior. A média de idade foi 24,1 anos (DP=3,2). Sobre a modalidade, 7 atletas praticavam artes marciais, 3 praticavam atletismo, 1 praticava ciclismo e 1 praticava vôlei. A aplicação do instrumento foi realizada coletiva e individualmente, dependendo da disponibilidade das instituições. O instrumento de medida foi construído com base no circumplexo de Russell (2003) e a escala foi baseada no diferencial semântico. Ao responder o instrumento, foi pedido ao atleta que pensasse no que ele sentia antes de competir.  Foram construídos 24 itens, sendo 2 itens-ancora e 22 itens que foram distribuídos em 3 categorias. Ao final do instrumento, foi pedido ao atleta que atribuísse um número de 1 a 3 para cada grupo de adjetivos, sendo a classificação 3 aplicada ao grupo que ele sentisse de forma mais intensa. Após todas as aplicações, houve 15 comentários relacionados ao instrumento de medida. Os comentários foram encaixados em três categorias: dificuldades com instruções (n=5), dificuldades com semântica das palavras (n=8) e dificuldades com coleta de dados clínicos (n=2). O instrumento manteve 24 itens e foram adicionadas 2 perguntas, uma lesão crônica e uma sobre o nível da última competição que o atleta participou (estadual, nacional ou internacional). Foi concluído que o instrumento obteve evidências de validade semântica para o seu público-alvo . Novos estudos para busca de evidências de validade da estrutura do construto e da sua relação com outras variáveis são necessários para identificar a validade e precisão do instrumento.