With the alignment of Swiss German Sign Language (Deutschschweizerische Gebärdensprache, DSGS) cu... more With the alignment of Swiss German Sign Language (Deutschschweizerische Gebärdensprache, DSGS) curricula in tertiary education to the Common European Framework of Reference for Languages (CEFR), the need for reliable and valid assessment instruments for adult learners arose. With the focus on beginning adult learners of DSGS and on the notion of vocabulary knowledge contributing to overall language proficiency, two vocabulary size tests for DSGS have been developed and evaluated. The dissertation at hand reports on this work. The first test is a web-delivered self-report, the second test represents a translation test from German to DSGS, which includes video-recording for later analysis. For both tests, the same set of items is used. The items have been sampled from existing DSGS teaching materials. For the main study, 20 DSGS adult learners of ages 24 to 55 (M = 39.3) were recruited as test takers. An item analysis of the test results yielded candidates for removal from the item set. An analysis of internal consistency (Cronbach’s Alpha) showed good results for both tests (>.90). Similarly, inter-rater reliability (Cohen’s Kappa) of the translation test indicated promising results. Evidence contributing to content and face validity was collected based on the sampling method of the test items and feedback from the test takers, respectively. Due to the lack of a second DSGS vocabulary test that could be used to establish concurrent validity, external variables (self-assessment of DSGS skills, number of DSGS courses attended, DSGS learning contexts) were identified and investigated as possible external criteria contributing to the performance of the test takers. Only one variable, number of DSGS courses attended, showed a statistically significant correlation with the test results. The results of this study confirm findings from studies of spoken language assessment and will contribute to future research in sign language testing and assessment.
The International Journal of Translation and Interpreting Research
In the interpreting profession, the term language direction (or directionality) is used to descri... more In the interpreting profession, the term language direction (or directionality) is used to describe interpreting from one’s native, dominant language (L1) into a second, non-dominant language (L2), or vice versa. Language direction has long been of interest to interpreting scholars in regards to the quality of the output. Spoken language interpreter educators have argued that high quality interpretations can only be produced when working from an L2 into an L1 (Nicodemus & Emmorey, 2013; Seleskovitch, 1978). Further, spoken language interpreters have reported a preference for working from their L2 into their L1 (Donovan, 2004). In contrast, signed language interpreters, particularly novices, report the opposite preference for language direction, that is, the majority indicate a preference to work from their L1 into their L2 (Nicodemus & Emmorey, 2013). Researchers have speculated about the factors underlying this direction asymmetry found between signed and spoken language interprete...
The Handbook of Language Assessment Across Modalities
Thinking about what is assessed—the construct—in any language assessment raises questions about t... more Thinking about what is assessed—the construct—in any language assessment raises questions about the nature of language use, the nature of developmental trajectories, and whose language patterns determine what is ‘standard’. The assessment of signed languages draws attention to assessment practices and understandings that are entrenched, for better or worse, in the assessment of spoken languages. Spoken language assessments of standardized varieties tend to value the written sentence as an ideal unit, a legacy of standardization. Signed language assessments, on the other hand, may be emerging alongside processes of standardization. Capturing semiotic complexity in the construct remains a significant challenge for both signed and spoken language assessments, despite the development of corpora which exemplify it. This chapter discusses these theoretical, ideological, and practical challenges for assessing signed and spoken language abilities. It brings together key ideas from chapters ...
This paper reports on work in animating Swiss German Sign Language (DSGS) fingerspelling sequence... more This paper reports on work in animating Swiss German Sign Language (DSGS) fingerspelling sequences and signs as well as on the results of a study evaluating the acceptance of the animations. The animated fingerspelling sequences form part of a fingerspelling learning tool for DSGS, while the animated signs are to be used in a study exploring the potential of sign language avatars in sign language assessment. To evaluate the DSGS fingerspelling sequences and signs, we conducted a focus group study with seven early learners of DSGS. We identified the following aspects of the animations as requiring improvement: non-manual features (in particular, facial expressions and head and shoulder movements), (fluidity of) manual movements, and hand positions of fingerspelling signs.
Abstract Sign language test development is a relatively new field within sign linguistics, motiva... more Abstract Sign language test development is a relatively new field within sign linguistics, motivated by the practical need for assessment instruments to evaluate language development in different groups of learners (L1, L2). Due to the lack of research on the structure and acquisition of many sign languages, developing an assessment instrument poses methodological challenges to test developers. Existing sign language tests use information and communication technologies (ICT) to different degrees for test delivery, for example Web-based tests or an application that is installed locally on a computer's hard drive. In order to gain an overview of the state of the art with respect to the delivery of sign language tests and the advantages/disadvantages that test developers see in the usage of ICT related to sign language tests, an international survey was launched (N = 19). The survey was distributed using the online survey tool LimeSurvey. The results of this exploratory study provide a first overview of issues related to the usage of ICT for sign language test delivery and indicate that some issues are comparable to those faced by computer-/Web-delivered tests for spoken languages, whereas others are sign language specific. For the future more research is needed to explore the potentials and challenges of ICT for sign language test development.
Despite the current need for reliable and valid test instruments in different countries in order ... more Despite the current need for reliable and valid test instruments in different countries in order to monitor the sign language acquisition of deaf children, very few tests are commercially available that offer strong evidence for their psychometric properties. This mirrors the current state of affairs for many sign languages, where very little research is available. No previous empirical study has focused explicitly on the linguistic, methodological, and theoretical issues involved in the process of adapting a test from a source sign language to a target sign language. Problems during the adaptation process can arise from linguistic differences between the source and the target language and differences in the source and the target cultures. Both are important aspects that need to be considered in the adaptation of a sign language test from a source to a target language. This study proposes a model for sign language test adaptation, based on the adaptation of the British Sign Language...
The Handbook of Language Assessment Across Modalities
There exists a scarcity in signed language assessment research, especially on scoring issues and ... more There exists a scarcity in signed language assessment research, especially on scoring issues and interrater reliability. This chapter describes two related assessment instruments, the SLPI and the NFA, which offer scoring criteria. Raters are provided with scales for evaluating the different components of the language production of the candidate. Through its use, the rating system has been proved successful; there is, however, hardly any data on interrater reliability. In this chapter, the authors describe reliability issues with attention to raters’ training and score resolution techniques and discuss how to identify and increase rater reliability. The dearth of knowledge on signed language assessment, and in particular its validity and reliability, indicates an urgent need for more research in this area.
With the alignment of Swiss German Sign Language (Deutschschweizerische Gebärdensprache, DSGS) cu... more With the alignment of Swiss German Sign Language (Deutschschweizerische Gebärdensprache, DSGS) curricula in tertiary education to the Common European Framework of Reference for Languages (CEFR), the need for reliable and valid assessment instruments for adult learners arose. With the focus on beginning adult learners of DSGS and on the notion of vocabulary knowledge contributing to overall language proficiency, two vocabulary size tests for DSGS have been developed and evaluated. The dissertation at hand reports on this work. The first test is a web-delivered self-report, the second test represents a translation test from German to DSGS, which includes video-recording for later analysis. For both tests, the same set of items is used. The items have been sampled from existing DSGS teaching materials. For the main study, 20 DSGS adult learners of ages 24 to 55 (M = 39.3) were recruited as test takers. An item analysis of the test results yielded candidates for removal from the item set. An analysis of internal consistency (Cronbach’s Alpha) showed good results for both tests (>.90). Similarly, inter-rater reliability (Cohen’s Kappa) of the translation test indicated promising results. Evidence contributing to content and face validity was collected based on the sampling method of the test items and feedback from the test takers, respectively. Due to the lack of a second DSGS vocabulary test that could be used to establish concurrent validity, external variables (self-assessment of DSGS skills, number of DSGS courses attended, DSGS learning contexts) were identified and investigated as possible external criteria contributing to the performance of the test takers. Only one variable, number of DSGS courses attended, showed a statistically significant correlation with the test results. The results of this study confirm findings from studies of spoken language assessment and will contribute to future research in sign language testing and assessment.
The International Journal of Translation and Interpreting Research
In the interpreting profession, the term language direction (or directionality) is used to descri... more In the interpreting profession, the term language direction (or directionality) is used to describe interpreting from one’s native, dominant language (L1) into a second, non-dominant language (L2), or vice versa. Language direction has long been of interest to interpreting scholars in regards to the quality of the output. Spoken language interpreter educators have argued that high quality interpretations can only be produced when working from an L2 into an L1 (Nicodemus & Emmorey, 2013; Seleskovitch, 1978). Further, spoken language interpreters have reported a preference for working from their L2 into their L1 (Donovan, 2004). In contrast, signed language interpreters, particularly novices, report the opposite preference for language direction, that is, the majority indicate a preference to work from their L1 into their L2 (Nicodemus & Emmorey, 2013). Researchers have speculated about the factors underlying this direction asymmetry found between signed and spoken language interprete...
The Handbook of Language Assessment Across Modalities
Thinking about what is assessed—the construct—in any language assessment raises questions about t... more Thinking about what is assessed—the construct—in any language assessment raises questions about the nature of language use, the nature of developmental trajectories, and whose language patterns determine what is ‘standard’. The assessment of signed languages draws attention to assessment practices and understandings that are entrenched, for better or worse, in the assessment of spoken languages. Spoken language assessments of standardized varieties tend to value the written sentence as an ideal unit, a legacy of standardization. Signed language assessments, on the other hand, may be emerging alongside processes of standardization. Capturing semiotic complexity in the construct remains a significant challenge for both signed and spoken language assessments, despite the development of corpora which exemplify it. This chapter discusses these theoretical, ideological, and practical challenges for assessing signed and spoken language abilities. It brings together key ideas from chapters ...
This paper reports on work in animating Swiss German Sign Language (DSGS) fingerspelling sequence... more This paper reports on work in animating Swiss German Sign Language (DSGS) fingerspelling sequences and signs as well as on the results of a study evaluating the acceptance of the animations. The animated fingerspelling sequences form part of a fingerspelling learning tool for DSGS, while the animated signs are to be used in a study exploring the potential of sign language avatars in sign language assessment. To evaluate the DSGS fingerspelling sequences and signs, we conducted a focus group study with seven early learners of DSGS. We identified the following aspects of the animations as requiring improvement: non-manual features (in particular, facial expressions and head and shoulder movements), (fluidity of) manual movements, and hand positions of fingerspelling signs.
Abstract Sign language test development is a relatively new field within sign linguistics, motiva... more Abstract Sign language test development is a relatively new field within sign linguistics, motivated by the practical need for assessment instruments to evaluate language development in different groups of learners (L1, L2). Due to the lack of research on the structure and acquisition of many sign languages, developing an assessment instrument poses methodological challenges to test developers. Existing sign language tests use information and communication technologies (ICT) to different degrees for test delivery, for example Web-based tests or an application that is installed locally on a computer's hard drive. In order to gain an overview of the state of the art with respect to the delivery of sign language tests and the advantages/disadvantages that test developers see in the usage of ICT related to sign language tests, an international survey was launched (N = 19). The survey was distributed using the online survey tool LimeSurvey. The results of this exploratory study provide a first overview of issues related to the usage of ICT for sign language test delivery and indicate that some issues are comparable to those faced by computer-/Web-delivered tests for spoken languages, whereas others are sign language specific. For the future more research is needed to explore the potentials and challenges of ICT for sign language test development.
Despite the current need for reliable and valid test instruments in different countries in order ... more Despite the current need for reliable and valid test instruments in different countries in order to monitor the sign language acquisition of deaf children, very few tests are commercially available that offer strong evidence for their psychometric properties. This mirrors the current state of affairs for many sign languages, where very little research is available. No previous empirical study has focused explicitly on the linguistic, methodological, and theoretical issues involved in the process of adapting a test from a source sign language to a target sign language. Problems during the adaptation process can arise from linguistic differences between the source and the target language and differences in the source and the target cultures. Both are important aspects that need to be considered in the adaptation of a sign language test from a source to a target language. This study proposes a model for sign language test adaptation, based on the adaptation of the British Sign Language...
The Handbook of Language Assessment Across Modalities
There exists a scarcity in signed language assessment research, especially on scoring issues and ... more There exists a scarcity in signed language assessment research, especially on scoring issues and interrater reliability. This chapter describes two related assessment instruments, the SLPI and the NFA, which offer scoring criteria. Raters are provided with scales for evaluating the different components of the language production of the candidate. Through its use, the rating system has been proved successful; there is, however, hardly any data on interrater reliability. In this chapter, the authors describe reliability issues with attention to raters’ training and score resolution techniques and discuss how to identify and increase rater reliability. The dearth of knowledge on signed language assessment, and in particular its validity and reliability, indicates an urgent need for more research in this area.
International Journal of Emerging Technologies in Learning, 2019
This study reports on the use of an open-source software for sign language learning and (self-) a... more This study reports on the use of an open-source software for sign language learning and (self-) assessment. A Yes/No vocabulary size test for Swiss German Sign Language (Deutschschweizerische Gebärdensprache, DSGS) was developed, targeting beginning adult learners. The Web-based test, which can be used for self-assessment or placement purposes, was administered to 20 DSGS adult learners of ages 24 to 55 (M = 39.3). The learners filled out a background questionnaire, took the Yes/No test tests, and filled out a feedback questionnaire. The comments provided by the learners about the suitability of the Web-based DSGS vocabulary self-assessment instrument provided concrete feedback towards improvement of the system.
In German Switzerland the learning and assessment of Swiss German Sign Language (Deutschschweizer... more In German Switzerland the learning and assessment of Swiss German Sign Language (Deutschschweizerische Gebärdensprache, DSGS) takes place in different contexts, for example, in tertiary education or in continuous education courses. By way of the still ongoing implementation of the Common European Framework of Reference for DSGS, different tests and assessment procedures are currently been developed and their potential is explored to support the learning and assessment of DSGS. Examples of this are two vocabulary size tests. The first is a web-delivered Yes/No Test, the second a Translation Test from written German to DSGS. For both tests, the same set of items was used. The items were sampled from DSGS teaching materials. For the development of the two vocabulary size tests, 20 DSGS adult learners of ages 24 to 55 (M = 39.3) were recruited as test takers. An item analysis of the test results yielded candidates for removal from the item set. Cronbach's Alpha showed good results for both tests (>.90), and inter-rater reliability of the Translation Test also indicated promising results (Cohen's Kappa = .613, p <.001). Evidence contributing to content validity was collected based on the sampling method of the test items. Due to the lack of a second DSGS vocabulary test that could be used to establish concurrent validity, external variables were identified and investigated as possible external criteria contributing to the performance of the test takers. One variable, number of courses attended, showed a significant correlation with the test results.
Simultanes Dolmetschen, ob zwischen zwei Lautsprachen, zwischen zwei Gebärdensprachen oder zwisch... more Simultanes Dolmetschen, ob zwischen zwei Lautsprachen, zwischen zwei Gebärdensprachen oder zwischen einer Gebärdensprache und einer Lautsprache, ist kognitiv höchst herausfordernd, da verschiedene komplexe Prozesse der Sprachproduktion und -rezeption gleichzeitig und sich überschneidend ablaufen. Weitgehend unbekannt sind die Faktoren, die mit dem Dolmetschprozess zwischen Laut- und Gebärdensprachen in Zusammenhang stehen. Ziel dieser Studie ist es, verschiedene Instrumente zu entwickeln -und mit Gebärdensprachdolmetscherinnen anzuwenden-, welche die Einflussfaktoren "Gebärdensprachkompetenz" und "kognitive Fähigkeiten" (wie z. B. Arbeitsgedächtniskapazität und visuell-räumliche Fähigkeiten) auf die Dolmetschkompetenz untersuchen. Weiter werden im Rahmen des Projekts Tests zur Überprüfung der Dolmetschkompetenz entwickelt und erprobt.
The Directive 2010/64/EU of the European Parliament and of the Council of 20 October 2010
establi... more The Directive 2010/64/EU of the European Parliament and of the Council of 20 October 2010 establishes common minimum rules for European Union (EU) countries on the right to interpretation and translation in criminal proceedings as well as in proceedings for the execution of the European arrest warrant. This provision as well as the right to sign language as a human right reiterated by the EUD in the Brussels Declaration ensure that deaf sign language users can access the justice system, typically through sign language interpreters. There is a growing body of literature that examines sign language interpreting provision and practices in legal contexts in various countries. The common theme in the results of all these studies is the limitations faced by deaf sign language users in gaining access to justice, either through inadequate interpreting provision, poor quality interpreting services, or lack of training, accreditation and standards for legal SLIs. This paper reports on a survey that was developed as part of the Justisigns project to provide an overview of the current status of sign language interpreting in legal settings across Europe to better understand what the training needs of interpreters, and other stakeholders such as police officers and deaf people themselves might be. Drawing on key themes from the European Commission survey on legal interpreting in the EU (Hertog & Van Gucht, 2008) and the survey of ASL legal interpreters in the United States (Roberson, Russell & Shaw, 2011), a questionnaire instrument was developed and delivered through an online survey tool. The findings reveal that there are inconsistencies in how legal sign language interpreting provision occurs across Europe.
Abstract:
In this paper, we report on interview data collected from 14 Deaf leaders across seven... more Abstract:
In this paper, we report on interview data collected from 14 Deaf leaders across seven countries (Australia, Belgium, Ireland, the Netherlands, Switzerland, United Kingdom, and the United States) regarding their perspectives on signed language interpreters. Using a semi-structured survey questionnaire, seven interpreting researchers interviewed two Deaf leaders each in their home countries. Following transcription of the data, the researchers conducted a thematic analysis of the comments. Four shared themes emerged in the data, as follows: (a) variable level of confidence in interpreting direction, (b) criteria for selecting interpreters, (c) judging the competence of interpreters, and (d) strategies for working with interpreters. The results suggest that Deaf leaders share similar, but not identical, perspectives about working with interpreters, despite differing conditions that hold regarding how interpreting services are provided in their respective countries. When compared to prior studies of Deaf leaders’ perspectives of interpreters, these data indicate some positive trends in Deaf leaders’ experience with interpreters; however, results also point to a need for further work in creating an atmosphere of trust, enhancing interpreters’ language fluency, and developing mutual collaboration between Deaf leaders and signed language interpreters.
Due to the increased demand to assess sign language learning in the context of secondary and tert... more Due to the increased demand to assess sign language learning in the context of secondary and tertiary education in many European countries, web- or mobile-based assessment solutions can provide a valuable resource for sign language learning and assessment. Most of the existing sign language online tests have been designed for specific tests only, whereas the newly developed sign language assessment portal aims at integrating different kinds tests (e.g., receptive, expressive) with different kinds of tasks and response formats. A preliminary version of this portal has been used within the EU-funded project SignMET. In the first part of the presentation, some key features of the portal will be presented, followed by a discussion on advantages and disadvantages as well as ethical issues in mobilebased sign language testing. Additionally, the potential of natural sign language processing (mostly automatic sign language recognition) and its value for sign language assessment will be addressed.
The goal of this study is to investigate the extent to which sign language Interpreter Education ... more The goal of this study is to investigate the extent to which sign language Interpreter Education Programs (IEPs) in Europe include the discipline of linguistics in their interpreter education courses.
Sign language test research is a fairly young area, which only started in the 1990s. It is nested... more Sign language test research is a fairly young area, which only started in the 1990s. It is nested within the fields of sign language linguistics, deaf education, and applied linguistics. Early research focused on the development of assessments for research purposes with only partial interest in their potential use in practice. Consequently, there has been limited discussion about the training of practitioners who plan to administer such tests within an educational or clinical context. However, the last decade has seen growing interest in sign language assessment (Haug & Hintermair, 2003; Mann & Prinz, 2006) and an increase in the number of tests being developed or adapted to measure deaf test takers’ sign language development. With this change in perception towards sign language assessment comes the need to consider the training required by practitioners to administering these tests effectively. Addressing this gap, we developed a 1-2 day training course for teachers of the deaf and other practitioners working with deaf children. This course focuses on three main areas: the need for sign language assessments, the process of test development (based on McNamara’s concept of testing cycle (2000)), and a discussion of issues pertaining to the assessment of signing populations and how this affects a test’s psychometrics. Additional topics include the use of new technologies (e.g., web-based testing, computer-adaptive testing, automatic sign recognition and generation) and particular challenges of sign language assessment (e.g., the lack of frequency lists). This course has been delivered over the last 10 years to practitioners and clinicians working in schools for the deaf in Germany, United Kingdom, and Switzerland. In our presentation, we discuss aspects related to the course format, feedback from participants, assessment of sign language in different groups of learners. In addition, we take a critical look at the future of sign language assessments.
The survey served to underpin our action research approach in two areas: curriculum development a... more The survey served to underpin our action research approach in two areas: curriculum development and pedagogy. In order to facilitate this, the survey design had the following research questions in focus: 1. To what extent is the CEFR implemented in higher education institutions that offer programs in sign language interpreting and Deaf Studies in Europe? 2. What is the current level of awareness of CEFR amongst sign language teachers working in higher education institutions in Europe? 3. What kind of supports are offered within these institutions to Deaf Studies and interpreter education programs aiming to implement CEFR-aligned curricula and assessment? 4. How are sign language assessments currently carried out in these institutions?
With the alignment of Swiss German Sign Language (Deutschschweizerische Gebärdensprache, DSGS) cu... more With the alignment of Swiss German Sign Language (Deutschschweizerische Gebärdensprache, DSGS) curricula in tertiary education to the Common European Framework of Reference for Languages (CEFR), the need for reliable and valid assessment instruments for adult learners arose. With the focus on beginning adult learners of DSGS and on the notion of vocabulary knowledge contributing to overall language proficiency, two vocabulary size tests for DSGS have been developed and evaluated. The dissertation at hand reports on this work. The first test is a web-delivered self-report, the second test represents a translation test from German to DSGS, which includes video-recording for later analysis. For both tests, the same set of items is used. The items have been sampled from existing DSGS teaching materials. For the main study, 20 DSGS adult learners of ages 24 to 55 (M = 39.3) were recruited as test takers. An item analysis of the test results yielded candidates for removal from the item set. An analysis of internal consistency (Cronbach’s Alpha) showed good results for both tests (>.90). Similarly, inter-rater reliability (Cohen’s Kappa) of the translation test indicated promising results. Evidence contributing to content and face validity was collected based on the sampling method of the test items and feedback from the test takers, respectively. Due to the lack of a second DSGS vocabulary test that could be used to establish concurrent validity, external variables (self-assessment of DSGS skills, number of DSGS courses attended, DSGS learning contexts) were identified and investigated as possible external criteria contributing to the performance of the test takers. Only one variable, number of DSGS courses attended, showed a statistically significant correlation with the test results. The results of this study confirm findings from studies of spoken language assessment and will contribute to future research in sign language testing and assessment.
Uploads
Papers by Tobias Haug
establishes common minimum rules for European Union (EU) countries on the right to
interpretation and translation in criminal proceedings as well as in proceedings for the execution of
the European arrest warrant. This provision as well as the right to sign language as a human right
reiterated by the EUD in the Brussels Declaration ensure that deaf sign language users can access
the justice system, typically through sign language interpreters. There is a growing body of
literature that examines sign language interpreting provision and practices in legal contexts in
various countries. The common theme in the results of all these studies is the limitations faced by
deaf sign language users in gaining access to justice, either through inadequate interpreting
provision, poor quality interpreting services, or lack of training, accreditation and standards for
legal SLIs. This paper reports on a survey that was developed as part of the Justisigns project to
provide an overview of the current status of sign language interpreting in legal settings across
Europe to better understand what the training needs of interpreters, and other stakeholders such
as police officers and deaf people themselves might be. Drawing on key themes from the European
Commission survey on legal interpreting in the EU (Hertog & Van Gucht, 2008) and the survey of
ASL legal interpreters in the United States (Roberson, Russell & Shaw, 2011), a questionnaire
instrument was developed and delivered through an online survey tool. The findings reveal that
there are inconsistencies in how legal sign language interpreting provision occurs across Europe.
In this paper, we report on interview data collected from 14 Deaf leaders across seven countries (Australia, Belgium, Ireland, the Netherlands, Switzerland, United Kingdom, and the United States) regarding their perspectives on signed language interpreters. Using a semi-structured survey questionnaire, seven interpreting researchers interviewed two Deaf leaders each in their home countries. Following transcription of the data, the researchers conducted a thematic analysis of the comments. Four shared themes emerged in the data, as follows: (a) variable level of confidence in interpreting direction, (b) criteria for selecting interpreters, (c) judging the competence of interpreters, and (d) strategies for working with interpreters. The results suggest that Deaf leaders share similar, but not identical, perspectives about working with interpreters, despite differing conditions that hold regarding how interpreting services are provided in their respective countries. When compared to prior studies of Deaf leaders’ perspectives of interpreters, these data indicate some positive trends in Deaf leaders’ experience with interpreters; however, results also point to a need for further work in creating an atmosphere of trust, enhancing interpreters’ language fluency, and developing mutual collaboration between Deaf leaders and signed language interpreters.
Addressing this gap, we developed a 1-2 day training course for teachers of the deaf and other practitioners working with deaf children. This course focuses on three main areas: the need for sign language assessments, the process of test development (based on McNamara’s concept of testing cycle (2000)), and a discussion of issues pertaining to the assessment of signing populations and how this affects a test’s psychometrics. Additional topics include the use of new technologies (e.g., web-based testing, computer-adaptive testing, automatic sign recognition and generation) and particular challenges of sign language assessment (e.g., the lack of frequency lists). This course has been delivered over the last 10 years to practitioners and clinicians working in schools for the deaf in Germany, United Kingdom, and Switzerland.
In our presentation, we discuss aspects related to the course format, feedback from participants, assessment of sign language in different groups of learners. In addition, we take a critical look at the future of sign language assessments.
1. To what extent is the CEFR implemented in higher education institutions that offer programs in sign language interpreting and Deaf Studies in Europe?
2. What is the current level of awareness of CEFR amongst sign language teachers working in higher education institutions in Europe?
3. What kind of supports are offered within these institutions to Deaf Studies and interpreter education programs aiming to implement CEFR-aligned curricula and assessment?
4. How are sign language assessments currently carried out in these institutions?
With the focus on beginning adult learners of DSGS and on the notion of vocabulary knowledge contributing to overall language proficiency, two vocabulary size tests for DSGS have been developed and evaluated. The dissertation at hand reports on this work. The first test is a web-delivered self-report, the second test represents a translation test from German to DSGS, which includes video-recording for later analysis. For both tests, the same set of items is used. The items have been sampled from existing DSGS teaching materials.
For the main study, 20 DSGS adult learners of ages 24 to 55 (M = 39.3) were recruited as test takers. An item analysis of the test results yielded candidates for removal from the item set. An analysis of internal consistency (Cronbach’s Alpha) showed good results for both tests (>.90). Similarly, inter-rater reliability (Cohen’s Kappa) of the translation test indicated promising results. Evidence contributing to content and face validity was collected based on the sampling method of the test items and feedback from the test takers, respectively. Due to the lack of a second DSGS vocabulary test that could be used to establish concurrent validity, external variables (self-assessment of DSGS skills, number of DSGS courses attended, DSGS learning contexts) were identified and investigated as possible external criteria contributing to the performance of the test takers. Only one variable, number of DSGS courses attended, showed a statistically significant correlation with the test results.
The results of this study confirm findings from studies of spoken language assessment and will contribute to future research in sign language testing and assessment.