Carlos Teixeira

IATIS 6th International Conference. Hong Kong. 3-6 July 2018. This presentation reports on the results of a usability study of a browser-based translation and post-editing tool that accepts multiple input modes and follows web accessibility principles. The tool was conceived to be used by professional translators and will include standard features that are necessary for integrating translation memories (TM) and machine translation (MT) synergetically. Its distinctive features include the option of using touch and voice commands, in addition to the typical keyboard and mouse commands. Another important feature is the inclusion of accessibility principles from the outset, with the aim of improving translation editing for professionals with special needs. In order to assess whether the usability and accessibility features included in the tool translate into improved performance and user experience, we have carried out experiments with a varied cohort of participants. The current presentation reports on the results of the accessibility tests with three blind translators. The results were analysed based on frustration reports, a usability & accessibility questionnaire and an interview. The findings will help understand how the possibilities offered by the newest technologies can improve the translation process from the user’s perspective and will inform future stages of the tool’s iterative development.

Multi-modal input devices that include touch-enabled screens and speech recognition can provide an ideal interface for correcting common MT errors, such as word order or capitalisation, as well as for repairing TM matches. Building on our experience in creating a prototype mobile post-editing interface for smartphones (Moorkens, O’Brien, and Vreeke 2016), we aim to create a software-as-a-service editing environment dedicated to post-editing with multiple input modes. On devices equipped with touch screens, such as tablets and select laptops, the tool allows translators to use touch commands in addition to the typical keyboard and mouse commands. On all devices, the tool also accepts voice input using system or standalone Automatic Speech Recognition engines. Another important feature in the tool is the inclusion of accessibility principles from the outset, with the aim of opening translation editing to professionals with special needs. In particular, the tool is being designed to cater for blind translators, whose difficulty in working with existing tools has been highlighted by Rodriguez Vazquez and Mileto (2016). The importance of making computer applications accessible in general is reinforced by recent news reports that estimate that only 16% of visually-impaired people in Ireland are employed (White, 2017). The project began in March 2017 and is progressing using an agile development process of short sprints. The planning phase involved sifting through the available frameworks and providing a breakdown of each. After analysis, the proposed solution was to use React, Facebook’s component based library for creating user interfaces. Using a modern library like React exposes the tool to a large repository of compatible, well developed technologies which greatly increase the ease and speed of development. An example of this is Draft JS, the rich text editor implemented by the tool. React provides the core functionality for building the web-based interface, allowing the tool to be used cross-platform. Moreover, React Native provides an implementation of React for building native applications, which opens the potential for the tool to be ported over to iOS and Android with minimal development costs. This paper reports on initial usability tests with an early version of the tool. The results include productivity measurements as well as data collected using satisfaction reports and suggest that the integration of voice input may increase user satisfaction. The use of touch input in the current stage of the tool development has not been well accepted by the participants, which has prompted us to rethink the design of that component. The findings will also provide relevant feedback for further stages of the iterative development. In a future stage, we will assess the effectiveness of the accessibility features that have been incorporated into the tool by conducting usability studies with professional blind translators. Ultimately, we will test whether non-desktop large-scale or crowd-sourced post-editing is feasible and productive and can help alleviate some of the pain points of the edit-intensive, mechanical task of desktop post-editing.

CIUTI Conference 2018. Heriot-Watt University. Edinburgh, 30-31 May 2018. Touch-based interactions are increasingly common for many people due to the widespread adoption of smartphones and other mobile devices. Likewise, voice commands and text input using automatic speech recognition (ASR) are becoming commonplace thanks to digital assistants on mobile devices and the Internet of Things. Although these technologies are becoming more and more ubiquitous, with much higher accuracy than ever before, mainstream translation tools are not taking advantage of them. We believe that voice and touch input could bring several benefits to translators in terms of ergonomics, usability, enjoyability, and satisfaction, with similar or improved productivity. In order to test this assumption, we have designed and developed a prototype translation tool that incorporates both technologies in addition to the more traditional keyboard and mouse. The tool runs on web browsers and has been developed to be modular, so that different connectors and APIs can be used to expand its features as well as to connect to alternative services, e.g. different machine translation (MT) engines and ASR systems. Building on the results of a previous usability study conducted in 2017 with an earlier version of the tool (Teixeira et al. 2017), we redesigned some of the features and now present the results of our second usability tests. We asked professional translators/post-editors to perform four different tasks using different interaction modes in the latest iteration of the tool. We then looked at quantitative performance indicators such as the amount of editing and the time on task, as well as qualitative satisfaction indicators. Preliminary results show a great improvement in usability as compared to the previous test round. In broad terms, the voice input is regarded as a useful feature by translators, while the touch interaction seems to be more suitable to specific use cases, such as when translators need to do quick edits and do not have a physical keyboard at hand. The ways that translators interact with the tool also depend on the type of edits they need to make, which in turn are related to the language combination and the type of MT being used (statistical vs. neural). The findings will provide relevant feedback for the next stages of the iterative development, where we will also assess the effectiveness of the accessibility features (Rodriguez Vazquez and Mileto 2016) that have been incorporated into the tool by conducting usability studies with blind and partially-sighted professional translators.

It is our contention that translation technology, specifically translation memory and machine translation, have traditionally been developed without much consideration of the cognitive ergonomic needs of translators. Despite the fact that it is more than two decades since the appearance of the first TM tools, translators are still dissatisfied with this technology. However, more recently, commercial translation tools have been incorporating new aesthetics, e.g. the replacement of the toolbar with the ‘ribbon' concept in some of the most used CAT tools in the market. They have also been integrating new features, such as an expanded integration of machine translation, either by connecting to additional MT systems or by using MT as an underlying technology for assembling sub-segment TM matches. Recent years have also seen the appearance of research-oriented translation tools, such as Casmacat, MateCAT or Lilt, the latter two having made it to the commercial level. These tools have different degrees of integration between TM and MT, although usually with a focus on post-editing. Innovations in these tools include interactive and adaptive machine translation, as well as indicators of confidence estimation for the MT suggestions, together with new ways of visualising and interacting with the MT output. Such innovations have prompted tool developers and advocates to say that we are moving ‘beyond post-editing' (Koehn 2015, Green 2015), implying that the innovations in the tools have created new ways of translating. By analysing how professional translators interact with tools in the workplace, we will address strengths and weaknesses of those innovations from a cognitive ergonomic perspective (Ehrensberger-Dow and O'Brien 2015). We use translation process methods such as eye tracking to identify patterns of visual attention, and we discuss whether certain innovations in the tools could represent a leap forward in meeting the needs of translators.

This paper reports on an empirical study that investigates the translation process in the workplace from a cognitive ergonomic perspective. In particular, the interaction between ten translators employed by a language service provider and the tools they deploy are examined. To that end, we recorded the translators’ workplace activities using keystroke logging, screen recording and eye tracking, combined with short retrospective interviews. We analysed their behaviour in terms of how they switched between the two screens on their desks, how they used different tools and where they invested their visual attention. Data related to productivity and quality are also presented. Among other findings, our data reveal that validation searches for terms and general expressions lead to considerable tool and task switching among professional translators.

Commercial software tools for translation have, until now, been based on the traditional input modes of keyboard and mouse, latterly with a small amount of speech recognition input becoming popular. In order to test whether a greater variety of input modes might aid translation from scratch, translation using translation memories, or machine translation postediting, we developed a web-based translation editing interface that permits multimodal input via touch-enabled screens and speech recognition in addition to keyboard and mouse. The tool also conforms to web accessibility standards. This article describes the tool and its development process over several iterations. Between these iterations we carried out two usability studies, also reported here. Findings were promising, albeit somewhat inconclusive. Participants liked the tool and the speech recognition functionality. Reports of the touchscreen were mixed, and we consider that it may require further research to incorporate touc...

This paper investigates the behaviour of ten professional translators when performing translation tasks with and without translation suggestions, and with and without translation metadata. The measured performances are then compared with the translators’ perceptions of their performances. The variables that are taken into consideration are time, edits and errors. Keystroke logging and screen recording are used to measure time and edits, an error score system is used to identify errors and post-performance interviews are used to assess participants’ perceptions. The study looks at the correlations between the translators’ perceptions and their actual performances, and tries to understand the reasons behind any discrepancies. Translators are found to prefer an environment with translation suggestions and translation metadata to an environment without metadata. This preference, however, does not always correlate with an improved performance. Task familiarity seems to be the most promin...

Despite considerable advances in machine translation (MT), adoption by professional translators still meets with resistance. Research on the human factors associated with MT (non-)adoption is required to understand this state of affairs. We investigate whether two specific groups of professional translators use MT, what reasons they advance for its use/non-use and what factors might explain the reasons given. Participants advanced an equally diverse set of reasons for using MT as for not using it, and this was strongly linked to text type, language pair, quality and trust. Using an agency theory lens, we found evidence of Pickering’s dialectics of resistance and accommodation in the focus group data. We also found that one group of translators is more open to the use of MT, and suggest that the socio-technical context of deployment might explain this finding.

The integration of machine translation (MT) and translation-memory (TM) systems in professional translation settings has turned pre-translation + post-editing into an attractive alternative in terms of productivity for all parties involved in the translation process. In some cases, source files are pre-translated using a combination of customised MT and TM before reaching the translators, who then become reviewers, or post-editors. But how does this actually affect productivity and how do translators feel when performing this new activity? In order to look for answers to those questions, I ran a pilot experiment comparing two different environments. The basic difference between the two is the availability of information on the provenance of the suggested translation for a particular segment (whether it comes from MT, TM, and at which match percentage). Data were collected using screen recording, keystroke logging and post-performance interviews.

This paper reports on an empirical study that investigates the translation process in the workplace from a cognitive ergonomic perspective. In particular, the interaction between ten translators employed by a language service provider and the tools they deploy are examined. To that end, we recorded the translators' workplace activities using keystroke logging, screen recording and eye tracking, combined with short retrospective interviews. We analysed their behaviour in terms of how they switched between the two screens on their desks, how they used different tools and where they invested their visual attention. Data related to productivity and quality are also presented. Among other findings, our data reveal that validation searches for terms and general expressions lead to considerable tool and task switching among professional translators.

This paper investigates the behaviour of ten professional translators when performing translation tasks with and without translation suggestions, and with and without translation
metadata. The measured performances are then compared with the translators’ perceptions of their performances. The variables that are taken into consideration are time, edits and errors. Keystroke logging and screen recording are used to measure time and edits, an error score system is used to identify errors and post-performance interviews are used to assess participants’ perceptions. The study looks at the correlations between the translators’ perceptions and their actual performances, and tries to understand the reasons behind any discrepancies. Translators are found to prefer an environment with translation suggestions and translation metadata to an environment without metadata. This preference, however, does not always correlate with an improved performance. Task familiarity seems to be the most prominent factor responsible for the positive perceptions, rather than any intrinsic characteristics in the tasks. A certain prejudice against MT is also present in some of the comments.

This thesis investigates whether and how translation metadata affect translator performance in a workflow that combines suggestions from translation memories and machine translation. The study is based on a translation process experiment with 10 professional translators working from English into Spanish in a workplace setting.
The keystroke logging tools Inputlog and MTeval allowed for the collection of data on translation times and typing effort. BB FlashBack was used for screen and face recording. A Tobii eye tracker was used to identify how the translators shifted their attention between different parts of the screen. The final translations were assessed for quality by two professional reviewers using an error-score system. Finally, interviews were used for eliciting opinions from participants about certain aspects of their performance.
The quantitative data were analysed with mixed-effects linear(ised) regression models. The results show that translation metadata affect translation time and typing effort, and that the effects vary according to the type of translation suggestion (exact matches, fuzzy matches, machine translation). As a complementary finding, the current study identified no significant correlation between the translators’ performances while typing and their performances while translating.
The qualitative data obtained from the interviews show a mismatch between the translators’ perceived performance and their measured performance. They tended to prefer an environment with translation suggestions and metadata, even when this environment did not correspond with better performance. The translators mentioned metadata as a helpful feature in the translation tool, among other reasons because metadata help them adapt their translation strategies more easily according to the suggestion type. Task familiarity was also identified as an important factor affecting translators’ perceptions.
The results obtained in this study suggest the need to advance research on how translators interact with translation tools, with a view to increase not only productivity but also job satisfaction. This thesis is expected to have also contributed to the field in terms of the methodology of workplace studies, by presenting some challenges and solutions. An important lesson is the need to find an optimal balance between ecological validity and data validity when conducting translation experiments in realistic scenarios.

In this chapter, several methods available for translation process research are explained and analysed, including keystroke and mouse logging, screen recording, eye tracking, retrospection, interviews, and quality revision. The use of the methods is constrained by the particularities of an experiment in which ten professional translators work in their everyday work environment with their computer of habitual use. This workplace setting aims at maximising the ecological validity of the research design, but it also creates difficulties for data collection and analysis. The chapter describes the particular conditions of the experiment and how the research methods were adapted to each translator’s office space. Without going into the particularities of the research variables, the chapter suggests solutions to the difficulties that may arise in this kind of experiment and presents possible alternatives to adjust to some of the unsolved problems. Finally, the chapter formulates some general principles on the most appropriate data-collection methods for similar research.

In this chapter, I map out how five translation tools present translation metadata. Although the spectrum of possible metadata elements runs to hundreds, the five tools combined display only around 15 such elements. I raise the question of whether this set of metadata elements and the way they are presented constitute the best combination in terms of translator productivity and translating effort. I take Pym’s minimalist approach to
translation competence and extrapolate it as a model for the translation process, indicating how translation tools can contribute to the generation/selection steps of this process.""

"Starting from an overview of multilingual systems, I point out the usefulness of machine translation in some translation contexts, especially in dynamic environments. Then I describe how electronic tools can be useful for the human translator and how machine translation can be integrated into translation memory systems in the translation workflow. After this general panorama,
I enumerate a series of studies in the field of translation studies that deal with translation process research and investigate the interaction between human translators and those technologies. I note that the main aspects being investigated are productivity, quality and effort. I also mention how the new technologies might affect the translation market and the activity of translation professionals. I conclude by indicating some areas for future
research, including tool usability and job satisfaction."

The integration of machine translation (MT) and translation-memory (TM) systems in professional translation settings has turned pre-translation + post-editing into an attractive alternative in terms of productivity for all parties involved in the translation process. In some cases, source files are pre-translated using a combination of customised MT and TM before reaching the translators, who then become reviewers, or post-editors. But how does this actually affect productivity and how do translators feel when performing this new activity? In order to look for answers to those questions, I ran a pilot experiment comparing two different environments. The basic difference between the two is the availability of information on the provenance of the suggested translation for a particular segment (whether it comes from MT, TM, and at which match percentage). Data were collected using screen recording, keystroke logging and post-performance interviews.

This presentation reports on a workplace study carried out during my PhD with ten professional translators working with translation memories and machine translation from English into Spanish. One of the goals is to compare the measured performances with the translators’ perceptions, to see if productivity metrics relate with increased task satisfaction. A second goal of my study is to compare the use of translation memories and machine translation in an environment that is at the same time ecologically valid and “fair” for both types of translation suggestions. The study uses keystroke logging and screen recording for measuring time and edits, eye tracking for determining where on screen the translators concentrated their attention when making specific types of edits, an error score system for assessing quality, and post-performance interviews for assessing perceptions. The participants were presented with three translation tasks, which were all performed within the same commercial translation memory system: in task 1, translators were asked to translate from scratch; in task 2, they were asked to translate in a normal TM workflow, with translation suggestions coming from a translation memory and machine translation; in task 3, they were asked to repair translation suggestions that were already pre-inserted in the segments, also coming from translation memories and machine translation. The study found that tasks 2 and 3, which presented translation suggestions, were faster, required less editing and produced less errors than task 1, which is not surprising. However, the study also found no significant difference between tasks 2 and 3 in terms of speed and quality, and found that translators typed considerably less in task 3. Therefore, it appears that the interactive translation workflow, with active insertion of translation suggestions and metadata about the suggestions, which is typical of TM systems, can actually lead to more typing, i.e. more physical effort by the translators, suggesting that TM-based translation could be more cognitively demanding than repairing pre-translated text with no translation metadata. When the results of the experiment are looked into in more detail, moving from the entire text to the individual types of segments, there is a clear difference in the strategies used by translators for handling exact matches, fuzzy matches and machine translation. This difference in strategies is confirmed by the eye-tracking data and corroborated by the translators themselves in the interviews. Despite the high degree of personal variation among the ten translators in terms of measured performance, the vast majority tended to prefer the traditional environment. Their answers in the interviews show that the metadata about the translation suggestions in task 2 allowed them to feel more comfortable and confident when performing the task. However, task familiarity also appears in their answers as a prominent factor for their feeling of comfort in task 2, which prompts to further investigation on how pre-translation and the absence of translation metadata affect translators who are more familiar with post-editing than were the participants in the current experiment.

"New directions in empirical translation process research" (Novos rumos da pesquisa empírica sobre os processos tradutórios) persegue o ideal proposto por Jakobsen de que a utilização de “ferramentas computacionais, estatísticas e analíticas adequadas” (viii) permite produzir “descrições e previsões confiáveis sobre” o processo tradutório, “contribuindo para gerar um modelo da tradução humana baseada em processos” (ibid.). Tais descobertas teriam muitas possíveis aplicações; por exemplo, a adaptação das ferramentas aos estilos de traduzir ou a criação de tecnologias de assistência personalizadas, tais como sugestões terminológicas ativadas pelo movimento ocular. Apesar do carácter inconclusivo ou não generalizável de muitos dos estudos, o livro demonstra as vantagens de se criar um banco de dados para armazenar dados dos processos tradutórios, de modo que o esforço investido em um estudo possa ser aproveitado por outros pesquisadores, talvez enfocando outras variáveis ou utilizando diferentes métodos de análise.

New directions in empirical translation process research (TPR) pursues Jakobsen’s vision that by using “the appropriate computational, statistical, and other analytical tools” (viii) it is possible to produce “reliable descriptions of and predictions about” the translation process, which will “contribute to generating a process-oriented model of human translation” (ibid.). This could have many applications, e.g. adapting tools to translation styles or designing personalised assistive technologies, such as gaze-based terminological prompts. Even if many of the results are inconclusive or not generalizable, the book demonstrates the advantages of building a database for storing TPR data, so that the effort invested in a study can be leveraged by other researchers, perhaps looking at different variables and even using different methods of analysis.

Michael Cronin’s Translation in the Digital Age builds on some of the topics explored in Translation and Globalization, published ten years earlier. In the new book, Cronin touches upon every imaginable aspect of technologies and social phenomena in our times, such as online machine translation, the semantic web, smartphone applications, social networks, user-based crowdsourced translation, flashmobs, lolcat and Wikileaks. By bringing different areas of thought together, he explores the historical, social and cultural implications of technology for translation while at the same time suggesting how Translation Studies could contribute to our understanding of what is going on in society in the digital age.

Alice Leal's "Is the glass half empty or half full? Reflections on translation theory and practice in Brazil" is a thorough study of many different topics that are relevant to Translation Studies these days. The book addresses topics as diverse as translator professionalization (and the status of the profession), the goals of Translation Studies, the institutionalisation of Translation Studies, essentialism vs. non-essentialism, theory vs. practice, and translator training. It also goes beyond the scope of Translation Studies by touching on epistemological issues such as modern vs. post-modern theories, scientism, and the role of higher education.

CIUTI Conference 2018. Heriot-Watt University. Edinburgh, 30-31 May 2018.

Touch-based interactions are increasingly common for many people due to the widespread adoption of smartphones and other mobile devices. Likewise, voice commands and text input using automatic speech recognition (ASR) are becoming commonplace thanks to digital assistants on mobile devices and the Internet of Things. Although these technologies are becoming more and more ubiquitous, with much higher accuracy than ever before, mainstream translation tools are not taking advantage of them. We believe that voice and touch input could bring several benefits to translators in terms of ergonomics, usability, enjoyability, and satisfaction, with similar or improved productivity. In order to test this assumption, we have designed and developed a prototype translation tool that incorporates both technologies in addition to the more traditional keyboard and mouse. The tool runs on web browsers and has been developed to be modular, so that different connectors and APIs can be used to expand its features as well as to connect to alternative services, e.g. different machine translation (MT) engines and ASR systems. Building on the results of a previous usability study conducted in 2017 with an earlier version of the tool (Teixeira et al. 2017), we redesigned some of the features and now present the results of our second usability tests. We asked professional translators/post-editors to perform four different tasks using different interaction modes in the latest iteration of the tool. We then looked at quantitative performance indicators such as the amount of editing and the time on task, as well as qualitative satisfaction indicators. Preliminary results show a great improvement in usability as compared to the previous test round. In broad terms, the voice input is regarded as a useful feature by translators, while the touch interaction seems to be more suitable to specific use cases, such as when translators need to do quick edits and do not have a physical keyboard at hand. The ways that translators interact with the tool also depend on the type of edits they need to make, which in turn are related to the language combination and the type of MT being used (statistical vs. neural). The findings will provide relevant feedback for the next stages of the iterative development, where we will also assess the effectiveness of the accessibility features (Rodriguez Vazquez and Mileto 2016) that have been incorporated into the tool by conducting usability studies with blind and partially-sighted professional translators.

Escola de Tradução do Porto: "Tradução cibernética: em busca do equilíbrio entre seres humanos e máquinas". 25-29 de junho de 2018.

Esta apresentação faz um breve mapeamento das pesquisas mais recentes relacionadas à interação entre tradutores e ferramentas, do ponto de vista da ergonomia cognitiva e da usabilidade das interfaces. Como ilustração, foram utilizados alguns estudos desenvolvidos pelo próprio palestrante abordando os seguintes aspectos:
- uso de equipamentos e ferramentas no ambiente de trabalho de agências de tradução;
- estudos de usabilidade de uma nova ferramenta multimodal (com tela sensível ao toque e reconhecimento de fala) que está sendo desenvolvida na DCU.

IATIS 6th International Conference. Hong Kong. 3-6 July 2018.

This presentation reports on the results of a usability study of a browser-based translation and post-editing tool that accepts multiple input modes and follows web accessibility principles. The tool was conceived to be used by professional translators and will include standard features that are necessary for integrating translation memories (TM) and machine translation (MT) synergetically. Its distinctive features include the option of using touch and voice commands, in addition to the typical keyboard and mouse commands. Another important feature is the inclusion of accessibility principles from the outset, with the aim of improving translation editing for professionals with special needs.
In order to assess whether the usability and accessibility features included in the tool translate into improved performance and user experience, we have carried out experiments with a varied cohort of participants. The current presentation reports on the results of the accessibility tests with three blind translators. The results were analysed based on frustration reports, a usability & accessibility questionnaire and  an interview.
The findings will help understand how the possibilities offered by the newest technologies can improve the translation process from the user’s perspective and will inform future stages of the tool’s iterative development.