research-article

Evaluation of English to Arabic Machine Translation Systems using BLEU and GTM

Authors:

Aljoharah Al-Rukban,

Abdul Khader Jilani SaudagarAuthors Info & Claims

ICETC '17: Proceedings of the 9th International Conference on Education Technology and Computers

Pages 228 - 232

https://doi.org/10.1145/3175536.3175570

Published: 20 December 2017 Publication History

Get Access

Abstract

The aim of this research study is to compare the effectiveness of three systems: Google Translator, Bing Translator and Golden Alwafi that are used to translate the corpus sentences from English language to Arabic language and then evaluate these sentences using two automatic evaluation methods viz., BLEU (Bilingual Evaluation Understudy) which is one of the most popular evaluation method and GTM (General Text Matcher) which is based on Recall, Precision, and F-measure. The scores obtained from the evaluation methods determine which one of these translation systems provides better translation. Higher the score means better translation and highly correlate with human translation. The results of this research study have revealed that Golden Alwafi achieves highest accuracy using BLEU and Google Translator attains highest accuracy with GTM method.

References

[1]

Al-Kabi, M., Hailat, T., Alsmadi, I. and Al-Shawakfa, E. 2013. Evaluating English to Arabic Machine Translation Using BLEU. International Journal of Advanced Computer Science and Applications. 4, 1 (2013), 66--73.

Google Scholar

[2]

Bassnett, S. 2014. Translation studies. Routledge.

Google Scholar

[3]

Gupta, V., Joshi, N. and Mathur, I. 2013. Subjective and Objective Evaluation of English to Urdu Machine Translation. In Proceedings of the International Conference on Advances in Computing, Communications and Informatics (Mysore, India, August 22-25, 2013). ICACCI' 13. IEEE, India, 1520--1525.

Google Scholar

[4]

Hailat, T., Al-Kabi, M., Alsmadi, I. and Al-Shawakfa, E. 2013. Evaluating English To Arabic Machine Translators. In Proceedings of the International Conference on Applied Electrical Engineering and Computing Technologies (Amman, Jordan, December 3-5, 2013). AEECT'13. IEEE, Jordan.

Google Scholar

[5]

Kalyani, A., Kumud, H., Singh, S.P., Kumar, A. and Darbari5, H. 2014. Evaluation and Ranking of Machine Translated Output in Hindi Language using Precision and Recall Oriented Metrics. International Journal of Advanced Computer Research. 4, 14 (Mar. 2014), 54--59.

Google Scholar

[6]

Most Widely spoken Languages of the World. http://www.nationsonline.org/oneworld/most_spoken_languages.htm. Accessed: 2017-03-26.

Google Scholar

[7]

Papineni, K., Roukos, S., Ward, T. and Zhu, W.-J. 2002. BLEU: A Method for Automatic Evaluation of Machine Translation. In Proceeding of the 40th Annual Meeting on Association for Computational Linguistics (Philadelphia, Pennsylvania, July 07-12, 2002, 2002). ACL'02. ACM, Stroudsburg, PA, USA, 311--318.

Digital Library

Google Scholar

[8]

Qin, Y., Wen, Q. and Wang, J. 2009. Automatic Evaluation of Translation Quality Using Expanded N-gram Co-occurrence. In Proceedings of the Natural Language Processing and Knowledge Engineering (Dalian, China, September 24-27, 2009). NLP-KE 2009. IEEE, China.

Google Scholar

[9]

Saikh, T., Naskar, S.K., Giri, C. and Bandyopadhyay, S. 2015. Textual Entailment Using Different Similarity Metrics. In Proceedings of the International Conference on Intelligent Text Processing and Computational Linguistics (Cairo, Egypt, April 14-20, 2015). CICling 2015. Springer, 491--501.

Google Scholar

[10]

"Science and Technology" Exerts Strenuous Efforts to Support and Enrich the Arabic Content on the Internet: 2012. https://www.kacst.edu.sa/eng/about/news/Pages/news4011216-2784.aspx. Accessed: 2016-11-30.

Google Scholar

[11]

Trujillo, A. 1999. Translation engines: techniques for machine translation. Springer.

Google Scholar

[12]

Turian, J.P., Shen, L. and Melamed, I.D. 2006. Evaluation of Machine Translation and its Evaluation. Report. Defense Technical Information Center. http://http://www.dtic.mil/get-tr-doc/pdf?AD=ADA453509. Accessed: 2017-03-20

Google Scholar

Cited By

View all

Naveen RKumar A(2024)Analysis of Neural Machine Translation for English to Hindi using Long Short-Term Memory Model and Transformer Model2024 4th International Conference on Sustainable Expert Systems (ICSES)10.1109/ICSES63445.2024.10763246(515-520)Online publication date: 15-Oct-2024
https://doi.org/10.1109/ICSES63445.2024.10763246
Al-Khalifa HAl-Khalefah KHaroon H(2024)Error Analysis of Pretrained Language Models (PLMs) in English-to-Arabic Machine TranslationHuman-Centric Intelligent Systems10.1007/s44230-024-00061-7Online publication date: 5-Feb-2024
https://doi.org/10.1007/s44230-024-00061-7
Calvo-Ferrer J(2023)Can you tell the difference? A study of human vs machine-translated subtitlesPerspectives10.1080/0907676X.2023.2268149(1-18)Online publication date: 12-Oct-2023
https://doi.org/10.1080/0907676X.2023.2268149
Show More Cited By

Index Terms

Evaluation of English to Arabic Machine Translation Systems using BLEU and GTM
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Machine translation
2. General and reference
  1. Cross-computing tools and techniques
    1. Evaluation

Recommendations

Discriminative Phrase-Based Models for Arabic Machine Translation

A design for an Arabic-to-English translation system is presented. The core of the system implements a standard phrase-based statistical machine translation architecture, but it is extended by incorporating a local discriminative phrase selection model ...
Inter-, Intra-, and Extra-Chunk Pre-Ordering for Statistical Japanese-to-English Machine Translation

A rule-based pre-ordering approach is proposed for statistical Japanese-to-English machine translation using the dependency structure of source-side sentences. A Japanese sentence is pre-ordered to an English-like order at the morpheme level for a ...
Machine Translation Errors: English and Iraqi Arabic

Errors in machine translations of English-Iraqi Arabic dialogues were analyzed using the methods developed for the Human Translation Error Rate measure (HTER). Human annotations were used to refine the Translation Error Rate (TER) annotations. The ...

Comments

Information & Contributors

Information

Published In

ICETC '17: Proceedings of the 9th International Conference on Education Technology and Computers

December 2017

270 pages

ISBN:9781450354356

DOI:10.1145/3175536

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 December 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ICETC 2017

ICETC 2017: 2017 9th International Conference on Education Technology and Computers

December 20 - 22, 2017

Barcelona, Spain

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
143
Total Downloads

Downloads (Last 12 months)20
Downloads (Last 6 weeks)2

Reflects downloads up to 15 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Naveen RKumar A(2024)Analysis of Neural Machine Translation for English to Hindi using Long Short-Term Memory Model and Transformer Model2024 4th International Conference on Sustainable Expert Systems (ICSES)10.1109/ICSES63445.2024.10763246(515-520)Online publication date: 15-Oct-2024
https://doi.org/10.1109/ICSES63445.2024.10763246
Al-Khalifa HAl-Khalefah KHaroon H(2024)Error Analysis of Pretrained Language Models (PLMs) in English-to-Arabic Machine TranslationHuman-Centric Intelligent Systems10.1007/s44230-024-00061-7Online publication date: 5-Feb-2024
https://doi.org/10.1007/s44230-024-00061-7
Calvo-Ferrer J(2023)Can you tell the difference? A study of human vs machine-translated subtitlesPerspectives10.1080/0907676X.2023.2268149(1-18)Online publication date: 12-Oct-2023
https://doi.org/10.1080/0907676X.2023.2268149
Li LTayir T(2021)Multimodal Machine Translation Enhancement by Fusing Multimodal-attention and Fine-grained Image Features2021 IEEE 4th International Conference on Multimedia Information Processing and Retrieval (MIPR)10.1109/MIPR51284.2021.00050(267-272)Online publication date: Sep-2021
https://doi.org/10.1109/MIPR51284.2021.00050
Zakraoui JSaleh MAl-Maadeed SAlja'am J(2021)Arabic Machine Translation: A Survey With Challenges and Future DirectionsIEEE Access10.1109/ACCESS.2021.31324889(161445-161468)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3132488
Han YLi LZhang JGurrin CÞór Jónsson BKando NSchoeffmann KChen PO'Connor N(2020)A Coordinated Representation Learning Enhanced Multimodal Machine Translation Approach with Multi-AttentionProceedings of the 2020 International Conference on Multimedia Retrieval10.1145/3372278.3390717(571-577)Online publication date: 8-Jun-2020
https://dl.acm.org/doi/10.1145/3372278.3390717
Tan LLi LHan YLi DHu KZhou DWang P(2020)An Empirical Study on Ensemble Learning of Multimodal Machine Translation2020 IEEE Sixth International Conference on Multimedia Big Data (BigMM)10.1109/BigMM50055.2020.00019(63-69)Online publication date: Sep-2020
https://doi.org/10.1109/BigMM50055.2020.00019

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Discriminative Phrase-Based Models for Arabic Machine Translation

Inter-, Intra-, and Extra-Chunk Pre-Ordering for Statistical Japanese-to-English Machine Translation

Machine Translation Errors: English and Iraqi Arabic

Comments

Published In

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Discriminative Phrase-Based Models for Arabic Machine Translation

Inter-, Intra-, and Extra-Chunk Pre-Ordering for Statistical Japanese-to-English Machine Translation

Machine Translation Errors: English and Iraqi Arabic

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations