A Genre-Based Quality Evaluation of Chinese-English Translation by Online Machine Translation Systems

by Qiong Yang
School of Foreign Studies, Tongji University, Shanghai 200092, China.

10.46679/9788196780593ch02

Yang, Q. (2024). A Genre-Based Quality Evaluation of Chinese-English Translation by Online Machine Translation Systems. In T. Chuanmao & D. Juntao, Translating the Future: Exploring the Impact of Technology and AI on Modern Translation Studies (pp 23-51). CSMFL Publications. https://dx.doi.org/10.46679/9788196780593ch02

Abstract

The past decades have witnessed remarkable progress in machine translation (MT) quality, sparking a heated debate within and beyond academia about whether human translators will be replaced by MT systems. This case study conducts a genre-based manual quality evaluation of Chinese-to-English translations produced by Google Translate, Baidu Translate, Sogou Translate, Youdao Translate, and human translators. The findings reveal that the quality of online machine translation output remains incomparable to that of human translation across expressive, informative, and vocative genres. Sogou Translate excels in expressive and informative genres but requires significant improvements in fidelity and comprehensibility for expressive genres, as well as in fidelity and genre function reproducibility for informative genres. For vocative genres, the quality of online machine translation varies across the four evaluation criteria and necessitates targeted enhancements. Notably, Google Translate, despite its reputation as a leading machine translation system, lags behind others in reproducing genre-specific functions and ensuring comprehensibility. The study suggests that machine translation developers should leverage advanced AI language technology to develop an AI-assisted MT training system capable of interpreting nuances for more sophisticated translations.

Keywords: Machine translation; genre; translation quality; manual evaluation; artificial intelligence language technology

This chapter is a part of: Translating the Future: Exploring the Impact of Technology and AI on Modern Translation Studies

© CSMFL Publications & its authors.
Published: November 12, 2024

References

  1. Abe, K., Matsubayashi, Y., Okazaki, N., & Inui, K. (2020) Multi-dialect Neural Machine Translation for 48 Low-resource Japanese Dialects. Journal of Natural Language Processing. https://doi.org/10.5715/jnlp.27.781
  2. Ahmed, M. A., Zaidan, B. B., Zaidan, A. A., Salih, M. M., & Lakulu, M. M. B. (2018) A review on systems-based sensory gloves for sign language recognition state of the art between 2007 and 2017. Sensors. https://doi.org/10.3390/s18072208
  3. Ali, M. A. (2020) Quality and Machine Translation: An Evaluation of Online Machine Translation of English into Arabic Genres. Open Journal of Modern Linguistics. https://doi.org/10.4236/ojml.2020.105030
  4. Alkhawaja, L., Ibrahim, H., Ghnaim, F., & Awwad, S. (2020) Neural Machine Translation: Fine-Grained Evaluation of Google Translate Output for English-to-Arabic Translation. International Journal of English Linguistics. https://doi.org/10.5539/ijel.v10n4p43
  5. Burchardt, A., Lommel, A., & Macketanz, V. (2020). A new deal for translation quality. Universal access in the information society, (7). https://doi.org/10.1007/s10209-020-00736-5
  6. Cao, R., Freitas, C., Chan, L., Sun, M., Jiang, H., & Chen, Z. (2017) ProLanGO: protein function prediction using neural machine translation based on a recurrent neural network. Molecules. https://doi.org/10.3390/molecules22101732
  7. Chatzikoumi, E. (2020). How to evaluate machine translation: A review of automated and human metrics. Natural Language Engineering, (2). https://doi.org/10.1017/S1351324919000469
  8. Dabre, R., Chu, CH. & Kunchukuttan, A. (2020). A Survey of Multilingual Neural Machine Translation. ACM Computing Surveys, (5). https://doi.org/10.1145/3406095
  9. Domingo, M. & Casacuberta, F. (2021) Two Demonstrations of the Machine Translation Applications to Historical Documents. arXiv e-prints.
  10. Fiederer, R. & O’Brien, S. (2009). Quality and machine translation: A realistic objective. The Journal of Specialized Translation, (11).
  11. Groves, M. & Mundt, K. (2021) A ghostwriter in the machine? Attitudes of academic staff towards machine translation use in internationalised Higher Education. Journal of English for Academic Purposes. https://doi.org/10.1016/j.jeap.2021.100957
  12. Guo, Y. (2020) Prospects for the teaching of translation majors in the new era. Babel-Amsterdam. https://doi.org/10.1075/babel.00179.guo
  13. Hutchins, J. & Somers, H. (1992). An Introduction to Machine Translation. London: Academic Press Limited.
  14. Kim, S. (2019) Playing with machine translation in language classroom: Affordances and constraints. Multimedia-Assisted Language Learning.
  15. King, M. (1997). Evaluating translation[A]. In Hauenschild, C. & Heizmann, S. (eds.) Machine Translation and Translation Theory. Berlin & New York: Mouton de Gruyter.
  16. Korolkova, S. & Novozhilova, A. (2021) Efficiency of Machine Translation in Urban Discourse. Vestnik Volgogradskogo gosudarstvennogo universiteta Serija 2 Jazykoznanije.
  17. Li, H. & Chen, H. (2019) Human vs. AI: An Assessment of the Translation Quality Between Translators and Machine Translation. International Journal of Translation Interpretation and Applied Linguistics.
  18. Li, J., Wang, X., Yang, B., Shi, S., & Tu, Z. (2020) Neuron Interaction Based Representation Composition for Neural Machine Translation. Proceedings of the AAAI Conference on Artificial Intelligence. https://doi.org/10.1609/aaai.v34i05.6334
  19. Li, X. & Hao, X. (2021) English Machine Translation Model Based on Artificial Intelligence. Journal of Physics: Conference Series. https://doi.org/10.1088/1742-6596/1982/1/012098
  20. Li, Y., Li, X., Yang, Y., & Dong, R. (2020) A Diverse Data Augmentation Strategy for Low-Resource Neural Machine Translation. Information (Switzerland). https://doi.org/10.3390/info11050255
  21. Macken, L., Prou, D., & Tezcan, A. (2020) Quantifying the Effect of Machine Translation in a High-Quality Human Translation Production Process. Informatics. https://doi.org/10.3390/informatics7020012
  22. Mendona, F. V., Catalo-Lopes, M., Marinho, R. T., & Figueira, J. R. (2020) Improving medical decision-making with a management science game theory approach to liver transplantation. Omega. https://doi.org/10.1016/j.omega.2019.03.008
  23. Olkhovska, A. & Frolova, I. (2020) Using Machine Translation Engines in The Classroom: A Survey of Translation Students’ Performance. Advanced Education. https://doi.org/10.20535/2410-8286.197812
  24. Pandey, V., Padmavati, D., & Kumar, R. (2021) Hindi Chhattisgarhi Machine Translation System Using Statistical Approach. Webology. https://doi.org/10.14704/WEB/V18SI02/WEB18067
  25. Popel, M., Tomkova, M., Tomek, J., Kaiser, Ł., Uszkoreit, J., Bojar, O., & Žabokrtský, Z. (2020) Transforming machine translation: a deep learning system reaches news translation quality comparable to human professionals. Nature communications. https://doi.org/10.1038/s41467-020-18073-9
  26. Prates, M. O. R., Avelar, P. H., Lamb, L. C. (2020) Assessing gender bias in machine translation: a case study with Google Translate. Neural Comput Appl. https://doi.org/10.1007/s00521-019-04144-6
  27. Qin, Y. (2018). An Analytical Study of Neural Network Machine Translation and Its Impacts on Translation Teaching. Technology Enhanced Foreign Language Education, (4).
  28. Reiss, K. (2000). Translation Criticism-potentials and Limitations: Categories and Criteria for Translation Quality Assessment. (Translated by Erroll, R.) Manchester: St. Jerome Publishing.
  29. Sanders, G. Przybocki, M. (2011). Madnani, N. & Snover, M. Human subjective judgments[A]. In Joseph, O. Caitlin, C. & John, M. (eds.) Handbook of Natural Language Processing and Machine Translation. New York: Springer.
  30. Si, J., Song, Dawo & Li, Y. (2016). Google Translate in the era of Big Data: Actuality and Challenge. Statistical Research, (5).
  31. Siddique, S., Ahmed, T., Talukder, M., & Uddin, M. M. (2020) English to Bangla Machine Translation Using Recurrent Neural Network. International Journal of Future Computer and Communication.
  32. Slype, G. (1979). Critical study of methods for evaluating the quality of machine translation. Prepared for the Commission of the European Communities[R]. Brussels: Bureau Marcel van Dijk.
  33. Sun, H., Wang, R., Utiyama, M., Marie, B., & Zhao, T. (2021) Unsupervised Neural Machine Translation for Similar and Distant Language Pairs: An Empirical Study. ACM Transactions on Asian and Low-Resource Language Information Processing. https://doi.org/10.1145/3418059
  34. Ye, Y. (2021) Translation Mechanism of Neural Machine Algorithm for Online English Resources. Complexity. https://doi.org/10.1155/2021/5564705
  35. Zhang, T. (2020) Recognition and Segmentation of English Long and Short Sentences Based on Machine Translation. International Journal of Emerging Technologies in Learning (iJET). https://doi.org/10.3991/ijet.v14i19.10182

[email protected]

Follow us @