Skip to main content

Research Repository

Advanced Search

The interaction effect between source text complexity and machine translation quality on the task difficulty of NMT post-editing from English to Chinese: A multi-method study

Jia, Yanfang; Zheng, Binghan

The interaction effect between source text complexity and machine translation quality on the task difficulty of NMT post-editing from English to Chinese: A multi-method study Thumbnail


Authors

Yanfang Jia



Abstract

This study explores the interaction effect between source text (ST) complexity and machine translation (MT) quality on the task difficulty of neural machine translation (NMT) post-editing from English to Chinese. When investigating human effort exerted in post-editing, existing studies have seldom taken both ST complexity and MT quality levels into account, and have mainly focused on MT systems used before the emergence of NMT. Drawing on process and product data of post-editing from 60 trainee translators, this study adopted a multi-method approach to measure post-editing task difficulty, including eye-tracking, keystroke logging, quality evaluation, subjective rating, and retrospective written protocols. The results show that: 1) ST complexity and MT quality present a significant interaction effect on task difficulty of NMT post-editing; 2) ST complexity level has a positive impact on post-editing low-quality NMT (i.e., post-editing task becomes less difficult when ST complexity decreases); while for post-editing high-quality NMT, it has a positive impact only on the subjective ratings received from participants; and 3) NMT quality has a negative impact on its post-editing task difficulty (i.e., the post-editing task becomes less difficult when MT quality goes higher), and this impact becomes stronger when ST complexity increases. This paper concludes that both ST complexity and MT quality should be considered when testing post-editing difficulty, designing tasks for post-editor training, and setting fair post-editing pricing schemes.

Citation

Jia, Y., & Zheng, B. (2022). The interaction effect between source text complexity and machine translation quality on the task difficulty of NMT post-editing from English to Chinese: A multi-method study. Across Languages and Cultures, 23(1), 36-55. https://doi.org/10.1556/084.2022.00120

Journal Article Type Article
Acceptance Date Oct 11, 2021
Online Publication Date May 9, 2022
Publication Date 2022-06
Deposit Date May 11, 2022
Publicly Available Date Mar 28, 2024
Journal Across Languages and Cultures
Print ISSN 1585-1923
Electronic ISSN 1588-2519
Publisher Akadémiai Kiadó
Peer Reviewed Peer Reviewed
Volume 23
Issue 1
Pages 36-55
DOI https://doi.org/10.1556/084.2022.00120

Files

Accepted Journal Article (733 Kb)
PDF

Copyright Statement
This file is not the final published version of the paper.





You might also like



Downloadable Citations