Jia, Yanfang and Zheng, Binghan (2022) 'The interaction effect between source text complexity and machine translation quality on the task difficulty of NMT post-editing from English to Chinese: A multi-method study.', Across Languages and Cultures, 23 (1). pp. 36-55.
This study explores the interaction effect between source text (ST) complexity and machine translation (MT) quality on the task difficulty of neural machine translation (NMT) post-editing from English to Chinese. When investigating human effort exerted in post-editing, existing studies have seldom taken both ST complexity and MT quality levels into account, and have mainly focused on MT systems used before the emergence of NMT. Drawing on process and product data of post-editing from 60 trainee translators, this study adopted a multi-method approach to measure post-editing task difficulty, including eye-tracking, keystroke logging, quality evaluation, subjective rating, and retrospective written protocols. The results show that: 1) ST complexity and MT quality present a significant interaction effect on task difficulty of NMT post-editing; 2) ST complexity level has a positive impact on post-editing low-quality NMT (i.e., post-editing task becomes less difficult when ST complexity decreases); while for post-editing high-quality NMT, it has a positive impact only on the subjective ratings received from participants; and 3) NMT quality has a negative impact on its post-editing task difficulty (i.e., the post-editing task becomes less difficult when MT quality goes higher), and this impact becomes stronger when ST complexity increases. This paper concludes that both ST complexity and MT quality should be considered when testing post-editing difficulty, designing tasks for post-editor training, and setting fair post-editing pricing schemes.
|Full text:||(AM) Accepted Manuscript|
Download PDF (716Kb)
|Publisher Web site:||https://doi.org/10.1556/084.2022.00120|
|Publisher statement:||This file is not the final published version of the paper.|
|Date accepted:||11 October 2021|
|Date deposited:||11 May 2022|
|Date of first online publication:||09 May 2022|
|Date first made open access:||11 May 2022|
Save or Share this output
|Look up in GoogleScholar|