Cookies

We use cookies to ensure that we give you the best experience on our website. By continuing to browse this repository, you give consent for essential cookies to be used. You can read more about our Privacy and Cookie Policy.


Durham Research Online
You are in:

The interaction effect between source text complexity and machine translation quality on the task difficulty of NMT post-editing from English to Chinese: A multi-method study

Jia, Yanfang and Zheng, Binghan (2022) 'The interaction effect between source text complexity and machine translation quality on the task difficulty of NMT post-editing from English to Chinese: A multi-method study.', Across Languages and Cultures, 23 (1). pp. 36-55.

Abstract

This study explores the interaction effect between source text (ST) complexity and machine translation (MT) quality on the task difficulty of neural machine translation (NMT) post-editing from English to Chinese. When investigating human effort exerted in post-editing, existing studies have seldom taken both ST complexity and MT quality levels into account, and have mainly focused on MT systems used before the emergence of NMT. Drawing on process and product data of post-editing from 60 trainee translators, this study adopted a multi-method approach to measure post-editing task difficulty, including eye-tracking, keystroke logging, quality evaluation, subjective rating, and retrospective written protocols. The results show that: 1) ST complexity and MT quality present a significant interaction effect on task difficulty of NMT post-editing; 2) ST complexity level has a positive impact on post-editing low-quality NMT (i.e., post-editing task becomes less difficult when ST complexity decreases); while for post-editing high-quality NMT, it has a positive impact only on the subjective ratings received from participants; and 3) NMT quality has a negative impact on its post-editing task difficulty (i.e., the post-editing task becomes less difficult when MT quality goes higher), and this impact becomes stronger when ST complexity increases. This paper concludes that both ST complexity and MT quality should be considered when testing post-editing difficulty, designing tasks for post-editor training, and setting fair post-editing pricing schemes.

Item Type:Article
Full text:(AM) Accepted Manuscript
Download PDF
(716Kb)
Status:Peer-reviewed
Publisher Web site:https://doi.org/10.1556/084.2022.00120
Publisher statement:This file is not the final published version of the paper.
Date accepted:11 October 2021
Date deposited:11 May 2022
Date of first online publication:09 May 2022
Date first made open access:11 May 2022

Save or Share this output

Export:
Export
Look up in GoogleScholar