We use cookies to ensure that we give you the best experience on our website. By continuing to browse this repository, you give consent for essential cookies to be used. You can read more about our Privacy and Cookie Policy.

Durham Research Online
You are in:

MuLD: The Multitask Long Document Benchmark

Hudson, G Thomas and Al Moubayed, Noura (2022) 'MuLD: The Multitask Long Document Benchmark.', 13th Conference on Language Resources and Evaluation (LREC 2022) Marseille, France, 20-25 June 2022.


The impressive progress in NLP techniques has been driven by the development of multi-task benchmarks such as GLUE and SuperGLUE. While these benchmarks focus on tasks for one or two input sentences, there has been exciting work in designing efficient techniques for processing much longer inputs. In this paper, we present MuLD: a new long document benchmark consisting of only documents over 10,000 tokens. By modifying existing NLP tasks, we create a diverse benchmark which requires models to successfully model long-term dependencies in the text. We evaluate how existing models perform, and find that our benchmark is much more challenging than their ‘short document’ equivalents. Furthermore, by evaluating both regular and efficient transformers, we show that models with increased context length are better able to solve the tasks presented, suggesting that future improvements in these models are vital for solving similar long document problems. We release the data and code for baselines to encourage further research on efficient NLP models.

Item Type:Conference item (Paper)
Additional Information:ISBN: 9791095546726
Full text:(AM) Accepted Manuscript
Download PDF
Full text:(VoR) Version of Record
Available under License - Creative Commons Attribution Non-commercial 4.0.
Download PDF
Publisher Web site:
Date accepted:04 April 2022
Date deposited:27 April 2022
Date of first online publication:20 June 2022
Date first made open access:26 June 2022

Save or Share this output

Look up in GoogleScholar