Thomas Hudson g.t.hudson@durham.ac.uk
Post Doctoral Research Associate
MuLD: The Multitask Long Document Benchmark
Hudson, G Thomas; Al Moubayed, Noura
Authors
Dr Noura Al Moubayed noura.al-moubayed@durham.ac.uk
Associate Professor
Contributors
Nicoletta Calzolari
Editor
Frederic Bechet
Editor
Philippe Blache
Editor
Khalid Choukri
Editor
Christopher Cieri
Editor
Thierry Declerck
Editor
Sara Goggi
Editor
Hitoshi Isahara
Editor
Bente Maegaard
Editor
Joseph Mariani
Editor
Helene Mazo
Editor
Jan Odijk
Editor
Stelios Piperidis
Editor
Abstract
The impressive progress in NLP techniques has been driven by the development of multi-task benchmarks such as GLUE and SuperGLUE. While these benchmarks focus on tasks for one or two input sentences, there has been exciting work in designing efficient techniques for processing much longer inputs. In this paper, we present MuLD: a new long document benchmark consisting of only documents over 10,000 tokens. By modifying existing NLP tasks, we create a diverse benchmark which requires models to successfully model long-term dependencies in the text. We evaluate how existing models perform, and find that our benchmark is much more challenging than their ‘short document’ equivalents. Furthermore, by evaluating both regular and efficient transformers, we show that models with increased context length are better able to solve the tasks presented, suggesting that future improvements in these models are vital for solving similar long document problems. We release the data and code for baselines to encourage further research on efficient NLP models.
Citation
Hudson, G. T., & Al Moubayed, N. (2022). MuLD: The Multitask Long Document Benchmark. In N. Calzolari, F. Bechet, P. Blache, K. Choukri, C. Cieri, T. Declerck, …S. Piperidis (Eds.),
Conference Name | 13th Conference on Language Resources and Evaluation (LREC 2022) |
---|---|
Conference Location | Marseille, France |
Start Date | Jun 20, 2022 |
End Date | Jun 25, 2022 |
Acceptance Date | Apr 4, 2022 |
Online Publication Date | Jun 20, 2022 |
Publication Date | 2022 |
Deposit Date | Apr 26, 2022 |
Publicly Available Date | Jun 26, 2022 |
Pages | 3675-3685 |
Public URL | https://durham-repository.worktribe.com/output/1138178 |
Publisher URL | http://www.lrec-conf.org/proceedings/lrec2022/index.html |
Related Public URLs | https://arxiv.org/pdf/2202.07362.pdf |
Files
Accepted Conference Proceeding
(197 Kb)
PDF
Published Conference Proceeding
(267 Kb)
PDF
Publisher Licence URL
http://creativecommons.org/licenses/by-nc/4.0/
You might also like
Is Unimodal Bias Always Bad for Visual Question Answering? A Medical Domain Study with Dynamic Attention
(2022)
Conference Proceeding
Towards Graph Representation Learning Based Surgical Workflow Anticipation
(2022)
Conference Proceeding
Does lossy image compression affect racial bias within face recognition?
(2022)
Conference Proceeding
Downloadable Citations
About Durham Research Online (DRO)
Administrator e-mail: dro.admin@durham.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search