Skip to main content

Research Repository

Advanced Search

ExBERT: An External Knowledge Enhanced BERT for Natural Language Inference

Gajbhiye, Amit; Al Moubayed, Noura; Bradley, Steven

ExBERT: An External Knowledge Enhanced BERT for Natural Language Inference Thumbnail


Authors

Amit Gajbhiye



Contributors

Igor Farkaš
Editor

Paolo Masulli
Editor

Sebastian Otte
Editor

Stefan Wermter
Editor

Abstract

Neural language representation models such as BERT, pretrained on large-scale unstructured corpora lack explicit grounding to real-world commonsense knowledge and are often unable to remember facts required for reasoning and inference. Natural Language Inference (NLI) is a challenging reasoning task that relies on common human understanding of language and real-world commonsense knowledge. We introduce a new model for NLI called External Knowledge Enhanced BERT (ExBERT), to enrich the contextual representation with realworld commonsense knowledge from external knowledge sources and enhance BERT’s language understanding and reasoning capabilities. ExBERT takes full advantage of contextual word representations obtained from BERT and employs them to retrieve relevant external knowledge from knowledge graphs and to encode the retrieved external knowledge. Our model adaptively incorporates the external knowledge context required for reasoning over the inputs. Extensive experiments on the challenging SciTail and SNLI benchmarks demonstrate the effectiveness of ExBERT: in comparison to the previous state-of-the-art, we obtain an accuracy of 95.9% on SciTail and 91.5% on SNLI.

Citation

Gajbhiye, A., Al Moubayed, N., & Bradley, S. (2021). ExBERT: An External Knowledge Enhanced BERT for Natural Language Inference. In I. Farkaš, P. Masulli, S. Otte, & S. Wermter (Eds.), Artificial Neural Networks and Machine Learning – ICANN 2021 30th International Conference on Artificial Neural Networks, Bratislava, Slovakia, September 14–17, 2021, Proceedings, Part V (460-472). Springer Verlag. https://doi.org/10.1007/978-3-030-86383-8_37

Acceptance Date Jul 1, 2021
Online Publication Date Sep 7, 2021
Publication Date 2021
Deposit Date Jul 20, 2021
Publicly Available Date Sep 18, 2021
Publisher Springer Verlag
Pages 460-472
Series Title Theoretical Computer Science and General Issues
Series Number 12895
Book Title Artificial Neural Networks and Machine Learning – ICANN 2021 30th International Conference on Artificial Neural Networks, Bratislava, Slovakia, September 14–17, 2021, Proceedings, Part V
ISBN 9783030863838
DOI https://doi.org/10.1007/978-3-030-86383-8_37

Files





You might also like



Downloadable Citations