Skip to main content

Research Repository

Advanced Search

CAM: A Combined Attention Model for Natural Language Inference

Gajbhiye, Amit; Jaf, Sardar; Al-Moubayed, Noura; Bradley, Steven; McGough, A. Stephen

CAM: A Combined Attention Model for Natural Language Inference Thumbnail


Authors

Amit Gajbhiye

Sardar Jaf

A. Stephen McGough



Contributors

Naoki Abe
Editor

Huan Liu
Editor

Calton Pu
Editor

Xiaohua Hu
Editor

Nesreen Ahmed
Editor

Mu Qiao
Editor

Donald Kossmann
Editor

Bing Liu
Editor

Kisung Lee
Editor

Jiliang Tang
Editor

Jingrui He
Editor

Jeffrey Saltz
Editor

Abstract

Natural Language Inference (NLI) is a fundamental step towards natural language understanding. The task aims to detect whether a premise entails or contradicts a given hypothesis. NLI contributes to a wide range of natural language understanding applications such as question answering, text summarization and information extraction. Recently, the public availability of big datasets such as Stanford Natural Language Inference (SNLI) and SciTail, has made it feasible to train complex neural NLI models. Particularly, Bidirectional Long Short-Term Memory networks (BiLSTMs) with attention mechanisms have shown promising performance for NLI. In this paper, we propose a Combined Attention Model (CAM) for NLI. CAM combines the two attention mechanisms: intraattention and inter-attention. The model first captures the semantics of the individual input premise and hypothesis with intra-attention and then aligns the premise and hypothesis with inter-sentence attention. We evaluate CAM on two benchmark datasets: Stanford Natural Language Inference (SNLI) and SciTail, achieving 86.14% accuracy on SNLI and 77.23% on SciTail. Further, to investigate the effectiveness of individual attention mechanism and in combination with each other, we present an analysis showing that the intra- and inter-attention mechanisms achieve higher accuracy when they are combined together than when they are independently used.

Citation

Gajbhiye, A., Jaf, S., Al-Moubayed, N., Bradley, S., & McGough, A. S. (2018). CAM: A Combined Attention Model for Natural Language Inference. In N. Abe, H. Liu, C. Pu, X. Hu, N. Ahmed, M. Qiao, …J. Saltz (Eds.), 2018 IEEE International Conference on Big Data (Big Data) ; proceedings (1009-1014). https://doi.org/10.1109/bigdata.2018.8622057

Conference Name IEEE International Conference on Big Data.
Conference Location Seattle, WA, USA
Start Date Dec 10, 2018
End Date Dec 13, 2018
Acceptance Date Oct 16, 2018
Online Publication Date Dec 10, 2018
Publication Date Dec 10, 2018
Deposit Date Nov 14, 2018
Publicly Available Date Mar 28, 2024
Publisher Institute of Electrical and Electronics Engineers
Pages 1009-1014
Book Title 2018 IEEE International Conference on Big Data (Big Data) ; proceedings.
DOI https://doi.org/10.1109/bigdata.2018.8622057
Public URL https://durham-repository.worktribe.com/output/1143597

Files

Accepted Conference Proceeding (473 Kb)
PDF

Copyright Statement
© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.





You might also like



Downloadable Citations