Skip to main content

Research Repository

Advanced Search

Recognising Human-Object Interactions Using Attention-based LSTMs

Almushyti, Muna; Li, Frederick W.B.

Authors

Muna Almushyti



Contributors

Franck P. Vidal
Editor

Gary K. L. Tam
Editor

Jonathan C. Roberts
Editor

Abstract

Recognising Human-object interactions (HOIs) in videos is a challenge task especially when a human can interact with multiple objects. This paper attempts to solve the problem of HOIs by proposing a hierarchical framework that analyzes human-object interactions from a video sequence. The framework consists of LSTMs that firstly capture both human motion and temporal object information independently, followed by fusing these information through a bilinear layer to aggregate human-object features, which are then fed to a global deep LSTM to learn high-level information of HOIs. The proposed approach applies an attention mechanism to LSTMs in order to focus on important parts of human and object temporal information.

Citation

Almushyti, M., & Li, F. W. (2019). Recognising Human-Object Interactions Using Attention-based LSTMs. In F. P. Vidal, G. K. . L. Tam, & J. C. Roberts (Eds.), Computer Graphics and Visual Computing (CGVC) (135-139). https://doi.org/10.2312/cgvc.20191269

Conference Name Computer Graphics and Visual Computing (CGVC)
Conference Location Bangor University, United Kingdom
Acceptance Date Jul 22, 2019
Online Publication Date Sep 12, 2019
Publication Date Sep 12, 2019
Deposit Date Oct 30, 2019
Publicly Available Date Mar 28, 2024
Pages 135-139
Book Title Computer Graphics and Visual Computing (CGVC).
DOI https://doi.org/10.2312/cgvc.20191269
Public URL https://durham-repository.worktribe.com/output/1141631
Related Public URLs https://diglib.eg.org/handle/10.2312/cgvc20191269