Skip to main content

Research Repository

Advanced Search

Using Compressed Audio-visual Words for Multi-modal Scene Classification

Kurcius, J.J.; Breckon, T.P.

Using Compressed Audio-visual Words for Multi-modal Scene Classification Thumbnail


Authors

J.J. Kurcius



Abstract

We present a novel approach to scene classification using combined audio signal and video image features and compare this methodology to scene classification results using each modality in isolation. Each modality is represented using summary features, namely Mel-frequency Cepstral Coefficients (audio) and Scale Invariant Feature Transform (SIFT) (video) within a multi-resolution bag-of-features model. Uniquely, we extend the classical bag-of-words approach over both audio and video feature spaces, whereby we introduce the concept of compressive sensing as a novel methodology for multi-modal fusion via audio-visual feature dimensionality reduction. We perform evaluation over a range of environments showing performance that is both comparable to the state of the art (86%, over ten scene classes) and invariant to a ten-fold dimensionality reduction within the audio-visual feature space using our compressive representation approach.

Citation

Kurcius, J., & Breckon, T. (2014). Using Compressed Audio-visual Words for Multi-modal Scene Classification. In Proc. International Workshop on Computational Intelligence for Multimedia Understanding (1-5). https://doi.org/10.1109/IWCIM.2014.7008808

Conference Name Proc. International Workshop on Computational Intelligence for Multimedia Understanding
Publication Date 2014
Deposit Date Dec 9, 2014
Publicly Available Date Mar 29, 2024
Publisher Institute of Electrical and Electronics Engineers
Pages 1-5
Book Title Proc. International Workshop on Computational Intelligence for Multimedia Understanding
DOI https://doi.org/10.1109/IWCIM.2014.7008808
Keywords multi-resolution, bag of words, MFCC, compressed sensing, audio-visual, multi-modal, random projection matrix
Public URL https://durham-repository.worktribe.com/output/1153679
Publisher URL https://breckon.org/toby/publications/papers/kurcius14audiovisual.pdf

Files

Accepted Conference Proceeding (411 Kb)
PDF

Copyright Statement
© 2014 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.





You might also like



Downloadable Citations