Skip to main content

Research Repository

Advanced Search

Social Behavioral Phenotyping of Drosophila with a 2D-3D Hybrid CNN Framework

Jiang, Ziping; Chazot, Paul L.; Celebi, M. Emre; Crookes, Danny; Jiang, Richard

Social Behavioral Phenotyping of Drosophila with a 2D-3D Hybrid CNN Framework Thumbnail


Authors

Ziping Jiang

M. Emre Celebi

Danny Crookes

Richard Jiang



Abstract

Behavioural phenotyping of drosphila is an important means in biological and medical research to identify genetic, pathologic or psychologic impact on animal behviour. Automated behavioural phenotyping from videos has been a desired capability that can waive long-time boring manual work in behavioral analysis. In this paper, we introduced deep learning into this challenging topic, and proposed a new 2D+3D hybrid CNN framework for drosphila’s social behavioural phenotyping. In the proposed multitask learning framework, action detection and localization of drosphila jointly is carried out with action classification, and a given video is divided into clips with fixed length. Each clip is fed into the system and a 2-D CNN is applied to extract features at frame level. Features extracted from adjacent frames are then connected and fed into a 3-D CNN with a spatial region proposal layer for classification. In such a 2D+3D hybrid framework, drosophila detection at the frame level enables the action analysis at different durations instead of a fixed period. We tested our framework with different base layers and classification architectures and validated the proposed 3D CNN based social behavioral phenotyping framework under various models, detectors and classifiers.

Citation

Jiang, Z., Chazot, P. L., Celebi, M. E., Crookes, D., & Jiang, R. (2019). Social Behavioral Phenotyping of Drosophila with a 2D-3D Hybrid CNN Framework. IEEE Access, 7, 67972-67982. https://doi.org/10.1109/access.2019.2917000

Journal Article Type Article
Acceptance Date Apr 8, 2019
Online Publication Date May 15, 2019
Publication Date 2019
Deposit Date May 7, 2019
Publicly Available Date Mar 28, 2024
Journal IEEE Access
Publisher Institute of Electrical and Electronics Engineers
Peer Reviewed Peer Reviewed
Volume 7
Pages 67972-67982
DOI https://doi.org/10.1109/access.2019.2917000

Files







You might also like



Downloadable Citations