Cookies

We use cookies to ensure that we give you the best experience on our website. By continuing to browse this repository, you give consent for essential cookies to be used. You can read more about our Privacy and Cookie Policy.


Durham Research Online
You are in:

Social behavioral phenotyping of Drosophila with a 2D-3D hybrid CNN framework.

Jiang, Ziping and Chazot, Paul L. and Celebi, M. Emre and Crookes, Danny and Jiang, Richard (2019) 'Social behavioral phenotyping of Drosophila with a 2D-3D hybrid CNN framework.', IEEE access., 7 . pp. 67972-67982.

Abstract

Behavioural phenotyping of drosphila is an important means in biological and medical research to identify genetic, pathologic or psychologic impact on animal behviour. Automated behavioural phenotyping from videos has been a desired capability that can waive long-time boring manual work in behavioral analysis. In this paper, we introduced deep learning into this challenging topic, and proposed a new 2D+3D hybrid CNN framework for drosphila’s social behavioural phenotyping. In the proposed multitask learning framework, action detection and localization of drosphila jointly is carried out with action classification, and a given video is divided into clips with fixed length. Each clip is fed into the system and a 2-D CNN is applied to extract features at frame level. Features extracted from adjacent frames are then connected and fed into a 3-D CNN with a spatial region proposal layer for classification. In such a 2D+3D hybrid framework, drosophila detection at the frame level enables the action analysis at different durations instead of a fixed period. We tested our framework with different base layers and classification architectures and validated the proposed 3D CNN based social behavioral phenotyping framework under various models, detectors and classifiers.

Item Type:Article
Full text:Publisher-imposed embargo
(AM) Accepted Manuscript
File format - PDF
(887Kb)
Full text:(VoR) Version of Record
Available under License - Creative Commons Attribution.
Download PDF (Advance online version)
(910Kb)
Full text:(VoR) Version of Record
Available under License - Creative Commons Attribution.
Download PDF
(5330Kb)
Status:Peer-reviewed
Publisher Web site:https://doi.org/10.1109/ACCESS.2019.2917000
Publisher statement:This work is licensed under a Creative Commons Attribution 3.0 License. For more information, see http://creativecommons.org/licenses/by/3.0/.
Date accepted:08 April 2019
Date deposited:07 May 2019
Date of first online publication:15 May 2019
Date first made open access:No date available

Save or Share this output

Export:
Export
Look up in GoogleScholar