Francis Xiatian Zhang xiatian.zhang@durham.ac.uk
Demonstrator (Ptt)
Towards Graph Representation Learning Based Surgical Workflow Anticipation
Zhang, Xiatian; Al Moubayed, Noura; Shum, Hubert P.H.
Authors
Dr Noura Al Moubayed noura.al-moubayed@durham.ac.uk
Associate Professor
Dr Hubert Shum hubert.shum@durham.ac.uk
Associate Professor
Abstract
Surgical workflow anticipation can give predictions on what steps to conduct or what instruments to use next, which is an essential part of the computer-assisted intervention system for surgery, e.g. workflow reasoning in robotic surgery. However, current approaches are limited to their insufficient expressive power for relationships between instruments. Hence, we propose a graph representation learning framework to comprehensively represent instrument motions in the surgical workflow anticipation problem. In our proposed graph representation, we maps the bounding box information of instruments to the graph nodes in the consecutive frames and build inter-frame/inter-instrument graph edges to represent the trajectory and interaction of the instruments over time. This design enhances the ability of our network on modeling both the spatial and temporal patterns of surgical instruments and their interactions. In addition, we design a multi-horizon learning strategy to balance the understanding of various horizons indifferent anticipation tasks, which significantly improves the model performance in anticipation with various horizons. Experiments on the Cholec80 dataset demonstrate the performance of our proposed method can exceed the state-of-the-art method based on richer backbones, especially in instrument anticipation (1.27 v.s. 1.48 for inMAE; 1.48 v.s. 2.68 for eMAE). To the best of our knowledge, we are the first to introduce a spatial-temporal graph representation into surgical workflow anticipation.
Citation
Zhang, X., Al Moubayed, N., & Shum, H. P. (2022). Towards Graph Representation Learning Based Surgical Workflow Anticipation. . https://doi.org/10.1109/bhi56158.2022.9926801
Conference Name | 2022 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI) |
---|---|
Conference Location | Ioannina, Greece |
Start Date | Sep 27, 2022 |
End Date | Sep 30, 2022 |
Acceptance Date | Jul 19, 2022 |
Online Publication Date | Nov 4, 2022 |
Publication Date | 2022 |
Deposit Date | Aug 9, 2022 |
Publicly Available Date | Oct 1, 2022 |
Series ISSN | 2641-3590,2641-3604 |
DOI | https://doi.org/10.1109/bhi56158.2022.9926801 |
Files
Accepted Conference Proceeding
(1.3 Mb)
PDF
Copyright Statement
© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
You might also like
Less is More: Reducing Task and Model Complexity for 3D Point Cloud Semantic Segmentation
(2023)
Conference Proceeding
Region-based Appearance and Flow Characteristics for Anomaly Detection in Infrared Surveillance Imagery
(2023)
Conference Proceeding
Hierarchical Graph Convolutional Networks for Action Quality Assessment
(2023)
Journal Article
UAV-ReID: A Benchmark on Unmanned Aerial Vehicle Re-Identification in Video Imagery
(2022)
Conference Proceeding
Downloadable Citations
About Durham Research Online (DRO)
Administrator e-mail: dro.admin@durham.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search