This repo contains the Pytorch implementation of the MICCAI'20 paper - "Ultrasound Video Summarization using Deep Reinforcement Learning" pdf.
We introduce a novel fully automatic video summarization method that is tailored to the needs of medical video data.
Our approach is framed as reinforcement learning problem and produces agents focusing on the preservation of important diagnostic information.
Environment:
- PyTorch 0.4
- python 2.7
- numpy 1.15.1
- h5py 2.7.0
- matplotlib 2.0.2
Pre-trained models:
The code utilize pre-trained sonoNet models for standard plane detection. More details for this model can be found here:
sononet.
Dataset: The features extracted from the ultrasound videos can be downloaded HERE. The raw ultrasound dataset is not public subject to approval by the project ethics board and under a formal Data Sharing Agreement between the collaboration institutaions, the hospital and the participants.
Example video summaries from an untrasound examination:
- reducing the video to 15% of it’s original length: demo1,
- unsupervised to 15% length: demo2
- to a more relaxed 45% of the original length: demo3.
@misc{liu2020ultrasound,
title={Ultrasound Video Summarization using Deep Reinforcement Learning},
author={Tianrui Liu and Qingjie Meng and Athanasios Vlontzos and Jeremy Tan and Daniel Rueckert and Bernhard Kainz},
year={2020},
eprint={2005.09531},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
This repo is implemented based on the the project https://github.com/Lorna-Liu/pytorch-vsumm-RL.