TSFD-Net: Nuclei Segmentation and Classification

Nuclei segmentation and classification using hematoxylin and eosin-stained histology images is a challenging task due a variety of issues, such as color inconsistency resulting from non-uniform manual staining operations, clustering of nuclei and blurry and overlapping nuclei boundaries. Existing approaches involve segmenting nuclei by drawing their polygon representations or by measuring the distances between nuclei centroids. In contrast, we leverage the fact that morphological features (appearance, shape and texture) of nuclei vary greatly depending upon the tissue type on which it is located. We exploit this information by extracting tissue specific (TS) features from raw histopathology images using our tissue specific feature distillation (TSFD) backbone. Then our bi-directional feature pyramid network (BiFPN) generates a robust hierarchical feature pyramid using these TS features. Next, our interlinked decoders jointly optimize and fuse these features to generate final predictions. We also propose a novel loss combination for joint optimization and faster convergence of our proposed network. Extensive ablation studies are performed to validate the effectiveness of each component of TSFD-Net. TSFD- Net achieves state-of-the-art performance on PanNuke dataset having 19 different tissue types and up to 5 clinically important tumor classes.

Full Paper

Pannuke Dataset:

The PanNuke dataset can be downloaded form here.

Dataset Preparation

You can follow the steps highlighted in the following repo to prepare the dataset for training.

Network Architecture:

Figure below shows full architecture of our proposed TSFD-Net.

Results

The Table below compare quantitative results of different models.

Visual Results

The figure below shows some qualitative results.

Evaluation

To evaluate the model we used the Panoptic Quality metric as introduced in HoverNet paper.

We use the official implementation provided by the authors of Pannuke dataset.

To see our implementation follow the link. We mainly follow the original implementation with some minor improvements for exception handelling, bug fixes and better visualization.

Citation

@article{ilyas2022tsfd,
  title={TSFD-Net: Tissue specific feature distillation network for nuclei segmentation and classification},
  author={Ilyas, Talha and Mannan, Zubaer Ibna and Khan, Abbas and Azam, Sami and Kim, Hyongsuk and De Boer, Friso},
  journal={Neural Networks},
  year={2022},
  publisher={Elsevier}
}