High-plex immunofluorescence imaging and traditional histology of the same tissue section for discovering image-based biomarkers

Jia-Ren Lin^1,2,*, Yu-An Chen^1,2,*, Daniel Campton^3,*, Jeremy Cooper³, Shannon Coy^1,4, Clarence Yapp^1,2, Juliann B. Tefft^1,2, Erin McCarty³, Keith L. Ligon⁴, Scott J. Rodig⁴, Steven Reese³, Tad George³, Sandro Santagata^1,2,4,±, Peter K. Sorger^1,2,±

Nature Cancer (2023). DOI: 10.1038/s43018-023-00576-1

* These authors contributed equally
± These authors contributed equally

Human Tissue Atlas Center
¹ Laboratory of Systems Pharmacology, Harvard Medical School, Boston, MA, 02115, USA.
² Ludwig Center at Harvard, Harvard Medical School, Boston, MA 02115, USA.
³ RareCyte, Inc., 2601 Fourth Ave., Seattle, WA, 98121, USA.
⁴ Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA 02115, USA.

---

Scientific summary

Precision medicine is critically dependent on better methods for diagnosing and staging disease and predicting drug response. Histopathology using Hematoxylin and Eosin (H&E) stained tissue - not genomics – remains the primary diagnostic method in cancer. Recently developed highly-multiplexed tissue imaging methods promise to enhance research studies and clinical practice with precise, spatially-resolved, single-cell data. Here we describe the “Orion” platform for collecting H&E and high-plex immunofluorescence images from the same cells in a whole-slide format suitable for diagnosis. Using a retrospective cohort of 74 colorectal cancer resections, we show that IF and H&E images provide human experts and machine learning algorithms with complementary information that can be used to generate interpretable, multiplexed image-based models predictive of progression-free survival. Combining models of immune infiltration and tumor-intrinsic features achieves a nearly 20-fold discrimination between rapid and slow (or no) progression, demonstrating the ability of multi-modal tissue imaging to generate high-performance biomarkers.

---

Data availability

Full-resolution images, derived imaging data, and single-cell spatial feature tables will be released by the National Cancer Institute sponsored repository for Human Tumor Atlas Network. While the public resource is still undergoing extensive development, we provided exemplar data via Synapse for demonstration purposes. Light-weight image viewing for all the bio-specimens used in this study is made available using Minerva and Scope2Screen [1].

Exemplar data (syn38990468)

To download, please refer to the Synapse documentation

• Files in the exemplar data

  P37_S29-CRC01/
  ├── quantification
  │   └── p37_s29_a24_c59kx_e15__at__20220106_014304_946511_cellringmask.csv
  │       # single-cell feature table
  ├── segmentation
  │   └── cellRingMask.tif
  │       # labeled mask used to quantify single-cell features
  └── registration
      ├── p37_s29_a24_c59kx_e15__at__20220106_014304_946511.ome.tiff
      │   # orion IF image (19-channel, 16-bit)
      └── 18459-lsp10353-us-scan-or-001 _093059-registered.ome.tif
          # post-orion H&E image registered to orion image (3-channel, 8-bit)

Image viewing using web browsers

Light-weight image viewing for all the bio-specimens used in this study is made available using Minerva and Scope2Screen [1]. Visit https://www.tissue-atlas.org/atlas-datasets/lin-chen-campton-2023/ to access the images.

Access the full dataset

All images at full resolution, derived image data (e.g., segmentation masks), and single-cell tables are stored and can be accessed through Amazon Web Services (AWS) S3. Detailed information and list of files are documented here.

---

Codes for imaging data processing

Highplex Orion whole-slide images were processed using MCMICRO modules with customizations to cope with the large X-Y dimension in the dataset. Registration of Orion immunofluorescence images and post-Orion H&E images was done with PALOM.

Files in the scripts-processing/ folder

To process Orion images into single tables, we ran unmicst and S3segmenter to generated labeled mask and run quantification using the segmentation mask and the 19-channel Orion image. Version and repositories of the processing modules is listed in the github_repo.md file.

• conda-env yaml files: environment specs for creating conda envs to run customized mcmicro modules
• github_repo.md file: version and repositories of the processing modules
• .py files: scripts for batch processing multiple input images
• file_list.csv file: file index for batch processing
• markers.csv file: antibody target names of channels in the Orion images

---

Codes for single-cell spatial analysis

Single-cell spatial analysis was performed using MATLAB 2019b. To run the provided scripts on your data, please contact authors for more details.

Demo for running the analysis and plotting scripts

Please contact the authors for detailed information.

%% Optimize ImmuneScore

markers = {'CD3','CD8','CD45','CD45RO','CD68','CD163','CD4','CD20','SMA'};
regions = {'R1','R2'};
figure;

for i = 1:length(markers)
    for j = 1:length(regions)
        subplot(3,6,(i-1)*length(regions)+j);
        marker1 = strcat('norm_',markers{i},regions{j});
        list1 = sumAllsample{:,marker1};
        list2 = sumAllsample.PFSDays;
        scatter(list1,list2,30,'b','fill');
        lsline;
        title(num2str(corr(list1,list2),'%0.2f'),'FontSize',16);
        set(gca,'xtick',[]);
        xlabel(marker1,'Interpreter','none');
        set(gca,'ytick',[]);
        ylabel('PFS Days');
    end
end

---

Funding

This work was supported by NCI grants U54-CA225088 and U2C-CA233262 (P.K.S. and S.S.), an NCI SBIR small business grant R41-CA224503 (RareCyte and P.K.S.) and commercial investment from RareCyte; image processing software and data science methods were developed with support from the Bill and Melinda Gates Foundation grant INV-027106 (P.K.S.), a Team Science Grant from the Gray Foundation (P.K.S. and S.S.), the David Liposarcoma Research Initiative (P.K.S. and S.S.), Emerson Collective (P.K.S.) and Ludwig Cancer Research (P.K.S. and S.S.). J.-R.L. is supported by an NCI Research Specialist Award (R50-CA274277), and S.C. by training grants T32-GM007748 from the NIGMS and T32-CA009216 from the NCI. S.S. is also supported by the BWH President’s Scholars Award.

---

References

[1] J. Jessup and R. Krueger et al., "Scope2Screen: Focus+Context Techniques for Pathology Tumor Assessment in Multivariate Image Data," in IEEE Transactions on Visualization and Computer Graphics, vol. 28, no. 1, pp. 259-269, Jan. 2022, doi: 10.1109/TVCG.2021.3114786.