Too-long column names cause Shiny render to fail silently

[wkumler]

Hi, super neat package you've got here and one I've found invaluable for data exploration and visualization. I ran into a super strange bug by trying to embed a heatmaply object into a Shiny app, but found that if the column names of the provided matrix were too long then the heatmap wouldn't render at all.

library(heatmaply)
library(shiny)
library(plotly)

Here's an MWE of the problem:

heatmaply_mat <- matrix(runif(36), nrow = 6)
long_colname <- paste(sample(letters, 50, replace = TRUE), collapse = "")
colnames(heatmaply_mat) <- c(head(letters, 5), long_colname)
hmc <- heatmaply(heatmaply_mat)
#hmc alone renders perfectly in RStudio "Viewer" pane and browser

ui <- fluidPage(
  plotlyOutput("heatmaply_object")
)
server <- function(input, output){
  output$heatmaply_object <- renderPlotly(hmc)
}
shinyApp(ui, server)
# Returns empty full-page plot window, but no heatmap
# Fails both in RStudio pane and "Open in browser", not tested on shinyapps.io as full website

The character limit seems to be set to 45 - rownames of length 45 render fine, but length 46 fail.

# This works - long row name has 45 characters in it
heatmaply_mat <- matrix(runif(36), nrow = 6)
long_colname <- paste(sample(letters, 45, replace = TRUE), collapse = "")
colnames(heatmaply_mat) <- c(head(letters, 5), long_colname)

ui <- fluidPage(plotlyOutput("heatmaply_object"))
server <- function(input, output){
  output$heatmaply_object <- renderPlotly(heatmaply(heatmaply_mat))
}
shinyApp(ui, server)

# This fails - long row name has 46 characters in it
heatmaply_mat <- matrix(runif(36), nrow = 6)
long_colname <- paste(sample(letters, 46, replace = TRUE), collapse = "")
colnames(heatmaply_mat) <- c(head(letters, 5), long_colname)

ui <- fluidPage(plotlyOutput("heatmaply_object"))
server <- function(input, output){
  output$heatmaply_object <- renderPlotly(heatmaply(heatmaply_mat))
}
shinyApp(ui, server)

It seems to only affect column names - row names can exceed the 45 character limit and not cause problems

# This works - changing the rownames behaves as expected
heatmaply_mat <- matrix(runif(36), nrow = 6)
long_colname <- paste(sample(letters, 50, replace = TRUE), collapse = "")
rownames(heatmaply_mat) <- c(head(letters, 5), long_colname)

ui <- fluidPage(plotlyOutput("heatmaply_object"))
server <- function(input, output){
  output$heatmaply_object <- renderPlotly(heatmaply(heatmaply_mat))
}
shinyApp(ui, server)

This does seem like a particularly weird heatmaply x shiny problem, as plotly alone can handle long column & row names:

heatmaply_mat <- matrix(runif(36), nrow = 6)
long_colname <- paste(sample(letters, 50, replace = TRUE), collapse = "")
hmc <- plot_ly(z=heatmaply_mat, type = "heatmap", 
               y=c(head(letters, 5), long_colname), 
               x=c(head(letters, 5), long_colname))
ui <- fluidPage(plotlyOutput("heatmaply_object"))
server <- function(input, output){
  output$heatmaply_object <- renderPlotly(hmc)
}
shinyApp(ui, server)

I'm not really familiar enough with the source code to go digging for a solution and it's not critical because I can just abbreviate the row names, but wanted to make sure this was documented for anyone else running into this bizarre problem.

> sessionInfo()
R version 4.3.1 (2023-06-16 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19045)

Matrix products: default


locale:
[1] LC_COLLATE=English_United States.utf8  LC_CTYPE=English_United States.utf8   
[3] LC_MONETARY=English_United States.utf8 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.utf8    

time zone: America/Los_Angeles
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] lubridate_1.9.2   forcats_1.0.0     stringr_1.5.0     dplyr_1.1.2       purrr_1.0.1      
 [6] readr_2.1.4       tidyr_1.3.0       tibble_3.2.1      tidyverse_2.0.0   shiny_1.7.4.1    
[11] heatmaply_1.4.2   viridis_0.6.3     viridisLite_0.4.2 plotly_4.10.2     ggplot2_3.4.2    

loaded via a namespace (and not attached):
 [1] gtable_0.3.3       bslib_0.5.0        htmlwidgets_1.6.2  lattice_0.21-8    
 [5] tzdb_0.4.0         vctrs_0.6.3        tools_4.3.1        crosstalk_1.2.0   
 [9] generics_0.1.3     parallel_4.3.1     ca_0.71.1          fansi_1.0.4       
[13] pkgconfig_2.0.3    Matrix_1.5-4.1     data.table_1.14.8  RColorBrewer_1.1-3
[17] assertthat_0.2.1   webshot_0.5.5      lifecycle_1.0.3    compiler_4.3.1    
[21] farver_2.1.1       textshaping_0.3.6  munsell_0.5.0      codetools_0.2-19  
[25] httpuv_1.6.11      seriation_1.5.1    htmltools_0.5.5    sass_0.4.7        
[29] yaml_2.3.7         lazyeval_0.2.2     crayon_1.5.2       pillar_1.9.0      
[33] later_1.3.1        jquerylib_0.1.4    ellipsis_0.3.2     cachem_1.0.8      
[37] iterators_1.0.14   TSP_1.2-4          foreach_1.5.2      nlme_3.1-162      
[41] mime_0.12          tidyselect_1.2.0   digest_0.6.33      stringi_1.7.12    
[45] reshape2_1.4.4     labeling_0.4.2     splines_4.3.1      fastmap_1.1.1     
[49] grid_4.3.1         colorspace_2.1-0   cli_3.6.1          magrittr_2.0.3    
[53] utf8_1.2.3         withr_2.5.0        scales_1.2.1       promises_1.2.0.1  
[57] bit64_4.0.5        timechange_0.2.0   registry_0.5-1     httr_1.4.6        
[61] bit_4.0.5          gridExtra_2.3      ragg_1.2.5         hms_1.1.3         
[65] memoise_2.0.1      mgcv_1.8-42        rlang_1.1.1        Rcpp_1.0.11       
[69] dendextend_1.17.1  xtable_1.8-4       glue_1.6.2         vroom_1.6.3       
[73] rstudioapi_0.15.0  jsonlite_1.8.7     R6_2.5.1           plyr_1.8.8        
[77] systemfonts_1.0.4 

[wkumler]

One more comment - it's not due to the dendrogram rendering either. We get the same result if we disable dendrogram things with Rowv and Colv = FALSE:

heatmaply_mat <- matrix(runif(36), nrow = 6)
long_colname <- paste(sample(letters, 46, replace = TRUE), collapse = "")
colnames(heatmaply_mat) <- c(head(letters, 5), long_colname)
hmc <- heatmaply(heatmaply_mat, Rowv = FALSE, Colv = FALSE)

ui <- fluidPage(plotlyOutput("heatmaply_object"))
server <- function(input, output){
  output$heatmaply_object <- renderPlotly(hmc)
}
shinyApp(ui, server)

[alanocallaghan]

Thanks for the bug report, the details are super useful and very much appreciated. I'll look into this later today but as I'm not a shiny expert I can't guarantee a quick fix

[alanocallaghan]

It seems like a plotly issue with (sub)plots that are too large for the given window, which I can replicate with:

heatmaply_mat <- matrix(runif(36), nrow = 6)
long_colname <- paste(sample(letters, 2000, replace = TRUE), collapse = "")
hmc <- plot_ly(z=heatmaply_mat, type = "heatmap", 
               y=c(head(letters, 6)), 
               x=c(head(letters, 6)))
sp <- subplot(replicate(20, hmc, simplify = FALSE))
ui <- fluidPage(plotlyOutput("heatmaply_object"))
server <- function(input, output){
  output$heatmaply_object <- renderPlotly(sp)
}
shinyApp(ui, server)

Probably plotly/plotly.js#4155

[wkumler]

Glad it's useful! The example you give above seems to maybe be a separate issue though, one relating to shiny not rendering more than 16 heatmap subplots no matter the size of the window independent of the length of the row/column names.

# Setup for all later plots
library(shiny)
library(plotly)

ui <- fluidPage(plotlyOutput("heatmaply_object"))
server <- function(input, output){
  output$heatmaply_object <- renderPlotly(sp)
}

This works as expected, with the heatmaps rendering both in RStudio and in browser:

sp <- subplot(replicate(16, {
  plot_ly(z=matrix(runif(36), nrow = 6), type = "heatmap")
}, simplify = FALSE), nrows = 4)
shinyApp(ui, server)

This one (with 20 subplots) fails in the Shiny app even though it renders just fine in the RStudio "Viewer" window

long_colname <- paste(sample(letters, 50, replace = TRUE), collapse = "")
sp <- subplot(replicate(20, {
  plot_ly(z=matrix(runif(36), nrow = 6), type = "heatmap")
}, simplify = FALSE), nrows = 4)
shinyApp(ui, server)

This one (16 subplots, long column names) seems to render just fine (although the long column name chaotically runs behind the plots below it):

sp <- subplot(replicate(16, {
  long_colname <- paste(sample(letters, 50, replace = TRUE), collapse = "")
  plot_ly(z=matrix(runif(36), nrow = 6), type = "heatmap", 
          x=c(head(letters, 5), long_colname), 
          y=c(head(letters, 6)))
}, simplify = FALSE), nrows = 4)
shinyApp(ui, server)

as does this one with long row names:

sp <- subplot(replicate(16, {
  long_rowname <- paste(sample(letters, 50, replace = TRUE), collapse = "")
  plot_ly(z=matrix(runif(36), nrow = 6), type = "heatmap", 
          y=c(long_rowname, head(letters, 5)), 
          x=c(head(letters, 6)))
}, simplify = FALSE), nrows = 4)
shinyApp(ui, server)

Does heatmaply use a bunch of small subplots in rendering the heatmap, maybe an additional one if the name is detected to be too long somehow?

[alanocallaghan]

Seems like a ggplotly thing, which is likely to make it a nightmare to debug. eg this works fine for me:

library(heatmaply)
library(shiny)
library(plotly)


heatmaply_mat <- matrix(runif(36), nrow = 6)
long_colname <- paste(sample(letters, 50, replace = TRUE), collapse = "")
colnames(heatmaply_mat) <- c(head(letters, 5), long_colname)
hmc <- heatmaply(heatmaply_mat, plot_method="plotly")

ui <- fluidPage(
  plotlyOutput("heatmaply_object")
)
server <- function(input, output){
  output$heatmaply_object <- renderPlotly(hmc)
}
shinyApp(ui, server)

[wkumler]

Oof, yep. Can confirm it works fine for me under plot_method="plotly" but fails under plot_method="ggplot". Good to know that I can just switch to the plotly option if necessary though, thank you!