input$table_rows_selected Returns Incorrect Indexes with Large Data & Slow Connection when Filter is set by default

[12sgintautas]

input$table_rows_selected Returns Incorrect Indexes with Large Data & Slow Connection when Filter is set by default

Description:

When a default filter is set on a DT::datatable (options -> searchCols -> search), the input$table_rows_selected can return incorrect row indexes, especially with large data frames (10k+ rows) and a slow internet connection (e.g., Slow 3G).

Steps to Reproduce:

Use the provided R code to set up a Shiny app with a DT::datatable.
Throttle the connection to "Slow 3G" in browser DevTools.
Refresh the page.
Click on any row in the table.

Expected Behavior:

The R console should print the correct row ID of the selected row.

Actual Behavior:

The R console prints incorrect rows_selected IDs.

Code to Reproduce:

library(shiny)
library(DT)

odd_even <- data.frame(is_odd = c(1:50000) %% 2 == 1)
ui <- fluidPage(dataTableOutput(outputId = "table"))
server <- function(input, output) {
  output$table <- renderDataTable({datatable(
    odd_even, filter = "top",
    options = list(searchCols = c(list(NULL), list(list(search = '["false"]'))))
  )})
  observe(print(input$table_rows_selected))
}
shinyApp(ui = ui, server = server)

xfun::session_info('DT')

> xfun::session_info('DT')
R version 4.2.3 (2023-03-15 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19045), RStudio 2022.2.3.492

Locale: LC_COLLATE=English_United Kingdom.utf8  LC_CTYPE=English_United Kingdom.utf8    LC_MONETARY=English_United Kingdom.utf8 LC_NUMERIC=C                            LC_TIME=English_United Kingdom.utf8    

Package version:
  base64enc_0.1.3   bslib_0.5.1       cachem_1.0.8      cli_3.6.1         crosstalk_1.2.0   digest_0.6.29     DT_0.29.2         ellipsis_0.3.2    evaluate_0.21     fastmap_1.1.1     fontawesome_0.5.2
  fs_1.6.3          glue_1.6.2        graphics_4.2.3    grDevices_4.2.3   highr_0.10        htmltools_0.5.5   htmlwidgets_1.6.2 httpuv_1.6.5      jquerylib_0.1.4   jsonlite_1.8.7    knitr_1.44       
  later_1.3.0       lazyeval_0.2.2    lifecycle_1.0.3   magrittr_2.0.3    memoise_2.0.1     methods_4.2.3     mime_0.12         promises_1.2.0.1  R6_2.5.1          rappdirs_0.3.3    Rcpp_1.0.8.3     
  rlang_1.1.1       rmarkdown_2.25    sass_0.4.7        stats_4.2.3       stringi_1.7.12    stringr_1.5.0     tinytex_0.46      tools_4.2.3       utils_4.2.3       vctrs_0.6.3       xfun_0.40        
  yaml_2.3.7  

Where is the problem?

As I understand the problem happens because of a race condition (see datatables.js Line 325 https://github.com/rstudio/DT/blob/main/inst/htmlwidgets/datatables.js#L325)

HTMLWidgets.widget({
  name: "datatables",
  type: "output",
  renderOnNullValue: true,
  initialize: function(el, width, height) {
    ...
  },
  renderValue: function(el, data, instance) {
    ...
    // use the dataSrc function to pre-process JSON data returned from R
    var DT_rows_all = [], DT_rows_current = [];
    if (server && HTMLWidgets.shinyMode && typeof options.ajax === 'object' &&
        /^session\/[\da-z]+\/dataobj/.test(options.ajax.url) && !options.ajax.dataSrc) {
      options.ajax.dataSrc = function(json) {
        DT_rows_all = $.makeArray(json.DT_rows_all);
        // !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
        DT_rows_current = $.makeArray(json.DT_rows_current);
        // !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
        var data = json.data;
        if (!colReorderEnabled()) return data;
        var table = $table.DataTable(), order = table.colReorder.order(), flag = true, i, j, row;
        for (i = 0; i < order.length; ++i) if (order[i] !== i) flag = false;
        if (flag) return data;
        for (i = 0; i < data.length; ++i) {
          row = data[i].slice();
          for (j = 0; j < order.length; ++j) data[i][j] = row[order[j]];
        }
        return data;
      };
    }
   ...
})

The line DT_rows_current = $.makeArray(json.DT_rows_current); get called 3 times, and depending on the internet speed the output can differ. (DT_rows_current is directly used for determining indexes of input$table_rows_selected)

Below you can see DT_rows_current values with slow and normal connection:

---

By filing an issue to this repo, I promise that

• I have fully read the issue guide at https://yihui.org/issue/.
• I have provided the necessary information about my issue.
  • If I'm asking a question, I have already asked it on Stack Overflow or RStudio Community, waited for at least 24 hours, and included a link to my question there.
  • If I'm filing a bug report, I have included a minimal, self-contained, and reproducible example, and have also included xfun::session_info('DT'). I have upgraded all my packages to their latest versions (e.g., R, RStudio, and R packages), and also tried the development version: remotes::install_github('rstudio/DT').
  • If I have posted the same issue elsewhere, I have also mentioned it in this issue.
• I have learned the Github Markdown syntax, and formatted my issue correctly.

I understand that my issue may be closed if I don't fulfill my promises.

[yihui]

Thanks for the report! I feel I don't have the expertise to fix it at the moment. Do you have an idea about how to fix it?