Download All doesn't work for specific data package

[mbjones]

Describe the bug
Clicking "Download All" on this data package results in a new blank tab, but the dataset is not downloaded. The individual file downloads work fine. Issue reported by a researcher in Redmine issue 27990

To Reproduce
Steps to reproduce the behavior:

1. Go to https://search.dataone.org/view/https%3A%2F%2Fpasta.lternet.edu%2Fpackage%2Fmetadata%2Feml%2Fknb-lter-arc%2F10006%2F7
2. Click on 'Download All' or the equivalent icon
3. See new empty tab appear, but no download

Expected behavior
The dataset zip file should be present on my disk

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: Mac OS
• Browser: Firefox
• Version: 124.0.1

Additional context

Same problem was seen for other datasets from LTER, ESS-DIVE, and ADC. The third one in this list, if accessed from the ADC site, works fine for download all, and produces the correct zip file.

• https://search.dataone.org/view/https%3A%2F%2Fpasta.lternet.edu%2Fpackage%2Fmetadata%2Feml%2Fknb-lter-nwt%2F264%2F3
• https://search.dataone.org/view/doi%3A10.15485%2F2336865
• https://search.dataone.org/view/doi%3A10.18739%2FA2F18SH0T

It is not clear to me how Download All would work on DataONE, as the download has to redirect to the member repository to get the data. For member repositories like ADC that support the package service, it would be fine to redirect to the ADC package service, but this doesn't work on all nodes. Is it possible that "Download All" should be disabled/not display for DataONE deploys?

This was verified on the newly released MetacatUI 2.28.0, but was originally reported on a prior version in Redmine.

[robyngit]

Something changed in how we construct these Download All URLs in the recent 2.28.0 release, see the comparison to the previous version:

1. Package: https://pasta.lternet.edu/package/metadata/eml/knb-lter-arc/10006/7

   • 2.27.0 URL: https://gmn.lternet.edu/mn/v2/packages/application%2Fbagit-097/doi%3A10.6073%2Fpasta%2F74407ca602bf8944e5152f7a74203ac4
   • 2.28.0 URL: https://search.dataone.org/cn/v2/packages/application%2Fbagit-1.0/doi%3A10.6073%2Fpasta%2F74407ca602bf8944e5152f7a74203ac4

2. Package: https://pasta.lternet.edu/package/metadata/eml/knb-lter-nwt/264/3

   • 2.27.0 URL: https://gmn.lternet.edu/mn/v2/packages/application%2Fbagit-097/doi%3A10.6073%2Fpasta%2F9deb9acebc7b5ed41f1bb13d6da7084c
   • 2.28.0 URL: https://search.dataone.org/cn/v2/packages/application%2Fbagit-1.0/doi%3A10.6073%2Fpasta%2F9deb9acebc7b5ed41f1bb13d6da7084c

3. Package: doi:10.15485/2336865

   • 2.27.0 URL: https://data.ess-dive.lbl.gov/catalog/d1/mn/v2/packages/application%2Fbagit-097/ess-dive-1bdd425d40f2167-20240416T194704825
   • 2.28.0 URL: https://search.dataone.org/cn/v2/packages/application%2Fbagit-1.0/ess-dive-1bdd425d40f2167-20240416T194704825

4. Package: doi:10.18739/A2F18SH0T

   • 2.27.0 URL: https://arcticdata.io/metacat/d1/mn/v2/packages/application%2Fbagit-097/resource_map_doi%3A10.18739%2FA2F18SH0T
   • 2.28.0 URL: https://search.dataone.org/cn/v2/packages/application%2Fbagit-1.0/resource_map_doi%3A10.18739%2FA2F18SH0T

Some, like example 2, worked before (2.27) but do not work now (2.28). For example 1, the one reported on Redmine, neither URL works.

It seems like we need to:

• Determine the best way to have an up to date list of nodes that support the package service and those that don't
• Exclude the download all button for datasets on DataONE that are on nodes without a packaging service
• Ensure we're constructing the download all links properly for DataONE
• Once fixed, contact the researcher who reported them problem to let them know (Redmine issue 27990)

[mbjones]

Is this due to the addition of the bagit-1.0.0 package service? How does metacatUI determine which nodes support which versions of the package service?

[robyngit]

We recently received a report of files being inaccessible in a package in DataONE (see ticket #28111: Error retrieving files); and I suspect this could be a related issue. In MetacatUI v 2.28.0 or later, the data package table shows the error "Error retrieving files for this data package", but on v 2.27.0 or earlier, the list of files appears as is.

Steps to reproduce

1. On a local instance of MetacatUI configured to the dataone theme, checkout the main branch.
2. Navigate to http://localhost:3000/view/doi%3A10.6085%2FAA%2FPISCO_kelpforest.1.6
3. See the error in the package table
4. Checkout MetacatUI version 2.27.0 in git: git checkout tags/2.27.0
5. Navigate to http://localhost:3000/view/doi%3A10.6085%2FAA%2FPISCO_kelpforest.1.6 again and see that the file list loads

I haven't dug into this enough to see exactly which request succeeds in the old version that now fails in the new version, but the list of requests that include package ID do differ between the two versions:

v 2.27.0

v 2.28.0

Note that both include some 404s & CORS errors.