Broken geojson attachment makes downloading new attachment impossible

[dbemke]

ODK Collect version

the store version 2024.1.3, the master version

Android version

10, 14

Device used

Redmi 9T, Pixel 7a

Problem description

After downloading to Collect a form with a broken geojson attachment it’s impossible to download the correct attachment afterwards. There is download failure (the notification and arrow with exclamation mark in Start new form). Only after deleting the project it’s possible to download the newest version.

Steps to reproduce the problem

1. In Central upload the form with the broken attachment.
   abc.geojson.txt
   geojsonUpdateBug.xlsx.txt
2. Download this form to Collect.
3. Change something in the geojson file.
4. In Central upload the changed geojson file.
5. Try to download the new version of the form to Collect.
   Additional steps
6. Delete the project.
7. Download the form with the newest version to Collect.

Expected behavior

It should be possible to download different version of form with changed media.

[seadowg]

This feels very specific, but I'm worried there's a more general problem lurking underneath so I'm scheduling it for this release.

[WKobus]

This issue also affects forms from other projects, that are using geojson with same name as the broken one.

[grzesiek2010]

It turns out we face a similar issue with external CSV files. For instance, when using the select_one_from_file. If we publish a form with a broken CSV file and try to download it, getting an updated version after fixing it becomes impossible.

The problem stems from the need to parse a form during the download process to retrieve metadata like title, formId, version etc. This parsing involves checking external instances:

collect/collect_app/src/main/java/org/odk/collect/android/formmanagement/ServerFormDownloader.java

Line 119 in ee9b02a

parsedFields = formMetadataParser

However, during the initial download, these instances aren't available yet. They're only copied to their proper location a few lines below.

Now, if a form with a broken external file (like geojson or CSV) is already downloaded and we attempt to redownload or update it, the parsing considers the existing instances. This isn't necessary and doesn't make sense because those instances might be replaced with new ones in the next step (copying newly downloaded media files).

There are a few things we can do here:

1. ODK Central should probably validate such files like geojson, csv and block uploading broken ones.
2. In ODK Collect, we could rearrange the sequence of steps mentioned earlier, placing the parsing of a form after the installation of media files. This adjustment ensures that parsing always checks the most updated files. By adopting this approach, the download of forms containing broken external instances would be prohibited. This adjustment appears sensible as attempting to download such forms would be futile, as they cannot be opened anyway.
3. We can try to implement changes in javarosa introducing a method for reading metadata without parsing the whole form with external instances as this seems to be unneeded.

All those options make sense to me. Number 2 seems to be the easiest one for a quick fix. Number 1 is something that should be done regardless of what we want to do in Collect. I'm not sure about the third option... on one hand it sounds like an overkill to parse the whole form during downloading to read metadata but on the other hand maybe it's a feature that allows us to catch different types of exceptions (bugs in forms) so that we don't download them at all?
@lognaturel @seadowg it would be good to hear what you think.

[seadowg]

All those options make sense to me. Number 2 seems to be the easiest one for a quick fix. Number 1 is something that should be done regardless of what we want to do in Collect. I'm not sure about the third option... on one hand it sounds like an overkill to parse the whole form during downloading to read metadata but on the other hand maybe it's a feature that allows us to catch different types of exceptions (bugs in forms) so that we don't download them at all?

I'm in full agreement here: 1 would be a nice add to Central, 2 is how we should fix this bug and 3 is something to consider for the future.