ENH: Better handling of bulk adds

[ekorman]

Feature Type

• Adding new functionality to valor

• Changing existing functionality in valor

• Removing existing functionality in valor

Problem Description

When bulk adding groundtruths or predictions, if there's some error (e.g. a network error in the post of one of the chunks or one of the groundtruths is bad), then the backend can get in a state where some of the requested data was added but not others. This makes it difficult for the user to know what data to retry.

Feature Description

Additional Context

No response

[rsbowman-striveworks]

If a batch of datums is sent to Valor some of which have been previously added, the whole add_groundtruths call returns 409 'Datum already exists' and no new datums are added (tested with 0.21.2-post1). This makes it difficult to ensure that all Chariot datums are added to the corresponding Valor dataset by using only the bulk interface.

In Chariot we have been using the one-at-a-time interface and ignoring the 'Datum already exists' errors in order to ensure that all datums are added to a possibly incomplete Valor dataset. (Datasets can be incomplete because of errors and also from canceled bulk inference operations or training runs.) It would be nice if we could have similar sematics for the bulk operations.