Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Duplicate handling during copy from one catalog to another #738

Open
pbeaucage opened this issue May 8, 2024 · 1 comment
Open

Duplicate handling during copy from one catalog to another #738

pbeaucage opened this issue May 8, 2024 · 1 comment

Comments

@pbeaucage
Copy link
Contributor

This issue is intended to centralize discussion around how to handle duplicate entries when using the functionality added by @danielballan in #735 to copy nodes from one catalog to another with tiled.client.sync.copy. The present behavior is that attempting to insert a node with an exactly-matching uuid to the destination catalog returns a HTTP 409 "Conflict". #737 is a basic stub that adds a feature setting to ignore such responses, but there are many (?) cases where performing a more principled sync, i.e, check for differences and do a differential update, always delete destination and overwrite, would be desirable.

Another question if implementing this is whether look-before-you-leap checking would be more performant, since asking the catalog for just node uuids is relatively cheap compared to accessing full metadata.

@danielballan
Copy link
Member

I imagine we'll eventually find use cases for ~all of the various rsync options here. I like that the approach in #737 leaves room for the API to grow.

Some things to consider and spin out into separate issues:

  • a CLI like tiled copy ...
  • a progress bar for the CLI and (maybe? optionally?) for the Python API
  • batch upload via new endpoint /nodes/metadata/ that accepts a list of nodes
  • how look-before-you-leap and partial update would interact with batch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants