ODC EP 002 Support for multiple DC databases
Brief paragraph on what the proposal is.
- Robert Woodcock (CSIRO)
- Under Discussion
- In Progress
- Completed
- Rejected
- Deferred
- Superceded
This EP has been superceded by EP-03
With multiple ODC deployments now underway there are often good collaborative and data sharing reasons to have access to multiple ODC instances. In addition, there is often private data and shared data, and the need to save your own results as a data cube. This leads to general requirement for multiple database support in the ODC code.
The idea will be to add support for multiple databases to be specified and used by the ODC API, in the database.conf file, dc.load() and the prototype dc.save(). By default the ODC code would behave in exactly the same way, simply iterating over all specified databases to search for the appropriate product and then directing calls to that database in order to retrieve data. Initially this would support a 1 to 1 mapping of product to database - that is, no duplication of product name and thus no merging of the products from across the databases.
Q: isn't this already possible with multiple Datacube instances?
WIP: More details are required, I just made a start...
18/7/2019 - @woodcockr Been working with a few users in a multi-database environment. Can confirm there is definitely a need for this but you can work around it using datacube environments. A few things to note:
- Product names tend to be unique and it is the default expectation that they should be. This could support the simple implementation discussed previously (per ODC SC: No clashes in product names, no duplicates -> simple union of products into same cube)
- It is a little confusing and error prone to use datacube environments. Not overly so but suggests a small amount of effort to reduce the burden would be beneficial
Next step would be to take a look at what code changes might be required and see if it is worth the effort (noting 2).
- One Individual
Welcome to the Open Data Cube