Cosmos DB saga storage

[gideonkorir]

Similar to this previous issue. I'd like to run rebus on Azure with Cosmos DB rather than use SQL Server since we are using currently using cosmos for all our data storage.

I've been looking into it and have an implementation here.

So far this is what I've come up with:

1. Partitioning by some correlation property (e.g. OrderId) means that looking up the saga data by any other correlation property will lead to a cross partition query. My current implementation only supports a single correlation property and throws if multiple properties are specified. This means all tests in Rebus.Tests.Contracts that have multiple correlation properties will fail.
2. Partitioning by saga data type (or some value derived from it) allows multiple correlation properties since we always know the partition key but will use up more RUs since we won't be doing a point query unless we are using the id property.

I've checked the MassTransit saga persistence and Nservicebus saga persistence and they all seem to be using (1) above.

Using (2) might be ok if one is ok with limits of a single partition

[mookid8000]

I'd definitely go for (1).

Regarding the tests, maybe it would make sense to simply not use the contract tests for this persister.

Or maybe it would make sense to split the contract tests out into separate fixtures for single-property and multi-property correlation scenarios, which would then make it possible for single-property persisters to include only the tests they can work with....?

[gideonkorir]

I have implemented both options (I was thinking that (2) might be better for someone migrating from other stores and they are ok with the limitations + it was quite simple to implement).

On the tests for (1) I copied over the contract tests and modified the correlation to get them to work. It was pretty straightforward.

I've pushed all the code + tests here.

[TimterLaare]

I would love this feature in a future release! @mookid8000 What is the status of this issue?

[mookid8000]

Not sure 🙂 @gideonkorir does it work for you?

[gideonkorir]

@mookid8000 I didn't get to use it in production :( but it was working for all initial tests that I had though I'd be happy to collaborate to get it in a good state

[TimterLaare]

@gideonkorir @mookid8000 Which steps need to be taken to get this implemented?

[mookid8000]

Hi @gideonkorir , did your CosmosDB saga persister work as expected? Would you be interested in contributing it to rebus-org?

[gideonkorir]

@mookid8000 The persister worked as expected for my local testing but as I discussed before I never got the chance to use it in prod. I am happy to contribute it to rebus-org let me know what I need to do :)

@TimterLaare the testing bit is what maybe needs improving I am not sure what other test cases I need to cover in order for them to be considered complete. I am also not generating a nuget package right now