Single vs Multiple sessions - performance

[mirzahassan]

I had an interesting case as I was messing around with patching some docs. I ran a simple test on patch 3 values on 20k documents.
I'm using 5.4.112 and executing all 20k document updates on a single session was twice as slow vs multiple sessions.

Below is the unit test. FYI it's using the RavenTestDriver.

I'm having trouble understanding why multiple sessions here is better than one. What is the guidance on session creation?

public class Record
{
    public int Amount { get; set; }
    public int Type { get; set; }
    public DateTime On { get; set; }
}

[Test]
public void SingleSessionUpdate()
{
    using (var session = _driver.DocumentStore.OpenSession())
    {
        for (var i = 0; i < 20000; i++)
            session.Store(new Record(), i.ToString());

        session.SaveChanges();
    }

    using (var session = _driver.DocumentStore.OpenSession())
    {
        var now = DateTime.Now;

        for (var i = 0; i < 20000; i++)
        {
            session.Advanced.Patch<Record, decimal>(i.ToString(), x => x.Amount, 100);
            session.Advanced.Patch<Record, decimal>(i.ToString(), x => x.Type, 90);
            session.Advanced.Patch<Record, DateTime>(i.ToString(), x => x.On, now);
        }

        session.SaveChanges();
    }
}

[Test]
public void MultiSessionUpdate()   //Almost 2x FASTER
{
    using (var session = _driver.DocumentStore.OpenSession())
    {
        for (var i = 0; i < 20000; i++)
            session.Store(new Record(), i.ToString());

        session.SaveChanges();
    }

    var now = DateTime.Now;

    for (var i = 0; i < 20000; i++)
    {
        using (var session = _driver.DocumentStore.OpenSession())
        {
            session.Advanced.Patch<Record, decimal>(i.ToString(), x => x.Amount, 100);
            session.Advanced.Patch<Record, decimal>(i.ToString(), x => x.Type, 90);
            session.Advanced.Patch<Record, DateTime>(i.ToString(), x => x.On, now);

            session.SaveChanges();
        }
    }
}

[ayende]

What you did here is submit a single 20K items task to the server.
Sessions aren't really intended for such sizes.

What happens is that you are building a buffer of 20K operations to run, send it to the server in one shot.
We now need to execute it (as a single transaction) and keep state around for all of that.

Note that with RavenTestDriver we aren't actually calling fsync, and you are running locally, so you aren't seeing some of the other costs associated with 20K separate transactions versus a single one.

In short, a session is meant to hold a reasonable amount of changes.

[yj7o5]

hey @ayende I am curious to know do you think it is worth to maintain such tests within our codebase where we occasionally try to test out patching thousands of documents and making sure Raven applies those update, such that it serves to make sure we are using Raven constructs appropriately such as in this case better utilization of sessions (since its all in-memory I believe it won't really provide a true test for us since as you mentioned we're skipping fsync and potentially other network related issues that might occur)? or that would be an overkill or maybe there's better to think about these tests on our side. Thanks

[ayende]

There is a place for stress testing, yes. But I don't think that this is what you are doing.
Are you trying to verify RavenDB's behavior? That is exploratory testing (very useful), but generally not something that you'll keep over the long run.

The question here is what it is that you are trying to test? The behavior of your code (keep the test) or RavenDB (in which case, we'll be happy to accept a PR with a test)

[yj7o5]

We are trying to test that our service ops are using the right constructs of RavenDB i.e. session vs patches for service ops that are expected to take from a few updates to a couple thousands and how we're making use of those constructs. For example, how service ops behave (or more specifically how DB reacts) when a flood of data comes across due to outage of an upstream system, etc. case in point this test above.

[ayende]

That is exploratory testing or benchmarks - don't add that as a regular part of your test suite.
In RavenDB, we have separate projects dedicated for that.