MySQL operations break after InvalidOperationException: Can't Replace Active Reader

[adanvdo]

Steps to reproduce

We are using .NET 8 and DI in our project. We have several categorized classes that create a new scope from the IServiceScopeFactory in order to access repository methods that query the database.

Some of these methods are called multiple times or at close to the same time from different points in our code.

testManager1.cs

public class TestManager(IServiceScopeFactory serviceScope)
{
  public async Task GetData()
  {
    using var scope = serviceScope.CreateScope();
    var testRepository = scope.ServiceProvider.GetRequiredService<ITestRepository>();
    var data = await testRepository.GetDataAsync();
  }
}

testManager2.cs

public class TestManager2(IServiceScopeFactory serviceScope)
{
  public async Task GetData2()
  {
    using var scope = serviceScope.CreateScope();
    var testRepository2 = scope.ServiceProvider.GetRequiredService<ITestRepository2>();
    var differentData = await testRepository2.GetDataAsync();
  }
}

etc..

every few seconds, call either testManager1.GetData() or testManager2.GetData2() randomly.

Eventually, we get the Can't Replace Active Reader error at MySqlConnector.MySqlDataReader.InitAsync
This happens subsequently after Pomelo.EntityFrameworkCore.MySql.Storage.Internal.MySqlExecutionStrategy.ExecuteAsync is called

The issue

MySql operations break after an exception is thrown by MySqlConnector. Once the exception occurs, we can no longer perform any reads on the db without restarting our app. After a restart, it will run fine for anywhere from a couple hours to a couple days, and then the exception occurs again.

System.InvalidOperationException: Can't replace active reader.
    at MySqlConnector.MySqlDataReader.InitAsync(CommandListPosition commandListPosition, ICommandPayloadCreator payloadCreator, IDictionary`2 cachedProcedures, IMySqlCommand command, CommandBehavior behavior, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlDataReader.cs:line 478
    at MySqlConnector.Core.CommandExecutor.ExecuteReaderAsync(CommandListPosition commandListPosition, ICommandPayloadCreator payloadCreator, CommandBehavior behavior, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/CommandExecutor.cs:line 56
    at MySqlConnector.MySqlCommand.ExecuteReaderAsync(CommandBehavior behavior, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlCommand.cs:line 357
    at MySqlConnector.MySqlCommand.ExecuteDbDataReaderAsync(CommandBehavior behavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlCommand.cs:line 350
    at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
    at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
    at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.InitializeReaderAsync(AsyncEnumerator enumerator, CancellationToken cancellationToken)
    at Pomelo.EntityFrameworkCore.MySql.Storage.Internal.MySqlExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func`4 operation, Func`4 verifySucceeded, CancellationToken cancellationToken)
    at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
    at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ToListAsync[TSource](IQueryable`1 source, CancellationToken cancellationToken)
    at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ToListAsync[TSource](IQueryable`1 source, CancellationToken cancellationToken)
    at Project.Repository.Test.SQLTestRepository.GetData() /Project/Repository/Test/SQLTestRepository.cs:line 1

Further technical details

MySQL version: 8.0.18-google
Operating system: Linux
Pomelo.EntityFrameworkCore.MySql version: 8.0.2
Microsoft.AspNetCore.App version: 8.0.4

Other details about my project setup:

our AppDbContext is based on DbContext
we are using services.AddDbContextPool<AppDbContext> when configuring services

[lauxjpn]

The Can't Replace Active Reader exception is thrown by MySqlConnector, when there is still another query operation happening for the same connection at the same time.

One of the following scenarios might lead to this exception:

• You are explicitly using the same connection in multiple threads.
• You are missing (intentionally or unintentionally) an await statement for one of your async method calls somewhere in your call stack, which could then lead to a second query operation while the first one is still in progress. Usually, the compiler warns you about that with CS4014, so look out for that warning when compiling.
• You are iterating over the result of a query operation in a foreach loop, with the query operation not using ToList[Async], ToArray[Async] or As[Async]Enumerable, leading to an open data reader in the foreach loop. If you then start a second query operation using the same DbContext from within that foreach loop, the data reader for the second/inner query operation could result in the exception you are experiencing.

EF Core and MySqlConnector both do not support parallel access from multiple threads to the same DbContext/MySqlConnection object.

(You can also take a look at possible similar issues (that throw a different exception) like #1436, #1512, #1608.)

[adanvdo]

• You are iterating over the result of a query operation in a foreach loop, with the query operation not using ToList[Async], ToArray[Async] or As[Async]Enumerable, leading to an open data reader in the foreach loop. If you then start a second query operation using the same DbContext from within that foreach loop, the data reader for the second/inner query operation could result in the exception you are experiencing.

Thank you. I will have to dig through the code to see if this is possibly happening. I am confident that the other two causes you listed would not be, but I can keep an eye out for those too.