Allow to Disable All Automatic Relationships

[jasonterando]

This is related to 11564 but my situation may be a bit different. If this is seen as a duplicate, please accept my apologies and close this out.

I've inherited a very nasty MySQL database with a large number of tables. It has landmines like tables without primary keys, inconsistent naming conventions, etc. To make things worse, there is enough "legacy code" sitting around that updating the schema is difficult.

I can make things work in EF.Core by setting everything up via Fluent syntax in DbContext.OnModelCreating. When my DbContext gets created though, there is a lot of wasted effort by EF.Core when my models are scanned and results in lots of warnings when it tries to create relationships that I already know won't work (and don't care about and don't need).

My ask: Give me something like a ModelBuilder.DisableAutomaticRelationships function. I don't mind putting in the work to define stuff fluently. This will help with cold start time and allow me to keep EF.Core warning log entries enabled, without having worry about signal-to-noise.

[roji]

This will help with cold start time and allow me to keep EF.Core warning log entries enabled, without having worry about signal-to-noise.

I'm not sure disabling automatic relationship detecting specifically would have a noticeable impact on startup time, and if it does, that's probably true only for pretty massive models with very large numbers of entities etc. In any case, you can use compiled models allow you to generate your model once, at design-time (or in CI), rather than each time your application starts.

Regarding the warnings, you can easily disable these - see the docs.

[jasonterando]

Thanks for the quick response! I'll look into setting up compiled models and how that would work in our environment (I'm building data access assemblies for use in multiple applications, so it sounds like I may need to stand up a "dummy" startup project to inject configuration information required to instantiate my DbContext classes to get the ef dbcontext optimize to work).

In the meantime, I'll go with disabling the warnings.

[smitpatel]

Also with #214 should be able to remove RelationshipDiscoveryConvention which will avoid adding any navigations/relationships by convention. (May be need to remove some other conventions too). cc: @AndriySvyryd

[jasonterando]

I'm going to follow up on this this a bit, even though I closed the issue. The notion that the automatic relationship building doesn't add much to startup time is definitely not true in my case. It's pretty significant.

To confirm this, I have a webservice that basically checks to see if an ID exists. The schema in question is a MySQL database with 769 tables, with 1,706 principal and 1,925 dependent relationships. Not huge, but not small. I have this running in a AWS Lambda function configured at 1 GB of RAM. Initially, I thought my cold start times were the overhead of .NET itself, but I wanted to confirm this, so I created a bare-bones application that does a single base call.

1. I created a controller which does nothing more than call a service that checks to see if the submitted record ID already exists, and track the time spent in the controller itself (which should isolate performance variance due to connectivity between me and AWS)
2. Deploy the application to Lambda using the full schema DbContext (all tables/DbSets) and again using a limited schema (only the six or so tables that are in scope for the API I am currently working on).
3. For each deployment perform three "cold starts" (forcing the Lambda to "reload") followed by three "warm runs" (calls after the initial cold start).

As shown in the table below, the overhead of the additional tables in DbContext added 20+ seconds of startup time. This is brutal.

Here's the thing. I would really like to have a "standard" data layer assembly that my applications can use when accessing the schema. I want to avoid having DbSets defined all over the place that I have to hunt down if we update the schema. But 20 seconds of overhead is a killer for microservices or an auto-scaled environment. And it's unnecessary. I'm more than willing to explicitly define my relationships, and have done so. There is no benefit, and apparently significant performance impact, to whatever is being done to by EF.Core when looking for relationships, unless there is something else going on that I can disable.

As of the moment, I'm not sure it makes sense to use EF.Core in an application that will be deployed as a microservice, or in an auto-scaled environment where 20+ seconds of extra init time are not acceptable.

Anyways, thanks for the work. I really like using the framework, and if this one tiny thing, disabling automatic relationships were a possible (by convention like in #214 or otherwise, I'd probably be set. I've also started looking into compiled models, but the lack of change tracking seems like a bit of a deal breaker.

	Total Seconds	Seconds In Controller
Limited Schema DBContext
Cold Start	2.81	0.799
Cold Start	2.77	0.798
Cold Start	2.79	0.835
Warm Run	0.165	0.081
Warm Run	0.129	0.03
Warm Run	0.089	0.06
Full Schema DBContext
Cold Start	23.36	21.237
Cold Start	23.52	21.535
Cold Start	22.68	20.624
Warm Run	0.479	0.085
Warm Run	0.112	0.023
Warm Run	0.087	0.004

[ajcvickers]

@jasonterando You're saying the model building is taking time; this is understandable. This is very different from saying "automatic relationship detecting", which is only a small part of model building, is the reason why building the model is taking time.

The lack of change tracking seems like a bit of a deal breaker

What are you referring to?

[roji]

To add to @ajcvickers's comments, we definitely have not seen 20 second of startup time due to model building; see this blog post, where a model with 449 entity types and 720 relationships takes around two seconds to start up - and that's without compiled models (with it, startup goes down to 257ms). You have more relationships and a different model so anything is possible, but that is still quite the gap.

In addition, if I read you comment correctly, you haven't compared your startup to a baseline without ef core (or with a minimal models), so it's impossible to know where exactly the 20 second are coming from.

To pursue this, please try reproducing the long startup time outside of Azure Functions, comparing to a proper baseline.

[jasonterando]

@ajcvickers - Thanks for the response. I believe "change tracking proxies" refers to an entity keep track of what fields where changed so updates are only for those fields. May be mistaken though, not yet super-fluent in EF.Core. If it's just the INotifyCollectionChanged events, though, I can definitely live without that in my use case.

@roji - The 2 second cold start time show in that table above are with minimal models.

As mentioned, I've inherited a bit of a mess of a database. We're trying to incrementally improve it, but it's a bit of a challenge because we've got code all over the place accessing it.

I think my answer is the compiled models, and I'll definitely start experimenting with those today. If we have a use case that requires change tracking, we can create a purpose-built DbContext for that.

Thanks again for the replies.

[ajcvickers]

@jasonterando Change-tracking proxies are rarely used, and I certainly wouldn't recommend them. They certainly are not needed for change tracking in general.

[jasonterando]

@ajcvickers - got it. one more question then, before I completely wear out my welcome ;)

I'm putting together a small console app to test compiled models. I'm using CreateHostBuilder to set up the DbContextFactory and that appears ok, but I'm getting the an error about "Microsoft.EntityFrameworkCore.Design.Internal.ICSharpRuntimeAnnotationCodeGenerator". I've included Microsoft.EntityFramework.Core/Design/Relational and Relational.Design packages. Did I miss one?

Compiling EF.Core data model
Build started...
Build succeeded.
info: Microsoft.EntityFrameworkCore.Infrastructure[10403]
      Entity Framework Core 6.0.3 initialized 'MyDbContext' using provider 'MySql.EntityFrameworkCore:6.0.0+MySQL8.0.28' with options: MaxPoolSize=1024 
System.InvalidOperationException: Unable to resolve service for type 'Microsoft.EntityFrameworkCore.Design.Internal.ICSharpRuntimeAnnotationCodeGenerator' while attempting to activate 'Microsoft.EntityFrameworkCore.Scaffolding.Internal.CSharpRuntimeModelCodeGenerator'.

[ajcvickers]

@jasonterando Not sure; @AndriySvyryd might know. However, if you're using MySQL, then I would recommend using the Pomelo provider: https://www.nuget.org/packages/Pomelo.EntityFrameworkCore.MySql/. People tend to have much better luck with it over the "official" provider from Oracle.

[jasonterando]

Thanks. Tried swapping Oracle for Pomelo, same error. Next thing I'll try is downgrading EF.Core assemblies to a an earlier version, maybe the issue is 6.0.3 specific

[jasonterando]

6.0.1 packages give same error about ICSharpRuntimeAnnotationCodeGenerator. Feels like I'm either missing a package or missing a line in ConfigureServices - but have no idea which.

The full stack trace in all of its glory:

Build started...
Build succeeded.
info: Microsoft.EntityFrameworkCore.Infrastructure[10403]
      Entity Framework Core 6.0.1 initialized 'MyDbContext' using provider 'MySql.EntityFrameworkCore:6.0.0+MySQL8.0.28' with options: MaxPoolSize=1024 
System.InvalidOperationException: Unable to resolve service for type 'Microsoft.EntityFrameworkCore.Design.Internal.ICSharpRuntimeAnnotationCodeGenerator' while attempting to activate 'Microsoft.EntityFrameworkCore.Scaffolding.Internal.CSharpRuntimeModelCodeGenerator'.
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.CreateArgumentCallSites(Type implementationType, CallSiteChain callSiteChain, ParameterInfo[] parameters, Boolean throwIfCallSiteNotFound)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.CreateConstructorCallSite(ResultCache lifetime, Type serviceType, Type implementationType, CallSiteChain callSiteChain)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.TryCreateExact(ServiceDescriptor descriptor, Type serviceType, CallSiteChain callSiteChain, Int32 slot)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.TryCreateEnumerable(Type serviceType, CallSiteChain callSiteChain)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.CreateCallSite(Type serviceType, CallSiteChain callSiteChain)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.CreateArgumentCallSites(Type implementationType, CallSiteChain callSiteChain, ParameterInfo[] parameters, Boolean throwIfCallSiteNotFound)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.CreateConstructorCallSite(ResultCache lifetime, Type serviceType, Type implementationType, CallSiteChain callSiteChain)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.TryCreateExact(ServiceDescriptor descriptor, Type serviceType, CallSiteChain callSiteChain, Int32 slot)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.TryCreateExact(Type serviceType, CallSiteChain callSiteChain)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.CreateCallSite(Type serviceType, CallSiteChain callSiteChain)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.CreateArgumentCallSites(Type implementationType, CallSiteChain callSiteChain, ParameterInfo[] parameters, Boolean throwIfCallSiteNotFound)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.CreateConstructorCallSite(ResultCache lifetime, Type serviceType, Type implementationType, CallSiteChain callSiteChain)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.TryCreateExact(ServiceDescriptor descriptor, Type serviceType, CallSiteChain callSiteChain, Int32 slot)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.TryCreateExact(Type serviceType, CallSiteChain callSiteChain)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.CreateCallSite(Type serviceType, CallSiteChain callSiteChain)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.GetCallSite(Type serviceType, CallSiteChain callSiteChain)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(Type serviceType)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Microsoft.EntityFrameworkCore.Design.Internal.DbContextOperations.Optimize(String outputDir, String modelNamespace, String contextTypeName)
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OptimizeContextImpl(String outputDir, String modelNamespace, String contextType)
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OptimizeContext.<>c__DisplayClass0_0.<.ctor>b__0()
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OperationBase.Execute(Action action)
Unable to resolve service for type 'Microsoft.EntityFrameworkCore.Design.Internal.ICSharpRuntimeAnnotationCodeGenerator' while attempting to activate 'Microsoft.EntityFrameworkCore.Scaffolding.Internal.CSharpRuntimeModelCodeGenerator'.

[jasonterando]

If relevant, I'm running .NET 6.0.201 on Linux:

$ dotnet --version
6.0.201
$ dotnet ef --version
Entity Framework Core .NET Command-line Tools
6.0.3

I've went ahead and restored packages and CLI tools back to 6.0.3

[ajcvickers]

@jasonterando Please attach a small, runnable project or post a small, runnable code listing that reproduces what you are seeing so that we can investigate.

[ajcvickers]

EF Team Triage: Closing this issue as the requested additional details have not been provided and we have been unable to reproduce it.

BTW this is a canned response and may have info or details that do not directly apply to this particular issue. While we'd like to spend the time to uniquely address every incoming issue, we get a lot traffic on the EF projects and that is not practical. To ensure we maximize the time we have to work on fixing bugs, implementing new features, etc. we use canned responses for common triage decisions.