How to fix Entity Framework issues with Nx and EF Core

[TheDevelolper]

When trying to perform a database update using dotnet ef database update I was seeing the following error:

error MSB4057: The target "GetEFProjectMetadata" does not exist in the project.

Unable to retrieve project metadata. Ensure it's an SDK-style project. 

If you're using a custom BaseIntermediateOutputPath or MSBuildProjectExtensionsPath values, 
Use the --msbuildprojectextensionspath option.

Long story short, this was caused by an issue where nx-dotnet forces all projects to be built into a dist folder regardless off whether they're dlls or applications etc. EntityFramework is unable to use the configuration to work out where it should go to read its metadata.

I came up with a solution that works for me and I wrote an article about it here:

https://dev.to/simplifycomplexity/using-entityframework-in-a-nxdotnet-environment-cfb

I hope this helps anyone else who is stuck with this problem as it took me a whole weekend to solve.

[bcallaghan-et]

This is a now-common error, ever since nx-dotnet chose to move the output directory for build intermediaries. The previous folder (following .NET defaults) was {projectRoot}/obj. The new folder (following Nx convention) is dist/intermediates/{projectRoot}/obj. This convention was adopted to more-properly handle artifact caching and Distributed Task Execution (DTE).

Previously, Nx and nx-dotnet would only cache the final build output. Though .NET defaults to storing final output in {projectRoot}/bin, the Nx convention is to store it in dist/{projectRoot}. It's worth mentioning that moving the bin folder did not break the dotnet or dotnet ef commands. By moving the intermediates into dist (and by updating some root-level Nx configuration), all .NET build output is cached. This improves the speed of re-building the workspace, but it also solves a problem that only exists in DTE. Projects further along the dependency graph require the obj folder of their dependencies, not just the final build output. If the obj folder is not properly cached, then it is not properly synced between the various machines of DTE, and later builds will fail.

The solution you provide in your article satisfies the dotnet ef commands, but it thoroughly breaks Nx caching, and thus re-breaks DTE. By changing the Directory.Build.props file (or the project-local copies), you changed where the build outputs are saved, but you did not tell Nx that you moved the outputs. Ultimately, Nx doesn't care where the outputs are saved (dist is only a convention), but it does need to be told where those outputs are.

The quick-and-dirty solution is to use the --msbuildprojectextensionspath option, as shown in the error message. Using your example above, the new command would be

dotnet ef database update --msbuildprojectextensionspath dist/intermediates/{projectRoot}/obj

where {projectRoot} is replaced with the full path to the project.

If typing that flag each time is too cumbersome, then the solution in your article is a step in the right direction. The two lines that should be removed from Directory.Build.props are

<BaseIntermediateOutputPath>$(RepoRoot)dist/intermediates/$(ProjectRelativePath)/obj</BaseIntermediateOutputPath>
<IntermediateOutputPath>$(BaseIntermediateOutputPath)</IntermediateOutputPath>

Then, the outputs property of all .NET build targets should be updated to include {projectRoot}/obj. The combination of these changes will resetting the intermediates folder to the .NET default of obj and instruct Nx to cache that obj folder.

A third possible option is to create a custom generator that automatically adds the -msbuildprojectextensionspath option with the correct path. We may want to consider adding this within nx-dotnet itself, but doing so on our end would require wrapping every possible dotnet ef sub-command, which is tedious at best.

It's a shame that Entity Framework does not respect the BaseIntermediateOutputPath or IntermediateOutputPath properties. If it did, then it would automatically look for the intermediates in the correct location. This is a known bug within EF Core, and there are other reasons to redirect the obj folder that are not Nx-related.

[AgentEnder]

Thanks for the solid write up Ben! This is all accurate - I really wish the EF Core bug was fixed.

[TheDevelolper]

Wow I knew I couldn't be the only person with this issue. Thank you for the explanation. I'll update my article an I'll also reference this discussion. Thank you.

[sharky98]

A word of caution (me losing few hours on this, before searching for answers), if you split your migration project and startup project (like... a typical Nx workspace, having the migrations and context in a Nx libs and the startup being the apps!), then the --msbuildprojectextensionspath is useless as it will be used for both --startup-project and --project.

A (very ugly) workaround that may work, is running the command twice, modifying the --msbuildprojectextensionspath parameter to point for the first project, then second project. It would create [project].csproj.EntityFrameworkCore.targets files in the obj directory. Then it may work; at least it worked once for me (but not anymore 🤷🏼)!

[sharky98]

Just to document better my maybe-working-workaroung.

I have an app (apps/my.backend/My.Backend.csproj) that initalize the context and make the database connection.
The context is in its own library (libs/my.context/My.Context.csproj), alongside its migrations.
I do the following steps.

1- rimraffing dist/ and nx/ (going for the total clean slate).
2- building apps/my.backend/My.Backend.csproj with dotnet build (or the executor).
3- building libs/my.context/My.Context.csproj with dotnet build (or the executor).

Now, I have dist/intermediates/apps/my.backend/obj and dist/intermediates/libs/my.context/obj that contains the build intermediates.

initial directory tree

dist/
└── intermediates/
    ├── apps/
    │   └── my.backend/
+   │       └── obj/
+   │           ├── net8.0/
+   │           ├── project.assets.json
+   │           ├── project.nuget.cache
+   │           ├── My.Backend.csproj.nuget.dgspec.json
+   │           ├── My.Backend.csproj.nuget.g.props
+   │           └── My.Backend.csproj.nuget.g.targets
    └── libs/
        └── my.context/
+           └── obj/
+               ├── net8.0/
+               ├── project.assets.json
+               ├── project.nuget.cache
+               ├── My.Context.csproj.nuget.dgspec.json
+               ├── My.Context.csproj.nuget.g.props
+               └── My.Context.csproj.nuget.g.targets

Then I run the following.

me@work:~> dotnet ef migrations add DullMonkeysDivide --msbuildprojectextensionspath dist/intermediates/apps/my.backend/obj --startup-project apps/my.backend/My.Backend.csproj --project libs/my.context/My.Context.csproj --context MyContext --no-build

libs/my.context/My.Context.csproj : error MSB4057: The target "GetEFProjectMetadata" does not exist in the project.
Unable to retrieve project metadata. Ensure it's an SDK-style project. If you're using a custom BaseIntermediateOutputPath or MSBuildProjectExtensionsPath values, Use the --msbuildprojectextensionspath option.

It fails, but it create a new file. Note that this is the context version of the file!

directory tree after pointing `--msbuildprojectextensionspath` to the app

dist/
└── intermediates/
    ├── apps/
    │   └── my.backend/
    │       └── obj/
    │           ├── net8.0/
    │           ├── project.assets.json
    │           ├── project.nuget.cache
    │           ├── My.Backend.csproj.nuget.dgspec.json
    │           ├── My.Backend.csproj.nuget.g.props
    │           ├── My.Backend.csproj.nuget.g.targets
+   │           └── My.Context.csproj.EntityFrameworkCore.targets
    └── libs/
        └── my.context/
            └── obj/
                ├── net8.0/
                ├── project.assets.json
                ├── project.nuget.cache
                ├── My.Context.csproj.nuget.dgspec.json
                ├── My.Context.csproj.nuget.g.props
                └── My.Context.csproj.nuget.g.targets

Now, let's just be bold, and re-run but with the other obj directory.

me@work:~> dotnet ef migrations add DullMonkeysDivide --msbuildprojectextensionspath dist/intermediates/libs/my.context/obj --startup-project apps/my.backend/My.Backend.csproj --project libs/my.context/My.Context.csproj --context MyContext --no-build

apps/my.backend/My.Backend.csproj : error MSB4057: The target "GetEFProjectMetadata" does not exist in the project.
Unable to retrieve project metadata. Ensure it's an SDK-style project. If you're using a custom BaseIntermediateOutputPath or MSBuildProjectExtensionsPath values, Use the --msbuildprojectextensionspath option.

It still fails, but now, it created two new files!

directory tree after pointing `--msbuildprojectextensionspath` to the lib

dist/
└── intermediates/
    ├── apps/
    │   └── my.backend/
    │       └── obj/
    │           ├── net8.0/
    │           ├── project.assets.json
    │           ├── project.nuget.cache
    │           ├── My.Backend.csproj.nuget.dgspec.json
    │           ├── My.Backend.csproj.nuget.g.props
    │           ├── My.Backend.csproj.nuget.g.targets
    │           └── My.Context.csproj.EntityFrameworkCore.targets
    └── libs/
        └── my.context/
            └── obj/
                ├── net8.0/
                ├── project.assets.json
                ├── project.nuget.cache
                ├── My.Context.csproj.nuget.dgspec.json
                ├── My.Context.csproj.nuget.g.props
                ├── My.Context.csproj.nuget.g.targets
+               ├── My.Backend.csproj.EntityFrameworkCore.targets
+               └── My.Context.csproj.EntityFrameworkCore.targets

Ok, let's be crazy for real now. Re-run the first command (where we target the my.backend obj directory)! And it worked!

me@work:~> dotnet ef migrations add DullMonkeysDivide --msbuildprojectextensionspath dist/intermediates/apps/my.backend/obj --startup-project apps/my.backend/My.Backend.csproj --project libs/my.context/My.Context.csproj --context MyContext --no-build

....
Done. To undo this action, use 'ef migrations remove'

And new files are again created!

directory tree after pointing `--msbuildprojectextensionspath` to the app again

dist/
└── intermediates/
    ├── apps/
    │   └── my.backend/
    │       └── obj/
    │           ├── net8.0/
    │           ├── project.assets.json
    │           ├── project.nuget.cache
    │           ├── My.Backend.csproj.nuget.dgspec.json
    │           ├── My.Backend.csproj.nuget.g.props
    │           ├── My.Backend.csproj.nuget.g.targets
+   │           ├── My.Backend.csproj.EntityFrameworkCore.targets
    │           └── My.Context.csproj.EntityFrameworkCore.targets
    └── libs/
        └── my.context/
            └── obj/
                ├── net8.0/
                ├── project.assets.json
                ├── project.nuget.cache
                ├── My.Context.csproj.nuget.dgspec.json
                ├── My.Context.csproj.nuget.g.props
                ├── My.Context.csproj.nuget.g.targets
                ├── My.Backend.csproj.EntityFrameworkCore.targets
                └── My.Context.csproj.EntityFrameworkCore.targets

---

EDIT 1: After another test, if I run the command pointing --msbuildprojectextensionspath on the library first, it works when running on the app afterwards.

And now that I have both files My.Backend.csproj.EntityFrameworkCore.targets and My.Context.csproj.EntityFrameworkCore.targets in both obj, it works, no matter which obj directory I point with --msbuildprojectextensionspath.

---

EDIT 2: I saw that I only had Microsoft.EntityFrameworkCore.Design in the My.Backend.csproj project. I did add it to the My.Context.csproj too for testing. And it did not change anything! (And added few details above to slim down the post)

---

EDIT 3: What is worse? The created file are just full of variable, nothing specific to the project. So, a really dirty workaround could be to create that file as-is for both the My.Backend and My.Context project and save it to both project intermediates...

<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <Target Name="GetEFProjectMetadata">
    <MSBuild Condition=" '$(TargetFramework)' == '' "
             Projects="$(MSBuildProjectFile)"
             Targets="GetEFProjectMetadata"
             Properties="TargetFramework=$(TargetFrameworks.Split(';')[0]);EFProjectMetadataFile=$(EFProjectMetadataFile)" />
    <ItemGroup Condition=" '$(TargetFramework)' != '' ">
      <EFProjectMetadata Include="AssemblyName: $(AssemblyName)" />
      <EFProjectMetadata Include="Language: $(Language)" />
      <EFProjectMetadata Include="OutputPath: $(OutputPath)" />
      <EFProjectMetadata Include="Platform: $(Platform)" />
      <EFProjectMetadata Include="PlatformTarget: $(PlatformTarget)" />
      <EFProjectMetadata Include="ProjectAssetsFile: $(ProjectAssetsFile)" />
      <EFProjectMetadata Include="ProjectDir: $(ProjectDir)" />
      <EFProjectMetadata Include="RootNamespace: $(RootNamespace)" />
      <EFProjectMetadata Include="RuntimeFrameworkVersion: $(RuntimeFrameworkVersion)" />
      <EFProjectMetadata Include="TargetFileName: $(TargetFileName)" />
      <EFProjectMetadata Include="TargetFrameworkMoniker: $(TargetFrameworkMoniker)" />
      <EFProjectMetadata Include="Nullable: $(Nullable)" />
      <EFProjectMetadata Include="TargetFramework: $(TargetFramework)" />
      <EFProjectMetadata Include="TargetPlatformIdentifier: $(TargetPlatformIdentifier)" />
    </ItemGroup>
    <WriteLinesToFile Condition=" '$(TargetFramework)' != '' "
                      File="$(EFProjectMetadataFile)"
                      Lines="@(EFProjectMetadata)" />
  </Target>
</Project>

---

EDIT 4 (hopefully the last of the night!): Running dotnet ef --msbuildprojectextensionspath dist/intermediates/apps/my.backend/obj --project apps/my.backend/My.Backend.csproj --no-build dbcontext list does create the needed file. So the workaround could run this command for all project that have Microsoft.EntityFrameworkCore.Design on build time. Then, when running dotnet ef migrations add, pointing --msbuildprojectextensionspath to the library should do the trick. Still the startup-project XML file will be copied over. But hey, it would kinda works!