Make Unity publishing easier

[reeseschultz]

@genaray I've been integrating Arch into Unity (for reasons), and noticed something peculiar.

Long-winded exposition

For Arch itself (edit: and non-codegen extensions), I only needed to pull these nuggets/transitive nuggets (2.7MB):

• Collections.Pooled
• CommunityToolkit.HighPerformance
• JobScheduler
• MessagePack
• Microsoft.Bcl.AsyncInterfaces
• Microsoft.Extensions.ObjectPool
• Microsoft.NET.StringTools
• System.Collections.Immutable
• System.Runtime.CompilerServices.Unsafe
• Utf8Json

And then Arch works beautifully. Note the absence of Roslyn (Microsoft.CodeAnalysis), despite the fact that IIncrementalGenerator is used in base Arch. So... how is codegen happening at all? Refer to sschmid/Entitas#1005 (comment). I'm assuming you must be intentionally using 4.1.0 for the same reason as the creator of Entitas (even though Unity's documentation is clearly wrong)? Or is this just a coincidence?

I figure the native Roslyn support was instigated by the extensive codegen in the Entities package for DOTS, which I might be familiar with. The project I'm testing Arch in is not directly using DOTS (yet), but Roslyn is still generally accessible. We know that DOTS is invisibly subsuming old editor/runtime features to fend off tech debt, so this makes sense.

The problem

All of that brings me to Arch.Extended. The codegen extensions are all doing this in their respective csproj:

<PackageReference Include="Microsoft.CodeAnalysis.CSharp.Workspaces" Version="4.1.0" PrivateAssets="all" />

Pay attention to PrivateAssets="all". Base Arch codegen does not include that key-value pair. Keeping it in the extensions, or at all, is a problem for Unity users, assuming you want to support them (I think you should, because Arch can actually help them transition away from being directly dependent on its capabilities). And remember that Unity has never properly addressed dependency management, so that leaves three options for pulling Roslyn and its transitive dependencies:

• UnityNuGet where Arch is not presently available in its artifact repository, so I did not test it
• NuGetForUnity which thrashes and crashes the editor repeatedly, and ends up not retrieving the dependencies it needs pertaining to SQLite (having to do with transitive SQLitePCLRaw.batteries_v2 not being where no human nor machine would intuit)
• Manually, which is what I actually did because I hate myself

So what has to be pulled down when PrivateAssets="all"? About 31MB of:

• Humanizer
• Microsoft.CodeAnalysis
• Microsoft.CodeAnalysis.CSharp
• Microsoft.CodeAnalysis.CSharp.Workspaces
• SQLitePCLRaw.batteries_v2 (which should actually come from SQLitePCLRaw.bundle_green, whatever the hell that means
• SQLitePCLRaw.core
• SQLitePCLRaw.provider
• System.Composition.AttributeModel
• System.Composition.Hosting
• System.Composition.Runtime
• System.Composition.TypedParts
• System.IO.Pipelines
• System.Reflection.Metadata

Not only does this make direct Arch-Unity integration a massive pain, but also: what if I want to supply a UPM package that uses Arch.Extended as a dependency? (Which maybe I do.) Packing 2.7MB of DLLs into a package, with correct author/license attribution, is not awesome, but not horrible. The additional 31MB is pretty bad, though, especially when NuGet itself can't determine where one dependency is supposed to come from. Fortunately all of the nuggets are Apache 2.0 or MIT-licensed (mostly MIT), so redistribution is perfectly fine from a legal perspective either way.

Concluding thoughts

Please consider my proposal to remove all instances of PrivateAssets="all" from your csproj files in the context of Roslyn. That's all I'm asking. Your users may end up using codegen anyway. Is it technically the right thing to expose your dependencies to them? Well, generally no, but this is a case where an exception should be made. It would seem I too am headed down codegen alley, despite regarding it as an anti-pattern in my past life as an enterprise dev. Seems like Rust is the only popular language properly geared for ECS due its trait system.

Obligatory regards: Arch is awesome. You're a veritable ECS prodigy. What you're doing is fantastic for Mono, Unity, Godot and other C# game devs, and you should be proud of yourself.

[genaray]

First of all, thank you for your interest in Arch, your kind words and this post here! :)

The only question I have is, would removing PrivateAssets="all" have any disadvantage? Should it be removed completely or set to another value instead? And could it lead to potential problems?

[genaray]

One more small question :)
What is required to run the build script? It tells me that no project was found: MSBUILD : error MSB1003: Specify a project or project map file. The current working directory does not contain a project or project map file.

For it to work, the script probably needs to be copied into the correct dictionary, right? ^^
Edit: A small usage example would be great, so i can add it to the wiki :)

[reeseschultz]

You should be able to run ./scripts/UnityPublish.sh from the terminal from the root directory of the project. Let me know if that works. It works for me.

[genaray]

Unfortunately it does not work on my end (windows). This error is thrown:

Projects to be restored are determined...
  All projects are up to date for recovery.
MSBuild version 17.4.0-preview-22470-08+6521b1591 for .NET
MSBUILD : error MSB1009: The project file does not exist.
Switch: t:Unity

Any idea why this happens? I was in the root and executed the .sh.

[reeseschultz]

Ah, probably because of this line in the script:

assemblyDir="`pwd`/dist/Assemblies"

Long ago, to be arbitrarily different, Microsoft decided Windows should use backslashes instead of forward slashes. So directory traversal on Windows is incompatible with pretty much every other operating system.

I recommend using Git BASH, and then you should no longer have a problem with the slashes. Note that the older build script uses forward slashes just as I have with the Unity publishing script. Generally for automation we target Linux since it's cheap and secure when running scripts in the cloud, but you run into compatibility issues when running them locally.

[genaray]

Ah thank you! Funnily enough I ran the script with git bash but the error remains the same :D
Pretty much out of ideas rn.