F-Bounded Tasks / Microservices

[jzabroski]

The idea here is to introduce a new way to encapsulate various Tasks that are used by the three FluentMigrator "Task Runners".

Currently, FluentMigrator not only has three Task Runners, but each one has a different API for the tasks themselves, which increases overall code complexity:

• FluentMigrator.Console
• FluentMigrator.DotNet.Cli
• FluentMigrator.MSBuild

FluentMigrator.Console primarily exists to support legacy .NET Framework. The other use case it supports is fully bundled dependencies, such that the nupkg file contains a zip of all libraries it needs, without needing to download them from a nuget package server.

FluentMigrator.DotNet.Cli exists to support .NET Core users. It does not support fully bundled dependencies, but we could always create a FluentMigrator.DotNet.Cli.Bundled re-packaging of FluentMigrator.DotNet.Cli.

FluentMigrator.MSBuild exists to provide "red blood" in the Windows Terminal so that when a migration fails, the error message is highlighted in red ink. MSBuild users can use FluentMigrator.DotNet.Cli to run their migrations, but they lose the error stream and red link that the FluentMigrator.MSBuild task provides.

Long-term, it would be nice to encapsulate each Task as a Micro-Service, and have FluentMigrator.MSBuild and FluentMigrator.DotNet.Cli use the same micro-services under the hood.

Goals

• Analyze lifetime scope of task dependencies
• Propose API. As hinted in title, it should implement F-Bounded Polymorphism such that the interface is something like FluentMigrator.Abstractions.ITaskDefinition<T> where T : FluentMigrator.Abstractions.ITask
• Demonstrate that, similar to .NET Task API, you can ContinueWith another FluentMigrator task.
• YAGNI - Prove why we also can't just use .NET Task API directly and implement Async All The Way.

Rough Sketch of API

    public interface ITask<TTaskContext>
    {
        ITask<TTaskContext> Run(TTaskContext taskContext);
    }

    public interface ITaskDefinition<TTask, TTaskContext> where TTask : ITask<TTaskContext>
    {
        string Key { get; }
        // Should task state belong to the definition, as opposed to the individual task?
        // The idea is that the task is just a process and shouldn't keep any state,
        // whereas the definition allows hydrating and dehydrating state.
        // The interface here exposes only a getter, but it does not guarantee immutability.
        // So, you could get a concrete TaskDefinition<MigrateUp, UpMigrationContext> instance
        // where the UpMigrationContext's properties then needs to be populated procedurally
        // Is there a better way to do this?
        // Need to provide examples of how two micro-tasks would feed into each other.
        // But the core idea is to build an API that provides:
        // (1) Processor(s)
        // (2) Processor Context
        TTaskContext { get; } 
    }

    // TODO: Provide API for how to glue consecutive tasks together.

Thoughts on Rough Sketch API

Why F-Bounded Polymorphism?

F-Bounded Polymorphism is a useful trick to allow important interface changes in derived types. In the case of FluentMigrator, each task has a unique set of parameters (inputs) and a unique set of return values/outputs.

The main reason not to use F-Bounded Tasks is if all the Tasks have the same homogeneous structure, and there is no scenario where one task may want to take its outputs and map them into the inputs of another task. Given FluentMigrator's current architecture, one could argue this is over-engineering. But what if we took all the existing tasks in FluentMigrator and broke them down into smaller pieces. For example, migrate:up task is really a composition of many other tasks:

• migrate:up range=1,4 --profile ProfileName --tags Dev,Alpha,Beta,Production --preview-only is actually four tasks in one, at minimum:
  1. migrate:up range=1,4 suggests to look at the VersionInfo table to determine what range of migrations need applying
  2. --profile ProfileName suggests to look at the ProfileSource dependency to determine what profiles to run and when to run them
  3. --tags Dev,Alpha,Beta,Production suggests to take the output from task one and apply an additional filter on the migrations by requiring they have these tags.
  4. --preview-only suggests to construct a "unit of work" where the persistent store is a text file

However, so far, FluentMigrator's internal APIs reflect how the consumers most typically use the task runners. This limits degrees of freedom, as there are no micro-tasks to easily create linear combinations of logic/features.

• In general, some people suggest F-Bound Polymorphism should be avoided, because:
  1. Each time a user wants to reference an object of an FBound type, they must also refer to the FBound type parameter.
  2. The compiler cannot infer FBound types. As the previous point suggests, it is up to the developer to tell the compiler the explicit type providing the FBound. In C#, you can smooth some of this out by only needing to provide the FBound once, but it is still a requirement. In OO languages with Inversion Of Control, the normal way I see developers overcome this limitation is by creating a Factory that "knows" the manual procedural process of constructing a proper FBound. (I believe) this is often referred to as "Assisted Inject" in advanced Inversion Of Control containers.

YAGNI - Why Not Just use .NET Task API?

There are some reasons why we shouldn't use the .NET Task API:

1. It doesn't have deterministic error handling. Namely, if your task throws an unobserved exception, there is no clean way to recover. UnobservedTaskException event won't run immediately after the failure.
2. "Async all the way" is the only way to guarantee deadlock-free code. In FluentMigrator, every Task should just be a "Value" and should not care "where" you run it.

[rballonline]

Long-term, it would be nice to encapsulate each Task as a Micro-Service, and have FluentMigrator.MSBuild and FluentMigrator.DotNet.Cli use the same micro-services under the hood.

This might be where I could help, not sure. But unifying all ways of producing SQL scripts would be ideal. I wonder if you could use a plug-in type of system like MEF to achieve this. Instead of including all runners, the end user just has a plug-in directory with the runner in it. I'm new to all of this though so I could just be saying a bunch of nonsense.

Instead of pointing to an assembly, it just takes whatever's in a directory and does it's thing.

[jzabroski]

There are two main open source plugin systems for .NET Core.

1. Nate McMaster's McMaster.DotNetCore.Plugins
2. Dapplo.Addons.

Nate goes into great detail on his blog explaining how .NET Core AssemblyLoadContext works and how it can (and can't) be used to create pluggable applications.

This particular discussion is a bit dated. I had private email discussion with another developer who convinced me a Tagless Final interface is a better microservice interface than f-bounded polymorphism. (If you're not familiar with Tagless Final Interpreters, reading up on them for the first time might be a bit head bending, but the basic idea is ultra strict adherence to a very generic interface).