Allows to redirect a Project reference or NuGet package to be consumed via local folder or NPM packages instead of inlining it inside of the main project output

[MangelMaxime]

Description

Explanation written for the JavaScript target as perhaps other target can work differently

When using Fable to publish libraries on NPM or any other dependency management tool, then all their Fable dependencies are inlined in them.

This can cause issues because if there is a shared project between these NPM packages then testing the equality of the same type will always fails.

#2488
#2282

This is why for example the NPM package fable-library has been created.

I made some exploration in this repository https://github.com/MangelMaxime/fable-npm-packages-exploration where after compiling the project, I am adapting the import instruction to use the output of a separation compilation for the library and it seems like adapting the import is not too difficult and works.

I think it would be beneficial to Fable, to allow the user to specify "redirection" or "alias" for some of the project reference. This would remove a lot of limitation there is currently when releasing Fable compiled output.

My idea would be to introduce a new argument to Fable, which is of the form --redirect MyLib.Core=mylib-core and then at compilation when Fable seems a reference to something coming from MyLib.Core it would use mylib-core as the base path for the import. And Fable would not generate the inline output of MyLib.Core as it would not be used.

This behaviour is similar to what happen when we use --fableLib fable-library, meaning that --fableLib could be deprecated in the future in favour of --redirect fableLib=fable-library if we want to keep one syntax only.

I believe some of the code to implementation this new feature already exist in Fable because --fableLib is implement and there is also --exclude which allows to exclude from the compilation the local reference to a Fable plugin.

Use case

• MyAwesomeMathLib - shared project
• ProjectA which uses MyAwesomeMathLib
• ProjectB which uses MyAwesomeMathLib and ProjectA

Both ProjectA and ProjectB are to be published on NPM.

Right now, there are issues in ProjectB because it will have its own version of MyAwesomeMathLib while ProjectA does too. With this feature, we could have a NPM package with this NPM dependencies:

• my-awesome-math-lib
  • fable-library
• project-a
  • fable-library
  • my-awesome-math-lib
• project-b
  • fable-library
  • my-awesome-math-lib
  • project-a

@ncave @nojaf @dbrattli

Do you see any blockers?

[nojaf]

Hi, I think this makes sense but I'm entirely sure I fully grasp the problem space.

One of more challenging aspects of Fable is the duality of F# and the output language.
I'm going to spitball some thought here:

So MyAwesomeMathLib.fsproj gets compiled to JavaScript. That JavaScript gets published to npm?
Should this not behave the same as any other npm package with typescript types?
I guess something like:

module Math

let sum a b = a + b

compiled

export function sum(a,b) { return a + b }

But to consume this, do we not want to generate something like:

module Math

[<Import("sum", "my-awesome-math-lib")>]
let sum: int * int -> unit = jsNative

// I'm not sure if this needs to be `int * int` or `int -> int`
// Anyway

If we have that, the MyAwesomeMathLib behaves like any other npm package we want to consume.
Both the runtime assets and the bindings would come with the npm package and you would no longer have a nuget package instead. For a Fable library this is probably the sweet spot.

The consumer would need to include the bindings a tad manually <Compile Item="node_modules/my-awesome-math-lib/Bindings.g.fs" />. But other than that I think there is something here.

[MangelMaxime]

So MyAwesomeMathLib.fsproj gets compiled to JavaScript. That JavaScript gets published to npm?

Yes

Should this not behave the same as any other npm package with typescript types?

It could but then you will be limited to only JavaScript features meaning that you can't use method overloading or you binding will need to have specifics binding for each generated function.

type Test () =

    member _.Log (txt : string) =
        ()

    member _.Log (txt : int) =
        ()

generates

import { class_type } from "fable-library/Reflection.js";

export class Test {
    constructor() {
    }
}

export function Test_$reflection() {
    return class_type("Test.Test", void 0, Test);
}

export function Test_$ctor() {
    return new Test();
}

export function Test__Log_Z721C83C5(_, txt) {
}

export function Test__Log_Z524259A4(_, txt) {
}

So your binding would be quite different from the original F# code something like:

// Pseudo code not tested

[<Import("Test", "my-module")>]
type Test () =
	[<Import("Test__Log_Z721C83C5", "my-module")>]
    abstract Log : string -> unit
	[<Import("Test__Log_Z524259A4", "my-module")>]
    abstract Log : int -> unit

Here the idea is just to replace a portion of the import statement so consume the code from another Folder. It has the benefit of not requiring it you to write a binding for your F# code which can be complex.

import { sum } from "./MyAwesomeMathLib/Math.fs.js";

// becomes

import { sum } from "my-awseome-math-lib/Math.fs.js";

[nojaf]

It has the benefit of not requiring it you to write a binding for your F# code which can be complex.

Fable would need to generate that, but I would assume it has all the information to do this correctly.

import { sum } from "my-awseome-math-lib/Math.fs.js";

So you would basically redirect the Math.fs from fable_modules to the node modules?
The problem I see there is that the compiler options could be different for the user project versus the precompiled library project. Things like #if DEBUG could potentially not be respected.

[MangelMaxime]

The problem I see there is that the compiler options could be different for the user project versus the precompiled library project. Things like #if DEBUG could potentially not be respected.

True, the idea is to consider the consumed code as a library so compiled in production mode.

It has the benefit of not requiring it you to write a binding for your F# code which can be complex.

Fable would need to generate that, but I would assume it has all the information to do this correctly.

I would assume so to, and that's something I didn't think about. I will try to come up with a manual POC of such approach to check how doable it is.

For example, I am wondering how well such approach will works for F# specifics types like DUs. Consuming a DUs generating by F# is doable in JavaScript because will we still have the compiler able to type check it I don't know.

[nojaf]

Yes, this needs a POC for sure. But given the fact that you have the source input code and the Fable AST it feels like something that could be done. Most likely easier said then done but it could bring a nice DX.

[MangelMaxime]

Most likely easier said then done but it could bring a nice DX.

Sure.

The problem is that right now there no way to create a binding for an F# DUs, because DUs, doesn't exist in JavaScript so we don't have API for that use case.

open Fable.Core

type Json =
    | Number of int
    | String of string

let number = Number 0

type JsonBinding =
    | [<Import("", "")>] Number of int
    // | [<Emit("")>] Number of int
    | String of string

let numberBinding = JsonBinding.Number 0

Right now, we have no way to write JsonBinding. ImportAttribute or EmitAttribute have no effects on DUs ATM.

Generating binding the F# type will need some tinkering.

I think as a first phase, we could just rewrite the import to play a little with what in means in term of constraint. And see how we could extends Fable to generate F# binding to consume such libraries.

And for as long as the feature is not stable, we would mark as Experimental.

[nojaf]

For my understanding, take this repl.

This does produce JavaScript, right? So, can we not produce the same user code but have the Json type come from the node_module?

[MangelMaxime]

This does produce JavaScript, right? So, can we not produce the same user code but have the Json type come from the node_module?

In the current state.

We can if we just rewrite the import statement (which is my proposition at the moment). For example, this is what is --fableLib does it rewrite the import statement.

import { Json } from "./MyLib/Json.js" // Local folder

// rewrote to

import { Json } from "my-lib/Json.js" // Consume from a NPM packages

We can't if we try to consume the code as a binding, currently you can't create a binding for a DUs.

[nojaf]

Well, that is where I'm slightly confused why the DU would need a binding.
The binding would add the same information for the compiler and the only thing that changes is the "my-lib/Json.js" part in the output. I don't think anything changes there.

The binding would be stripped-down version of the source and Fable would need to know that the output file lives inside the node_modules folder.

[MangelMaxime]

Well, that is where I'm slightly confused why the DU would need a binding.

You spoke about bindings here

But to consume this, do we not want to generate something like:

module Math

[<Import("sum", "my-awesome-math-lib")>]
let sum: int * int -> unit = jsNative

// I'm not sure if this needs to be `int * int` or `int -> int`
// Anyway

If we have that, the MyAwesomeMathLib behaves like any other npm package we want to consume. Both the runtime assets and the bindings would come with the npm package and you would no longer have a nuget package instead. For a Fable library this is probably the sweet spot.

The consumer would need to include the bindings a tad manually <Compile Item="node_modules/my-awesome-math-lib/Bindings.g.fs" />. But other than that I think there is something here.

Original comment

Which lead me to say if we go in this direction then any types / functions / variables would needs to have a binding associated to it.

Perhaps, I misunderstood something.

What I am proposing is we ask the user to provide --redirect MyLib=my-lib in the CLI and then Fable knows that import { Json } from "./MyLib/Json.js" needs to be rewritten as import { Json } from "my-lib/Json.js".

This is what it currently do for --fableLib fable-library.

It knows that import { ofArray } from "./fable_modules/fable-library/List.js"; needs to be rewritte as import { ofArray } from "fable-library/List.js";.

[nojaf]

Yep, I definitely sidetracked the conversation there. I think I'm talking about taking your suggestion one step further and avoiding the need for the source files to exist inside fable_modules in the first place.

[MangelMaxime]

After a call with @nojaf, we confirmed that we were speaking about 2 different features.

We think it is a good idea to try implements:

What I am proposing is we ask the user to provide --redirect MyLib=my-lib in the CLI and then Fable knows that import { Json } from "./MyLib/Json.js" needs to be rewritten as import { Json } from "my-lib/Json.js".

As it can open new doors for scenario where we control dependencies of several NPM packages. And it will also allow us to explore what limitation comes with consuming pre-compiled Fable libraries.

[nojaf]

After reading https://fable.io/blog/2022/2022-01-09-Faster-compilation-Fable-3-7.html, I do believe the redirect might have the limitation that an assumption must be made that both projects are using the exact sample fable compiler version.
Still worth exploring for sure!