[Discussion] Transitive dependencies #856
Comments
|
My rough guidelines for this would be:
|
|
We could also still consider using Nuget to distribute Fable libraries. Advantages of using npm:
Advantages of using Nuget:
Other, for example Github (Elm is doing like that, I believe):
|
|
Also, just to summarize the current situation with Fable libraries in Fable 1.0 beta:
|
|
Personnaly, I would prefer stick to npm. I am not sure it's a good thing to make a "distinction between JS and F# dependencies". I love to think that way:
However, we still have a minimal use of nuget on the client side when doing |
|
In my head, peer deps are things you require, but you want the user to be able to uprade or downgrade at will. It is a dep if you require it and want to control the version that gets installed along with your library. It should be up to the developer which to use in a given scenario. So elmish might have fable as a dep, but powerpack as a peer dep. The main difference to me, is that with dep it is hidden from me, with peer dep I retain control. If PowerPack updates, as a peer dep, I can upgrade it, as a dep, I have to wait for you to upgrade it or install my own version of it. To me we should just be providing guidance on when to use both and the implications of that. BTW, the first thing i do after git clone or "scripts": {
"build": "webpack",
"start": "webpack-dev-server",
"test": "echo \"Error: no test specified\" && exit 1",
"postinstall": "dotnet restore",
"fable": "dotnet fable npm-run",
"bld": "yarn fable -- build",
"dev": "yarn fable -- dev"
},It annoys me to immensely, that Fable stole start and build from me. IMO, it should look like this instead but I can't without breaking it. "scripts": {
"webpack": "webpack",
"webpack-dev-server": "webpack-dev-server",
"test": "echo \"Error: no test specified\" && exit 1",
"postinstall": "dotnet restore",
"fable": "dotnet fable npm-run",
"build": "yarn fable -- build",
"start": "yarn fable -- start"
},The advantage to adding postinstall, is now I get dotnet restore executed automatically any time I install something with yarn, the downside is that it runs every time I install something with yarn, even if there is nothing for it to do. I would prefer it if the FS code was not in npm at all and instead was in Nuget, because neither Visual Studio or the new Rider IDE EAP will look in node modules for .fs code even if it is specified in the .fsproj. Also, I want to write the same F# no matter where I am, meaning not following JS conventions of lowerCamelCase. JavaScript to me, is the IL for Fable, but Fable is still F#. Meaning we should be using F# conventions for everything. I don't want to write a dotnet core console app and use UpperCamelCase, and then have to remember that it is lowerCamelCase in Fable. Just my opinion. I personally don't think any source should be in node_modules, that is intended for code that is ready to go, which is why most IDE's ignore and hide it by default. |
|
@mizzle-mo I use this "scripts": {
"postinstall": "dotnet restore Main.fsproj",
"build": "dotnet fable webpack",
"watch": "dotnet fable webpack-dev-server"
}Now I can just use |
|
Okay, so, this is going to be a long post... First, some facts:
There is, however, a solution to the above problem: package sets. A package set is basically just some metadata which lists out all of the available packages + a version for each package + the dependencies of each package. This solution is used by Haskell (with Stack) and PureScript (with Here is the standard PureScript package set. It might seem intimidating, but it's actually really simple. This is the basic structure: {
"foo": {
"dependencies": [
"bar",
"qux"
],
"repo": "https://github.com/foo/foo.git",
"version": "v3.0.0"
},
"bar": {
"dependencies": [
"qux"
],
"repo": "https://github.com/bar/bar.git",
"version": "v2.1.0"
},
"qux": {
"dependencies": [],
"repo": "https://github.com/qux/qux.git",
"version": "v5.0.0"
}
}Each key is the name of the package (e.g. Notice that it doesn't list the version for the dependencies. That's because each dependency will use whatever version is specified in the JSON file. As an example, The benefit of this is that all of the packages + their dependencies are listed in one spot. So let's say that But with package sets, you only need to bump the version number in one spot, so it's much easier. This also means that the package set is always consistent, so you don't need to worry about dependency hell. It's also important to note that the package set itself also has a version. Haskell and PureScript package sets are immutable and reproducible: the only way to change anything in the package set is to release a new version of the package set. And they don't use semver either: all versions are exact. This has the same benefit as a lock file: it guarantees that the packages will work correctly. You never need to worry about one package releasing a new version and then breaking other packages. But even though they don't use semver, it's not difficult to keep the packages up to date, because the versions are all specified in one place: the package set. So you just bump the version in the package set and you're done. There's some other benefits as well: the package set serves as a curated list of packages, so it's possible to exclude packages which aren't high enough quality (if you want). And it also makes it easier for users to find packages, because all of the available packages are listed in the package set. You can also do things like run automatic continuous integration over the entire package set (to run unit tests and catch regressions). Or you can display a webpage which lists all of the packages + documentation for each package (both Haskell and PureScript do this). And you can add in (optional) checksums which ensure that the packages haven't been corrupted or tampered with. This increases security and reproducibility. It's a really fantastic system that completely solves many problems: it ensures that there is only a single version of each package, it guarantees no dependency hell, it guarantees reproducible builds, it's searchable, it's consistent, and it's easy to manage. Of course, the tradeoff with package sets is that you are essentially creating a Fable-specific package manager. PureScript's package manager is really simple, so the idea of "custom package manager" doesn't need to be scary. But it is a tradeoff that needs to be considered. I can explain more of the details about package sets, if you want. There's some other awesome stuff like Nix, which I believe is similar to package sets, but I don't have any experience with it. |
|
Also, just to be clear, even though I really like package sets, and I think they are absolutely the right way to handle packages, I'm okay with using npm or paket instead. It won't be as good as package sets, but it'll have better integration with existing packages / systems / tools. If we decide to use npm, we probably want to use |
|
Being a Haxe user, I put that here. It may give ideas: Haxelib |
|
Thanks a lot for the useful comments! @mizzle-mo About visibility of the dependencies, this is tricky. It's true that in Fable 1.0 they're not code "ready to go" but at the moment we are considering them as normal dependencies which most packages hide somehow. I don't think this would change even if we move them to Nuget. We could also consider installing dependency Fable projects side by side to the user project but this would likely bring another set of questions (like should the dependency projects be added to the user repo, etc). @Pauan The idea of package sets looks very interesting and as you say it can help stabilize the Fable ecosystem, make sure the packages work well with each other and provide better visibility. I assume this comes with a cost in term of flexibility, for example, if a package author wants to publish a new major version with some breaking changes they must be sure the dependent packages still work before bumping the package set version. Am I right? The only thing that puts me off is the idea of having a custom package manager for this. Not only because of the extra maintenance overhead for Fable, but also because users would have to deal with 3 package managers as we cannot avoid using both npm/yarn and Nuget/Paket as of now. @whitetigle |
100% agreed.
Indeed, haxelib is a custom package manager Packages are hosted as zip files on a central server managed by the Haxe Foundation but it is possible to get packages from git or mercurial repositories as well or host a private server. Useful for --next or private libs for instance. Then when we install packages, each one solves its own dependencies like this: {
"name": "flixel",
"url" : "http://haxeflixel.com",
"license": "MIT",
"tags": ["game", "openfl", "flash", "html5", "neko", "cpp", "android", "ios", "cross"],
"description": "HaxeFlixel is a 2D game engine based on OpenFL that delivers cross-platform games.",
"version": "4.2.1",
"releasenote": "Fixed rendering with Haxe 3.4.0 and OpenFL Next.",
"contributors": ["haxeflixel"],
"dependencies": { "lime": "2.9.1", "openfl": "3.6.1" }
}
Regarding @Pauan remark:
Unless specified, haxelib installs the latest version of the package but we may have dozen different versions especially when we still need that old deprecated feature... They use a custom semver. But if I understand well there does not seem something consistent like a package set which seems great! |
|
@alfonsogarciacaro Here's how it works in PureScript: there is a package set maintainer, who manages the package set repo. When a library author creates a new version of their library, they send a pull request to upgrade the library version in the There is some discussion about making this process easier, including the possibility of having a bot that will automatically accept pull requests. Even after the pull request is accepted, this doesn't affect anybody, because everybody is still using the old package set version. On regular intervals, the package set maintainer will release a new package set version, and then application authors can switch to the new package set version. Of course applications can continue to use the old version of the package set. In fact applications can use any version of the package set, even very old versions, and it's still guaranteed to be consistent, because package sets are immutable. Essentially, each package set version is an immutable snapshot of the Making a snapshot just means using It's possible to have multiple maintainers for a package set, and it's possible to have multiple package sets, but those situations are rarely needed. If a library author releases a new major version, any libraries that depend upon that library will have to be updated. The library authors are given some time (usually a few weeks or so) to upgrade to the new major version. If they don't upgrade in time, their package will be dropped from the package set. After all of the dependent libraries are either upgraded or dropped, then a new major version of the package set is released. As an example:
This guarantees that every version of the package set is consistent. The above is pretty much the worst-case scenario, the process is a lot easier for non-major version bumps:
The above discussion is about the package set repo and the library authors, but it's also worth talking about the experience of an application developer. As an application developer, you create a {
"name": "my-project",
"set": "psc-0.10.2",
"source": "https://github.com/purescript/package-sets.git",
"depends": [
"foo",
"bar"
]
}
You can use Upgrading to a new version of the package set is as easy as changing the As you can see, it's super easy for application developers, especially because they get guaranteed consistency. So package sets put a tiny bit more burden on library authors and package set maintainers, but in exchange it makes things so much better for application authors. Having three package managers definitely is bad, but if Nix is similar to package sets, it's also based on immutable consistent snapshots, and it already has support for npm. I'm not sure if it has support for Paket though. |
|
@Pauan Can't we do 3 package managers with yarn now? We can dotnet restore in the postinstall step which could also kick off the 3rd package manager, I like the idea of package sets. |
|
I'm confused, what constitutes a set, and how dropping a library from one keeps it useful? Maybe an example from fable ecosystem would help me understand this. |
|
@et1975 My understanding is that it would enable packaging similar to a mono repo. Where Elmish, Fable, Powerpack, would be packaged together with one version number. So when I need to update my deps, I am getting all 3 at a version that is known to work together (with testing). So consumers would update one dependency instead of 3. Then we could do the same for Electron, where it depends on the centralized package, but is itself also distributed as one package with all 4. @Pauan will correct me if I am wrong. He's good at that. :) |
|
@mizzle-mo Yes, we can use Then again, a package set system can be designed in a simple and robust way, so breakage should be pretty rare. You're right that the concept of package sets is similar to a monorepo, except it doesn't require all of the packages to be in a single Git repository, so it scales to many developers / repos / packages. Haskell's package sets contain over 2,300 packages, which is obviously infeasible with a monorepo. @et1975 There's many ways to implement package sets, but the way that PureScript does it is that there is a git repo which contains a {
"aether": {
"dependencies": [],
"repo": "https://github.com/Prolucid/fable-aether.git",
"version": "927a4083e78f41b62669ac4030747478cd2f5b50"
},
"elmish": {
"dependencies": [
"powerpack"
],
"repo": "https://github.com/fable-elmish/elmish.git",
"version": "v0.8.0"
},
"powerpack": {
"dependencies": [],
"repo": "https://github.com/fable-compiler/fable-powerpack.git",
"version": "19b64ecf670e015ec23fb11d6f9069f4c3c23e75"
},
"unit-test": {
"dependencies": [
"powerpack"
],
"repo": "https://github.com/fable-compiler/fable-unit-test.git",
"version": "f63db8efa7b07be56982b966c0df9866838d232d"
}
}This Generally speaking, a package set will contain as many packages as possible. You can think of the package set as serving a similar purpose as the npm registry: it's a collection of metadata about all available packages. In the case of Fable, it would contain all of the available Fable libraries (including but not limited to elmish, powerpack, unit-test, aether, etc.). A package set is just a git repo with a And it's also possible to use a package set + custom packages as well. For example, you can say "I want to use the packages in the Fable 1.0 package set, and in addition to that I want to use a package The Because these snapshots are immutable, that means if you switch to an older snapshot, you will have access to everything which was available at that time. For instance, if a package is removed in the Or if you switch to an older snapshot, the versions of the packages will be downgraded. Or if you switch to a newer snapshot, the versions of the packages will be upgraded. Basically, you can think of it as being similar to the Except unlike Also unlike As for why it's useful to drop a package, the big benefit of package sets is that they contain up-to-date, curated, maintained, working packages. In other words, when using a package set, everything should Just Work(tm). You don't need to worry about a package being outdated (e.g. the package works with Fable 0.75 but hasn't been upgraded to Fable 1.0). You don't need to worry about one package breaking another package. You don't need to worry about multiple versions of the same package. You don't need to worry about conflicting versions. You don't need to deal with dependency hell. Packages only get dropped when they break, so even if a package is unmaintained, it won't get dropped as long as it keeps working. Also, even if a package does get dropped, that doesn't mean that it's banned, it just means that it's not available in that particular snapshot. This is done to ensure that the package set is always consistent and doesn't have any breakage. After the breakage is fixed, the package can be re-added in a future snapshot. It's not a big deal. It's also possible to delay the release of a new snapshot until after the broken package is fixed, rather than dropping the package. With Stack, most breakages are fixed by the package authors within the deadline, so it's rare for packages to get dropped. They also have a system where somebody can volunteer to be a maintainer for a package, even if they aren't the author of that package. That way they can fix breakages even if the package is unmaintained by the author. Having said that, the removal of broken packages is a social thing, not a technical thing, so the package set maintainer can choose to keep broken packages in the package set if they want. From a technical standpoint, a package set is just a bunch of metadata about packages, versions, and dependencies. All of the other benefits (consistency, non-breakage, etc.) have to be enforced by the package set maintainer, so it's ultimately up to the package set maintainer how they choose to manage their package set. And because there can be multiple package sets, different package sets might have different maintainers, each with their own personal decisions about how to manage their package set. That's fine, in that case the application developer just chooses which package set aligns closest with their goals. But I imagine that there would be a single official Fable package set, which would be the default, so most users would just use that. In which case we would need to decide as a community how to manage that package set (e.g. whether broken packages should be removed or not, how much time we give, etc.) |
|
Package sets is a nice concept and solves many practical issues with compat of transitives. But AFAIK it doesn't solve the issue of resolution in your project. If transitives are used in multiple sets or you use uncommon deps then things become complicated again. We should not say "it all works". It's really a mess and I personally think dependency sets won't help us at all. It's not the problem we need to solve right now. |
|
@forki I think the general idea is that you don't use multiple sets: each application uses a single package set, so there isn't any resolution problems. I agree that it can still break in certain situations, such as when using npm / paket dependencies. I wonder if that can be solved by checking in a At the very least, it solves the problem of consistency with Fable dependencies, so it's still better than the status quo. I agree that having to manage many package managers is a huge pain. Regardless of what package manager we choose, we need a single simple way for users to use it. That might mean using multiple package managers under the hood and providing a simple API for users, or it might mean using a single package manager. We need a single tool system, not a mishmash of many different tools. I also wonder if it would be possible to use Nix. It's a lot more complicated than PureScript package sets, but it also has many very useful features: they've already solved many complicated problems like caching, binary builds, continuous integration, checksums, etc. Nix already has a package for F# 4.1, thousands of npm packages, and hundreds of NuGet packages (including FAKE) Just like package sets, Nix provides consistent snapshots, even when using npm or NuGet packages. So we get the benefits of a single package manager, we get the benefits of using existing package ecosystems (e.g. npm, NuGet, etc.), and we get consistent snapshots. Nix has packages for libraries, but it also has packages for tools, compilers, samples, documentation, applications, etc. So we can have a single consistent system for everything in Fable. The only thing that really worries me is that I'm not sure how good the Windows support is for Nix... |
|
You can't use just one public set. This would not know about private deps.
So you are back to managing your own sets. Which is like resolving
yourself.
Am 03.05.2017 18:20 schrieb "Pauan" <notifications@github.com>:
… @forki <https://github.com/forki> I think the general idea is that you
*don't* use multiple sets: each application uses a single package set, so
there isn't any resolution problems.
I agree that it can still break in certain situations, such as when using
npm / paket dependencies. I wonder if that can be solved by checking in a
yarn.lock file to the package set repo, that way npm dependencies will
also be consistent.
At the very least, it solves the problem of consistency with Fable
dependencies, so it's still better than the status quo.
I agree that having to manage many package managers is a huge pain.
Regardless of what package manager we choose, we need a single simple way
for users to use it.
That might mean using multiple package managers under the hood and
providing a simple API for users, or it might mean using a single package
manager.
We need a single tool system, not a mishmash of many different tools.
------------------------------
I also wonder if it would be possible to use Nix. It's a lot more
complicated than PureScript package sets, but it also has many very useful
features: they've already solved many complicated problems like caching,
binary builds, continuous integration, checksums, etc.
Nix already has a package for F# 4.1
<https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/compilers/fsharp41/default.nix>,
thousands of npm packages
<https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/node-packages/node-packages-v6.nix>,
and hundreds of NuGet packages (including FAKE)
<https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/dotnet-packages.nix>
Just like package sets, Nix provides consistent snapshots, even when using
npm or NuGet packages. So we get the benefits of a single package manager,
we get the benefits of using existing package ecosystems (e.g. npm, NuGet,
etc.), and we get consistent snapshots.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#856 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AADgNGj_UpcDGh2YGCqMaxyD-0X8UOfwks5r2KlAgaJpZM4NKObO>
.
|
|
Regarding nix. Using just another standard would not solve a single issue.
Especially if you have lots of private packages.
Am 03.05.2017 18:36 schrieb "Steffen Forkmann" <sforkmann@gmail.com>:
… You can't use just one public set. This would not know about private deps.
So you are back to managing your own sets. Which is like resolving
yourself.
Am 03.05.2017 18:20 schrieb "Pauan" ***@***.***>:
> @forki <https://github.com/forki> I think the general idea is that you
> *don't* use multiple sets: each application uses a single package set,
> so there isn't any resolution problems.
>
> I agree that it can still break in certain situations, such as when using
> npm / paket dependencies. I wonder if that can be solved by checking in a
> yarn.lock file to the package set repo, that way npm dependencies will
> also be consistent.
>
> At the very least, it solves the problem of consistency with Fable
> dependencies, so it's still better than the status quo.
>
> I agree that having to manage many package managers is a huge pain.
> Regardless of what package manager we choose, we need a single simple way
> for users to use it.
>
> That might mean using multiple package managers under the hood and
> providing a simple API for users, or it might mean using a single package
> manager.
>
> We need a single tool system, not a mishmash of many different tools.
> ------------------------------
>
> I also wonder if it would be possible to use Nix. It's a lot more
> complicated than PureScript package sets, but it also has many very useful
> features: they've already solved many complicated problems like caching,
> binary builds, continuous integration, checksums, etc.
>
> Nix already has a package for F# 4.1
> <https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/compilers/fsharp41/default.nix>,
> thousands of npm packages
> <https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/node-packages/node-packages-v6.nix>,
> and hundreds of NuGet packages (including FAKE)
> <https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/dotnet-packages.nix>
>
> Just like package sets, Nix provides consistent snapshots, even when
> using npm or NuGet packages. So we get the benefits of a single package
> manager, we get the benefits of using existing package ecosystems (e.g.
> npm, NuGet, etc.), and we get consistent snapshots.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <#856 (comment)>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AADgNGj_UpcDGh2YGCqMaxyD-0X8UOfwks5r2KlAgaJpZM4NKObO>
> .
>
|
|
@forki If all of the Fable libraries are in the package set, then most projects won't need any private dependencies. And for the projects that do need private dependencies, they shouldn't have too many. But yes, private packages do require you to do the resolution yourself. If you have many private packages, you can create your own private package set. It's easy to fork a package set, since it's just a Git repo. I don't see why Nix doesn't solve the problem. Nix has support for private packages. In Nix, private packages use exactly the same system as public packages, so you get the same benefits. |
|
We always have private dependencies in our products. This would not fix anything for us. It would only increase pain since now packages need to conform to nix. |
|
@forki How is that any different from requiring private dependencies to conform to npm, or to conform to NuGet, or to conform to Paket? Whatever package manager we decide to use, packages will need to conform to that package manager. I guess you're saying that "we have private dependencies for npm, and we don't want to change them to Nix, since that's extra work"? |
|
I was asked to help with nix once. It was a nightmare. I don't want to
force my colleagues on this. I guess then I'd go my own way.
Am 03.05.2017 18:44 schrieb "Pauan" <notifications@github.com>:
… @forki <https://github.com/forki> How is that any different from
requiring private dependencies to conform to npm, or to conform to NuGet,
or to conform to Paket? Whatever package manager we decide to use, packages
will need to conform to that package manager.
I guess you're saying that "we have private dependencies for npm, and we
don't want to change them to Nix, since that's extra work"?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#856 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AADgNLwxzUSLZM3SLK2A_qeu_3BigtTRks5r2K8GgaJpZM4NKObO>
.
|
|
@forki I haven't used Nix, so I'm genuinely curious: when did you try Nix? Were you trying to use Nix on Windows? What problems did you run into? |
|
It was when we tried to add fake. Most issues where mono based and mostly
about making the build process repeatable. IIRC we gave up at that first
try.
Am 03.05.2017 18:51 schrieb "Pauan" <notifications@github.com>:
… @forki <https://github.com/forki> I haven't used Nix, so I'm genuinely
curious: when did you try Nix? Were you trying to use Nix on Windows? What
problems did you run into?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#856 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AADgNEffp6-ZASShsev6WRfWxTmzeFjqks5r2LChgaJpZM4NKObO>
.
|
|
There are several things being discussed here. I think that's good because it gives us an opportunity to discuss the tooling and ecosystem before the stable 1.0 version is released. That's the purpose of the beta and that's what we did with Fable 0.7. I've been thinking a lot about this. I'll try now to break down some of the current issues and possible solutions, it may be easier to tackle them individually instead of trying to fix everything at once: Friction between .NET and JSNovelty of dotnet SDK also affects this. People need to know about two ecosystems, sync dependencies from two registers, invoke .NET and JS tools at the same time, etc. Possible solutions:
Distributing F# libraries as sources and use project referencesF# tooling is not designed for distributing libraries as sources and obviously npm wasn't designed to distribute F# libraries so frictions (which actually motivated this thread) are expected here too. Possible solutions:
Dependency version managementA package set could be beneficial to give stability and visibility to Fable's ecosystem, but we should be careful of the maintenance and cognitive overhead this would entail. Also, I believe if we can solve the issues above (for example, using just one package manager) this will likely be less problematic. Please comment if you agree with some of these solutions (letting things be as they're is also an alternative) or if you have any other suggestion. |
|
@alfonsogarciacaro I'm not sure how reliable Yarn's Also, when you create a package and you specify |
|
@alfonsogarciacaro @Pauan I like the idea of using Paket with modifications. I think if we could add the ability to link project references, and maybe later add the ability to diff public APIs to enforce semver, that would be rocking and tooling would have an easier time if I'm not mistaken. |
|
Sorry if this is completely irrelevant (I sadly haven't used fable quite jet, but hopefully will soon). But might this be a good point to think about how to distribute fable packages? What I'm reading all over the place with distributing source and fsproj files looks more like a hack from the early days? Can't we package dll and javascript code (or something similar) instead? |
|
That is actually a good question @alfonsogarciacaro. Why are we packaging the source? |
|
@mizzle-mo This has been discussed before and it has no bearing on the choices above. Among other things: because it allows the consumer a free choice of the output: js version/modules system/etc is a matrix, and in the past the library author had to produce all of it; and if your desired combination is not there - good luck. Personally I really like sources distribution, let's not reopen that conversation. |
|
Yes, one of the main problems of packaging the JS sources is that if the compiler adds a new feature (say, more detailed reflection info for the types), it forces the package authors to republish their packages just to bundle the generated JS with the new feature. And it adds to the version management hell, because you don't only need to take the Fable lib version into the account, but the Fable compiler version that was used to generate the code. So yeah, I think we should discard that choice. However, .dll + F# sources distributed with Nuget/Paket is still an option. There may be some difficulties though: if the lib has JS dependencies, how do you mark them; if Fable has to collect all source files, how it can resolve the dependency graph... but we could take that direction anyways. |
|
@alfonsogarciacaro I know @forki doesn't like Nix, but that's one of the really cool features of Nix: if the Fable compiler changes, all Fable packages will be automatically recompiled to use the new Fable compiler. So even with DLL compilation, library authors don't need to manually republish on every change. It's also possible to configure Nix packages in various ways. For example, the consumer of a Fable library can choose the module type, polyfill, Fable version, fable-core version, etc. You aren't stuck with whatever choice the library author made. Nix has a global cache, and Nix packages are pure and immutable, so packages with custom configuration can be cached. Right now, Fable libraries have to be recompiled over and over again. With Nix, they would only need to be compiled once and then cached. That means you get incredible flexibility, more developer convenience, and faster build times. They've really thought about how to solve these difficult packaging problems. I know I'm really pushing for package sets, but there is another option: package sets for Fable, with additional npm metadata: {
"foo": {
"dependencies": [
"bar",
"qux"
],
"npm-dependencies": {
"webpack": "^1.0.0"
},
"repo": "https://github.com/foo/foo.git",
"version": "v3.0.0"
}
}It's the same as before, but it has an additional Installing a package is as simple as This means that users only need to learn one system, one configuration file, one set of tools. |
|
While I personally love Nix and use it every day (doing F# and Fable development on NixOS) my personal experience is that Nix has a steep learning curve that could turn out to be disadvantageous for new users and the project. This will be especially overwhelming for those kinds of people who are curious and just want to try things out quickly, because not only will they have to learn the language (F#) and tooling, but also the Nix expression language, how to compose package sets and how to set up development environments. Without saying that my experiences are representative for others, but it took me quite a while to grok all this and become productive. Thus, a deep integration with Paket is - in my view - preferable. My 2¢. |
Its great that this discussion has already emerged before. Could you please link it?
Yes this means we would need to look out for breaking changes and mitigate them. But every compiler on the world needs to do that. What I'm really afraid of is that this stuff works now that we have only some small projects and not a big number of fable packages (so explicitly using a large number of fable packages with a large number of transitive dependencies)? |
|
@matthid There's an issue with the discussion before publishing 0.7, I don't remember which but anyways there it was actually decided to go for .dll + precompiled JS files. I think the discussions about the problems that arose later (more or less outlined above) and motivated the development of Fable 1.0 happened in Gitter so we cannot link to them :/ It's tricky because the problem I mentioned above affects not only breaking changes but also additive changes, bug fixes, etc in the compiler. It was very hard with Fable 0.7 to keep all the packages (and there were not many) up-to-date with the compiler development. As the author of the most popular Fable package, @et1975 has something to say on that regard ;) The only advantage of distributing precompiled JS files is you don't have to compile the dependencies by yourself. This is something we're used to in .NET (not in other communities) and it's true that the first compilation takes some time for a simple fable-elmish-react project with Fable 1.0 because of the dependencies, but developers normally use watch mode with Fable so these extra seconds you have to wait only happen once. Also, in some cases you do want to distribute your library as JS, this happens with thegamma-script and it's what we want to do with the JS compiled FCS+Fable, but in those cases because of the way bundling works the API is different from when you have JS files mapped 1:1 to F#. So it can be a bit confusing to have libraries with JS only prepared to be used from F# and others with JS prepared for other JS libraries. |
|
@alfonsogarciacaro I don't quite remember the full details, but I think the decision to go for sources was because of issues with file paths in the JS files? Maybe that's been solved by Webpack, I don't know. Also, watch mode is fantastic and I wish every language supported it, but it's not a substitute for precompilation. If the initial compilation takes 5 minutes, you have to wait 5 minutes every time you restart watch mode. There's many reasons why you might need to restart watch mode: restarting your computer, issues with the cache, adding new dependencies, making a change to I use watch mode all the time on various JavaScript projects, and even waiting 30 seconds feels too much, since I often need to restart watch mode (for the reasons listed above). I think it would be cool if Fable could somehow cache Fable libraries, so they only need to be compiled once. The compilation would still happen on the user side, not the library side, so it shouldn't cause any problems. |
|
@Pauan I also believe that using the source instead of the .dll help Fable to have full access to the AST of F# where it was limited when using DLL. (even with the fablemap file) |
|
@MangelMaxime Ah, yeah, I think there was also some problems with |
@Pauan That's a great idea! And also a good PR opportunity :) |
|
I will be merging soon PR #884. With that, Fable libraries will be distributed in the form of dll + F# sources through Nuget and managed by Paket. Hopefully this will solve most of the problems outlined above 😸 If you have any comment, could you please write them in the PR? Thx! |
Continuing the conversation from gitter about transitive dependencies support @alfonsogarciacaro is adding, started in elmish/react#1:
I'm assuming the situation is as follows:
devdependency.peeror regular, let the user figure it out and install".dotnet fable addwe get to examine dependencies of the library being added and potentially trigger installation of its (transitive) dependencies.Question: should we?
Question: if yes, what is the appropriate section
devorpeer?Question: what is the predicate for triggering a transitive dependency install?
Question: what is the desired behaviour if installed with
yarninstead ofdotnet fable add?The text was updated successfully, but these errors were encountered: