Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packaging solutions #497

Open
intergrav opened this issue Aug 6, 2023 · 80 comments
Open

Packaging solutions #497

intergrav opened this issue Aug 6, 2023 · 80 comments
Labels
enhancement New feature or request

Comments

@intergrav
Copy link

intergrav commented Aug 6, 2023

For discussion, see the thread in the #development channel of the Modrinth Discord. You can also continue discussing here if you prefer that.

About

At the time of writing, there are only two official ways for using the Modrinth App on Linux - AppImages and Debian packages. While this is fine for people on Debian-based distributions, and those who like using AppImages, some may prefer to use a package manager such as flatpak or dnf as it's easier to manage packages that way. According to this comment, the current compiled Debian packages aren't very great right now because they don't include proper dependency info and leave out some important stuff.

Along with AppImages and Debian packages, there are various packaging solutions to consider. If there's other packaging types that you think would be a good idea, suggest them in this thread. I personally think that the Flatpak should be maintained by Modrinth, and the rest could be maintained by members of the community if they wanted to do so. I think this is especially important as the Modrinth App is now in an open beta.

This issue serves as a place to discuss packaging, and I'll also treat this as a sort of "unofficial download page" for Linux users with updated information on what packages to use and where to get them.

Indicators

  1. a package exists and works properly
  2. that package is considered ready for daily use
  3. that package is available on commonly used repositories, such as a Flatpak being on Flathub for example

🔃

  1. same as 1 and 2 in the above
  2. however, that package is not available on commonly used repositories, but is available on a developer's personal repository. this may not be applicable for some solutions and will use the checkmark icon instead, such as COPR

Unix/Unix-like packaging

Flathub logo Flatpak

Package is available and maintained on Flathub - thank you @getchoo and other contributors!

Package that can be used on most common Linux distributions, and also good for security as Flatpaks are sandboxed unlike other types. This will be the package that most Linux users are looking for.

Notable mentions/issues
Who's working on this?

Arch

Packages are available and maintained on the AUR - modrinth-app and modrinth-app-bin - thank you Antti!

Any ideas on this?
Notable mentions/issues
Who's working on this?
  • Antti has made two AUR packages
    • modrinth-app
    • modrinth-app-bin
  • @CompeyDev is working on a package
  • Nixuge has made two AUR packages, however they could be broken as they miss many dependencies
    • modrinth-app-git
    • modrinth-cli-git

Nix logo Nix

Package is available and maintained on Nixpkgs - modrinth-app - thank you @getchoo and other contributors!

Nix can be used on both Linux and Darwin systems and is the package manager for NixOS.

Who's working on this?

Fedora logo Fedora Copr

User-maintained package repositories for Fedora, CentOS Stream, and RHEL. Can be enabled with dnf copr enable <author>/<repo>

Notable mentions/issues

Homebrew logo Homebrew cask

Homebrew is a package manager for MacOS (and Linux). Idea from #560. It would allow for easy installation on MacOS.

Windows packaging

🔃 🍨 Scoop

Package is available and maintained on brawaru/bucket - thank you @brawaru!

Scoop is a command-line installer for Windows. It has many benefits, such as installing packages in a portable way (by default, it installs packages to ~/scoop). Package manifests are just simple JSON files - this shouldn't be too hard to set up.

WinGet logo WinGet

Package is available and maintained on WinGet - thank you @TheBossMagnus and other contributors!

WinGet is Microsoft's own package manager.

Other information

Prism Launcher, for example, also uses many packages types maintained by users - see their Linux install page for more info. I think that Modrinth doing the same would be great for the community. I don't expect there to be as many packages as Prism of course - Theseus is very complex to package and doing so for many different formats at a time would be a little chaotic. I just think that the common package types like Flatpak, Arch and Nix packages should be done.

I am not a packager and do not have too much experience with this process, however I think that using multiple packaging formats can be very beneficial. As said before, it does not need to all be maintained by Modrinth - the community can maintain other packages.

Leave your thoughts below!

For discussion, see the thread in the #development channel of the Modrinth Discord. You can also continue discussing here if you prefer that.
@intergrav intergrav added the enhancement New feature or request label Aug 6, 2023
@Hyphrio
Copy link

Hyphrio commented Aug 6, 2023

For Linux these two Tauri issues are related:
tauri-apps/tauri#4402
tauri-apps/tauri#3619

@offbeat-stuff
Copy link

At least make a PKGBUILD that extracts the deb package to be used as an archlinux pacman package.

@offbeat-stuff
Copy link

Here is a PKGBUILD for archlinux that extracts a .deb package

https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=linux-xanmod-linux-bin-x64v3

@getchoo
Copy link

getchoo commented Aug 6, 2023

At least make a PKGBUILD that extracts the deb package to be used as an archlinux pacman package.

this isn't a good method for the long term, especially as arch is known to make abi incompatible changes to packages frequently, or at the very least push versions of libraries usually incompatible with those found on debian/ubuntu based distros (there are also problems between those, but usually less so). here's a good example

for a temporary thing it would probable be fine? but ideally all community packages should be built against their distro's packages to ensure the final binaries actually work

@getchoo
Copy link

getchoo commented Aug 6, 2023

I personally think that the Flatpak should be maintained by Modrinth, and the rest could be maintained by members of the community if they wanted to do so.

i 100% agree with this. it's very hard and completely impractical for upstream to support the endless amounts of linux distributions out there; universal solutions should be the main focus here, along with basic files for distribution that can be used there and in downstream packages

Flatpak/Flathub

the biggest issue here is that networking may need to be enabled during building due to flatpak's build tools lacking support for pnpm lock files. this is a problem since flathub doesn't really like that and it could make it very difficult for it to be uploaded there

Nixpkgs

this is pretty much complete and fully working on any linux distro. macos support for is being worked on. the problem with pnpm and offline builds also showed up here, but since nix is a lot more flexible with it's build process, i was able to figure out a way to maintain reproducibility while still using pnpm's lock file. i will try to upstream this in nixpkgs once a stable release is made

Fedora Copr
Arch User Repository
Alpine Linux packages

like you said, these should probably be community packages. i would like to note though that some may be difficult (mainly fedora rpms) due to the issue with pnpm and offline builds as mentioned before. arch is very lose with this though so it should work fine; i can't say anything about alpine since i've never used/packaged for it personally

the only other thing i would like to note is that tauri's native package export feature really shouldn't be used here. the current debian package for example seems to strip out the other tools theseus provides such as it's cli, doesn't include dependency information (i'm assuming the direct dependencies of theseus are compiled statically...which isn't great but works ig) which can cause issues with even launching minecraft, and in general is very inflexible for the future where manual intervention for packaging may be required. i'm shocked the deb package works in it's current state, let alone if it can even actually start minecraft ootb. we should be following the standard of other linux packages (specifically packages for other launchers) by supporting the formats themselves instead of relying on a tool that in all honestly, doesn't produce good packages. most of this will fall on the community of individual distros, but for a download link on the upstream maintainer's website, poorly auto generated debs and appimages aren't really acceptable for "linux support" imo. the best solution to fix this is flatpak

edit: macos support for nix is available now. discussion about upstreaming to nixpkgs will be in NixOS/nixpkgs#247468 (thanks PC!)

@Hyphrio
Copy link

Hyphrio commented Aug 6, 2023

Tauri issue and PR for Arch Linux:
tauri-apps/tauri#3728
tauri-apps/tauri#4301
PR seems to be related for Pacman but seems to have stalled

@CompeyDev
Copy link

I'd be willing to draft PKGBUILDs for the AUR, just lmk.

@TheoCGaming
Copy link

If you do go for Flatpak, please, PLEASE include proper permissions. I'm tired of having to deal with broken permissions on Flatpak packages.

@MagmaBro123
Copy link

MagmaBro123 commented Aug 6, 2023

If you do go for Flatpak, please, PLEASE include proper permissions. I'm tired of having to deal with broken permissions on Flatpak packages.

Flatseal can help with fixing those, but agreed.

@intergrav
Copy link
Author

Flatseal can help with fixing those, but agreed.

The user shouldn't have to use Flatseal to fix broken permissions provided by the package, that's just bad design

@MagmaBro123
Copy link

Flatseal can help with fixing those, but agreed.

The user shouldn't have to use Flatseal to fix broken permissions provided by the package, that's just bad design

Again, I said I agreed

@blt-r
Copy link

blt-r commented Aug 7, 2023

There is a tool to generate flatpak builder sources definitions from package-lock.json or yarn.lock, but it doesn't support pnpm-lock.yaml. The only way I found to convert pnpm-lock.yaml to something that the tool can consume is this npm package, but I didn't manage to make it work. We need to get either yarn or npm lockfile with right dependency versions. (Or implement support for pnpm in that tool, but that's very not easy).

Besides that, we probably will need to disable auto updating for the flatpak package. If there is no better way, that could be done with some sed in the build script.

Since the whole app is bundled in a single executable, packaging it as a flatpak would probably be pretty easy.

@blt-r
Copy link

blt-r commented Aug 7, 2023

Actually, on a closer look, the resulting binary has quite a few dynamic dependencies. I'll try to see if I can put all of them into the flatpak

@intergrav
Copy link
Author

intergrav commented Aug 8, 2023

I just made a forum thread in the Modrinth Discord server #development channel for discussing about this: https://discord.com/channels/734077874708938864/1138510665947041876
I'll also edit the issue and place it around the top, you can continue discussing here aswell :)

@AnOpenSauceDev
Copy link

3 days and still no mention of snap...
/s

@MagmaBro123
Copy link

Ew, Snaps.

@getchoo
Copy link

getchoo commented Aug 9, 2023

https://discord.com/channels/734077874708938864/1138510665947041876

quoting a message i made here:

nix packaging for linux and darwin is basically complete, just needs a few little touchups. i have flatpak almost working, but as it stands it would only be able to be distributed through flathub - only the website. this is due to no flatpak build tools supporting pnpm’s lockfile format; it’s a similar case with nix, but there are a few diy methods already in nixpkgs i was able to use so not an issue there. for flatpak, networking needs to be enabled during the build but that’s a big nono

only other “issue” is that tauri (or what i’m hoping to be only the specific version of tauri being used) is using an old version of webkitgtk to build so…i need to compile all of webkitgtk for it to work, which causes my pc to oom

it seems that upstream is willing to switch to npm (source) which solves the first issues; everything else with the flatpak seems pretty good.

one thing i thought of though was making some packaging guidelines for the app, as currently some things such as actual naming can be unclear. maybe this could be done with upstream sources for stuff like desktop files?

@MagmaBro123
Copy link

MagmaBro123 commented Aug 9, 2023

https://discord.com/channels/734077874708938864/1138510665947041876

quoting a message i made here:

nix packaging for linux and darwin is basically complete, just needs a few little touchups. i have flatpak almost working, but as it stands it would only be able to be distributed through flathub - only the website. this is due to no flatpak build tools supporting pnpm’s lockfile format; it’s a similar case with nix, but there are a few diy methods already in nixpkgs i was able to use so not an issue there. for flatpak, networking needs to be enabled during the build but that’s a big nono

only other “issue” is that tauri (or what i’m hoping to be only the specific version of tauri being used) is using an old version of webkitgtk to build so…i need to compile all of webkitgtk for it to work, which causes my pc to oom

it seems that upstream is willing to switch to npm (source) which solves the first issues; everything else with the flatpak seems pretty good.

one thing i thought of though was making some packaging guidelines for the app, as currently some things such as actual naming can be unclear. maybe this could be done with upstream sources for stuff like desktop files?

You and the Modrinth team would probably be better at chatting about how this would work imo. I mostly broght up Flatpak due to it being more secure thanks to its sandboxed nature.

And, Snaps also suck, Debs are kinda the stone-age of package formats, and AppImages are kinda annoying due to you having to manually download each new AppImage per update. Also, both the Deb and AppImage Modrinth provides seem to be quite buggy. At least, afaik.

@AnOpenSauceDev
Copy link

And, Snaps also suck, Debs are kinda the stone-age of package formats, and AppImages are kinda annoying due to you having to manually download each new AppImage per update. Also, both the Deb and AppImage Modrinth provides seem to be quite buggy. At least, afaik.

I was joking about snaps (unfortunately Canonical isn't), and I've had tonnes of issues with installing the DEB version at all, plus packaging systems that bundle everything like AppImages and Flatpak tend to take up more space than native (10x more for the modrinth app), which might be an issue for some people.

@MagmaBro123
Copy link

And, Snaps also suck, Debs are kinda the stone-age of package formats, and AppImages are kinda annoying due to you having to manually download each new AppImage per update. Also, both the Deb and AppImage Modrinth provides seem to be quite buggy. At least, afaik.

I was joking about snaps (unfortunately Canonical isn't), and I've had tonnes of issues with installing the DEB version at all, plus packaging systems that bundle everything like AppImages and Flatpak tend to take up more space than native (10x more for the modrinth app), which might be an issue for some people.

Eh, true. But there's no real universal way of packaging apps easily.

@getchoo
Copy link

getchoo commented Aug 9, 2023

Flatpak tend to take up more space than native (10x more for the modrinth app),

this is basically a made up number. in fact, most of modrinth app's dependencies are already in the runtime - this means unlike some flatpaks, there are very few dependencies specific to it that need to be bundled. it should be a similar size to the deb + the size of the runtime

@MagmaBro123
Copy link

@AnOpenSauceDev
Copy link

AnOpenSauceDev commented Aug 9, 2023

Flatpak tend to take up more space than native (10x more for the modrinth app),

this is basically a made up number. in fact, most of modrinth app's dependencies are already in the runtime - this means unlike some flatpaks, there are very few dependencies specific to it that need to be bundled. it should be a similar size to the deb + the size of the runtime

I was more talking about the appimage, I'm aware that flatpak has measures to keep its size down.

@cryolithic
Copy link

I'll add that flatpak is what the Steam Deck is using, so that's a big gaming base there

@AnOpenSauceDev
Copy link

AnOpenSauceDev commented Aug 10, 2023

I'll add that flatpak is what the Steam Deck is using, so that's a big gaming base there

flatpak is supported by almost all modern distros in general, but the deck can run the AppImage.

@AnOpenSauceDev
Copy link

ARM64 does work... but only on the frontend. Modrinth fails to retrieve the ARM64 version of java, and i think i also saw an error related to launch options, but it was cut off by the small display that i used for my PI. It's also likely that modrinth wont retrieve the correct natives too, but the fact the GUI works at all is a good start.

@Parilia
Copy link

Parilia commented Aug 29, 2023

I would love to see a flatpak version, it would make a lot of things more simple and for the end user it would be easier to understand to install. Persoanlly I want to as i can more easily set its launch env as running in hybrid mode getting it to use the correct gpu is annoying

@MagmaBro123
Copy link

MagmaBro123 commented Aug 29, 2023

I would love to see a flatpak version, it would make a lot of things more simple and for the end user it would be easier to understand to install. Persoanlly I want to as i can more easily set its launch env as running in hybrid mode getting it to use the correct gpu is annoying

I'd recommend checking out the thread for this stuff on the Discord.

@Geometrically
Copy link
Member

FWIW we are fine to switch to NPM for development if it makes linux package easier. if PNPM is a limitation we are fine dropping it

@getchoo
Copy link

getchoo commented Aug 30, 2023

pnpm has mostly been worked around (or permanently solved in the case of nix) so i wouldn't consider it needed just yet. the bigger issue here would probably be with tauri's dependency on an older version of webkitgtk that has caused us to fall back to the gnome 42 runtime, since my own machine and gha are running into OOMs from compiling webkitgtk; the ci time before it fails has also reached close to 5 hours. there isn't really anything we can do here as even the latest stable version of tauri is still dependent on this, so the biggest help would probably be upgrading to tauri 2.0.0 as soon as it's stable

@Boxersteavee
Copy link

Oh and don't forget the deb package requires a deprecated library that is no longer in ubuntu's apt repository for 22.04. I had to go to their website and find the 20.04 version, which luckily still worked.

@intergrav intergrav changed the title Linux (and Darwin) packaging solutions Packaging solutions Sep 24, 2023
@intergrav
Copy link
Author

intergrav commented Sep 24, 2023

I'm now including Windows packaging in this issue, although I've noted it as low priority. I've added WinGet and Scoop, but let me know if there's any other significant options you would want added.

@osfanbuff63
Copy link

osfanbuff63 commented Sep 24, 2023

For a Scoop package, we'd pretty much need a portable version of Theseus, which as far as I know isn't possible easily with Tauri but I could be wrong. this is incorrect, see below

@intergrav
Copy link
Author

For a Scoop package, we'd pretty much need a portable version of Theseus

I don't think the package has to be portable, although I only really install CLI programs with Scoop which are portable, and other software with WinGet, so I wouldn't know. I assume this because they have a nonportable bucket with a few apps that I know for sure don't have portable versions

@AnOpenSauceDev
Copy link

I don't do much with Windows anymore, but Chocolatey exists.

@getchoo
Copy link

getchoo commented Sep 24, 2023

I don't think the package has to be portable

this is true. the prism launcher package for example was using the regular, non portable version of the package for a while

@osfanbuff63
Copy link

Alright, I'll look into that.

@intergrav
Copy link
Author

intergrav commented Oct 1, 2023

I've added my own Scoop package to the issue

edit: i've replaced it with braw's because his is better

@CompeyDev
Copy link

I've added my own Scoop package to the issue

Bravo!

@jptrzy
Copy link

jptrzy commented Nov 13, 2023

I did create a Alpine Linux package that works.

Thou minecraft won't run because of alpine not using java with gnu core utils.
It looks like, if you want to play minecraft on that distro you need to use flatpak,
which I learned only after compiling theseus too many times 😅

PS. I don't see any reason using alpine package at the current state of alpine java package, but I will include my efforts bellow 😁

Modrinth-App APKBUILD
modrinth-app-0.6.1-r0.apk

@Danik1601
Copy link

Danik1601 commented Nov 13, 2023

It looks like, if you want to play minecraft on that distro you need to use flatpak, which I learned only after compiling theseus too many times 😅

They have been actively working on it, but the work has stalled being blocked by Tauri dependencies. It seems that the work on Flatpak version will continue after Tauri 2.0 is released (after Q1 2024)

@CeruleanDerpo
Copy link

The issue for tauri apps as RPM has been recently closed and compatibility has been added. Now modrinth can be packaged for Fedora.

@getchoo
Copy link

getchoo commented Feb 15, 2024

i've opened a PR for nixpkgs; flathub is next
NixOS/nixpkgs#289149

@RokeJulianLockhart
Copy link

RokeJulianLockhart commented Feb 19, 2024

#497 (comment)

@intergrav, don't merely package RPMs for Fedora — make them compatible with openSUSE (and others) too, using universal RPM packaging, as https://forums.opensuse.org/t/proton-bridge-installation-error/172223/5 explains. I'd also build them at the OBS (as chidiwilliams/buzz#466 (comment) explains).

@getchoo
Copy link

getchoo commented Feb 20, 2024

The issue for tauri-apps/tauri#4402 has been recently closed and compatibility has been added. Now modrinth can be packaged for Fedora.

this shouldn't be used. it will have the same issues as the current deb package, as tauri has no idea it's meant to also launch minecraft

make them compatible with openSUSE (and others) too, using universal RPM packaging

this can get messy very fast. i've attempted it before with prismlauncher, and while it did work, it made the spec files more much difficult to maintain with little benefit. i would highly advise against anyone doing this, especially if you are also targeting leap releases as well as tumbleweed or fedora + enterprise linux.

I'd also build them at the OBS (as chidiwilliams/buzz#466 (comment) explains).

this is a good idea though!...just not for fedora. copr is much better for fedora users with it's built in support, and likewise obs is great for suse users with it's built in support but not so great on fedora. in my experience, users of both distros also overwhelmingly prefer their "native" platforms

@getchoo
Copy link

getchoo commented Feb 20, 2024

i've opened a PR for nixpkgs; flathub is next NixOS/nixpkgs#289149

for anyone who's missed it: the flathub pr is up! you can find the link in the original post for this thread

@MagmaBro123
Copy link

i've opened a PR for nixpkgs; flathub is next NixOS/nixpkgs#289149

for anyone who's missed it: the flathub pr is up! you can find the link in the original post for this thread

Still has a bunch of unfixed stuff though

@getchoo
Copy link

getchoo commented Feb 22, 2024

Still has a bunch of unfixed stuff though

it works fine outside of bugs found in webkitgtk (so nothing specific to the package). anything mentioned there is only about meeting flathub's guidelines rather than getting a working flatpak period

@MagmaBro123
Copy link

MagmaBro123 commented Feb 22, 2024

Ah.

@intergrav
Copy link
Author

I did create a Alpine Linux package that works.

Thou minecraft won't run because of alpine not using java with gnu core utils. It looks like, if you want to play minecraft on that distro you need to use flatpak, which I learned only after compiling theseus too many times 😅

PS. I don't see any reason using alpine package at the current state of alpine java package, but I will include my efforts bellow 😁

Modrinth-App APKBUILD modrinth-app-0.6.1-r0.apk

@jptrzy you think I should just remove the mention of an Alpine package in this issue considering that it doesn't make sense to use it in it's current state?

I'll remove it for now but lmk if you want me to add it back to the issue.

@intergrav
Copy link
Author

I'm currently working on cleaning up and updating the issue with latest info

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Development

No branches or pull requests