Issues with managing plugins on NixOS with Home Manager

[Anomalocaridid]

I use NixOS and I am trying to manage xplr plugins using the Home Manager module. However, for some reason, only the first plugin listed in programs.xplr.plugins gets loaded, preventing me from using two at once, even if they do not appear to otherwise conflict.

Strangely, it also appears that the first plugin gets loaded, even if I do not require it or run its setup function in programs.xplr.extraConfig

Also, since there are currently no packages for xplr plugins, I declare the plugins' repositories as inputs to my config flake and it is these that I put in programs.xplr.plugins.

Am I doing something wrong? I have only recently started using xplr and I know very little about lua.

My xplr config:

{ inputs, ... }: {
  programs.xplr = {
    enable = true;
    plugins = [
      inputs."icons.xplr"
      inputs."extra-icons.xplr"
      inputs."tri-pane.xplr"
    ];
    extraConfig =
      #lua
      ''
        require("icons").setup()
        require("extra-icons").setup()
        require("tri-pane").setup()
      '';
  };
}

The relevant flake.nix inputs

    "tri-pane.xplr" = {
      url = "github:sayanarijit/tri-pane.xplr";
      flake = false;
    };

    "icons.xplr" = {
      url = "github:Ashvith10/icons.xplr/fix-nerd-fonts-v3.0.0-changes";
      flake = false;
    };

    "extra-icons.xplr" = {
      url = "github:dtomvan/extra-icons.xplr";
      flake = false;
    };

[ChanceHarrison]

Hi! I am encountering the same issue as you are (at least I think I am anyway). I am not managing my plugins with flake inputs (yet), but the issue that you described of "only the first plugin gets picked up" still applies to me anyway.

The home-manager module for xplr sets the package.path with the following Nix code:

pluginPath = if cfg.plugins != [ ] then
  (''
    package.path=
  '' + (concatStringsSep " ..\n"
    (map (p: ''"${p}/init.lua;${p}/?.lua;"'') cfg.plugins)) + ''
       ..
      package.path
    '')
else
  "\n";

Like you, I do not have a wealth of experience with lua, but have found myself at the conclusion that the module is at fault; it seems incorrect to have a component of package.path which does not include an interrogation mark, except maybe at the end if it is desired as a default option. At least, that's what is suggested by this page (emphasis mine):

A component does not need to have interrogation marks; it can be a fixed file name, such as the last component in the following path: ?;?.lua;/usr/local/default.lua. In this case, whenever require cannot find another option, it will run this fixed file. (Of course, it only makes sense to have a fixed component as the last component in a path.)

The first part of the map in the HM module creates ${p}/init.lua (where p is the value from the cfg.plugins list) which does not include a question mark (interrogation mark).

As an example:

With the following directory structure:

.
├── plugin-1
│  └── init.lua
├── plugin-2
│  └── init.lua
└── test.lua

Where test.lua is:

package.path="./plugin-1/init.lua;./plugin-2/init.lua"..package.path

require "plugin-1".setup()
require "plugin-2".setup()

and plugin-<1 or 2>.lua look like:

local function setup()
  print("hello from plugin <1 or 2>")
end

return { setup = setup }

Invoking test.lua will produce the following output:

hello from plugin 1
hello from plugin 1

Thus seeming to confirm my impression that the current HM module setup results in one of the plugin's init.lua files being treated as the default that will be associated with all calls to require.

I spent a considerable amount of time thinking about how this could be fixed, and I think I might have a winning idea.

Instead of the HM module setting the package.path, it could instead call dofile(<plugin-path>) to load each plugin. Amazingly, subsequent require(<plugin>) calls work as expected, so long as the path provided to require is the same as the path provided to dofile. [Edit: I played myself. The example below was done such that plugin-1 and plugin-2 were on the default package.path. This is not generally applicable]

To illustrate, in the lua interpreter using the example directory and code from above:

> dofile("plugin-1/init.lua")
> dofile("plugin-2/init.lua")
> require("plugin-1").setup()
hello from plugin 1
> require("plugin-2").setup()
hello from plugin 2

So, as long as we can safely make the assumption/assertion that plugins will follow the convention established by upstream (that plugins have an init.lua at the root of their directory), I think this could be a viable fix.

I plan on making the fix and making a PR to home-manager if said fix goes according to plan.

[Edit: In light of the above edit, I think this could still be a viable fix, but I think the output of dofile would have to be saved to a lua variable to be used in extraConfig and/or the HM module needs to provide a new way to specify arguments to setup() so that way HM can add a setup() call (or one w/ arguments) directly off of the dofile call.]

[NotAShelf]

That's something plugins should resolve, no? Hardcoding ~/.config is not very healthy imo.

[sayanarijit]

I mean, since we're already hard coding .config/xplr/init.lua, it doesn't seem too wrong (from my limited knowledge). Also, we can avoid putting symlinks in .config by maybe putting symlinks in /nix/store.

Let's say we use a wrapper:

wrappedPkg = wrapPkg inputs."icons.xplr";

Where

wrappedPkg points to /nix/store/hash/icons -> /nix/store/hash-icons.xplr

And we add the path of wrappedPkg (/nix/store/hash/?.lua) in package.path, I think it'd be solved.

[sayanarijit]

Now this wrapping can be done internally... we need to send inputs."icons.xplr" along with the repo name "icons" to the plugins argument.

[sayanarijit]

I may not be very clear at communication, so I can try drafting a pr if you need help with visualisation.

[sayanarijit]

See #668 (reply in thread)

[ChanceHarrison]

[ Edit: this was meant to be a reply to my original comment; my apologies for splitting the discussion! ]

@NotAShelf Thanks for hopping in here!

lists is a subset of types - you'd just do types.lists.unique - which is already overlayed to types.unique so just unique should work as intended if types is in the scope

I just tried, and even though types is in-scope, calling just unique causes an error. types.unique is callable, but then I get the error: value is a list while a set was expected. And types.lists.unique throws an error about types not having attribute lists. So we seem to be back at where we started with pulling in lists?

you can specify absolute paths in the plugins list with both derivations and flakes

Yes, but issues arise when using those absolute paths to update Lua's package.path which requires components of that path to have some kind of wildcard (e.g., file name or some part of the path) that corresponds to how the user would reference the plugin in the require call as part of the extraConfig. We could potentially adapt the HM module to reduce the dependency/reliance on the extraConfig escape hatch for configuring plugins, but that feels like it might be best as a separate conversation?

@sayanarijit I am having trouble following with the wrapping concept you proposed. Would that necessitate any changes on the xplr side or is this all Nix-specific?

[Anomalocaridid]

@sayanarijit: I mean, since we're already hard coding .config/xplr/init.lua, it doesn't seem too wrong (from my limited knowledge).

@sayanarijit I think the two cases are pretty different though. ~/.config/xplr/init.lua isn't an external dependency like a plugin is.

@NotAShelf: That's something plugins should resolve, no? Hardcoding ~/.config is not very healthy imo.

I agree with @NotAShelf that it might be at least worth considering that we have plugins address this by not hard coding absolute paths into packages.path. Especially since we seem to be running into some significant complications trying to work around it instead.

Although if that is not an option, maybe we could try using something like substituteInPlace to replace the hardcoded paths ourselves?

[ChanceHarrison]

Hi all. I hope you have had a good weekend.

I have taken @sayanarijit's most recent version of wrapPlugin and the seemingly prevailing desire to have the input be in the form plugins."name" = "path" and turned it into a working patch for the HM module. This patch is tested and working on my end.

Of note, I modified wrapPlugin so that it works for out-of-store paths as well. I think this harmonizes both solutions well.

As always, your time and thoughts are appreciated!

diff --git a/modules/programs/xplr.nix b/modules/programs/xplr.nix
index 22aca02..99e0ab2 100644
--- a/modules/programs/xplr.nix
+++ b/modules/programs/xplr.nix
@@ -1,8 +1,7 @@
 { config, lib, pkgs, ... }:
 let
   inherit (lib)
-    concatStringsSep types mkIf mkOption mkEnableOption mkPackageOption
-    literalExpression;
+    types mkIf mkOption mkEnableOption mkPackageOption literalExpression;
 
   cfg = config.programs.xplr;
 
@@ -10,21 +9,39 @@ let
     version = '${cfg.package.version}'
   '';
 
-  # We provide a default version line within the configuration file, which is
-  # obtained from the package's attributes. Merge the initial configFile, a
-  # mapped list of plugins and then the user defined configuration to obtain the
-  # final configuration.
-  pluginPath = if cfg.plugins != [ ] then
-    (''
-      package.path=
-    '' + (concatStringsSep " ..\n"
-      (map (p: ''"${p}/init.lua;${p}/?.lua;"'') cfg.plugins)) + ''
-         ..
-        package.path
-      '')
+  # If `value` is a Nix store path, create the symlink `/nix/store/newhash/${name}/*` 
+  # to `/nix/store/oldhash/*` and returns `/nix/store/newhash`.
+  wrapPlugin = name: value:
+    if lib.isStorePath value then
+      pkgs.symlinkJoin {
+        name = name;
+        paths = [ value ];
+        postBuild = ''
+          mkdir ${name}
+          mv $out/* ${name}/
+          mv ${name} $out/
+        '';
+      }
+    else
+      builtins.dirOf value;
+
+  makePluginSearchPath = p: "${p}/?/init.lua;${p}/?.lua";
+
+  pluginPath = if cfg.plugins != { } then
+    let
+      wrappedPlugins = lib.mapAttrsToList wrapPlugin cfg.plugins;
+      searchPaths = map makePluginSearchPath wrappedPlugins;
+      pluginSearchPath = lib.concatStringsSep ";" searchPaths;
+    in (''
+      package.path = "${pluginSearchPath};" .. package.path
+    '')
   else
     "\n";
 
+  # We provide a default version line within the configuration file, which is
+  # obtained from the package's attributes. Merge the initial configFile, a
+  # mapped list of plugins and then the user defined configuration to obtain
+  # the final configuration.
   configFile = initialConfig + pluginPath + cfg.extraConfig;
 in {
   meta.maintainers = [ lib.maintainers.NotAShelf ];
@@ -35,15 +52,20 @@ in {
     package = mkPackageOption pkgs "xplr" { };
 
     plugins = mkOption {
-      type = with types; nullOr (listOf (either package str));
-      default = [ ];
-      defaultText = literalExpression "[]";
+      type = with types; nullOr (attrsOf (either package str));
+      default = { };
+      defaultText = literalExpression "{ }";
       description = ''
-        Plugins to be added to your configuration file.
+        An attribute set of plugin paths to be added to the `[package.path](https://www.lua.org/manual/5.4/manual.html#pdf-package.path)` of the `~/config/xplr/init.lua` configuration file.
 
-        Must be a package, an absolute plugin path, or string to be recognized
-        by xplr. Paths will be relative to
-        {file}`$XDG_CONFIG_HOME/xplr/init.lua` unless they are absolute.
+        Must be a package or string representing the plugin directory's path. 
+        If the path is not absolute, it will be relative to {file}`$XDG_CONFIG_HOME/xplr/init.lua`.
+      '';
+      example = literalExpression ''
+        {
+          wl-clipboard = inputs."wl-clipboard.xplr";
+          local-plugin = "/home/user/.config/plugins/local-plugin";
+        };
       '';
     };

Expand for representative HM module usage and `init.lua` output

programs.xplr = {
  enable = true;
  plugins = {
    wl-clipboard = inputs."wl-clipboard.xplr";
    trashy = "/home/ch/.config/xplr/plugins/trashy";
  };
  extraConfig = ''
    xplr.config.general.show_hidden = true;

    require("wl-clipboard").setup{
      copy_command = "wl-copy -t text/uri-list",
      paste_command = "wl-paste",
      keep_selection = false,
    }
    require("trashy").setup()
  '';
};

Resulting ~/.config/xplr/init.lua:

version = '0.21.3'
package.path = "/home/ch/.config/xplr/plugins/?/init.lua;/home/ch/.config/xplr/plugins/?.lua;/nix/store/880jcj7ks0kx5355zmkgzwdbl064g798-wl-clipboard/?/init.lua;/nix/store/880jcj7ks0kx5355zmkgzwdbl064g798-wl-clipboard/?.lua;" .. package.path
xplr.config.general.show_hidden = true;

require("wl-clipboard").setup{
  copy_command = "wl-copy -t text/uri-list",
  paste_command = "wl-paste",
  keep_selection = false,
}
require("trashy").setup()

[NotAShelf]

That seems like a good temporary workaround, though I must express my distaste towards the buildPhase in that function. It may have implications that we don't like (and I definitely don't like the unnecessary read/writes). As such I'd like to propose opening a PR/issue on the HM repository and asking for committers' opinion on it.

[ChanceHarrison]

See nix-community/home-manager#4520 and nix-community/home-manager#4521

[NotAShelf]

The most succinct way that I can say it is that the current HM module implementation does not work for loading more than one xplr plugin.

I had tested the module while working on it, but can't say I've tested with multiple plugins. Probably an oversight on my behalf. My understanding of the problems, however, is that plugins themselves look for certain paths that they cannot resolve as they live in the store. This is aside from the fact that the mapping logic is likely incorrect.

The fix is (seemingly) the least amount of added complexity to correctly add plugins to the Lua packages.path that allows them to be referenced (when loaded/configured with the extraConfig option) with a stable, user-known name (i.e., not the /nix/store path with an unknown hash and unknown (+ potentially non-unique) package name.

The user doesn't need to know the package path though, no? String interpolation, when used correctly, will automatically point at the correct (parent) path of whatever the plugin is, whether it is a package or input with ${pkgs.somePackage} or ${inputs.someInput}. Am I misinterpreting what you mean here?

My question, then, is do we want to support the following input types as well?

• types.pathInStore "A path that is contained in the Nix store. This can be a top-level store path like pkgs.hello or a descendant like "${pkgs.hello}/bin/hello"."
• types.path "A filesystem path is anything that starts with a slash when coerced to a string." (i.e., not limited to the Nix store)

I am under the impression that the module already allows for those types with with types; nullOr (listOf (either package str));

types.str allows store paths and regular paths with no problems

(As an aside, @NotAShelf you mentioned you were fatigued. I hope you are doing well and are able to get some rest if needed. From my perspective, feel free to de-prioritize this if that would help your well-being.)

Thank you for your concern. I'll be taking a much needed rest for the weekend soon enough.