Synchronizing installed packages across machines?

[DarwinAwardWinner]

I use pallet in my Emacs config which is shared across multiple machines. When I uninstall a package on one machine, Pallet removes it from the cask file, which gets propagated to the other machines. However, nothing tells the other machines to uninstall this package. Would it be possible to provide a command to uninstall every package except for the packages explicitly requested in the cask file and their dependencies?

I could maybe try my hand at implementing it if you're interested.

[rdallasgray]

Hi Ryan -- if you're up for trying an implementation, I'd certainly be interested. The main difficulty I can foresee is that it's impossible to distinguish between top-level dependencies and sub-dependencies. You could get round this by deleting packages not referenced in the Cask file and then running pallet-install to reinstall any missing dependencies. I leave it up to you though.

Thanks a lot!

[DarwinAwardWinner]

I'm hoping it will be possible to collect the packages referenced in the Cask file and then recursively walk the dependency graph to get all their dependencies, and finally just uninstall everything else. But I don't know much about the internals of package.el. Is there anything that would prevent this approach from working?

[rdallasgray]

There's certainly no simple, public API way to do it using package.el. I think there might be a way in epl, though: https://github.com/cask/epl/blob/master/epl.el#L73

[DarwinAwardWinner]

Here's a POC dependency walking function that works for my Cask file:

(defun bundle-recursive-deps (bundle &optional universe)
  (unless universe
    (setq universe (epl-available-packages)))
  (let* ((universe-pkgnames
          (delete-dups (mapcar #'epl-package-name universe)))
         (dep-list nil)
         (dep-ring (make-ring (length universe))))
    ;; Initialize the ring with the explicitly required package names
    ;; from the Cask file
    (cl-loop
     for dep in (cask-bundle-runtime-dependencies bundle)
     do (ring-insert dep-ring (cask-dependency-name dep)))
    ;; Pop each package off the ring, push it into the dependency
    ;; list, and then push its not-previously-seen dependencies into
    ;; the ring.
    (cl-loop
     while (> (ring-length dep-ring) 0)
     ;; Pop the next package off the ring and add it to the list
     for pkgname = (ring-remove dep-ring)
     do (push pkgname dep-list)
     for seen-pkgnames = (nconc (ring-elements dep-ring) dep-list)
     ;; Select all packages in universe with the specified name (there
     ;; might be multiple ones)
     for pkgs = (cl-remove-if-not (lambda (pkg) (eq (epl-package-name pkg) pkgname)) universe)
     ;; Get all dependencies of selected packages, then filter out
     ;; built-in packages and already-seen packages
     for new-pkg-deps =
     (cl-set-difference
      (delete-dups
       (cl-remove-if
        #'epl-built-in-p
        (cl-mapcan (lambda (pkg)
                     (mapcar #'epl-requirement-name
                             (epl-package-requirements pkg)))
                   pkgs)))
      seen-pkgnames)
     ;; Push the new dependencies into the ring
     do (cl-loop for depname in new-pkg-deps
                 do (ring-insert dep-ring depname)))
    dep-list))

;; Simple test
(setq x (cask-initialize))
;; Direct requirements
(setq reqs (mapcar #'cask-dependency-name (cask-bundle-runtime-dependencies x)))
;; Direct requirements plus all their non-builtin recursive
;; dependencies
(setq recreqs (bundle-recursive-deps x (epl-installed-packages)))
;; Should be nil
(cl-set-difference reqs recreqs)
;; Indirect requirements only
(cl-set-difference recreqs reqs)

I have no idea if I'm doing things idiomatically, since epl has lots of struct-based layers of indirection. And also maybe this function belongs in epl itself, or Cask, instead of Pallet. Let me know what you think.

[DarwinAwardWinner]

I should add, I haven't actually implemented the cleanup function, but getting the list of packages to uninstall is just a simple set-difference operation of all installed packages against the return value of the above dependency walker.

[rdallasgray]

Sorry it's taken me so long to get to this. OK, it looks doable, and the way you're using epl seems fine to me. Few points:

• I'm not keen on the heavy use of cl -- it should be simple enough to do this with standard elisp, and would be more idiomatic and readable. I'd be happy to bring in dash if it makes the list munging simpler.
• I'd prefer that all variables internal to the functionality be let rather than set.
• I think it would be more comprehensible if separated into a few smaller functions with informative names.
  If you can get to that kind of implementation, and add tests, then I'm happy to bring this functionality into Pallet.

Thanks again!

[DarwinAwardWinner]

Sure, I'm happy to rewrite it more idiomatically. That was just my proof of concept.

[rdallasgray]

Great stuff, thanks.