Skip to content

Commit

Permalink
Backport bits of CLI rewrite
Browse files Browse the repository at this point in the history 
The rewrite for Doom's CLI is taking a while, so I've backported a few
important changes in order to ease the transition and fix a couple bugs
sooner.

Fixes #2802, #2737, #2386

The big highlights are:

- Fix #2802: We now update recipe repos *before* updating/installing any
  new packages. No more "Could not find package X in recipe repositories".

- Fix #2737: An edge case where straight couldn't reach a pinned
  commit (particularly with agda).

- Doom is now smarter about what option it recommends when straight
  prompts you to make a choice.

- Introduces a new init path for Doom. The old way:
  - Launch in "minimal" CLI mode in non-interactive sessions
  - Launch a "full" interactive mode otherwise.
  The new way
  - Launch in "minimal" CLI mode *only* for bin/doom
  - Launch is a simple mode for non-interactive sessions that still need
    access to your interactive config (like async org export/babel).
  - Launch a "full" interactive mode otherwise.

  This should fix compatibility issues with plugins that use the
  async.el library or spawn child Emacs processes to fake
  parallelization (like org's async export and babel functionality).

- Your private init.el is now loaded more reliably when running any
  bin/doom command. This gives you an opportunity to configure its
  settings.

- Added doom-first-{input,buffer,file}-hook hooks, which we use to queue
  deferred activation of a number of packages. Users can remove these
  modes from these hooks; altogether preventing them from loading,
  rather than waiting for them to load to then disable them,
  e.g. (after! smartparens (smartparens-global-mode -1)) -> (remove-hook
  'doom-first-buffer #'smartparens-global-mode)

  Hooks added to doom-first-*-hook variables will be removed once they
  run.

  This should also indirectly fix #2386, by preventing interactive modes
  from running in non-interactive session.

- Added `doom/bump-*` commands to make bumping modules and packages
  easier, and `doom/bumpify-*` commands for converting package!
  statements into user/repo@sha1hash format for bump commits.

- straight.el is now commit-pinned, like all other packages. We also
  more reliably install straight.el by cloning it ourselves, rather than
  relying on its bootstrap.el.

  This should prevent infinite "straight has diverged from master"
  prompts whenever we change branches (though, you might have to put up
  with it one more after this update -- see #2937 for workaround).

All the other minor changes:

- Moved core/autoload/cli.el to core/autoload/process.el
- The package manager will log attempts to check out pinned commits
- If package state is incomplete while rebuilding packages, emit a
  simpler error message instead of an obscure one!
- Added -u switch to 'doom sync' to make it run 'doom update' afterwards
- Added -p switch to 'doom sync' to make it run 'doom purge' afterwards
- Replace doom-modules function with doom-modules-list
- The `with-plist!` macro was removed, since `cl-destructuring-bind`
  already serves that purpose well enough.
- core/autoload/packages.el was moved into core-packages.el
- bin/doom will no longer die if DOOMDIR or DOOMLOCALDIR don't have a
  trailing slash
- Introduces doom-debug-variables; a list of variables to toggle on
  doom/toggle-debug-mode.
- The sandbox has been updated to reflect the above changes, also:
  1. Child instances will no longer inherit the process environment of
     the host instance,
  2. It will no longer produce an auto-save-list directory in ~/.emacs.d
  • Loading branch information
@hlissner
hlissner committed May 15, 2020
1 parent 043a561 commit 0e851ac
Show file tree
Hide file tree
Showing 31 changed files with 1,325 additions and 1,158 deletions.
114 changes: 55 additions & 59 deletions bin/doom
View file
Expand Up @@ -5,25 +5,25 @@
:; case "$VERSION" in *\ 2[0-5].[0-9]) echo "Detected Emacs $VERSION"; echo "Doom only supports Emacs 26.1 and newer"; echo; exit 2 ;; esac
:; DOOMBASE="$(dirname "$0")/.."
:; [ "$1" = -d ] || [ "$1" = --debug ] && { shift; export DEBUG=1; }
:; [ "$1" = run ] && { cd "$DOOMBASE"; shift; exec $EMACS -q --no-splash -l bin/doom "$@"; exit 0; }
:; [ "$1" = run ] && { cd "$DOOMBASE"; shift; exec $EMACS -q --no-splash -l init.el "$@"; exit 0; }
:; exec $EMACS --no-site-file --script "$0" -- "$@"
:; exit 0

(let* ((load-prefer-newer t)
(loaddir (file-name-directory (file-truename load-file-name)))
(let* ((loaddir (file-name-directory (file-truename load-file-name)))
(emacsdir (getenv "EMACSDIR"))
(user-emacs-directory
(abbreviate-file-name (or emacsdir (expand-file-name "../" loaddir)))))

;;
(load (expand-file-name "core/core.el" user-emacs-directory) nil t)

;; HACK Load `cl' and site files manually so we can stop them from polluting
;; CLI logs with deprecation and file load messages.
(let ((inhibit-message t))
(when (> emacs-major-version 26)
(require 'cl))
(load "site-start" t t))
(quiet! (when (> emacs-major-version 26)
(require 'cl))
(load "site-start" t t))

(push (expand-file-name "core" user-emacs-directory) load-path)
(require 'core)
(doom-log "Initializing Doom CLI")
(require 'core-cli)

(defcli! :main
Expand All @@ -45,10 +45,10 @@ with a different private module."
(setq user-emacs-directory (file-name-as-directory emacsdir))
(print! (info "EMACSDIR=%s") localdir))
(when doomdir
(setenv "DOOMDIR" doomdir)
(setenv "DOOMDIR" (file-name-as-directory doomdir))
(print! (info "DOOMDIR=%s") localdir))
(when localdir
(setenv "DOOMLOCALDIR" localdir)
(setenv "DOOMLOCALDIR" (file-name-as-directory localdir))
(print! (info "DOOMLOCALDIR=%s") localdir))
(when debug-p
(setenv "DEBUG" "1")
Expand All @@ -75,54 +75,50 @@ with a different private module."
(when (or emacsdir doomdir localdir)
(load! "core/core.el" user-emacs-directory))

(cond ((not noninteractive)
(print! "Doom launched out of %s (test mode)" (path user-emacs-directory))
(load! "init.el" user-emacs-directory)
(doom-run-all-startup-hooks-h))

((null command)
(doom-cli-execute "help"))

((condition-case e
(let ((start-time (current-time)))
(and (doom-cli-execute command args)
(print! (success "Finished! (%.4fs)")
(float-time
(time-subtract (current-time)
start-time)))))
(user-error
(print! (error "%s\n") (error-message-string e))
(print! (yellow "See 'doom help %s' for documentation on this command.") (car args))
(error "")) ; Ensure non-zero exit code
((debug error)
(print! (error "There was an unexpected error:"))
(print-group!
(print! "%s %s" (bold "Type:") (car e))
(print! (bold "Message:"))
(print-group!
(print! "%s" (get (car e) 'error-message)))
(print! (bold "Data:"))
(print-group!
(if (cdr e)
(dolist (item (cdr e))
(print! "%S" item))
(print! "n/a")))
(when (featurep 'straight)
(when (string-match-p (regexp-quote straight-process-buffer)
(error-message-string e))
(print! (bold "Straight output:"))
(print-group! (print! "%s" (straight--process-get-output))))))
(unless debug-on-error
(terpri)
(print!
(concat "Run the command again with the -d (or --debug) switch to enable debug\n"
"mode and (hopefully) generate a backtrace from this error:\n"
"\n %s\n\n"
"If you file a bug report, please include it!")
(string-join (append (list (file-name-nondirectory load-file-name) "-d" command)
args)
" "))
(error ""))))))) ; Ensure non-zero exit code
(let (print-level print-gensym print-length)
(condition-case e
(if (null command)
(doom-cli-execute "help")
(let ((start-time (current-time)))
(and (doom-cli-execute command args)
(print! (success "Finished! (%.4fs)")
(float-time
(time-subtract (current-time)
start-time))))))
(user-error
(print! (error "%s\n") (error-message-string e))
(print! (yellow "See 'doom help %s' for documentation on this command.") (car args))
(error "")) ; Ensure non-zero exit code
((debug error)
(print! (error "There was an unexpected error:"))
(print-group!
(print! "%s %s" (bold "Type:") (car e))
(print! (bold "Message:"))
(print-group!
(print! "%s" (get (car e) 'error-message)))
(print! (bold "Data:"))
(print-group!
(if (cdr e)
(dolist (item (cdr e))
(print! "%S" item))
(print! "n/a")))
(when (and (bound-and-true-p straight-process-buffer)
(string-match-p (regexp-quote straight-process-buffer)
(error-message-string e)))
(print! (bold "Straight output:"))
(print-group! (print! "%s" (straight--process-get-output)))))
(unless debug-on-error
(terpri)
(print!
(concat "Run the command again with the -d (or --debug) switch to enable debug\n"
"mode and (hopefully) generate a backtrace from this error:\n"
"\n %s\n\n"
"If you file a bug report, please include it!")
(string-join (append (list (file-name-nondirectory load-file-name) "-d" command)
args)
" "))
;; Ensure non-zero exit code
(error ""))))))

(doom-cli-execute :main (cdr (member "--" argv)))
(setq argv nil))
23 changes: 14 additions & 9 deletions core/autoload/config.el
View file
Expand Up @@ -55,14 +55,19 @@ And jumps to your `doom!' block."

(defmacro doom--if-compile (command on-success &optional on-failure)
(declare (indent 2))
`(with-current-buffer (compile ,command)
(add-hook
'compilation-finish-functions
(lambda (_buf status)
(if (equal status "finished\n")
,on-success
,on-failure))
nil 'local)))
(let ((windowsym (make-symbol "doom-sync-window")))
`(with-current-buffer (compile ,command t)
(let ((,windowsym (get-buffer-window (current-buffer))))
(select-window ,windowsym)
(add-hook
'compilation-finish-functions
(lambda (_buf status)
(if (equal status "finished\n")
(progn
(delete-window ,windowsym)
,on-success)
,on-failure))
nil 'local)))))

;;;###autoload
(defun doom/reload ()
Expand All @@ -89,7 +94,7 @@ Runs `doom-reload-hook' afterwards."
(doom-initialize-modules 'force)
(general-auto-unbind-keys t)))
(run-hook-wrapped 'doom-reload-hook #'doom-try-run-hook)
(print! (success "Config successfully reloaded!")))
(message "Config successfully reloaded!"))
(user-error "Failed to reload your config")))

;;;###autoload
Expand Down
158 changes: 90 additions & 68 deletions core/autoload/debug.el
View file
@@ -1,5 +1,36 @@
;;; core/autoload/debug.el -*- lexical-binding: t; -*-

;;
;;; Doom's debug mode

;;;###autoload
(defvar doom-debug-variables
'(doom-debug-mode
init-file-debug
debug-on-error
garbage-collection-messages
use-package-verbose
jka-compr-verbose
lsp-log-io
gcmh-verbose
magit-refresh-verbose
url-debug)
"A list of variable to toggle on `doom/toggle-debug-mode'.")

;;;###autoload
(defun doom/toggle-debug-mode (&optional arg)
"Toggle `debug-on-error' and `doom-debug-mode' for verbose logging."
(interactive (list (or current-prefix-arg 'toggle)))
(let ((value
(cond ((eq arg 'toggle) (not doom-debug-mode))
((> (prefix-numeric-value arg) 0)))))
(mapc (doom-rpartial #'set value) doom-debug-variables)
(message "Debug mode %s" (if value "on" "off"))))


;;
;;; Hooks

;;;###autoload
(defun doom-run-all-startup-hooks-h ()
"Run all startup Emacs hooks. Meant to be executed after starting Emacs with
Expand All @@ -17,18 +48,12 @@
;;
;;; Helpers

(defun doom-template-insert (template)
"TODO"
(let ((file (expand-file-name (format "templates/%s" template) doom-core-dir)))
(when (file-exists-p file)
(insert-file-contents file))))

(defsubst doom--collect-forms-in (file form)
(when (file-readable-p file)
(let (forms)
(with-temp-buffer
(insert-file-contents file)
(delay-mode-hooks (emacs-lisp-mode))
(let (emacs-lisp-mode) (emacs-lisp-mode))
(while (re-search-forward (format "(%s " (regexp-quote form)) nil t)
(let ((ppss (syntax-ppss)))
(unless (or (nth 4 ppss)
Expand All @@ -45,7 +70,7 @@ ready to be pasted in a bug report on github."
(require 'vc-git)
(require 'core-packages)
(let ((default-directory doom-emacs-dir)
(doom-modules (doom-modules)))
(doom-modules (doom-module-list)))
(letf! (defun sh (&rest args) (cdr (apply #'doom-call-process args)))
`((emacs
(version . ,emacs-version)
Expand Down Expand Up @@ -243,48 +268,62 @@ Some items are not supported by the `nsm.el' module."
(file (make-temp-file "doom-sandbox-")))
(require 'package)
(with-temp-file file
(insert
(prin1-to-string
(macroexp-progn
(append `((setq noninteractive nil
init-file-debug t
load-path ',load-path
package--init-file-ensured t
package-user-dir ,package-user-dir
package-archives ',package-archives
user-emacs-directory ,doom-emacs-dir)
(with-eval-after-load 'undo-tree
;; undo-tree throws errors because `buffer-undo-tree' isn't
;; corrrectly initialized
(setq-default buffer-undo-tree (make-undo-tree))))
(pcase mode
(`vanilla-doom+ ; Doom core + modules - private config
`((load-file ,(expand-file-name "core.el" doom-core-dir))
(doom-initialize)
(doom-initialize-core)
(add-hook 'window-setup-hook #'doom-display-benchmark-h)
(setq doom-modules ',doom-modules)
(maphash (lambda (key plist)
(let ((doom--current-module key)
(doom--current-flags (plist-get plist :flags)))
(load! "init" (doom-module-locate-path (car key) (cdr key)) t)))
doom-modules)
(maphash (lambda (key plist)
(let ((doom--current-module key)
(doom--current-flags (plist-get plist :flags)))
(load! "config" (doom-module-locate-path (car key) (cdr key)) t)))
doom-modules)
(run-hook-wrapped 'doom-init-modules-hook #'doom-try-run-hook)
(doom-run-all-startup-hooks-h)))
(`vanilla-doom ; only Doom core
`((load-file ,(expand-file-name "core.el" doom-core-dir))
(doom-initialize)
(doom-initialize-core)
(doom-run-all-startup-hooks-h)))
(`vanilla ; nothing loaded
`((package-initialize)))))))
"\n(unwind-protect (progn\n" contents "\n)\n"
(format "(delete-file %S))" file)))
(prin1 `(progn
(setq noninteractive nil
process-environment ',doom--initial-process-environment
exec-path ',doom--initial-exec-path
init-file-debug t
load-path ',load-path
package--init-file-ensured t
package-user-dir ,package-user-dir
package-archives ',package-archives
user-emacs-directory ,doom-emacs-dir)
(with-eval-after-load 'undo-tree
;; undo-tree throws errors because `buffer-undo-tree' isn't
;; correctly initialized
(setq-default buffer-undo-tree (make-undo-tree)))
(ignore-errors
(delete-directory ,(expand-file-name "auto-save-list" doom-emacs-dir) 'parents)))
(current-buffer))
(prin1 `(unwind-protect
(defun --run-- () ,(read (concat "(progn\n" contents "\n)")))
(delete-file ,file))
(current-buffer))
(prin1 (pcase mode
(`vanilla-doom+ ; Doom core + modules - private config
`(progn
(load-file ,(expand-file-name "core.el" doom-core-dir))
(setq doom-modules-dirs (list doom-modules-dir))
(let ((doom-init-modules-p t))
(doom-initialize)
(doom-initialize-core-modules))
(setq doom-modules ',doom-modules)
(--run--)
(maphash (lambda (key plist)
(let ((doom--current-module key)
(doom--current-flags (plist-get plist :flags)))
(load! "init" (doom-module-locate-path (car key) (cdr key)) t)))
doom-modules)
(maphash (lambda (key plist)
(let ((doom--current-module key)
(doom--current-flags (plist-get plist :flags)))
(load! "config" (doom-module-locate-path (car key) (cdr key)) t)))
doom-modules)
(run-hook-wrapped 'doom-init-modules-hook #'doom-try-run-hook)
(doom-run-all-startup-hooks-h)))
(`vanilla-doom ; only Doom core
`(progn
(load-file ,(expand-file-name "core.el" doom-core-dir))
(let ((doom-init-modules-p t))
(doom-initialize)
(doom-initialize-core-modules))
(--run--)
(doom-run-all-startup-hooks-h)))
(`vanilla ; nothing loaded
`(progn
(package-initialize)
(--run--))))
(current-buffer)))
(let ((args (if (eq mode 'doom)
(list "-l" file)
(list "-Q" "-l" file))))
Expand Down Expand Up @@ -343,7 +382,7 @@ to reproduce bugs and determine if Doom is to blame."
(doom-sandbox-emacs-lisp-mode)
(setq-local default-directory doom-emacs-dir)
(unless (buffer-live-p exists)
(doom-template-insert "VANILLA_SANDBOX")
(insert-file-contents (doom-glob doom-core-dir "templates/VANILLA_SANDBOX"))
(let ((contents (substitute-command-keys (buffer-string))))
(erase-buffer)
(insert contents "\n")))
Expand Down Expand Up @@ -378,20 +417,3 @@ will be automatically appended to the result."
(profiler-report)
(profiler-stop))
(setq doom--profiler (not doom--profiler)))

;;;###autoload
(defun doom/toggle-debug-mode (&optional arg)
"Toggle `debug-on-error' and `doom-debug-mode' for verbose logging."
(interactive (list (or current-prefix-arg 'toggle)))
(let ((value
(cond ((eq arg 'toggle) (not doom-debug-mode))
((> (prefix-numeric-value arg) 0)))))
(setq doom-debug-mode value
debug-on-error value
garbage-collection-messages value
use-package-verbose value
jka-compr-verbose value
lsp-log-io value
gcmh-verbose value
magit-refresh-verbose value)
(message "Debug mode %s" (if value "on" "off"))))

0 comments on commit 0e851ac

Please sign in to comment.