Skip to content

publicimageltd/delve

Repository files navigation

delve.el

So what is Delve?

Delve (currently version 0.9.6) is a package on top of Org Roam. It provides tools to collect, inspect and edit Org Roam Nodes in a separate application buffer. It helps you to quickly establish and maintain a project-specific subset of Org Roam Nodes, e.g. when writing a paper or for collecting information on a particular topic. Those subsets (called ‘collections’) can be stored persistently in separate files. Delve also offers functions to ‘edit’ nodes remotely, e.g. by adding or removing tags without having to open the node’s file itself.

Here’s a slightly outdated gif:

./screenshots/delve-intro-tour.gif

Table of Contents

State of Development

Presently, development is mostly aiming at stabilizing the current basic set of features and covering the present functionality with automated tests.

The one big feature still lacking, in my view, is a decent undo function. I’m staring at that goal intensively each time I work on the repo for some other reasons, and I hope the relevant code will thus materialize itself. Until then, Delve will not be added to MELPA yet.

Feature List

  • Provide a Dashboard with customizable queries for quick access (and overview) of nodes with certain tags and of nodes with no links.
  • Sort any list with a spiffy interactive transient menu.
  • Insert or delete Org Roam nodes interactively in the Delve buffer, or add them directly from any open Org Roam file using delve-minor-mode.
  • Re-arrange nodes manually by moving them around or by creating hierarchical trees (outlines).
  • Copy and paste Delve nodes.
  • Yank copied Zettel into Org buffers (they will be rendered as links),
  • Preview an Org Roam node in Delve for a rough glance at its content without having to leave the collection buffer.
  • Toggle between a short node view and a more extended one.
  • Edit the node’s tags from within the Delve buffer without leaving it.
  • Add backlinks or ‘fromlinks’ of a specific node into the current Delve buffer.
  • Hide away nodes by creating “piles”. I do not like this feature and will most likely remove it in 1.0.
  • Move nodes into a new collection.
  • Bookmark stored collections for quick access.

Installation

Delve requires lister, which is available on Melpa. Delve itself, however, is not yet on Melpa. For the time being, you will have to install it manually.

Delve is currently following the org-roam source without taking care of the releases. The only ‘strong’ dependency is the DB scheme used by Org Roam, which has to conform to the Org Roam scheme. If there is a DB error or something similar, thus consider updating org-roam from source even if there is not a new official release yet.

Delve profits from all-the-icons. If it is installed, items will be displayed with nice icons. Install it from there.

Here’s an example installation using straight.el:

(use-package delve
  :straight (:repo "publicimageltd/delve" :host github :type git)
  :after (org-roam)
  ;; this is necessary if use-package-always-defer is true
  :demand t
  :custom
  ;; set meaningful tag names for the dashboard query
  (delve-dashboard-tags '("Tag1" "Tag2"))
  :hook
  (delve-mode . delve-compact-view) ; turn on compact view per default
  (delve-mode . hl-line-mode)   ; nicer to use with hl-line
  :bind
  ;; the main entry point, offering a list of all stored collections
  ;; and of all open Delve buffers:
  (("<f12>" . delve)))

(use-package delve-minor-mode
  :straight (:repo "publicimageltd/delve" :host github :type git)
  :init
  (setq delve-minor-mode-prefix-key (kbd "C-."))
  :config
  (delve-global-minor-mode +1))

The notion of a ‘collection’

The core idea of Delve is to add stuff to editable “collections”. A collection is a list of Org Roam nodes, and it can be stored in a file or be visited in a Delve buffer. All commands which ask you to do something “with a collection”, e.g. to add a node to a collection, first ask you to select the collection to act on. In these cases, you can always choose either an open Delve buffer or a storage file to act on. Selecting a storage file effectively causes the file to be read in a new buffer, reading it ‘on the fly’.

Example: Imagine you have a stored collection of nodes referring to the topic Artificial Intelligence. Then when visiting an Org Roam file, you find an interesting node which you would like to add to that collection. You press M-n c (delve-minor-mode-collect) from within the file’s buffer and select the Delve file which contains that collection on AI. Now a new Delve buffer has been created in the background and the node at point been added to it. To explicitly visit that buffer after adding the Org Roam node, use C-u f12 (C-u M-x delve). Don’t forget to save the modified collection to persist these changes.

Storing collections in files

The top-level command Delve (M-x delve) a list of all collections, stored and currently open. This is useful e.g. when adding Org roam nodes to an existing collection.

Delve tries to mimic Emacs’s established behavior of storing buffers into files. To store a collection, save any Delve buffer with M-x delve-save-buffer (or C-x s from within a Delve buffer). To open a collection (that is, to visit it in a new buffer), respectively use M-x delve-find-storage-file (or C-x f from within a Delve buffer). To save a buffer which is already linked to a file, use M-x delve-write-buffer (or C-x w from within a Delve buffer).

All storage files will be recognized by the file extension .delve. The extension can be changed by setting delve-storage-suffix.

Once a buffer is associated with a file, the file name will be displayed in the header. An asterisk indicates that the buffer content has been modified.

The default storage directory is defined in the variable delve-storage-paths. It is initially set to a directory delve-store within the local emacs user directory. It will be created when you use the storage feature for the first time. But since it’s Emacs, you can customize it:

;; one file name -- one directory:
(setq delve-storage-paths "~/path/to/directory")
;; a list of file names -- multiple locations:
(setq delve-storage-paths '("~/path1" "~/path2"))

Note that if you provide multiple paths, you will have to make sure by yourself that these directories do actually exist.

All stored files can be bookmarked. Simply set a bookmark in the visiting buffer. Calling the bookmark will jump to an existing buffer containing that collection or load it.

Changed at =0.9.4=: If you had used Delve prior to v 0.9.4, you might want to convert all existing files in the storage directory to the new file name format. You can use M-x delve-convert-storage-directory for that. The function is interactive and will guide you through the conversion process in two steps: It first asks you for the directory name (the default should be fine if you did not yet change delve-storage-paths) and then gives you some information about the files found in this directory. Only after you confirm that will your files be changed. In any case, the function is just a bulk rename, so you can just do it manually.

How items are presented

All nodes which refer to a file (and not to a subheading) look like that:

./screenshots/file-node-no-tags.png

Here the node has been created as a backlink from the node “Künstliche Intelligenz (AI)”. If you press RET on the button linking to that original node, Delve will jump to it.

The other type of nodes (i.e., headings) looks like that:

./screenshots/heading-node-tags.png

Also note the list of tags which are associated with that specific node.

Per default, heading notes are displayed with their outline path, including the file title. The variable delve-display-path controls this behaviour, set it to nil to turn this off. Here’s two nodes first with path and then without:

./screenshots/node-with-and-without-path.png

In the dashboard buffer, you’ll find queries:

./screenshots/query.png

Press <TAB> to add its contents into the current buffer’s collection.

Keybindings / Usage

Delve

Navigation / The Tab Key

Use the usual nagivation commands.

<TAB> does the following:

  • If there is a sublist (indented nodes) below the node at point, hide them outline-style.
  • If the current node hides a sublist, open it.
  • If there is no sublist, and the current item is a Zettel, insert all backlinks and fromlinks as a sublist.
  • If there is no sublist, and the current item is a query, execute the query and insert the results as a sublist.
  • If there is no sublist, and the current item is a a pile, insert the contents of the pile and remove the pile (!).

As with most commands inserting stuff, pressing C-u before executing the command offers you to add that result into another collection.

Preview or visit the node at point

./screenshots/node-with-preview.png

Command / KeysFunction
o, C-returnVisit the node at point (its original file)
vToggle display of node (long view vs. short view with only basic infos)
RETIf on a node, toggle preview

The preview buffer recognizes all Org Roam links in the previewed text and turns them into ‘buttons’. Press RET or click on these buttonized links to visit the node they are referring to. Press i on the links in the preview to directly add the node referred to the current collection.

Marking / unmarking nodes

Command / KeysFunction
mMark node at point and move to next one
C-u mMark all nodes below current nodes, if they form a “sublist”
uUnmark node at point and move to next one
C-u uUnmark sublist bewlow
UUnmark all items

Most functions which work with “marked nodes” also accept regions.

Choosing and inserting nodes

Per default, offer to insert a node from a given list of nodes per completion. If consult is installed, all of the following commands allow to insert multiple nodes at once. Support for other completion packages is lacking, contributions are welcome.

Command / KeysFunction
nnInsert new node(s)
ntInsert node(s), limit selection to a specific tag or tags
nbInsert node(s) from all backlinks of that node below
nfinsert node(s) from all fromlinks of that node below

Insert nodes directly

Command / KeysFunction
tabIf current node is not hiding a sublist, insert backlinks and fromlinks
f, C-rightInsert fromlinks of current node as a sublist
b, C-leftInsert backlinks to current node as a sublist

Deleting nodes

Command / KeysFunction
<delete>Delete marked nodes or node at point

Copy and Paste

There is a rudimentary support of copy/paste. Use the usual commands to copy the items within the active region into the kill ring, such as M-w, or to copy and kill them (C-w). A string representing the selected items is pushed onto the kill ring. The yank command (M-y) is remapped to an internal function which interprets this string data and inserts it at point.

There is currently no replacement for yank-pop.

Refresh / Update

Command / KeysFunction
gSync all nodes
C-u gForce update of marked nodes or node at point

Press g to sync all nodes with the Org Roam DB. Delve items which have no corresponding DB entry will be removed, queries will be updated. Use C-u g to just update the node at point (or some marked nodes).

Piling Zettel

Like on any good real desktop, you can pile the Zettels:

Command / KeysFunction
m, uMark or unmark first the nodes you want to pils
pThen create a pile
iInsert contents of pile and remove the pile

If you press p while the region is active, pile the nodes in that region.

To insert a pile, either press <TAB> or i.

Piling will most likely be removed in 1.0.

Insert headings

Use h to insert a heading. A heading is just a simple text item which you can use to internally structure your nodes.

Remote Editing of Org Roam Nodes

Command / KeysFunction
+Add tag(s) remotely
-Remove tag(s) remotely

Remote editing either applies to all marked nodes and the nodes in the currently active region, or, if nothing is marked, to the node at point.

If editing multiple nodes, you can choose between all tags which are present in all nodes (union of sets). Attempts to remove a tag in a node which does not have this tag are silently skipped.

Press g to refresh after editing.

Sorting

The key s gives access to some sorting commands, which are presented as a transient menu. Sorting (or reversing) applies to the current sublist at point. If there is no sublist, the whole list is sorted.

Delve Minor Mode

If you enable the delve-global-minor-mode, a delve-minor-mode will be automatically enabled when visting an Org Roam file. This binds some keys which facilitate ‘collecting’ stuff. All keys are on a transient prefix defaultsing to M-n. You can change the binding for this transient by setting the variable delve-minor-mode-prefix-key manually (or using customize).

;; set this /before/ loading Delve!, e.g. in the :init section of a
;; use-package declaration:
  (setq delve-minor-mode-prefix-key (kbd "C-c d"))

Collecting vs. Inspecting

Delve offers two distinct ways of collecting nodes, corresponding to different workflows.

One variant is to collect Org Roam nodes while browsing through your note files. The imagined workflow is that you visit Org Roam files and think ‘Yes, that’s interesting, I will use it later!’ Thus you copy this node into a list which remains in the background and move on looking through your notes. (Very much like “open tab in background” in web browsers).

For this workflow, Delve Minor Mode commands which have the word collect in their function name are your friends. Per default, they add the nodes to the last selected Delve buffer in the background, not disturbing your evaluation of th nodes.

These collecting commands accept the prefix key (usually C-u) to finetune the selection of the target Delve buffer. Per default (no prefix), Delve uses the last selected buffer or asks you to select one if there is none yet. Using one prefix (C-u) unconditionally prompts you to select the target collection. Using two prefixes (C-u C-u) creates a new buffer for you. Note that in this case, since this automatically generated buffer remains in the background, it will not be recognized as the “last selected buffer” by the following operations.

The second workflow supported is to inspect nodes in order to further explore their relations to other nodes within a Delve buffer. That is, you encounter an interesting node and think: ‘Hey, I want to look at this node’s backlinks, and their backlinks, and just generally check where this node leads me too!’ In this case, you want to switch immediately to the buffer in which you have just collected the nodes. Functions offering this kind of functionality have the word inspect in their function name. They add the nodes to an automatically created Delve buffer and then switch to it. Additionally, this buffer is always marked as the “last selected buffer” so that all further collection commands recognize it.

Minor Mode Keys

M-n opens a transient menu offering the user to either edit, inspect or collect the node at point. However, the functions finally reached through the transients can also be bound separately. Have a look at how the transients are defined or post an issue. It is planned to enable the collection keys also in Org Roam Mode buffers.

For collecting the node at point, use these commands:

Command / KeysFunction
M-n c nAdd node at point to a Delve collection
M-n c aAdd all nodes of current Org Roam file to a Delve collection
M-n c bCollect backlinks from current node
M-n c fFind the node at point in currently open Delve buffers

For inspecting nodes, these commands are available:

Command / KeysFunction
M-n i nAdd node to an automatically created collection and open it
M-n i aInspect all nodes of the current Org Roam file
M-n i bInspect backlinks

Furthermore, delve-minor-mode offers some convenience functions for editing the node at point, which are basically wrappers around the corresponding Org Roam and Org Mode functions:

Command / KeysFunction
M-n e .Create an ID link for the current heading
M-n e +Add tag to the heading at point
M-n e -Remove tag from the heading at point

Customizations

delve-dashboard-queries

A list of functions determining the initial dashboard queries (e.g., for “TODO Items” or for tags). Each function must return a delve--query struct (see the docstring). If this variable is set to nil, do not add any non-tag queries to the Dashboard.

Default setting:

(defcustom delve-dashboard-queries (list #'delve--create-todo-query
                                         #'delve--create-unlinked-query
                                         #'delve--create-last-modified-query)

This adds:

  • A query for TODO items
  • A query for uninked items
  • A query for the 10 last modified items

Please open an issue if you want any specific query to be added.

delve-dashboard-tags 

List of strings (or of lists of strings), from which the initial Dashboard queries are built. E.g., with the setting (setq delve-dashboard-tags '("relevant")), the Dashboard will offer a query for all Delve nodes tagged with the tag relevant.

delve-last-modified-limit

Number of nodes to be displayed in the pre-configured Dashboard query ‘last modified items’.

delve-compact-view-shows-node-path

In compact view, show the complete path to the node if it is not a file node (that is, if it is a subtree). Defaults to t.

Testing Delve

Delve uses the excellent Eldev for development. There is a tests/ directory with tests. To run the automated tests, use:

eldev test

Not eveything is covered by the tests. In particular, interactive stuff is tested ‘manually’. For interactive testing, the repository ships with a live environment which does not interfere with your own local Emacs setup. From the root directory of the repository, just call:

bin/test-emacs 

The script will upgrade the dependencies in an isolated environment and then start a new Emacs instance with only Delve, Org Roam and some basic packages for completion installed.

If you want to have nice icons in the live environment, you have to manually install the icons on your system using M-x all-the-icons-install-fonts from within the testing Emacs instance. Note that even though you install them from your test instance, these icons will be installed system-wide and will not be restricted to the test environment. In general, however, this should not be a problem, since all this command does is to install the fonts and to update the font cache.

The testing enviroment provides a small pseudo Org Roam Zettelkasten to browse, with links, tags, and all.

Changelog

0.9.6.

  • Add two-way syncing Delve <-> DB
  • Refactor deletion
  • Sync with lister v0.9.6.

0.9.5

0.9.4

  • Bookmarks.
  • Lift storage system restriction to one predefined directory; force all storage files to end in .delve.
  • Toggle between extended view (default) and a shorter one.
  • Rudimentary copy/paste.
  • Yank copied or pasted Org Roam node zettel into org buffers.
  • Bugfix because org-roam somwhere lost the function org-roam-node-find-file-noselect.
  • Sorting.

0.9.3

  • Refactor collecting nodes from outside Delve.
  • For non-file nodes, display the outline path. Depends now on Org Roam with DB 18 (merged in Nov., 10th, 21).
  • Introduce new item type “Heading” (key h)
  • Allow remote editing (add, remove tags) of multiple items.
  • Mark list as “modified” if items are deleted, inserted or updated. Storing the list removes that flag.
  • Somewhere in between is 0.9.2, I forgot to update all version numbers in all files.

0.9

Complete rewrite; now based on Org Roam v2.