Adopt more error reporting features available in Clojure 1.10+ #3338
Replies: 9 comments 15 replies
-
|
There's a bit of backstory here #2443 for whoever is interested. CIDER stacktrace functionality predates Clojure 1.10 by a few years and we never made a focused effort to update it once those changes landed for various reasons. In general I agree it'd better to support properly the extra info added in Clojure 1.10, so contributions in that direction would be most welcome. Back when it was introduced it was hard to parse Clojure (EDN) in Emacs, but that problem was solved by |
Beta Was this translation helpful? Give feedback.
-
Thanks, I didn't realize this had been on the back burner for close to 5 years.
It's possible but unlikely that I'll be able to directly contribute a PR for this. But for folks who might be able to contribute, the first question I imagine would be to understand prior work. Does CIDER currently do anything at all with this data?
I think I either don't understand or don't agree. We already have the ability to collapse stack frames, which works well. What's important – it seems to me, and I'd really like other folks to chime in here – is to de-emphasize the stacktrace entirely and present an error summary based on data, as Alex suggested in 2018. Basically, the minimum is to match clojure.main's functionality, plus the ability to see the whole stacktrace. As it says in the ticket Alex linked:
This is a longer-term concern, but the configuration described in that ticket feels like the way Magit has config options for various UI elements to be verbose, concise, collapsed, or hidden entirely. In their case it's whether to show the remote URL and "Untracked files", in CIDER we might want to do the same for things like error category and the standardized |
Beta Was this translation helpful? Give feedback.
-
I simply meant it'd be nice to collapse that huge error info map that started to appear in the stack frames in Clojure 1.10. Obviously it'd be even better if we parse it and use it to improve the error messages, but for me the map made the raw stacktraces more unwieldy as the frame that contains it looks huge.
Agreed. I assume for many people it'd be fine if they just see a short error summary in an overlay or the REPL buffer instead a dedicated stacktrace buffer. |
Beta Was this translation helpful? Give feedback.
-
|
@daveliepmann thanks for opening this, I'd love to see things improved in cider as well. Somewhat related, I've noticed that if I augment and rethrow an Exception (using ex-cause) my cider will only show the cause, not the more useful original exception that's thrown:
If I collapse the cause and expand the original exception it looks good:
|
Beta Was this translation helpful? Give feedback.
-
|
Just pointing out that there's a configuration setting Here's what that looks like when successively eval-ing forms from the buffer (note: |
Beta Was this translation helpful? Give feedback.
-
|
Yeah, I think this might be a nice step in the right direction. Although, I guess it might become also a point of friction, given it's going to affect people quite visibly. |
Beta Was this translation helpful? Give feedback.
-
|
I'm only seeing the error message in the mini-buffer, not overlaid. Is there a separate setting to get the overlay behavior? |
Beta Was this translation helpful? Give feedback.
-
|
If we go in this direction, yuhan0's That said, I'm not sure I agree that this is the right solution. Aren't location of the error and behavior of opening a new buffer orthogonal concerns? I like that an error pops up I don't think the clear, concise error message should only be available in the minibuffer or in an overlay which disappears when I start navigating. |
Beta Was this translation helpful? Give feedback.
-
|
@jackrusher perhaps you have I didn't intend this post as a solution per se for the original discussion, and totally agree that the orthogonal problem of improving cider-error's UX / middleware error extraction needs closer attention. In fact this feature came about from personal frustration that the popup buffer was so frequently unhelpful and verbose, that I would begin instinctively dismisssing the window as it appeared. This led to realising how more useful the concise error messages that were being emitted in my hidden repl buffer (exactly the point being made by the OP), and finding a way to surface these stderr messages without too much context switching. Now the act of bringing up the detailed stacktrace buffer is a deliberate decision ("expanding" is a good metaphor) instead of feeling like an annoying intrusion in those situations where I can just intuit the source of error ("agh, forgot to add :else to that last clause, go away Related point re. overlays disappearing on navigation: this was another of my nits with eval results being too ephemeral, but when submitting a PR I don't feel licensed to affect others by changing the defaults? So many users probably never realise |
Beta Was this translation helpful? Give feedback.
-
|
great tip, though it seems it also makes it so it doesn't show the stacktrace in cider debugger when the user explicitly requests it. I added to my config: |
Beta Was this translation helpful? Give feedback.
-
|
@yuhan0 👍🏻 that was it, yeah. I wasn't thinking of this as a solution to the problem statement at the start of the thread, but I did want to try out that setup because I might prefer the ergonomics of a brief, persistent, in-place error message. :) |
Beta Was this translation helpful? Give feedback.
-
|
I'm interested in revisiting this with some concrete solutions which are within easy reach. It's possible I have tunnel vision for the idea of an error summary so it would be good to see alternatives, or to flesh out the idea. Setting aside radical UI changes and existing workarounds using different configs: what is the simplest change which would address any of the examples from Alex's list? For instance, how easy/hard would it be to provide a summary in the existing |
Beta Was this translation helpful? Give feedback.
-
|
It's a matter of updating the existing Recently much of the related functionality was extracted into https://github.com/clojure-emacs/haystack by @r0man, so perhaps he might be willing to help with this as well? For the "proper" solution - I guess we should have some alternative op that returns are "human-friendly" summary of the error instead of the stacktrace. And a some configuration options and wrapper commands for this in CIDER to select the behavior you want with respect to errors. That would require more work on the Emacs side of things (but relatively simple work) and probably a bit more work on the Clojure side of things as well. |
Beta Was this translation helpful? Give feedback.
-
|
Hello, I had a quick look at the haystack analyzer using the From what I can tell, we already send data for the following keys to the client:
Those keys and values are available in the Looking at the code we should also have a It looks we have a bug or outdated code and this (or (:clojure.spec/problems data)
(:clojure.spec.alpha/problems data)
(:clojure.spec/failure data)
(:clojure.spec.alpha/failure data))in order for the I think we have most things in place and it is mostly a matter of trying all those examples and making sure we provide all the necessary data in a usable form. This is how I tried out the I have 2 other NREPL projects going on at the moment, and will be on vacation soon, so I won't have time to work on this myself right now. But I'm happy to assist anyone looking into it. [1] https://github.com/clojure-emacs/haystack/blob/master/src/haystack/analyzer.clj#L337 |
Beta Was this translation helpful? Give feedback.
-
|
I've created #3418 which seems to me a very minimalistic and effective way of tackling 90% of the problem. To be honest, I don't believe this typically is a "stacktrace" problem. A stacktrace is not the best medium of conveying The good news is, as seen in the #3338 (comment) screenshot, CIDER already does a more than decent job at summarizing the issue and presenting it with a concise, overlay-based UI. So, we just need to dynamically choose when, and when not to, display stacktraces. That's what #3418 is about. Still, there are some nice to haves though:
|
Beta Was this translation helpful? Give feedback.
-
|
FWIW, here is the behavior in CIDER 1.8.2. Section (ns repl.cider-error-examples)
;;; error during reading
:::5
;; => Invalid token: :::5
;;; error during macroexpansion caught by a macro spec
(let [x])
;; => [x] - failed: even-number-of-forms? at: [:bindings] spec: :clojure.core.specs.alpha/bindings
;; CLI has an addition line (which seems helpful):
;; Syntax error macroexpanding clojure.core/let at (REPL:1:1).
;; [x] - failed: even-number-of-forms? at: [:bindings] spec: :clojure.core.specs.alpha/bindings
;;; error during macroexpansion thrown by the macro
(cond 1)
;; => cond requires an even number of forms
;; Again, CLI has an additional line:
;; Syntax error macroexpanding cond at (REPL:1:1).
;; cond requires an even number of forms
;;; unexpected error during macroexpansion
(defmulti 5 class)
;; => Unexpected error (ClassCastException) macroexpanding defmulti at (src/repl/cider_error_examples.clj:31:1).
;; class java.lang.Long cannot be cast to class clojure.lang.IObj (java.lang.Long is in module java.base of loader 'bootstrap'; clojure.lang.IObj is in unnamed module of loader 'app')
;;; compilation error
(def 5)
;; The CLI's extra line feels less necessary in this situation, to me:
;; Syntax error compiling def at (REPL:1:1).
;; First argument to def must be a Symbol
;;; evaluation errorNote — these open a *cider-error* buffer on my machine. (/ 1 0)
;; Unhandled java.lang.ArithmeticException
;; Divide by zero
;; Numbers.java: 190 clojure.lang.Numbers/divide
;; Numbers.java: 3911 clojure.lang.Numbers/divide
;; REPL: 46 repl.cider-error-examples/eval19614
(+ 1 :a)
;; Unhandled java.lang.ClassCastException
;; class clojure.lang.Keyword cannot be cast to class java.lang.Number
;; (clojure.lang.Keyword is in unnamed module of loader 'app'; java.lang.Number
;; is in module java.base of loader 'bootstrap')
;; Numbers.java: 155 clojure.lang.Numbers/add
;; Numbers.java: 3743 clojure.lang.Numbers/add
;; REPL: 58 repl.cider-error-examples/eval19616
;;; printing errorAnother *cider-error* and a stacktrace (deftype T [a])
(defmethod print-method T [_ w] (throw (Exception. "boom")))
(->T 1)
I wonder what other folks think of the "Syntax error macroexpanding clojure.core/let" lines which the CLI has but CIDER does not. |
Beta Was this translation helpful? Give feedback.
-
This might be simply because CIDER doesn't always force scrolling so that all output is visible. This is what I see in the repl: ...The first line indeed is not shown until you scroll to it.
This message could be handled a touch better, for which I've filed #3521. Thanks!
Yes, that's expected behavior since those are runtime exceptions, not compilation/read/macroexpansion ones. Our design/plan for them is as follows:
|
Beta Was this translation helpful? Give feedback.
-
I don't think this is a scrolling issue. I see the same thing you do in the REPL: That seems fine. What I believe is an issue is that like many CIDER users I almost never work in the REPL buffer, and am more concerned with behavior from a source buffer. I ran these examples from the 
All the rest sounds excellent, thanks! |
Beta Was this translation helpful? Give feedback.
-
|
There's some intentional dropping, intended for the overlays, and that indeed affects the minibuffer (since it's considered equivalent to overlays, per I foresee that the vast majority of times (It's redundant because in the context of an interactive evaluation, the file and line generally always match the current ones) As you hint, the full output is printed to the repl, but not to other places. It seems a fair compromise - even if you don't use the repl a lot, at least it's a 'log' (you probably can also use a dedicated buffer for output). This concept is further expanded in today's #3523 |
Beta Was this translation helpful? Give feedback.
-
Hmm. I see your point that the first line is somewhat verbose. Some words don't change across distinct error kinds. At the same time, most of the expressions I evaluate are more than one line, so I see some worth in preserving line numbers. (And isn't this a concern in the context of #3495? I'm thinking of a scenario where a user with overlays turned off (so the error message only appears in the minibuffer) evaluates an entire buffer. Aren't line numbers vital in that case?) I also see value in preserving the classification of the error — "Syntax error reading source" contains information distinguishing it from "Syntax error macroexpanding" and the others. One ready-to-hand example is I can figure out what happened given only the second line, but the first line points to the problem ( Or, consider 
The information necessary to understand what went wrong and find it is precisely what has been removed: "Syntax error macroexpanding clojure.core/fn at (src/repl/cider_error_examples.clj:178:11)." Lastly, the upside (preserving a single line of screen real estate) is not compelling to my preferences. Most of the time 1. I have only a small number of buffers open, 2. I'm on a large screen, and 3. if there's an error I want all the relevant info in one place. Maybe this is worth letting folks configure according to their own tradeoffs?
I think I must respectfully disagree. To me, "the full output is printed to the repl" is not ideal, since I almost never look at the REPL buffer. My impression is that this is a fairly common CIDER workflow, and why not? Errors pop up, results go where I tell them to; the only info I can only get in the REPL buffer is a warning, which is by definition ignoreable. |
Beta Was this translation helpful? Give feedback.
-
|
Another example. Say I fat-finger 
The hidden part of the error message seems pretty useful: "Syntax error macroexpanding letfn at (src/repl/cider_error_examples.clj:184:3)." |
Beta Was this translation helpful? Give feedback.
-
|
Alright! Fair enough, I've created #3525 |
Beta Was this translation helpful? Give feedback.
-
|
Today's round of input has been much useful, both in new info we've provided confirming the current/intended behavior starting from 1.8.0, and pointing out some necessary tweaks (many of which are already in transit). Other than that, I'll lock this discussion simply b/c it's completed - I wouldn't want it to become a catch-all for future development. |
Beta Was this translation helpful? Give feedback.
-
Since Clojure 1.10, Clojure provides ways to greatly improve error reporting in REPL tools. As today's standard-bearer for interactive development, CIDER should take full advantage of these new capabilities.
Problem Statement
People consider error messages the top reason Clojure isn't more popular than it is, and the most important area of improvement for the Clojure experience. Year after year in the annual State of Clojure survey, the community ranks "error messages" a top priority for improvement and a major source of frustration:
When introducing people to programming in Clojure, long and complicated stacktraces are a leading complaint. Experienced Clojurians often don't feel it so acutely, but obscure stacktraces are a pain point in the Clojure experience.
Regarding Clojure tooling, CIDER is top dog. It is the most popular Clojure development environment, so a lot of the bad experiences with Clojure errors happen in CIDER.
New possibilities
For a long time, a better error message experience was impossible (or prohibitively difficult) due to limitations in the language itself: an error produced stacktraces with mostly Strings instead of data. As of Clojure 1.10, this is no longer the case: Clojure now categorizes errors, returns information about the error as consistent
ex-data, and provides functions likeex-triage,ex-str, andThrowable->map.The clojure.main REPL takes advantage of these, making many error messages extremely useful and concise. Alex Miller helpfully provided a short list of examples:
:::5- error during reading(let [x])- error during macroexpansion caught by a macro spec(cond 1)- error during macroexpansion thrown by the macro(defmulti 5 class)- unexpected error during macroexpansion(def 5)- compilation error(/ 1 0)or (+ 1 :a) - evaluation error(deftype T [a])(defmethod print-method T [_ w] (throw (Exception. "boom")))(->T 1)- printing errorThe first of that list (
:::5) doesn't look too bad in my CIDER, because the stacktrace isn't long and some of it is minimized.The second,
(let [x]), looks like this in the Clojure CLI:Short and to the point. If I run the same in CIDER's REPL buffer, I get almost the same result...but that's not how I work. I almost never enter forms into the REPL buffer. Like many Clojurians, I prefer to work in an interactive style, sending forms from namespace buffers. With this, the CIDER error experience is not good:
The relevant part doesn't even fit on my screen:
How can we provide a nicer, more concise CIDER error UI?
Beta Was this translation helpful? Give feedback.
All reactions