Operators revisited

[2colours]

Hello,

since operators are one of my favorite topics (and also a topic that involves many misunderstandings and documentation issues), I decided to open this brainstorming session about possible directions to rework documentation involving operators.

Quick disclaimer: when I use the word "productive", I usually mean the linguistic sense: the ability to create previously undefined concepts (here, apparent operators) via regular deduction.

Issues and challenges so far

To get the hang of it, I went through all open issues in the repo, to collect some thoughts. I'm adding a little "tldr" to all of them.

#4160 -> how do we relate to the "decont operator" that isn't even standalone syntax, just a mental framework?
#4058 -> what can/should we say about _intermediary_ computation in an expression? how to glue it to the evaluation model?
#4073 -> how to better distinguish the two identically-looking assignments that otherwise can have different properties?
#2861 -> how to improve the op= situation (and also the meta-operator situation) so that these synthetic operators will be easier to look up and understand?
#2337 -> how to define and categorize operators and related content; productivity and custom definition concerns
#2129 -> bring in identity value as a systematically described aspect of all operators
#1730 -> customizability of user-defined operators
#4033 -> what can/should we say about the underlying behavior of core operators; how can we establish better communication than "not a real operator"?

The following issues are also "operator issues" but I think they are merely content issues; nothing systemic about them:

#4326
#4133
#4077 (this is about hyper meta-operators which can still be tough stuff...)
#3839
#3815
#3756 (is this even an operator, but anyway)
#3333 (I think .= as an intersection of two special situations will require some attention actually)
#3346
#3189
#3037

My ideas

Right now, operators mostly reside in a big monolith at https://docs.raku.org/language/operators. I would like to break this page into two: language/operators would mostly become a guide to Raku operators, talking about several aspects (associativity, infix/prefix/etc. positions, thunkiness, meta-operators/productivity, underlying behavior, etc.) It would contain rationale for "methodoperators" and "thunkiness" at the very least. To remove the big table and list of operators, we could consider URI redirects (using the #component as well for distinguishing, if that's possible), or we could keep that part of the site, preferably moving it out of the way, at the end of the page.
The other part would have a better place under Type in my opinion, but if that's unwelcome (since these are not literal "types"), it could still be a separate thing under Language. It would have roughly the same operators that are listed currently but with much more qualitative information for each: again, associativity, thunkiness, composability with meta-operators, underlying behavior, precedence level and whatnot. Of course, then the really good thing would be to optionally also provide qualitative information about operators in a huge table that can be at least ordered - but let's say, that's a big wish for now.

The point is, separate the "guide" purpose and the "reference" purpose and make them grow in their logical ways (the "guide" deserves definitions, examples and rationale about certain concepts; the "reference" needs accurate, "measurable" information easily accessible).

[doomvox]

Looking over the language/operators page, it does seem like a bit of an info dump up top, with the useful examples buried down below. There might be some advantage in trying to list which operators a new user might really want to see documentation for, (perhaps a synopsis section with some exemplar operators worked through?).

I note that some things on that page, it wouldn't normally occur to us to think of as operators, e.g. using curly braces to create a Hash or a Block. That feels more like a language syntax feature than an operator...

[2colours]

Good points!

Personally, an "example-first approach" (if that makes sense?) doesn't really sit with me - but maybe something like snippets for each individual topic ("this is how you can use a meta-operator with a newly defined operator", "this is an example for chain associativity", etc.) can be useful without feeling too much out of context. I think it's not easy to add the right amount of examples and not fall to one of the extremes.

You are absolutely right about the things that appear as operators but you might not notice or expect them to show up, like [...] or {...}, especially when there are also the "quoting structures" that are apparently not operators, including <...>. I have been thinking about this as well and the best idea I have is a sort of cross-referencing; that is, if you look up Lists, it would be reinforced that they are created using the comma operator, if you look up Arrays, it would mention that usually they are defined using the [...] operator - I think the latter might be "trap-worthy" as well because it's an idempotent operator (didn't look up if it's already there) - and so on. It's realistic and probably not too annoying that with a common type there can be a little description of its usual appearance, "literal" if you will, and that would hopefully increase the discoverability.

[codesections]

I really like this idea overall, but I'd like to take it even further. Right now, docs.raku.org/reference lists around 47 pages and separates them into just two categories: "Fundamental topics" and "General reference". Looking at "Fundamental topics", there's a huge disparity between just how "fundamental" some of the topics are. For example Control flow is genuinely fundamental to reading/writing Raku. Conversely, the MOP or Raku's newline handling are … much less fundamental.

What I'd like to see is for the /reference section to be split into several different categories and to have just a few pages in a "Fundamental topics" category – those pages that a new Rakoon really ought to read to grasp Raku's fundamentals.

I'm bringing this up because one of the Operators pages you propose sounds like it'd fit in that narrower version of "Fundamental topics" and the other one wouldn't. If we reorganized along those lines, then I'd really like the idea of having two Operators pages – basically, a "Operators fundamentals" and an "Advanced operators". But I'm less sold on the idea of having two pages if they're both classified as "Fundamental topics" (not opposed; just less sold – maybe we could pick a name that made the distinction clear).

What do others think about idea of reorganizing /reference into different sub-categories with the goal of getting the list of "fundamental topics" down to a manageable size?

[codesections]

(By the way, I'm intentionally avoiding the label "Guides" or "Tutorials" for any of the pages in /reference. docs.raku.org/introduction seems like the home for "Guides" or "Tutorials". But even within /reference, there are some fundamental references, and some references to more advanced topics.

That said, if we do reorganize a bit, it might make sense to move some of the pages out of /introduction. some of the "Tutorials" are (appropriately!) fairly advanced; having them live under /introduction feels a bit weird to me.)

[2colours]

I think what you say does make a lot of sense but I feel this will lead us to a dead end again.

The current categorization is very recent. If you read into the comments, I for one already noted that I didn't have the same concept of "reference" in mind. However, previously, it really was just "Language" and an unreasonably promoted "Programs". I think getting categorization right is such a large and broad topic, especially when our hands are tied with the URL's at the very least, that it will lead to endless bikeshedding, no agreement, and based on my mindset, quite possibly people who feel they weren't listened to and that the "whole thing was a waste of time" or "worse than leaving as it was".

Frankly, I believe much more in cross-referencing than some hierarchic structure, especially given the existing conditions. Somebody can find the suitable topic in any way - mostly via search - and then naturally discover anything else that might be related.

All in all, I'd like to be more concerned with the possible content and format of covering the topic of operators here. It's almost a mere technicality that I'd like to split it into two and I'm deliberately using the concepts of "reference" and "guide" here: regardless where we put them, the content should be separated based on this duality. Some people want to learn about operators in general - for them, a big table full of properties is near useless; other people want to check some exact properties of particular operators - for them, a guide is just noise that gets in the way. This is the basic idea - but what should be the content? How to structure it within the two parts?