Undocumented syntax for "'{;' Comprehensions" #6264

smoge · 2024-04-22T15:11:14Z

SuperCollider has had the great feature of implementing List Comprehension for decades.

But taking a look around the source code, I found a strange syntax that I don't remember being documented anywhere. And it's not an "exotic" curiosity, it's a sophisticated feature.

Have you ever seen List Comprehensions using {; instead of {:? These aren't your typical list comprehensions, but they've got some similar vibes. I will call it "Monad Comprehension", because that's a way it could be interpreted in Haskell.

Look:

Normal List Comprehension syntax: {:

{: x*2, x <- (2..6)}.all  // Gives you: [4, 6, 8, 10, 12]

This one gives you control over what gets yielded and when.

{; x*2, x <- (2..6)}  // Immediately outputs the first result: 2

Use {; when you only want to output specific results or need to control outputs based on certain conditions, present of past states, etc.

It doesn’t exist as List Comprehension in Haskell, but can be achieved as a Monad Comprehension. See:

import Control.Monad (guard)

doubleIfGreaterThanFive :: Int -> Maybe Int
doubleIfGreaterThanFive x = do
    guard (x > 5)  -- If x is not greater than 5, the computation outputs Nothing
    return (x * 2)  -- Otherwise, return Just the doubled value 

processList :: [Int] -> [Maybe Int]
processList xs = do
    x <- xs            
    return (doubleIfGreaterThanFive x) 

main :: IO ()
main = print $ processList [3, 6, 8, 2, 10, 1]

Another Example in SuperCollider:

r = Routine { loop { {; if (x*2 > 10) { yield (x*2)}, x <- (2..6) } } };

r.nextN(30); 
-> a Routine
-> [ 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12 ]

It deserves proper documentation.

--

The text was updated successfully, but these errors were encountered:

JordanHendersonMusic · 2024-04-22T15:24:05Z

I could only get this to output a constant value. Could you please provide an example of where you might use this?

smoge · 2024-04-22T15:52:25Z

I could only get this to output a constant value. Could you please provide an example of where you might use this?

It's because 2 * 2, 2 * 3, 2* 4 and 2 * 5 are false for (> 10), only 6 returns true, and the output is x * 2, there is only one option

It's a demonstration. Would you like more examples?

(It's also related to Python Generators in a way, you might know it)

EDIT: List comprehension is a bit like recursions for those who always use for loops, sort of.... the same problems can be done both ways, but sometimes you just prefer one way over another

JordanHendersonMusic · 2024-04-22T16:24:40Z

It's called a termination termination clause apparently!

supercollider/HelpSource/Guides/ListComprehensions.schelp

Line 175 in 28a0b12

subsection:: termination clause

Edit: Oh wait ignore me! This is wrong.

smoge · 2024-04-22T16:26:57Z

It's called a termination termination clause apparently!

supercollider/HelpSource/Guides/ListComprehensions.schelp

Line 175 in 28a0b12

subsection:: termination clause

Ok, but that's also a SC terminology, right? I don't remember this elsewhere.

Another thing is that it does not just terminate, it can continue as the second example. Let's see what's that.

JordanHendersonMusic · 2024-04-22T16:44:16Z

Sorry I messed up, ignore that last thing.

Here is the grammer from the help doc.

[ ] - optional
{ } - zero or more

<list_compre> ::= "{:" <body> ',' <qualifiers> "}"

<body> ::= <exprseq>

<exprseq> ::= <expr> { ";" <expr> }

<qualifiers> ::= <qualifier> { ',' <qualifiers> }

<qualifier> ::= <generator> | <guard> | <binding> | <side_effect> | <termination>

<generator> ::= <name> "<-" <exprseq>

<guard> ::= <exprseq>

<binding> ::= "var" <name> "=" <exprseq>

<side_effect> ::= "::" <exprseq>

<termination> ::= ":while" <exprseq>

{; isn't valid according to that. Either the doc is wrong, or the implementation is wrong, probably important to figure out which before making this a feature.

I'm also quite confused by lines like this

{;x, x <- (5..15), x > 10}

which returns 5.

I also still can't see why you'd

r = Routine { loop { {; if (x*2 > 10) { yield (x*2)}, x <- (2..6) } } };

over

r = {: x*2, x <- (2..6), x*2 > 10 }
r.all or r.next

smoge · 2024-04-22T16:46:00Z

It's deep in the parser of the language. In what sense it is "wrong"?

JordanHendersonMusic · 2024-04-22T17:00:49Z

<list_compre> ::= "{:" <body> ',' <qualifiers> "}"

This line means list comprehension must begin with the literal {:.

Either the documentation is wrong, or the actual parser is wrong. I'm just saying, before considering documenting the behaviour of {; it is important to figure out which is wrong — it looks like the docs are wrong.

Here the bit of the parser for reference

generator : '{' ':' exprseq { pushls(&generatorStack, $3); pushls(&generatorStack, 1); } ',' qual '}'
			{
				PyrSlot slot;
				SetSymbol(&slot, getsym("r"));
				PyrSlotNode* selectornode = newPyrSlotNode(&slot);

				PyrParseNode *block = (PyrParseNode*)newPyrBlockNode(0, 0, (PyrParseNode*)$6, false);
				PyrParseNode *blocklit = (PyrParseNode*)newPyrPushLitNode(NULL, block);
				$$ = (intptr_t)newPyrCallNode(selectornode, (PyrParseNode*)blocklit, 0, 0);
			}
		| '{' ';' exprseq { pushls(&generatorStack, $3); pushls(&generatorStack, 2); } ',' qual '}'
			{
				$$ = $6;
			}
		;

It is clearly there.
Its also in sparkler - https://solitarybees.us/hadron/sparkler/-/blob/main/grammar/SCParser.g4?ref_type=heads

listComp : CURLY_OPEN COLON exprSeq COMMA qualifiers CURLY_CLOSE
         | CURLY_OPEN SEMICOLON exprSeq COMMA qualifiers CURLY_CLOSE
         ;

smoge · 2024-04-22T17:01:03Z

Can you have an interactive control flow, or when you can combine timing and selection to alter the result?

r = Routine {
loop {
{; if ((x2).mod(7) == 0) { yield (x2) }, x <- (1..100)}
}
}

smoge · 2024-04-22T17:02:59Z

So JMC wrote a bad parser that does this thing? It looks like like it was just "forgotten", but was indeed implemented. It's part of the language, it's clear.

Many ideas never went to the foreground for one reason or another. We can just forget about it too )))) But some curiosity would be nice.

I will ask him, maybe he remembers.

JordanHendersonMusic · 2024-04-22T17:09:27Z

Can you have an interactive control flow, or when you can combine timing and selection to alter the result?

There is a good chance I'm missing something, but I just can't see the use-case?

This does the same but is much easier to read.

x = {: x*2, x <- (1..100), (x*2).mod(7) == 0 }
x.next

Many ideas never went to the foreground for one reason or another. We can just forget about it too )))) But some curiosity would be nice.

I wasn't suggesting not to do this! Just that it is new to me and violates what the existing docs say. I do think a use-case where there is some clear benefit over using {: is needed to support this issue however.

smoge · 2024-04-22T17:14:14Z

It's not a list comprehension, it's something else. But related. People experiment with other "comprehensions" in other languages, like Haskell. Maybe it was just forgotten. Not even LC is known in the SC community. Not rarely do people find it strange in the code, which is unfortunate.

Let's give some time to this one, no need to rush. ))))

smoge · 2024-04-22T17:35:25Z

I wasn't suggesting not to do this! Just that it is new to me and violates what the existing docs say. I do think a use-case where there is some clear benefit over using {: is needed to support this issue, however.

I think, just playing a little and without any reference, that it is more flexible in terms of control inside the comprehension, how much and when then will deliver new values independently etc. Am I wrong?

JordanHendersonMusic · 2024-04-23T09:12:54Z

I think, just playing a little and without any reference, that it is more flexible in terms of control inside the comprehension, how much and when then will deliver new values independently etc. Am I wrong?

The examples you've posted so far can more concisely be reproduce in the {: syntax.

It is also a little confusing because it returns the value, not the routine, but perhaps there are situations where this is useful?

It should definitely be documented — assuming it isn't an unfinished feature —, the question is to what extent?

smoge · 2024-04-23T09:32:51Z

I will come up with more examples exploring it, and we will analyze them and see if it makes sense to resurrect it after two decades, or let it be forgotten forever )) (No time today, though)

Anyway, we need to figure all out before writting documentation

smoge added the enhancement label Apr 22, 2024

smoge changed the title ~~Undocumented unique syntax for "Monad Comprehension" in SuperCollider~~ Undocumented syntax for "'monad' Comprehension" in SuperCollider Apr 22, 2024

smoge changed the title ~~Undocumented syntax for "'monad' Comprehension" in SuperCollider~~ Undocumented syntax for "'{;' Comprehensions" Apr 23, 2024

JordanHendersonMusic added the comp: class library SC class library label May 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Undocumented syntax for "'{;' Comprehensions" #6264

Undocumented syntax for "'{;' Comprehensions" #6264

smoge commented Apr 22, 2024

JordanHendersonMusic commented Apr 22, 2024

smoge commented Apr 22, 2024 •

edited

JordanHendersonMusic commented Apr 22, 2024 •

edited

smoge commented Apr 22, 2024 •

edited

JordanHendersonMusic commented Apr 22, 2024

smoge commented Apr 22, 2024

JordanHendersonMusic commented Apr 22, 2024

smoge commented Apr 22, 2024

smoge commented Apr 22, 2024 •

edited

JordanHendersonMusic commented Apr 22, 2024 •

edited

smoge commented Apr 22, 2024 •

edited

smoge commented Apr 22, 2024

JordanHendersonMusic commented Apr 23, 2024

smoge commented Apr 23, 2024 •

edited

Undocumented syntax for "'{;' Comprehensions" #6264

Undocumented syntax for "'{;' Comprehensions" #6264

Comments

smoge commented Apr 22, 2024

JordanHendersonMusic commented Apr 22, 2024

smoge commented Apr 22, 2024 • edited

JordanHendersonMusic commented Apr 22, 2024 • edited

smoge commented Apr 22, 2024 • edited

JordanHendersonMusic commented Apr 22, 2024

smoge commented Apr 22, 2024

JordanHendersonMusic commented Apr 22, 2024

smoge commented Apr 22, 2024

smoge commented Apr 22, 2024 • edited

JordanHendersonMusic commented Apr 22, 2024 • edited

smoge commented Apr 22, 2024 • edited

smoge commented Apr 22, 2024

JordanHendersonMusic commented Apr 23, 2024

smoge commented Apr 23, 2024 • edited

smoge commented Apr 22, 2024 •

edited

JordanHendersonMusic commented Apr 22, 2024 •

edited

smoge commented Apr 22, 2024 •

edited

smoge commented Apr 22, 2024 •

edited

JordanHendersonMusic commented Apr 22, 2024 •

edited

smoge commented Apr 22, 2024 •

edited

smoge commented Apr 23, 2024 •

edited