Alternative line wrap mode

[broneill]

It looks like the line wrap syntax with the leading backslash is there mainly to prevent erroneously defining a new pipeline stage. In other contexts (tuples, etc.) no special line wrap character is needed. Personally, I find the use of the backslash to be visually distracting, and knowing when to use it might be confusing for new users.

Is it possible instead to require that a new pipeline stage be non-indented, within the current scope? Thus the example in the book would simply be:

from artists
select is_europe =
  country == "DE"
  || country == "FR"
  || country == "ES"

Because each "country" line is indented, it's clear that it's an extension of the is_europe expression.

I can see how this approach does cause some issues, because indenting currently has no effect on syntax. Leaving it out does simplify the parser. But the parser already treats newlines as syntax, which special rules, and so there is some precedent for treating whitespace as syntax.

[max-sixty]

FWIW I am sympathetic to indentation-sensitive languages, I rather like them! We did have some discussion about this in the very early days of PRQL.

But they're a totally different type of language; it's not viable to have "just a bit of indentation-sensitivity" given how pervasive it is — even more so for whether users should have to worry about indentation when writing PRQL than us overhauling the parser. Newlines are quite different from indentation in that respect.

[max-sixty]

One note — something like any_of could take a tuple of values, so this could work:

from artists
select is_europe = any_of {
  country == "DE",
  country == "FR",
  country == "ES",
}

(would need to think about the name. Notably any is an aggregate function...)

[broneill]

There might be areas in which the parser can be a little smarter with respect to newlines, when it "knows" that a | isn't a valid next token.

Legal:

let is_europe = c -> c == "DE" || c == "FR" || c == "ES"

Illegal:

let is_europe = c -> c == "DE"
|| c == "FR" || c == "ES"

If I had used an explicit |, the parser complains with "unexpected | while parsing function call"

let is_europe = c -> c == "DE" | || c == "FR" || c == "ES"

If a newline is encountered where it's not syntactically legal for a | to appear, could it just be converted to a line wrap automatically?

Note: I edited the above code snippet.

[broneill]

I understand there has to be a tradeoff, and I agree that a context-free grammar is generally preferred.

I've been experimenting with various oddball cases in the playground, which because of the "auto pipe" feature look very confusing at first. Like this:

let a = (3
math.pow 2)

I'm wondering if I'm at fault for trying to break stuff...

[max-sixty]

Ha, breaking stuff is great!! Please post examples of things that break if there isn't one already!

---

Slightly adjusted:

from x
derive y = (3
math.pow 2)

Yes, I agree that looks odd!

Though the criteria for what to highlight should be something like "there isn't a good way of expressing this" or "this looks deceptively like something else". Rather than "there exists a bad way of expressing this"...

[broneill]

I'm still trying to understand the rules for pipes and newlines, I'm very confused.

Illegal: unexpected | while parsing function call

let a = 1 | from artists

Legal:

let a = 1
from artists

It seems that newline is more than a pipe, but also an expression terminator?

[max-sixty]

Check out the list at https://prql-lang.org/book/reference/syntax/pipes.html :)

[broneill]

Yeah, there's just too many exceptions to the rule.

[broneill]

I'm looking at the other pipelined query languages, and they've all adopted a syntax which requires the pipe character to be specified. If PRQL adopted a similar strategy, then newlines aren't syntax anymore, and the need for word wrapping goes away too.

[max-sixty]

Appreciate the thoughts. To set expectations: I don't think there's a much capacity to make sweeping changes to the syntax, and it would require a strong consensus built with many examples. The decision to have semantic newlines was quite deliberate.

That said, happy to leave this open and see if any consensus builds, and appreciate any questions and directed suggestions backed with examples...

[broneill]

I noticed that if the first keyword on a line is into, the line break isn't interpreted as a pipe. The docs only list func, let or from as receiving special treatment.

[max-sixty]

I noticed that if the first keyword on a line is into, the line break isn't interpreted as a pipe.

It is! The query is piped into into...

[broneill]

What about these forms:

1 into a  # legal

1
into a  # legal

1 | into a  # illegal -- pipeline expected one of !, (, +, -, ->, ==, [, an identifier, keyword case or {, but found keyword into

If the form with an explicit pipe is illegal, then I would expect it also be illegal when the pipe is inserted with a line break.

[max-sixty]

1 | into a # illegal -- pipeline expected one of !, (, +, -, ->, ==, [, an identifier, keyword case or {, but found keyword into

Ah, this is a bug. Thank you!

[broneill]

What's the correct behavior supposed to be? Are all forms supposed to be legal?

[max-sixty]

Yes I wasn't clear — I updated #4291