Added the nested, updating, and chained subqueries CIP #100

petraselmer · 2016-06-22T16:01:11Z

This is part of the redesign of Cypher for adding support for working with multiple graphs that targets Cypher 10.

Rendered:

https://github.com/petraselmer/openCypher/blob/CIP-nested-subqueries/cip/1.accepted/CIP2016-06-22-nested-updating-and-chained-subqueries.adoc

aseemk · 2016-06-23T14:31:45Z

I'm not informed enough to offer meaningful feedback here, but as a user of Neo4j, this looks great! Thank you for putting this together. =)

aseemk · 2016-06-23T14:35:22Z

The only thing that made me go "hmm, interesting" was that subqueries' variables (variable bindings) persist into the outer query. How does that work if you have multiple independent subqueries that happen to (coincidentally) share a variable name? (E.g. something generic like user or tweet).

It would feel a bit more intuitive to me if only the RETURN variables persisted. Then, you could naturally rename those with WITH before running another subquery, etc.

But again, I'm not informed enough to know if this feedback is reasonable or flawed.

cleishm · 2016-06-23T16:32:47Z

@aseemk Based on the examples, I'm guessing that it is only the RETURN values that are made available to the outer query. It's just that the language isn't very clear when it says "...Any new variable bindings produced by evaluating the subquery will augment the variable bindings of the initial record...".

boggle · 2016-06-24T10:58:10Z

Even more so, the intent is to not allow any shadowing, i.e. you cannot rebind variable from the outer query in the RETURN of the nested subquery. This is in line with CALL ... YIELD... and UNWIND ... AS ... semantics which also do not allow to do that. I think it really helps reduce confusion and hopefully nudges the users towards picking more meaningful (distinguishing) names.

zazi · 2016-06-27T15:55:36Z

+1
(but I guess in SPARQL I can do the same (i.e. I doubt a bit the SPARQL comparison in your proposal ;) ))

thobe · 2016-07-12T11:34:02Z

I like the simplicity of this proposal, but I wonder how well it scales to other things we would/might want to do with/as subqueries in the future.

I was actually hoping we would be able to replace union with subqueries, rather than just allowing post-union processing by using union in a subquery, because I find union-queries hard to read with the constituent parts not being strongly delineated.

I've also held some hope that we would be able to use sub-queries as parameters to procedures, and there might be other use cases we would want to use sub-queries for that I can't think of right now.

petraselmer · 2016-07-20T14:26:46Z

I have added the 'NOT READY FOR REVIEW' label only because we still need to consider path bindings for subqueries. We'll add this in a fortnight or so, when @boggle and I are back from leave.

In the meantime, much content regarding the new DO syntax has been added, so comments welcome.

petraselmer · 2016-07-20T14:59:21Z

Hi @zazi

Thanks for your comment. I was wondering if you could provide an example (or link) in which correlated subqueries are supported in SPARQL? According to the W3C link in the CIP, it appears to be the case that 'vanilla' SPARQL cannot support correlated subqueries, but it may very well be the case that a vendor-specific implementation does. If this is the case, I would love to know, and, of course, this shall be added to the CIP.

zazi · 2016-07-24T20:53:04Z

@petraselmer I'm not 100% an expert in this field. Nevertheless, (from my understanding) I'm supporting the answer to this question http://answers.semanticweb.com/questions/24508/does-sparql-11-support-correlated-subqueries ;)

zazi · 2016-07-24T21:09:05Z

cip/CIP2016-06-22-nested-subqueries.adoc

+
+Owing to the bottom-up nature of SPARQL query evaluation, the subqueries are evaluated logically first, and the results are projected up to the outer query.
+
+Only variables projected out of the subquery will be visible, or in scope, to the outer query.


I would tend to say that this is not a disadvantage, but rather then a (good) feature.

zazi · 2016-07-24T21:15:35Z

For readability issues, I would recommend to add all queries in plain, natural language sentences as well.

cleishm · 2016-07-25T07:25:21Z

cip/CIP2016-06-22-nested-subqueries.adoc

+  MERGE (c:Child {id: x})
+  MERGE (r)-[:PARENT]->(c)
+}
+----


Might be nice to explain the difference between the preceding example and what it would be without the DO and just having the MERGE calls following UNWIND. I assume it's simply keeping the same cardinality (i.e. the DO itself has a cardinality of 1) and not introducing any new identifiers, but it might be helpful to be explicit about that and why it's helpful.

zazi · 2016-07-26T15:03:23Z

Furthermore, some example data that illustrate the queries might be helpful (i.e. a small data set and the result sets of the different queries).

Mats-SX · 2016-07-28T09:11:54Z

@zazi

Furthermore, some example data that illustrate the queries might be helpful (i.e. a small data set and the result sets of the different queries).

This would be solved by the addition of TCK scenarios for this feature (which is planned). Do you agree?

boggle · 2016-09-26T10:05:50Z

@thobe passing subqueries as arguments to procedures is out of scope for this CIP.

@thobe How do you envision replacing UNION with subqueries? I'm not seeing that. Besides having UNION is very common for people with a SQL background and opens up a natural trajectory for adding other set operations (INTERSECT SET DIFFERENCE)

Tin-Nguyen · 2016-09-30T08:24:52Z

+1, I'm waiting for this

ric81 · 2016-12-13T18:20:55Z

In which Neo4j release is/will this change be included ?

Mats-SX · 2016-12-14T08:20:41Z

@ric81 That is currently not known. Do note that this repository manages the specification and standardisation of Cypher as a language, which is separate from Neo4j the graph database (although this decoupling is fairly new and not yet fully mature).

thobe · 2017-03-30T15:21:46Z

cip/CIP2016-06-22-nested-subqueries.adoc

+
+The inner mandatory query is any inner match query.
+
+Moreover, any inner match query from which the trailing final `RETURN` clause has been omitted may also be used as an inner mandatory query.


I think this needs examples to demonstrate the use of these. As well as the difference between having a RETURN clause and leaving it out.

In my understanding MANDATORY without RETURN would be like EXISTS, but as a clause with the capability of failing the query rather than a boolean expression. Interestingly that use of MANDATORY would not affect the cardinality of the query, whereas MANDATORY with RETURN could potentially increase the cardinality of the query (although never decrease it, since that would mean that no matches were found).

We're actually thinking about making RETURN mandatory on MANDATORY match based on the reason that this will be the 95% use case and having to name in the remaining 5% isn't a big deal.

The benefit would be less special casing. Then again, maybe someone might find it very natural to leave off RETURN.

Mats-SX · 2017-04-24T09:51:34Z

This CIP doesn't contain any formal syntax description in EBNF format.

- Clear separation between additive and replacing semantics - Additive semantics for nesting with {} - Replacing semantics for flat composition - Use THEN for discard cardinality - Use WITH|RETURN|YIELD NOTHING for discard fields

petraselmer added language feature CIP labels Jun 22, 2016

petraselmer mentioned this pull request Jun 23, 2016

Cypher feature request: post-UNION processing neo4j/neo4j#2725

Closed

petraselmer added the NOT READY FOR REVIEW label Jul 20, 2016

zazi reviewed Jul 24, 2016
View reviewed changes

cleishm reviewed Jul 25, 2016
View reviewed changes

Mats-SX removed the language feature label Feb 3, 2017

petraselmer mentioned this pull request Feb 6, 2017

Extended subqueries for Cypher #181

Open

petraselmer removed the NOT READY FOR REVIEW label Mar 30, 2017

thobe reviewed Mar 30, 2017

View reviewed changes

Petra Selmer added 3 commits October 16, 2017 13:47

Added the nested subqueries CIP

5c6f9d7

Sundry content edits to the subquery CIP

d7d0a83

Clarified the syntax wrt OPTIONAL

bf71712

Petra Selmer and others added 10 commits October 16, 2017 13:47

Addressing comments; making clarifications

4caeb54

Sketched out additional forms of nested subqueries.

5b5b9cc

Homogeneous syntax for OPTIONAL, MANDATORY, MATCH, DO WHEN

fe21475

Address feedback and introduce new syntactic short forms

80a1ce4

Add chained subqueries with THEN and overhaul document

b8f49d6

Reflect discussion; add new conditional form of DO and WHERE shorthand

bf53252

Textual improvements

70a91cd

Refer to Query Combinator CIP

1f02e2b

Wording

cc176e8

Rework CIP

2921112

- Clear separation between additive and replacing semantics - Additive semantics for nesting with {} - Replacing semantics for flat composition - Use THEN for discard cardinality - Use WITH|RETURN|YIELD NOTHING for discard fields

boggle force-pushed the CIP-nested-subqueries branch 2 times, most recently from 746478d to 061b2fd Compare October 16, 2017 15:30

Clarify precedence rules

3ed1ca9

boggle force-pushed the CIP-nested-subqueries branch from 061b2fd to 3ed1ca9 Compare October 16, 2017 15:31

boggle changed the title ~~Added the nested subqueries CIP~~ Added the Nested, updating, and chained subqueries CIP Oct 16, 2017

boggle and others added 3 commits October 16, 2017 21:55

Add ammending nested subqueries and fix query combinator precedence

2d2435f

Fix definition of chained queries and move to right directory

0156bc3

Textual edits

acfac59

boggle changed the title ~~Added the Nested, updating, and chained subqueries CIP~~ Added the nested, updating, and chained subqueries CIP Oct 17, 2017

boggle and others added 7 commits October 17, 2017 15:22

Clarified query combinator semantics

7554cc9

Fixed erroneous queries

10fa182

Reformatted title

1ca70bf

Reworking/incorporating alternative CIP

cfc2a43

Fused with nested subqueries CIP from multigraph work

6199430

Added stand-alone nested calls and some clarifications/fix-ups

5b6a333

Grammar fix

077fb18

boggle added enhancement NOT READY FOR MERGE oCIG cypher10 This work targets Cypher 10 labels May 8, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added the nested, updating, and chained subqueries CIP #100

Added the nested, updating, and chained subqueries CIP #100

petraselmer commented Jun 22, 2016 •

edited by boggle

aseemk commented Jun 23, 2016

aseemk commented Jun 23, 2016

cleishm commented Jun 23, 2016

boggle commented Jun 24, 2016

zazi commented Jun 27, 2016

thobe commented Jul 12, 2016

petraselmer commented Jul 20, 2016

petraselmer commented Jul 20, 2016

zazi commented Jul 24, 2016

zazi Jul 24, 2016

zazi commented Jul 24, 2016

cleishm Jul 25, 2016

zazi commented Jul 26, 2016

Mats-SX commented Jul 28, 2016 •

edited

boggle commented Sep 26, 2016 •

edited

Tin-Nguyen commented Sep 30, 2016

ric81 commented Dec 13, 2016

Mats-SX commented Dec 14, 2016

thobe Mar 30, 2017

boggle Apr 13, 2017

boggle Apr 13, 2017

Mats-SX commented Apr 24, 2017


		Owing to the bottom-up nature of SPARQL query evaluation, the subqueries are evaluated logically first, and the results are projected up to the outer query.

		Only variables projected out of the subquery will be visible, or in scope, to the outer query.


		The inner mandatory query is any inner match query.

		Moreover, any inner match query from which the trailing final `RETURN` clause has been omitted may also be used as an inner mandatory query.

Added the nested, updating, and chained subqueries CIP #100

Are you sure you want to change the base?

Added the nested, updating, and chained subqueries CIP #100

Conversation

petraselmer commented Jun 22, 2016 • edited by boggle

aseemk commented Jun 23, 2016

aseemk commented Jun 23, 2016

cleishm commented Jun 23, 2016

boggle commented Jun 24, 2016

zazi commented Jun 27, 2016

thobe commented Jul 12, 2016

petraselmer commented Jul 20, 2016

petraselmer commented Jul 20, 2016

zazi commented Jul 24, 2016

zazi Jul 24, 2016

Choose a reason for hiding this comment

zazi commented Jul 24, 2016

cleishm Jul 25, 2016

Choose a reason for hiding this comment

zazi commented Jul 26, 2016

Mats-SX commented Jul 28, 2016 • edited

boggle commented Sep 26, 2016 • edited

Tin-Nguyen commented Sep 30, 2016

ric81 commented Dec 13, 2016

Mats-SX commented Dec 14, 2016

thobe Mar 30, 2017

Choose a reason for hiding this comment

boggle Apr 13, 2017

Choose a reason for hiding this comment

boggle Apr 13, 2017

Choose a reason for hiding this comment

Mats-SX commented Apr 24, 2017

petraselmer commented Jun 22, 2016 •

edited by boggle

Mats-SX commented Jul 28, 2016 •

edited

boggle commented Sep 26, 2016 •

edited