Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for XQuery Tumbling and Sliding Window #4529

Open
wants to merge 74 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
058c750
[test] Add Tumbling Window basic example
gabriele-tomassetti Jun 28, 2022
818e2ff
[feature] Add basic version of windowClause
gabriele-tomassetti Jun 28, 2022
b5b00fc
[feature] Add support for start condition for tumbling windowClause
gabriele-tomassetti Jun 29, 2022
3711e81
[feature] Add support for windowEndCondition
gabriele-tomassetti Jun 29, 2022
6834883
[feature] Add WindowExpr, WindowCondition to support WindowClause
gabriele-tomassetti Jul 4, 2022
cad39e4
[refactor] Improve parsing of Tumbling Window
gabriele-tomassetti Jul 4, 2022
d18e61d
[feature] Add handling of intermediate AST for Tumbling Window
gabriele-tomassetti Jul 4, 2022
c99ffea
[test] Improve testing for Tumbling Window
gabriele-tomassetti Jul 4, 2022
efb0e4e
[feature] Add support for Sliding Window Expression
gabriele-tomassetti Jul 6, 2022
fbbdbc4
[test] Add test for Sliding Window Expression
gabriele-tomassetti Jul 6, 2022
24fd165
[refactor] Rename to match XQuery spec
adamretter Jul 16, 2022
4c50f49
[refactor] Mark nullable class members
adamretter Jul 16, 2022
e2f8bdd
[refactor] Reorder WindowCondition members to match XQuery spec
adamretter Jul 16, 2022
f54c973
[refactor] Variables used by Binding Expressions should be EQName acc…
adamretter Jul 16, 2022
3bacea4
[refactor] Move conflict check between for-expr binding variable name…
adamretter Jul 16, 2022
21078d5
[refactor] Move conflict check between window binding variables names…
adamretter Jul 16, 2022
5179316
[feature] Implement Sliding Window Expressions
adamretter Jul 16, 2022
d786e13
[bugfix] Use the correct error code for a variable reference to an un…
adamretter Jul 19, 2022
d545743
[bugfix] Ensure the correct scope of the Window variables
adamretter Jul 19, 2022
4dae8c0
[bugfix] Fix bug in handling current item when positional var is present
gabriele-tomassetti Jul 20, 2022
a6bc38d
[test] Add more examples from reference
gabriele-tomassetti Jul 20, 2022
2dc9a2b
[refactor] Remove need for starting eXist-db server from test
adamretter Jul 20, 2022
87fedc4
[bugfix] Preserve the Error Code from the initial XQuery parser stage
adamretter Jul 20, 2022
ec5415d
[bugfix] Add missing ErrorCode on type checks
adamretter Jul 20, 2022
3af0d4c
[bugfix] Further type checks for Window Expr
adamretter Jul 20, 2022
f87e20b
[bugfix] 'declare variable' expressions should not remove the context…
adamretter Jul 23, 2022
91ea433
[bugfix] Correctly implement start-next-item and end-next-item for Wi…
adamretter Jul 23, 2022
8b079af
[bugfix] Fix window type checks
adamretter Jul 23, 2022
9be9ec2
[refactor] Simplify previous item handling
adamretter Jul 23, 2022
506e94a
[bugfix] Add missing error codes
adamretter Jul 23, 2022
62e4cd6
[bugfix] Enable sorting of windows
adamretter Jul 23, 2022
8d90fb8
[bugfix] Correctly use start condition when end condition is absent
adamretter Jul 25, 2022
dd83306
[feature] Sliding Window support
adamretter Jul 25, 2022
6dc74fc
[bugfix] Fix use of group by in Windows
adamretter Jul 25, 2022
eff6049
[bugfix] group by clause must atomize the group keys in the output tu…
adamretter Jul 25, 2022
91b810d
[bugfix] group by clause must adhere to type declarations on group va…
adamretter Jul 25, 2022
1ca3cf1
[bugfix] In the GroupingSpec of a Group By clause, the type declarati…
adamretter Jul 26, 2022
ccde1c2
[bugfix] Collator should be used when comparing node values in fn:dee…
adamretter Aug 7, 2022
be6fede
[refactor] Implement fn:deep-equal in terms of deepCompare (needed fo…
adamretter Aug 7, 2022
f50f1c7
[optimise] Only evaluate arguments to fn:codepoint-equal once
adamretter Aug 7, 2022
acf52bf
[bugfix] Correctly implement op:same-key#2 for XQuery Map types
adamretter Aug 7, 2022
9534cba
[bugfix] Fix comparison of INF and -INF values for xs:float and xs:do…
adamretter Aug 7, 2022
f23615f
[bugfix] Fix comparison of NaN values for xs:float and xs:double
adamretter Aug 7, 2022
0167c37
[bugfix] Group By keys must be compared using fn:deep-equal when grou…
adamretter Aug 7, 2022
ea94968
[bugfix] Reimplement fn:sort so that it complies with the W3C specifi…
adamretter Aug 9, 2022
29f5dde
[bugfix] Group By must preserved the ordering of the tuple stream
adamretter Aug 9, 2022
69b07df
[bugfix] Group By Clause cannot group by variables that do not exist …
adamretter Aug 15, 2022
ec7f1f1
[bugfix] DateTime values must consider timezone in Group By keys
adamretter Aug 23, 2022
a86c93b
[bugfix] Window expr must analyze the start and end window conditions
adamretter Aug 23, 2022
c855d8c
[bugfix] Correct Sliding Window positional calculation
adamretter Aug 26, 2022
acfa9ae
[ignore] cleanup
adamretter Aug 27, 2022
b5618c0
[bugfix] Fix issues with reserved words for sliding and tumbling windows
adamretter Aug 30, 2022
658d5a2
[bugfix] Use correct error code when order by clause has more than on…
adamretter Sep 3, 2022
c901c47
[bugfix] Use correct error code for unknown Collation (XQTS: prod-Ord…
adamretter Sep 3, 2022
38cfc21
[bugfix] Use correct Error Code when fn:exactly-one is called upon a …
adamretter Sep 3, 2022
ed14b96
[bugfix] xs:hexBinary must support both upper and lower-case [a-f] ch…
adamretter Sep 3, 2022
4d29903
[bugfix] Further fixes to comparison of INF and -INF values for xs:fl…
adamretter Sep 3, 2022
2fd8324
[bugfix] Relative Collation URIs must be resolved relative to the bas…
adamretter Sep 3, 2022
4aed1a2
[bugfix] Order By expression may need access to the Context Item (XQT…
adamretter Oct 10, 2022
46b115e
[refactor] Optimise for the positive path
adamretter Nov 6, 2022
b3b5569
[bugfix] Avoid an NPE when calling toString() after clear()
adamretter Nov 6, 2022
4889566
[bugfix] Fix toString representation of an OrderSpec
adamretter Nov 11, 2022
886be90
[optimise] Avoid unnecessary array allocation
adamretter Nov 11, 2022
853dceb
[optimise] Reduce collection calls by using Deque functions
adamretter Nov 11, 2022
21819a8
[optimise] Avoid over sizing the OrderedValueSequence
adamretter Nov 11, 2022
58247f5
[bugfix] Mixed primitive types must be coerced before Order By Clause…
adamretter Nov 9, 2022
d4e7494
[bugfix] Return the correct error code from XQuery Functions that can…
adamretter Jun 2, 2023
476410e
[bugfix] Need to bind the variables in the output tuple stream, now p…
adamretter Jun 6, 2023
dbeeba4
[bugfix] Correct a bad test - groupby:atomize-group-vars#0
adamretter Jun 8, 2023
7340308
[refactor] Address Codacy static analysis issues
adamretter Jun 9, 2023
5e7b332
[refactor] Address Code Review comments from @reinhapa
adamretter Jun 9, 2023
1c00798
[refactor] Address SonarCloud static analysis issues
adamretter Jun 9, 2023
49d16e5
[refactor] Address Code Review comments from @dizzzz
adamretter Jun 15, 2023
d118a3b
[refactor] Address further SonarCloud static analysis issues
adamretter Oct 29, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
84 changes: 76 additions & 8 deletions exist-core/src/main/antlr/org/exist/xquery/parser/XQuery.g
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ options {

public XQueryParser(XQueryLexer lexer) {
this((TokenStream)lexer);
this.lexer= lexer;
this.lexer = lexer;
setASTNodeClass("org.exist.xquery.parser.XQueryAST");
}

Expand All @@ -95,20 +95,27 @@ options {
}

public String getErrorMessage() {
StringBuilder buf= new StringBuilder();
for (Iterator i= exceptions.iterator(); i.hasNext();) {
StringBuilder buf = new StringBuilder();
for (Iterator i = exceptions.iterator(); i.hasNext();) {
buf.append(((Exception) i.next()).toString());
buf.append('\n');
}
return buf.toString();
}

public Exception getLastException() {
if (!foundError) {
return null;
}
return (Exception) exceptions.get(exceptions.size() - 1);
adamretter marked this conversation as resolved.
Show resolved Hide resolved
}

public String getXQDoc() {
return lexer.getXQDoc();
}

protected void handleException(Exception e) {
foundError= true;
foundError = true;
exceptions.add(e);
}
}
Expand Down Expand Up @@ -180,6 +187,11 @@ imaginaryTokenDefinitions
PRAGMA
GTEQ
SEQUENCE
TUMBLING_WINDOW
CURRENT_ITEM
PREVIOUS_ITEM
NEXT_ITEM
WINDOW_VARS
;

// === XPointer ===
Expand Down Expand Up @@ -690,7 +702,7 @@ expr throws XPathException

exprSingle throws XPathException
:
( ( "for" | "let" ) DOLLAR ) => flworExpr
( ( "for" | "let" ) ("tumbling" | "sliding" | DOLLAR ) ) => flworExpr
| ( "try" LCURLY ) => tryCatchExpr
| ( ( "some" | "every" ) DOLLAR ) => quantifiedExpr
| ( "if" LPAREN ) => ifExpr
Expand Down Expand Up @@ -797,7 +809,9 @@ flworExpr throws XPathException

initialClause throws XPathException
:
( forClause | letClause )
( ( "for" DOLLAR ) => forClause
| ( "for" ( "tumbling" | "sliding" ) ) => windowClause
| letClause )
;

intermediateClause throws XPathException
Expand All @@ -820,6 +834,11 @@ letClause throws XPathException
"let"^ letVarBinding ( COMMA! letVarBinding )*
;

windowClause throws XPathException
:
"for"! ("tumbling"|"sliding") "window"^ inVarBinding windowStartCondition ( windowEndCondition )?
;

inVarBinding throws XPathException
{ String varName; }
:
Expand All @@ -844,6 +863,37 @@ allowingEmpty
"allowing"! "empty"
;

windowStartCondition throws XPathException
:
"start"^ windowVars "when" exprSingle
;

windowEndCondition throws XPathException
:
( "only" )? "end"^ windowVars "when" exprSingle
;

windowVars throws XPathException
{ String currentItemName = null, positionalVarName = null, previousItemName = null, nextItemName = null; }
:
( DOLLAR! currentItemName=eqName! )?
( "at"! DOLLAR! positionalVarName=eqName! )?
( "previous"! DOLLAR! previousItemName=eqName! )?
( "next"! DOLLAR! nextItemName=eqName! )?
{
windowVars_AST = (org.exist.xquery.parser.XQueryAST)astFactory.create(WINDOW_VARS);
if (currentItemName != null)
windowVars_AST.addChild(astFactory.create(CURRENT_ITEM,currentItemName));
if (positionalVarName != null)
windowVars_AST.addChild(astFactory.create(POSITIONAL_VAR,positionalVarName));
if (previousItemName != null)
windowVars_AST.addChild(astFactory.create(PREVIOUS_ITEM,previousItemName));
if (nextItemName != null)
windowVars_AST.addChild(astFactory.create(NEXT_ITEM,nextItemName));
currentAST.root = (org.exist.xquery.parser.XQueryAST) windowVars_AST;
}
;

letVarBinding throws XPathException
{ String varName; }
:
Expand Down Expand Up @@ -876,7 +926,6 @@ orderModifier
groupByClause throws XPathException
:
"group"! "by"! groupingSpecList
// "group"! toGroupVarRef "as"! groupVarBinding "by"! groupSpecList
{ #groupByClause= #([GROUP_BY, "group by"], #groupByClause); }
;

Expand All @@ -888,7 +937,7 @@ groupingSpecList throws XPathException
groupingSpec throws XPathException
{ String groupKeyVarName; }
:
DOLLAR! groupKeyVarName=varName! ( COLON! EQ! exprSingle )? ( "collation" STRING_LITERAL )?
DOLLAR! groupKeyVarName=varName! ( ( typeDeclaration )? COLON! EQ! exprSingle )? ( "collation" STRING_LITERAL )?
{ #groupingSpec = #(#[VARIABLE_BINDING, groupKeyVarName], #groupingSpec); }
;

Expand Down Expand Up @@ -2228,8 +2277,27 @@ reservedKeywords returns [String name]
"empty-sequence" { name = "empty-sequence"; }
|
"schema-element" { name = "schema-element"; }
|
"tumbling" { name = "tumbling"; }
|
"sliding" { name = "sliding"; }
|
"window" { name = "window"; }
|
"start" { name = "start"; }
|
"end" { name = "end"; }
|
"only" { name = "only"; }
|
"previous" { name = "previous"; }
|
"next" { name = "next"; }
|
"when" { name = "when"; }
;


/**
* The XQuery/XPath lexical analyzer.
*/
Expand Down