Support for XQuery Tumbling and Sliding Window

[adamretter]

This implements both tumbling window and sliding window.

Along the way to make sure it passes the Window tests in the XQTS. We also had to fix a number of other things:

1. Several fn:deep-equal bug-fixes
2. Only evaluate arguments to fn:codepoint-equal once
3. Complete re-implementation of fn:sort
4. Almost a complete rewrite of group by expression implementation
5. Fix comparison of NaN values for xs:float and xs:double
6. Fix comparison of INF and -INF values for xs:float and xs:double
7. Fix and implement op:same-key#2 for XQuery Map types
8. Re-implement fn:deep-equal in terms of op:deepCompare
9. Several Error Code fixes
10. Several XDM Type fixes

By running exist-xqts-runner (from its compat/eXist-7.0.0-SNAPSHOT branch):

exist-xqts-runner-assembly-1.4.0-SNAPSHOT.jar --xqts-version HEAD --output-dir /tmp/xqts-output --test-case fn-codepoint-equal,fn-deep-equal,fn-sort,prod-GroupByClause,prod-WindowClause

We see the results:

The 7 remaining Window failures are all related to the count clause which still needs to be implemented first in a separate PR, see: #4530.

---

This open source contribution to the eXist-db project was commissioned by the Office of the Historian, U.S. Department of State, https://history.state.gov/.

[sonarcloud]

SonarCloud Quality Gate failed.   

10 Bugs
0 Vulnerabilities
0 Security Hotspots
56 Code Smells

66.5% Coverage
3.5% Duplication

[joewiz]

Very exciting! This works with all of the code from XQuery for Humanists in the sections on Windowing except for one, which apparently isn't tested in XQTS - and which I'll submit as a new test there. This query is:

let $sentence := "This is a a test of of windowing"
let $words := fn:tokenize($sentence, " ")
for tumbling window $window in $words
    start $ws at $wsn previous $wp next $wn 
        when $wp eq $ws
return
    <window ws="{$ws}" wsn="{$wsn}" wp="{$wp}" wn="{$wn}">{ $window }</window>

The results from this branch:

<window ws="of" wsn="7" wp="of" wn="windowing">a test of</window>
<window ws="windowing" wsn="8" wp="of" wn="">of windowing</window>

The expected results, as returned by BaseX and Saxon:

<window ws="a" wsn="4" wp="a" wn="test">a test of</window>
<window ws="of" wsn="7" wp="of" wn="windowing">of windowing</window>

(The window is selected correctly, but the variables bound by the start, previous, next, and at clauses are wrong.)

Also, there seems to be one parser issue. The query:

<x/>/@when

... returns the error:

error found while executing expression: org.exist.xquery.XPathException: err:XPST0003 unexpected token: when [at line 1, column 7]

From exist.log:

2022-08-30 08:53:17,523 [qtp1304614894-67] ERROR (XQueryServlet.java [process]:549) - Cannot compile xquery: err:XPST0003 org.exist.xquery.XPathException: err:XPST0003 unexpected token: when [at line 1, column 7]
 [at line 7, column 1] 
org.exist.EXistException: Cannot compile xquery: err:XPST0003 org.exist.xquery.XPathException: err:XPST0003 unexpected token: when [at line 1, column 7]
 [at line 7, column 1]
...
Caused by: org.exist.xquery.StaticXQueryException: err:XPST0003 org.exist.xquery.XPathException: err:XPST0003 unexpected token: when [at line 1, column 7]
 [at line 7, column 1]
	at org.exist.xquery.XQuery.compile(XQuery.java:235) ~[exist-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
	at org.exist.xquery.XQuery.compile(XQuery.java:180) ~[exist-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
	at org.exist.xquery.XQuery.compile(XQuery.java:138) ~[exist-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
	at org.exist.http.servlets.XQueryServlet.process(XQueryServlet.java:439) ~[exist-core-6.1.0-SNAPSHOT.jar:6.1.0-SNAPSHOT]
	... 66 more

[joewiz]

@adamretter I can confirm that with the last commit the <x/>/@when issue is no longer present. Thank you!

[sonarcloud]

SonarCloud Quality Gate failed.   

10 Bugs
0 Vulnerabilities
0 Security Hotspots
65 Code Smells

67.5% Coverage
3.5% Duplication

[duncdrum]

Nice change. This needs a rebase to resolve conflicts. It’s there content work still missing to no longer mark it as draft?

[adamretter]

@reinhapa each of the 5 bugs listed by SonarCloud are false positives. I fixed a few small code smell issues highlighted by SonarCloud. The remaining suggestions from SonarCloud and Codacy otherwise IMHO make the code worse!

[reinhapa]

@adamretter there is a compile error on the test source:

/D:/a/exist/exist/exist-core/src/main/java/org/exist/xquery/BasicExpressionVisitor.java:[34,8] org.exist.xquery.BasicExpressionVisitor is not abstract and does not override abstract method visitWindowExpression(org.exist.xquery.WindowExpr) in org.exist.xquery.ExpressionVisitor

[dizzzz]

compilation issue