Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(spanner/spansql): fix DATE and TIMESTAMP parsing. #4480

Merged
merged 4 commits into from Jul 23, 2021
Merged

fix(spanner/spansql): fix DATE and TIMESTAMP parsing. #4480

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f10b604
fix(spanner/spansql): fix DATE, TIMESTAMP parse.
Jul 22, 2021
0cb3b33
fix(spanner/spansql): fix allFuncs index, and add more Date, Timestamp
Jul 23, 2021
007bcf2
fix(spanner/spansql): fix parse DATE, Timestamp literals.
Jul 23, 2021
4586857
Merge branch 'master' into fix/spansql-date-ts-parse
Jul 23, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions spanner/spansql/keywords.go
View file
Open in desktop
Expand Up @@ -158,6 +158,7 @@ var allFuncs = []string{
"BYTE_LENGTH", "CHAR_LENGTH", "CHARACTER_LENGTH",
"CODE_POINTS_TO_BYTES", "CODE_POINTS_TO_STRING",
"CONCAT",
"DATE",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Do not add this to the list of string functions (see comment above the list)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
Fixed and added more date timestamp functions at 0cb3b33

"ENDS_WITH",
"FORMAT",
"FROM_BASE32", "FROM_BASE64", "FROM_HEX",
Expand Down Expand Up @@ -190,4 +191,5 @@ var allFuncs = []string{
"ARRAY_REVERSE",
"ARRAY_IS_DISTINCT",
"SAFE_OFFSET", "SAFE_ORDINAL",
"TIMESTAMP",
}
12 changes: 8 additions & 4 deletions spanner/spansql/parser.go
View file
Open in desktop
Expand Up @@ -2714,11 +2714,15 @@ func (p *parser) parseLit() (Expr, *parseError) {
p.back()
return p.parseArrayLit()
case tok.caseEqual("DATE"):
p.back()
return p.parseDateLit()
if !p.sniff("=") && !p.sniff(",") && !p.sniff("FROM") && !p.sniff("AS") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be implemented using a positive look-ahead instead of a negative-look ahead. In other words: If the DATE keyword is followed by a string literal, then it is a date literal. Otherwise it is not. The current implementation does not work for for example the following valid query:

SELECT UNIX_DATE(DATE "2008-12-25")

Also, adding additional exceptions to the current if-not list will not solve that permanently, as there might be changes to the query syntax that would add additional keywords (i.e. not only AS and FROM) that could follow an identifier.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreeed.

Fixed at 007bcf2

p.back()
return p.parseDateLit()
}
case tok.caseEqual("TIMESTAMP"):
p.back()
return p.parseTimestampLit()
if !p.sniff("=") && !p.sniff(",") && !p.sniff("FROM") && !p.sniff("AS") {
p.back()
return p.parseTimestampLit()
}
}

// TODO: struct literals
Expand Down
37 changes: 37 additions & 0 deletions spanner/spansql/parser_test.go
View file
Open in desktop
Expand Up @@ -83,6 +83,28 @@ func TestParseQuery(t *testing.T) {
},
},
},
{`SELECT date, timestamp as timestamp FROM Packages WHERE date = DATE '2014-09-27' AND timestamp = TIMESTAMP '2014-09-27 12:30:00'`,
Query{
Select: Select{
List: []Expr{ID("date"), ID("timestamp")},
From: []SelectFrom{SelectFromTable{Table: "Packages"}},
Where: LogicalOp{
Op: And,
LHS: ComparisonOp{
Op: Eq,
LHS: ID("date"),
RHS: DateLiteral{Year: 2014, Month: 9, Day: 27},
},
RHS: ComparisonOp{
Op: Eq,
LHS: ID("timestamp"),
RHS: TimestampLiteral(timef(t, "2006-01-02 15:04:05", "2014-09-27 12:30:00")),
},
},
ListAliases: []ID{"", "timestamp"},
},
},
},
{`SELECT SUM(PointsScored) AS total_points, FirstName, LastName AS surname FROM PlayerStats GROUP BY FirstName, LastName`,
Query{
Select: Select{
Expand Down Expand Up @@ -336,7 +358,14 @@ func TestParseExpr(t *testing.T) {

// Date and timestamp literals:
{`DATE '2014-09-27'`, DateLiteral(civil.Date{Year: 2014, Month: time.September, Day: 27})},
{`TIMESTAMP '2014-09-27 12:30:00'`, TimestampLiteral(timef(t, "2006-01-02 15:04:05", "2014-09-27 12:30:00"))},

// date and timestamp funclit
{`DATE('2014-09-27')`, Func{Name: "DATE", Args: []Expr{StringLiteral("2014-09-27")}}},
{`TIMESTAMP('2014-09-27 12:30:00')`, Func{Name: "TIMESTAMP", Args: []Expr{StringLiteral("2014-09-27 12:30:00")}}},
// date and timestamp identifier
{`DATE = '2014-09-27'`, ComparisonOp{LHS: ID("DATE"), Op: Eq, RHS: StringLiteral("2014-09-27")}},
{`TIMESTAMP = '2014-09-27 12:30:00'`, ComparisonOp{LHS: ID("TIMESTAMP"), Op: Eq, RHS: StringLiteral("2014-09-27 12:30:00")}},
// Array literals:
// https://cloud.google.com/spanner/docs/lexical#array_literals
{`[1, 2, 3]`, Array{IntegerLiteral(1), IntegerLiteral(2), IntegerLiteral(3)}},
Expand Down Expand Up @@ -796,3 +825,11 @@ func TestParseFailures(t *testing.T) {
}
}
}

func timef(t *testing.T, format, s string) time.Time {
ti, err := time.ParseInLocation(format, string(s), defaultLocation)
if err != nil {
t.Errorf("parsing %s [%s] time.ParseInLocation failed.", s, format)
}
return ti
}