/
still.todo
89 lines (65 loc) · 2.34 KB
/
still.todo
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
# summary
* [ ] Summarize fields, infer data types
# schema
* [ ] Version
* [ ] Comments
* [ ] @rules (e.g. @ignoreCase)
* [ ] Configure functions to allow for explicit column references
* [ ] Figure out how to handle NA columns @ignoreNA ?
* [ ] Optional directive
# directives
* [ ] @explicit
* [ ] @ordered
# @vectorops
-> Enable vector operations and functions?
-> Loads entire dataset and passes vectors
* [ ] cum_mean
* [ ] last
# @bloom_uniq
# input
* [x] excel
* [x] gzipped files
# output
* [ ] Filter invalid records?
* [ ] JSON with invalids
* [ ] HTML with invalids
* [ ] Text file with invalids
# Data dictionary
* [ ] Sumarize fields and infer data types and output
* [ ] Output formats
* [ ] HTML
* [ ] Markdown
* [ ] TSV
* [ ] Cardinality
# config
* [ ] Raise errors on malformed configuration
# CLI
* [ ] Enable multi-file running (still validate schema csv1 csv2 csv3...)
* [ ] ansi-style log (e.g. nextflow?)
# Performance
* [ ] Pre-process evaluation of rules
* [ ] Parallelize rules
# Functions
* [ ] VersionDecl ::= ("version 1.0" | "version 1.1")
* [ ] QuotedDirective ::= DirectivePrefix "quoted"
* [ ] TotalColumnsDirective ::= DirectivePrefix "totalColumns" PositiveNonZeroIntegerLiteral
* [ ] PermitEmptyDirective ::= DirectivePrefix "permitEmpty"
* [ ] NoHeaderDirective ::= DirectivePrefix "noHeader"
* [ ] IgnoreColumnNameCaseDirective ::= DirectivePrefix "ignoreColumnNameCase"
* [x] Comment ::= SingleLineComment | MultiLineComment
* [ ] SingleLineComment ::= //[\S\t ]* /* xgc:regular-expression */
* [ ] MultiLineComment ::= \/\*(?:[^*\r\n]+|(?:\r?\n))*\*\/ /* xgc:regular-expression */
* [ ] QuotedColumnIdentifier ::= StringLiteral
* [x] ColumnRule ::= ColumnValidationExpr* ColumnDirectives
* [ ] ColumnDirectives ::= OptionalDirective? MatchIsFalseDirective? IgnoreCaseDirective? WarningDirective? /* xgc:unordered */
* [ ] OptionalDirective ::= DirectivePrefix "optional"
* [ ] MatchIsFalseDirective ::= DirectivePrefix "matchIsFalse"
* [ ] IgnoreCaseDirective ::= DirectivePrefix "ignoreCase"
* [ ] WarningDirective ::= DirectivePrefix "warningDirective"
* [ ] ExplicitContextExpr ::= ColumnRef "/"
* [ ] NumericOrAny ::= NumericLiteral | WildcardLiteral
* [ ] PositiveIntegerOrAny ::= PositiveIntegerLiteral | WildcardLiteral
* [ ] EmptyExpr ::= "empty"
* [ ] NotEmptyExpr ::= "notEmpty"
# Misc.
* [ ] Logo