New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(bigtable/cbt): cbt 'import' cmd to parse a .csv file and write to CBT #5072
Conversation
9d8f2d3
to
f767730
Compare
f767730
to
fe1b8b6
Compare
Looks like another linter error showed up.
Also, if you'd like, the script that is running is |
df22379
to
77b3f18
Compare
77b3f18
to
e70df18
Compare
bigtable/cmd/cbt/cbt_test.go
Outdated
if !tc.fail && err != nil { | ||
t.Errorf("parseCsvHeaders() failed. input:%+v, error:%s", tc, err) | ||
continue | ||
} | ||
if tc.fail && err == nil { | ||
t.Errorf("parseImportArgs() did not fail. input:%+v, error:%s", tc, err) | ||
continue | ||
} | ||
if tc.fail { | ||
continue |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: I think more typically the tests just catch this via an OR or two separate conditionals, collapsing this a bit? I may be missing nuance here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This case would allow continuation since the fail is expected, but we don't want to do any comparisons below. It can't be paired with any of the above because they add errors were as this is expected.
I can put the if err == nil {t.Error} inside the tc.fail block, just wasn't sure if go-style preferred the nested ifs.
if tc.fail {
if err == nil {
t.Errorf("parseImportArgs() did not fail. input:%+v, error:%s", tc, err)
}
continue
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this to me seems like we are trying to check if fail end err match. I don't see why they would have different messages. Also one of these has 'parseImportArgs' which I assume is leftover from a copy paste?
Are we in essence checking this
if tc.fail != (err != nil)
Alternatively, rather than storing tc.fail as a bool, store the err string to do a deeper check: possibly kind of like https://github.com/googleapis/google-cloud-go/blob/master/datastore/datastore_test.go#L1987
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
updated to error matching pattern.
…o CBT (#5072) Co-authored-by: Christopher Wilcox <crwilcox@google.com>
Add 'import' command to 'cbt' and unit/emulator tests
Runtime expectations:
2MB file with 12588 rows
190MB file in 4s @ 47.5MB/s when using 'batch-size=5' 'workers=20'