You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Might want to add an optional flag to enable gzip, and write plaintext ndjson files otherwise (as we do now)
get_open_file_for_writing actually already supports this if you override the suffix argument with .ndjson.gz rather than the default .ndjson. We just need to have parse_and_write_files accept a gzip boolean flag from the main parse function, or just hardcode it to use that suffix.
The text was updated successfully, but these errors were encountered:
Might want to add an optional flag to enable gzip, and write plaintext ndjson files otherwise (as we do now)
get_open_file_for_writing
actually already supports this if you override thesuffix
argument with.ndjson.gz
rather than the default.ndjson
. We just need to haveparse_and_write_files
accept agzip
boolean flag from the main parse function, or just hardcode it to use that suffix.The text was updated successfully, but these errors were encountered: