Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor inline in gather to be callable by other flowcbs #919

Open
andreleblanc11 opened this issue Feb 2, 2024 · 1 comment
Open

Refactor inline in gather to be callable by other flowcbs #919

andreleblanc11 opened this issue Feb 2, 2024 · 1 comment
Assignees
Labels
efficiency improve performance and/or reduce overhead Refactor change implementation of existing functionality. wishlist would be nice, not pressing for any particular client.

Comments

@andreleblanc11
Copy link
Member

Currently, inline and inlineEncoding logic is built-in the file gather, flowcb/gather/file.py. It would be nice to have this logic be callable by other flowcbs.

This would be especially useful for the AM server. We currently have our own code in the AM server to determine the character encoding. It is very similar to what we already have in the file gather.

What would need to be done

  • Separate the inline* logic from post_file method in flowcb/gather/file.py. Would be good to place new method under __init__
  • For AM
    • Call this new method from the code.
@andreleblanc11 andreleblanc11 added efficiency improve performance and/or reduce overhead wishlist would be nice, not pressing for any particular client. Refactor change implementation of existing functionality. labels Feb 2, 2024
@petersilva
Copy link
Contributor

The file logic covers text which is always utf-8 and if it isn't then it's binary.
It doesn't permit iso-8859-1 encoding. The default value is "guess" . so when it a file in iso-8859-1 with french accents in it is received it will:

  • try to interpret it as utf-8, fail
  • encode it as base64 instead.

So the data will still be sent with fidelity, you just won't be able to read the data in the content (meaning the base64 encoding isn't human readable.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
efficiency improve performance and/or reduce overhead Refactor change implementation of existing functionality. wishlist would be nice, not pressing for any particular client.
Projects
None yet
Development

No branches or pull requests

2 participants