Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for byte[] to String/FixedString types #138

Open
MaceWindu opened this issue Jul 3, 2022 · 3 comments
Open

Add support for byte[] to String/FixedString types #138

MaceWindu opened this issue Jul 3, 2022 · 3 comments
Assignees
Labels
wontfix This will not be worked on

Comments

@MaceWindu
Copy link

ClickHouse String/FixedString types store binary data, but provider supports only UTF8 string representation.

It doesn't work well in some cases:

  • use of `byte[]' results in "System.Byte[]" value in database (here)
  • not all binary data represents valid utf8 sequence and doesn't survive conversion to string as UTF8 and back without data loss. E.g. 0xFF will be 0xEF 0xBF 0xBD after binary->string->binary conversion
@DarkWanderer
Copy link
Owner

DarkWanderer commented Jul 3, 2022

I understand the desire to use string values as storage for byte arrays, but it is not possible to resolve without introducing ambiguity which cannot be resolved at driver level. Specifically, even if I write the value as a byte array to database, I will not be able to read it back as it's not possible to understand whether the value is a valid string or a byte array in general case

Can you use Array(UInt8) in ClickHouse instead? See #110

@DarkWanderer DarkWanderer self-assigned this Jul 3, 2022
@DarkWanderer DarkWanderer added the wontfix This will not be worked on label Jul 3, 2022
@MaceWindu
Copy link
Author

MaceWindu commented Jul 4, 2022

Well, actually this is not an issue for me - I don't use ClickHouse.

I'm just implementing support for it in linq2db and report back issues, discovered during testing process.

Currently I'm working with 3 providers:

  • your provider
  • Octonica binary provider
  • mysql interface provider

and binary support present only in Octonica:

  • in data reader they allow to get raw data if you call GetFieldValue<byte[]> and utf-8 string for GetString call (your provider currently doesn't implement GetFieldValue)
  • in bulk copy they allow to specify used Type for column: string or byte[] (which I find unnecessary as both types should be already recognized as valid values, but that's not really important).
  • as for query parameters - I don't use them for ClickHouse, so I have nothing to add here

MySql interface also has issues with binary data reads, but this is due incorrect implementation on server side.

@DarkWanderer
Copy link
Owner

What is the scenario you use/plan to use this in linq2db for?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wontfix This will not be worked on
Projects
None yet
Development

No branches or pull requests

2 participants