Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to preserve Unicode #168

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

joyfullservice
Copy link

In many cases we may want to preserve the Unicode text in our JSON content. This adds a non-default option to preserve the Unicode content.

In many cases we may want to preserve the Unicode text in our JSON content. This adds a non-default option to preserve the Unicode content.
@sabatale
Copy link

sabatale commented Feb 7, 2021

Not working for me.

For example: l'\u00C9glise instead of l'Église.

@joyfullservice
Copy link
Author

Not working for me.

For example: l'\u00C9glise instead of l'Église.

Did you set the option to allow Unicode characters? AllowUnicodeChars = True

@ddez
Copy link

ddez commented May 17, 2022

Works perfectly well on my end,
Thanks for the change, I support the pull request!

Indeed, before using the ConvertToJson function, remember to set :
JsonConverter.JsonOptions.AllowUnicodeChars = True

@joyfullservice
Copy link
Author

Yes, it has worked great in my projects for the last two years. I am hopeful that @timhall will consider merging this PR for the benefit of those that are working with UTF-8 content. (Which is standard practice in most software development and integrations, from my experience.)

@IlBaroneJaja
Copy link

It worked like a charm for me as well if we call it with the new option: JsonConverter.JsonOptions.AllowUnicodeChars = True. Thank you ! Upvote for this PR!

@SoccerDitter
Copy link

maybe this helps someone else, although the repository is already old and does not get an update anymore

the change as suggested by @joyfullservice worked for me.
1 - change in the jsonconverter, inserted by @joyfullservice.
2 - set in own code, JsonConverter.JsonOptions.AllowUnicodeChars = True.
3 - in addition, because my httpRequest is executed with ("WinHttp.WinHttpRequest.5.1"), I added set to:
httpRequest.Option(WinHttpRequestOption_UrlEncodeAmps) = True.
4.- then:
httpRequest.ResponseBody edited with the function
Function

BytesToUTF8String(bytes() As Byte) As String
                'converter Utf8
    Dim stream As Object
    Set stream = CreateObject("ADODB.Stream")
    
    stream.Type = 1 ' adTypeBinary
    stream.Open
    stream.Write bytes
    
    stream.Position = 0
    stream.Type = 2 ' adTypeText
    stream.Charset = "UTF-8" '
    
    BytesToUTF8String = stream.ReadText(-1) ' Read all text
    
    stream.Close
    Set stream = Nothing
End Function

When converting JSON back to a string, Line Feed characters should be converted back to line feed characters, not Carriage Return & Line Feed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants