Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add charset converter to convert non utf-8 char #1705

Merged
merged 1 commit into from
May 12, 2024

Conversation

anisurrahman75
Copy link
Contributor

Database name

MS SQL-Server

Pull request description

Linux OS:

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_DESCRIPTION="Ubuntu 22.04.4 LTS"
VERSION="22.04.4 LTS (Jammy Jellyfish)"

SQL Server Version: 2022-CU12-ubuntu-22.04

Error:

> ERROR: 2024/05/10 16:11:34.598412 proxy: failed to read blocklist xml: EOF

During the backup process, SQL Server always provides data encoded in UTF-8(Tested for Linux: 2022-CU12-ubuntu-22.04 and Windows: SQL Server 2022).
However, inside the wal-g codebase, we always assume that the data is encoded in UTF-16. As a result, during the decoding process, we encounter an EOF error.
So, I added a decoder which convert any non UTF-16 character to UTF-8.

Please provide steps to reproduce (if it's a bug)

Just try to back up SQL-Server Latest by using Wal-g

Please add config and wal-g stdout/stderr logs for debug purpose

2024/05/10 16:11:34 WARN: You specified both instance name and port in the connection string, port will be used and instance name will be ignored
INFO: 2024/05/10 16:11:34.190430 running proxy at backup.local:443
INFO: 2024/05/10 16:11:34.436030 database [demo] size is 6478336, required blob count 1
INFO: 2024/05/10 16:11:34.436046 starting backup database [demo] to URL = 'https://backup.local/basebackups_005/base_20240510T101134Z/demo/blob_000'
ERROR: 2024/05/10 16:11:34.598412 proxy: failed to read blocklist xml: EOF
ERROR: 2024/05/10 16:11:34.598761 proxy: failed to read blocklist xml: EOF
ERROR: 2024/05/10 16:11:34.599061 proxy: failed to read blocklist xml: EOF
ERROR: 2024/05/10 16:11:34.599358 proxy: failed to read blocklist xml: EOF
ERROR: 2024/05/10 16:11:36.497753 proxy: failed to read blocklist xml: EOF
ERROR: 2024/05/10 16:11:38.498794 proxy: failed to read blocklist xml: EOF
^X^CINFO: 2024/05/10 16:11:40.777150 Received interrupt signal. Shutting down
INFO: 2024/05/10 16:11:40.777175 stopping proxy
^C^Zfish: Job 1, 'sudo wal-g backup-push --config…' has stopped

Signed-off-by: Anisur Rahman <sunny.cse7575@gmail.com>
Copy link
Contributor

@mialinx mialinx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mialinx mialinx merged commit 9bff846 into wal-g:master May 12, 2024
81 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants