Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pass Context around Folder upload functions, add PG WAL push timeout #1546

Merged
merged 12 commits into from
Sep 11, 2023

Conversation

ne2pit
Copy link
Contributor

@ne2pit ne2pit commented Sep 4, 2023

Database name

Timeout is added to Postgres WAL push. Changes in Upload is common for all DBs.

Pull request description

  • Folder interface is extended with PutObjectWithContext method for gradual migration.
  • Folder implementations supports cancelation by context for new method.
  • Context passing added to some Upload logic.
  • Timeout is added to PG WAL Push handle.

Side changes

  • github.com/ncw/swift bumped to v2 major.
  • Swift storage integration docker test
  • SSH storage integration docker test

Pavel Tolstikov added 2 commits September 5, 2023 02:21
# Conflicts:
#	cmd/common/st/put_object.go
#	internal/storagetools/put_object_handler.go
internal/databases/postgres/bguploader.go Outdated Show resolved Hide resolved
pkg/storages/swift/folder.go Outdated Show resolved Hide resolved
pkg/storages/memory/mock/folder.go Outdated Show resolved Hide resolved
@usernamedt
Copy link
Member

I also suggest to add the new setting to the WAL-G daemon docs.

@ne2pit
Copy link
Contributor Author

ne2pit commented Sep 11, 2023

todo: Add Swift and SSH docker tests to make pg_integration_test. Something like:

	@if [ "$(TEST)" = "pg_tests" ]; then\
		docker-compose build ssh &&\
		docker-compose up --exit-code-from pg_storage_swift_test pg_storage_swift_test &&\
		docker-compose up --exit-code-from pg_storage_ssh_test pg_storage_ssh_test ;\
	fi

Similar changes are made in #1551 . Doing so in parallel will lead to merge conflicts. It is possible to add it to one of two PRs that will be merged second.

@usernamedt usernamedt merged commit 5805948 into wal-g:master Sep 11, 2023
72 checks passed
@ne2pit ne2pit deleted the upload_timeout branch September 12, 2023 10:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants