Release/2.8.2 #2
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* CBG-758: Reliably flush to console on Fatalf * Switch waitgroup to pointer * Fix typo
* CBG-724: Deletions of empty documents are not imported * CBG-724: Deletions of empty documents are not imported When the document is a delete, the bodyBytes would be replaced with “{}” and we can’t use bodyBytes in CRC-32 checksum comparison to determine whether the mutation was an SG write and the document has already been imported. Need to set the current value of CRC-32 checksum from bodyBytes when the document is non-delete and revert back to the correct CRC-32 checksum “0x00” before we replace the bodyBytes otherwise (if the document is delete). * CBG-724: Deletions of empty documents are not imported - Added additional unit tests (TestUpdateXattrWithDeleteBodyAndIsDelete, TestUpdateXattrWithDeleteBodyAndIsNotDelete) in bucket_gocb_test.go to validate the new behaviour of UpdateXattr. - Refactored XattrMacroCrc32cDelete to DeleteCrc32c and added short comment describes what it represents. - Removed obsolete comments left over from working session; added a brief description about the situation and need to modify currentBodyCrc32c in the delete case.
This speeds up full integration test runs significantly, by moving the
bucket setup/teardown into an async worker which gets buckets ready in
the background while tests are running using a different bucket.
## Usage:
```
export SG_TEST_BACKING_STORE=Couchbase
export SG_TEST_COUCHBASE_SERVER_URL=couchbase://localhost
go test -v -p=1 -count=1 ./...
```
### Configuration
#### Existing
- `SG_TEST_BACKING_STORE` (default `Walrus`)
- Can be set to `Couchbase` to enable integration tests
- `SG_TEST_COUCHBASE_SERVER_URL` (default `http://localhost:8091`)
- The connection string to connect with Couchbase Server
#### New
- `SG_TEST_USERNAME` (default `"Administrator"`)
- The username to use when connecting to the test server
- `SG_TEST_PASSWORD` (default `"password"`)
- The password to use when connecting to the test server
- `SG_TEST_BUCKET_POOL_SIZE` (default `3`)
- Specify how many buckets to create and use.
- `SG_TEST_BUCKET_QUOTA_MB` (default `150`)
- Specify how many MB each bucket's RAM quota should be
- `SG_TEST_BACKING_STORE_RECREATE` (default `false`)
- Drops any existing test buckets, and creates new ones
before continuing to add them to the pool. If this is
not set, previous buckets are just readied as normal.
- `SG_TEST_BUCKET_POOL_PRESERVE` (default `false`)
- In the event of a failing test, the bucket used is not emptied
and put back into the pool. This may result in exhaustion of
buckets if more tests fail than the pool size.
- `SG_TEST_BUCKET_POOL_DEBUG` (default `false`)
- If "true", verbose bucket pool logging is turned on,
to see what is happening to each bucket whilst running.
- `SG_TEST_DISABLE_GSI`
- Always set to true until CBG-818
---
* CBG-625 - Add bucket pooling test framework.
* Simplify LoggingBucket log code, and include bucket name in log context
* Prevent test failures when using walrus buckets
* Check for TestBucket pointer in AsGoCBBucket
* Fix WaitGroup data race by incrementing counter before sending bucket over bucketReadierQueue
* Prevent data race with multiple concurrent cluster.Authenticate() calls by using a pre-authed cluster reference
* Fix data race around initialisation of bucketReadierWaitGroup
* Add getTestKeyNamespace to force unique doc IDs for xattr tests (avoids KV tombstone issues)
* Increase timeout on retry loop for WaitForNumDocsViaChanges
* Bump gocb/gocbcore for cluster ops pre-bucket open
* Add SG_TEST_USE_VIEWS flag, but force it to be true all the time until 6.5.1
* Fix TestQuerySequencesStatsN1ql bucket type/view check
* Use TestsUseViews in RestTester setup
* Avoid vet catching unreachable code
* Use TestsUseViews for ServerContext test
* rename TestsUseViews to TestsDisableGSI and use in PostUpgrade
* Handle View and Query stats in TestTombstoneCompaction
* Bump bucket pool size from 100MB to 150MB
* Parameterise bucket quota
* Improve comments, rename things to be TBP scoped, rearrange to tidy
* More cleanup
* Tweak values in TestAddRawTimeoutRetry to avoid exceeding 150MB bucket quota, whilst still reproducing issue
* Add retry loop around UpsertDesignDocument to handle sporadic Erlang 500 errors
* Bump sg-bucket/walrus revisions
* Handle non-nil empty error from gocb's GetBuckets for 401 status code
* Add SG_TEST_USERNAME/SG_TEST_PASSWORD env vars
* Add benchmark for GetCallersName
* Skip printStats if no buckets opened (e.g. benchmarks)
* Remove unused GetTestBucketSpec
* Remove alignment chars from test logging
* Add comments for unused functions
* Drop default test pool size to 3
* Use t.Name() from getTestKeyNamespace
* Simplify document keys in single-doc tests
* Remove unnesesary query result Close
* Clean up TestBucketPool naming/API
* Move primary index creation out into test-only code
* Make mockClaims expire far into the future
* Fix missed InitializeIndex paramter removal
* Refer to global TestBucketPool from db TestMain
* Unskip TestPostUpgradeIndexesSimple
Switch to compareAndSwap when toggling compactRunning on channelCacheImpl
The change to include the active-only criteria in the N1QL query, instead of post-cache retrieval filtering, resulted in invalid cache contents. Active-only query results should not be cached.
…n non-default (#4551) * CBG-801: Auto-generated OIDC callback URL should include provider when non-default * CBG-801: Added changes to handle updating the provider value * CBG-801: Updated the error code returned by failures to set parameters to URL query string * CBG-801: Added changes in getOIDCCallbackURL to use SetURLQueryParam while building the callback URL * CBG-801: Changes track URL parsing errors and test case for provider name that would need URL encoding. - Added test case for a provider name that would need URL encoding. - Added changes to track underlying error * Fixed issue in adding provider to callback URL and included Unit tests * CBG-801: Handle error when writing mocked response body for discovery endpoints in TestGetOIDCCallbackURL * CBG-801: Revised getOIDCCallbackURL to remove nesting if clauses while building the callback URL.
Adds query pagination for channel queries triggered by changes processing. Default query limit for pagination is 5000, and can be modified with a new ‘query_limit’ config property in channel cache config. Pagination is done while building the per-channel feed (go-channel). This avoids storing the entire cross-query resultset in memory, allows streaming of results as soon as the first query completes, and optimizes interaction with any changes request limit. This has implications for cache warming - only the last (query_limit) entries will be prepended to the cache per changes request.
* Make GetTestKeyNamespace public * Remove getTestKeyNamespace(t) in favour of t.Name()
* PrometheusRules: Added Prometheus directory * Address PR comments * Comment out SGR1 row * Remove extra space
Sort recent sequences prior to updating syncData.
* CBG-1085 Re-register node when not present on cluster update When a SG node receives a cluster cfg update that doesn't it's own node UUID (e.g. it's been removed by another node due to heartbeat expiry), it should re-register itself to the node set. * Simplify localNodePresent calculation
If an import node is temporarily removed from the set of cbgt node definitions (e.g. due to temporary dropped heartbeats), have nodes re-register themselves.
* CBG-1127 Don't treat imported SDK resurrections as conflicts When a tombstoned document is resurrected via the SDK, the metadata history is lost and the document is resurrected as a new revision on a disconnected branch. When replicated via sg-replicate, this revision shouldn't be treated as being in conflict with a tombstoned document (no active leaves). Concurrent updates and deletes to the same revision continue to be treated as conflicts, as this doesn't meet the 'disconnected branch' criteria. * Port test fixes from master * Avoid timing issues for TestActiveReplicatorPushAndPullConflict
…ding blipsync websocket request (#4841)
…iggered by change for existing channels (#4910)
…ption (#4909) * [2.8.1 Backport] Fix empty webhook payload logging due to buffer-reuse (#4891) * [2.8.1 Backport] Use standard config schema validation for event handler config validation (#4898) * [2.8.1 Backport] Add winning_rev_only option to DocumentChanged event handler config (#4901) * Add String method to EventType for better logging * Add winning_rev_changed event handler * Rework winning rev only option for DocumentChanged event to use generic options supplied to all event handlers/webhooks. * Add missing params in event/webhook functions * Remove old event type * Address PR comments
…after cfgEventCallback (#4922)
…rom non-admin requests (#4935) * [2.8.1 Backport] CBG-759 - Add option to hide product version info from non-admin requests (#4928) * CBG-759 - Add option to hide product info from non-admin requests * Scope placeholders in version.go to prevent accidental usage * Have default ProductName const (for use in testing) * Omit product name from Basic auth realm when hide_product_info=true * Hide product version only * Fix test assertions for last change * Address PR comments (add rootResponse struct and build wwwAuthenticateHeader ahead of time) * Fix unused import from cherry-pick
…pty documents (#4884) (#4937) * CBG-1200: Tombstones v2 * Possibly handle pre-6.6 * Alter returned cas values * Additional Test * Added manual test * Remove debug * Test fix * Missed skip on manual test * Address PR comments and import handling * Remove commented out block * Additional cleanup * Change if requirements * Bump manifest * Assert specific error in tests
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Updating multicookie branch from commit 80946a2 to (2.8.2) in mainline github.com/couchbase/sync_gateway