Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: Upgrade Bigtable to V2 #7317

Merged
merged 38 commits into from
Jun 3, 2024
Merged

feat!: Upgrade Bigtable to V2 #7317

merged 38 commits into from
Jun 3, 2024

Conversation

saranshdhingra
Copy link
Contributor

@saranshdhingra saranshdhingra commented May 16, 2024

This PR takes care of the following:

BREAKING_CHANGE_REASON=RC-1 for the version 2 of the Bigtable library.

@bshaffer
Copy link
Contributor

For the --prefer-lowest, you need to update Bigtable's minimum google/cloud-core version to one that contains the DetectProjectIdTrait:

PHP Fatal error:  Trait "Google\Cloud\Core\DetectProjectIdTrait" not found in /home/runner/work/google-cloud-php/google-cloud-php/Bigtable/src/BigtableClient.php on line 43
.................Running Bigtable Snippet Tests
PHP Fatal error:  Trait "Google\Cloud\Core\ApiHelperTrait" not found in /home/runner/work/google-cloud-php/google-cloud-php/Bigtable/src/Table.php on line 53

Copy link
Contributor

@bshaffer bshaffer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the failing tests:

See the GAPIC generator: the logic that adds the emulator environment variable is now handed by GAPIC
you don't need to call modifyClientOptions anymore, you just need to add the spanner client to the EmulatorSupportGenerator, and you can remove SpannerEmulatorTrait.

To fix the unit tests, just add the same logic to bypass finals to the bootstrap file in Core/unit-bootstrap.php, which is the bootstrap file used in the root phpunit.xml.dist

@saranshdhingra
Copy link
Contributor Author

googleapis/gapic-generator-php#717 is needed for emulator tests to pass.

@saranshdhingra saranshdhingra marked this pull request as ready for review May 23, 2024 11:51
@saranshdhingra saranshdhingra requested a review from a team as a code owner May 23, 2024 11:51
@saranshdhingra saranshdhingra requested review from a team as code owners May 23, 2024 11:51
Copy link
Contributor

@bshaffer bshaffer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to take a look at Bigtable/owlbot.py because some very strange things are happening there. I am hoping that the following likes can be removed entirely:

# fix unit test namespace
s.replace(
'tests/Unit/Admin/*/*.php',
r'namespace Google\\Cloud\\Bigtable\\Admin\\Tests\\Unit',
r'namespace Google\\Cloud\\Bigtable\\Tests\\Unit\\Admin')
# fix test group
s.replace(
'tests/**/Admin/**/*Test.php',
'@group admin',
'@group bigtable' + '\n' + ' * bigtable-admin')
# Fix class references in gapic samples
for version in ['V2']:
pathExpr = [
'src/' + version + '/Gapic/BigtableGapicClient.php',
'src/Admin/' + version + '/Gapic/*GapicClient.php'
]
types = {
'new BigtableClient': r'new Google\\Cloud\\Bigtable\\' + version + r'\\BigtableClient',
'new BigtableInstanceAdminClient': r'new Google\\Cloud\\Bigtable\\Admin\\' + version + r'\\BigtableInstanceAdminClient',
r'\$instance = new Instance': r'$instance = new Google\\Cloud\\Bigtable\\Admin\\' + version + r'\\Instance',
'= Type::': r'= Google\\Cloud\\Bigtable\\Admin\\' + version + r'\\Instance\\Type::',
'new FieldMask': r'new Google\\Protobuf\\FieldMask',
r'\$cluster = new Cluster': r'$cluster = new Google\\Cloud\\Bigtable\\Admin\\' + version + r'\\Cluster',
'new AppProfile': r'new Google\\Cloud\\Bigtable\\Admin\\' + version + r'\\AppProfile',
'new Policy': r'new Google\\Cloud\\Iam\\V1\\Policy',
'new BigtableTableAdminClient': r'new Google\\Cloud\\Bigtable\\Admin\\' + version + r'\\BigtableTableAdminClient',
'new Table': r'new Google\\Cloud\\Bigtable\\Admin\\' + version + r'\\Table',
}
for search, replace in types.items():
s.replace(
pathExpr,
search,
replace
)
### [START] protoc backwards compatibility fixes
# roll back to private properties.
s.replace(
"src/**/V*/**/*.php",
r"Generated from protobuf field ([^\n]{0,})\n\s{5}\*/\n\s{4}protected \$",
r"""Generated from protobuf field \1
*/
private $""")
# Replace "Unwrapped" with "Value" for method names.
s.replace(
"src/**/V*/**/*.php",
r"public function ([s|g]\w{3,})Unwrapped",
r"public function \1Value"
)
### [END] protoc backwards compatibility fixes

Also, we'll want to make sure googleapis/gapic-generator-php#714 is merged and generated for the Admin client, as it contains LRO's (cc @noahdietz)

Bigtable/tests/Snippet/bootstrap.php Outdated Show resolved Hide resolved
Bigtable/MIGRATING.md Show resolved Hide resolved
Bigtable/src/ResumableStream.php Outdated Show resolved Hide resolved
Bigtable/src/ResumableStream.php Outdated Show resolved Hide resolved
Bigtable/src/ResumableStream.php Outdated Show resolved Hide resolved
Bigtable/src/ResumableStream.php Outdated Show resolved Hide resolved
Comment on lines +150 to +153
$this->bigtableClient->mutateRows(
Argument::type(MutateRowsRequest::class),
Argument::type('array')
)->shouldBeCalled()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to check if the OptionalConfiguration also passed correctly?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So it seems like we should change this to

Suggested change
$this->bigtableClient->mutateRows(
Argument::type(MutateRowsRequest::class),
Argument::type('array')
)->shouldBeCalled()
$this->bigtableClient->mutateRows(
Argument::type(MutateRowsRequest::class),
$this->options + $options
)->shouldBeCalled()

This will test that the custom options we expect will show up.

I see that you've changed it from key1 => value1 to transportOptions, to make sure they're valid call options. That works for me!

@bshaffer
Copy link
Contributor

bshaffer commented May 29, 2024

After running the tests in php-docs-samples/bigtable against this PR, I received the following exceptions:

Next Google\ApiCore\ValidationException: Error decoding message: cannot handle unknown field row_filter on message google.bigtable.v2.ReadRowsRequest in vendor/google/gax/src/Serializer.php:149
Stack trace:
#0 vendor/google/cloud/Bigtable/src/Table.php(332): Google\ApiCore\Serializer->decodeMessage(Object(Google\Cloud\Bigtable\V2\ReadRowsRequest), Array)
#1 vendor/google/cloud/Bigtable/src/Table.php(358): Google\Cloud\Bigtable\Table->readRows(Array)
#2 src/hello_world.php(130): Google\Cloud\Bigtable\Table->readRow('greeting0', Array)
#3 {main}
  thrown in vendor/google/gax/src/Serializer.php on line 149

This takes place in the test bigtableTest::testHelloWorld

Fixed in d9c4fd0

Bigtable/composer.json Outdated Show resolved Hide resolved
@bshaffer bshaffer enabled auto-merge (squash) June 3, 2024 22:49
@bshaffer bshaffer merged commit 446c61f into main Jun 3, 2024
25 checks passed
@bshaffer bshaffer deleted the bigtable-v2 branch June 3, 2024 23:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants