Unrecognized fields when copying a field mask when trying to update a custom audience

[caspercrause]

Describe the bug:
Attempted to update a custom audience, had to copy a field mask with

fm = protobuf_helpers.field_mask(None,custom_audience._pb) # Field Mask
         
  custom_audience_operation.update_mask.CopyFrom(fm) # Update Field Mask

This created an invalid field called type_
When attempting to mutate:

custom_audience_response = custom_audience_service.mutate_custom_audiences(customer_id=customer_id, operations=Operation)

The following error occured:

WARNING:google.ads.googleads.client:Request made: ClientCustomerId: REDACTED, Host: googleads.googleapis.com, Method: /google.ads.googleads.v8.services.CustomAudienceService/MutateCustomAudiences, RequestId: tEtk-QUSW2PFBpQ7idVkTg, IsFault: True, FaultMessage: The field mask contained an invalid field: 'type_'.
ERROR:root:Internal Python error in the inspect module.
Below is the traceback from this internal error.

Steps to Reproduce:
You will need to update an existing custom audience – currently there exists no such script – I created my own. It worked under google-ads==9.0.0 and API version (V6) bot not with API version (V8) and google-ads==12.0.0
But in short:

           # Create a custom audience operation.
            custom_audience_operation = client.get_type("CustomAudienceOperation")
            # Update a custom audience
            custom_audience = custom_audience = custom_audience_operation.update
            custom_audience.resource_name = custom_audience_service.custom_audience_path(customer_id, __id__)
            custom_audience.name = __audiencename__
            # What will the Audience Type be ?
            CustomAudienceEnum = client.get_type("CustomAudienceTypeEnum")
            member_type_enum   = client.get_type("CustomAudienceMemberTypeEnum").CustomAudienceMemberType
            custom_audience.type_ = CustomAudienceEnum.CustomAudienceType.SEARCH
            member1             = _create_custom_audience_member(client,member_type_enum.KEYWORD)
            member1.keyword = keyword
            custom_audience.members.append(member1)
            fm = protobuf_helpers.field_mask(None,custom_audience._pb) # Field Mask
            # pdb.set_trace()
            custom_audience_operation.update_mask.CopyFrom(fm) # Update Field Mask
            #BREAKS HERE:
            custom_audience_response = custom_audience_service.mutate_custom_audiences(customer_id=customer_id, operations=Operation)

Expected behavior:

print ("Updated custom audience with resource name: "
       f"'{custom_audience_response.results[0].resource_name}'")

Client library version and API version:
Client library version: 12.0.0
Google Ads API version: V8

packages_used.txt

Request/Response Logs:

logs.txt

Anything else we should know about your project / environment:

[laurachevalier4]

Hey @caspercrause, can you try using the new copy_from helper method? As of v10, this is considered best practice because it handles both native and wrapped protobuf messages. So your code would change to look something like:

fm = protobuf_helpers.field_mask(None, custom_audience._pb)  # Field Mask   
client.copy_from(custom_audience_operation.update_mask, fm)  # Update Field Mask

You may find this doc useful as well. Let us know how that works out!

[BenRKarl]

It looks like the issue here is that the field mask helper generates a field mask using "type_" instead of "type" which I can reproduce:

custom_audience.type_ = 2
print(ca)
# type_: AUTO
fm = protobuf_helpers.field_mask(None, custom_audience)
print(fm)
# paths: "type_"

This is incorrect behavior, I'll check with the proto-plus maintainers to see if this can be fixed.

[caspercrause]

@BenRKarl Yes, this exactly.

Thanks.

[BenRKarl]

I filed this bug with the team that maintains the generated layer for our library, hopefully they can provide some guidance. I'll leave this Issue open until that other bug is resolved.

[BenRKarl]

FYI the discussion around this is centering on this change to the proto-plus-python library. The side effects of this change are that the native protobuf instances will not have a trailing underscore for fields that are reserved words, which I think makes more sense semantically. To illustrate:

# Wrapped CustomAudience instance
custom_audience.type_ = 2
print(ca)
# >>> type_: AUTO

# Native instance
native = custom_audience._pb
print(native)
# >>> type: AUTO

fm = protobuf_helpers.field_mask(None, custom_audience._pb)
print(fm)
# paths: "type"

[caspercrause]

FYI the discussion around this is centering on this change to the proto-plus-python library. The side effects of this change are that the native protobuf instances will not have a trailing underscore for fields that are reserved words, which I think makes more sense semantically. To illustrate:

# Wrapped CustomAudience instance
custom_audience.type_ = 2
print(ca)
# >>> type_: AUTO

# Native instance
native = custom_audience._pb
print(native)
# >>> type: AUTO

fm = protobuf_helpers.field_mask(None, custom_audience._pb)
print(fm)
# paths: "type"

I'm sorry @BenRKarl but I cannot recreate the above. I still get paths: "type_" instead of # paths: "type"

my version of proto-plus is the latest which is 1.19.0 as verified here

Image:

Package versions:

I might be missing something.

[BenRKarl]

@caspercrause sorry for the misunderstanding, my example is what the interface will likely be like. It's still currently in progress. I'll update this Issue once it's been fixed in this library.

[BenRKarl]

Here's a PR that will fix this issue. The solution that was landed on was to strip trailing underscores in the field mask helper. So there won't be any changes to the protobuf interface as I described above.

Once that's published @caspercrause you should be able to install the latest version to fix this issue. At that point we'll update this library to set that version as the minimum.

[caspercrause]

Here's a PR that will fix this issue. The solution that was landed on was to strip trailing underscores in the field mask helper. So there won't be any changes to the protobuf interface as I described above.

Once that's published @caspercrause you should be able to install the latest version to fix this issue. At that point we'll update this library to set that version as the minimum.

@BenRKarl Any updates on this?

[BenRKarl]

@caspercrause we're working on a new release that will set the minimum version of google-api-core to 1.31.2, which includes a fix for this. In the meantime if you want to manually install this version into your application that should fix the problem now.

[BenRKarl]

This issue is fixed in version 14.0.0 of this library.

[caspercrause]

@BenRKarl

Thanks so much for attending to this matter. All is working.

Much appreciated!