Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversion from bq schema to proto #543

Open
richwomanbtc opened this issue Dec 26, 2022 · 1 comment
Open

Conversion from bq schema to proto #543

richwomanbtc opened this issue Dec 26, 2022 · 1 comment
Assignees
Labels
api: bigquerystorage Issues related to the googleapis/python-bigquery-storage API. priority: p3 Desirable enhancement or fix. May not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@richwomanbtc
Copy link

Is your feature request related to a problem? Please describe.

The java implementation can convert bigquery's schema to proto and write json without preparing proto files.
I think it would be very convenient to have something equivalent to these in the python implementation.
References:

Describe the solution you'd like

I would like to add API implementation or sample code for it.
It will look like this (although I haven't actually run it).

bq_table_schema_type_map = {
    TableFieldSchema.Type.BOOL: FieldDescriptorProto.Type.TYPE_BOOL,
    ...
}

bq_table_schema_mode_map = {
    TableFieldSchema.Mode.NULLABLE: FieldDescriptorProto.Label.LABEL_OPTIONAL,
    ...
}

def convert_bq_field_to_proto_field(bq_field: TableFieldSchema, index: int) -> FieldDescriptorProto:
    mode = bq_field.mode
    field_name = bq_field.name
    type_ = bq_field.type_
    return FieldDescriptorProto(
        name=field_name,
        type=bq_table_schema_type_map[type_],
        label=bq_table_schema_mode_map[mode],
        number=index
    )

def convert_bq_table_schema_to_descriptor_proto(table_schema: TableSchema, scope: str,
                                                dependency_map: Dict = None) -> DescriptorProto:
    index = 1
    fields: List[FieldDescriptorProto] = []
    for bq_field in table_schema.fields:
        index += 1
        fields.append(convert_bq_field_to_proto_field(bq_field, index))

    descriptor_proto = DescriptorProto(name=scope, field=fields)
    return descriptor_proto

...

Are there any plans to add such a feature? Also, would it be possible for me to contribute?

@product-auto-label product-auto-label bot added the api: bigquerystorage Issues related to the googleapis/python-bigquery-storage API. label Dec 26, 2022
@shollyman shollyman added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. priority: p3 Desirable enhancement or fix. May not be included in next release. labels Mar 3, 2023
@shollyman
Copy link
Contributor

FR related to making type conversion easier. Assigning to the rosie who's working on the updated client.

@rosiezou rosiezou assigned shollyman and unassigned rosiezou May 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquerystorage Issues related to the googleapis/python-bigquery-storage API. priority: p3 Desirable enhancement or fix. May not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

3 participants