Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add basic explain get_metadata function for tf2. #507

Merged
merged 22 commits into from Jul 8, 2021

Conversation

ji-yaqi
Copy link
Contributor

@ji-yaqi ji-yaqi commented Jun 24, 2021

Basic explain get_metadata function, for tf2.

Usage:

builder = google.cloud.aiplatform.explain.metadata.tf.v2.saved_model_metadata_builder.SavedModelMetadataBuilder(
    "MODEL_PATH"
)
metadata = builder.get_metadata()

The main difference from Vertex SDK and XAI SDK's get_metadata function:

  1. In Vertex SDK, users the camelCase while in XAI, variables users snake_case
    e.g. Vertex: inputTensorName, XAI: input_tensor_name

  2. If modality and encoding in inputs are not set, XAI SDK populates the default:

      'modality': 'categorical',
      'encoding': 'identity'

Vertex SDK doesn't populate the default values. It only populates the users's settings.
The two methods will not make any difference in the actual settings.

@ji-yaqi ji-yaqi requested a review from sasha-gitg June 24, 2021 04:40
@ji-yaqi ji-yaqi requested review from a team as code owners June 24, 2021 04:40
@product-auto-label product-auto-label bot added the api: aiplatform Issues related to the AI Platform API. label Jun 24, 2021
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Jun 24, 2021
@ji-yaqi
Copy link
Contributor Author

ji-yaqi commented Jun 24, 2021

/hold

@ji-yaqi
Copy link
Contributor Author

ji-yaqi commented Jun 24, 2021

/unhold

self,
model_path: str,
signature_name: Optional[str] = None,
outputs_to_explain: Optional[List[str]] = (),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Optional type is a List while the default is an empty tuple. It seems like the default can be None instead.

Comment on lines 104 to 115
if tensor_spec.dtype.is_floating:
input_mds[
name
] = explanation_metadata.ExplanationMetadata.InputMetadata(
input_tensor_name=name
)
else:
input_mds[
name
] = explanation_metadata.ExplanationMetadata.InputMetadata(
input_tensor_name=name, modality="categorical",
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Option to reduce the size of this block.

Suggested change
if tensor_spec.dtype.is_floating:
input_mds[
name
] = explanation_metadata.ExplanationMetadata.InputMetadata(
input_tensor_name=name
)
else:
input_mds[
name
] = explanation_metadata.ExplanationMetadata.InputMetadata(
input_tensor_name=name, modality="categorical",
)
input_mds[name] = explanation_metadata.ExplanationMetadata.InputMetadata(
input_tensor_name=name,
modality=None if tensor_spec.dtype.is_floating else "categorical"
)

setup.py Outdated
@@ -31,7 +31,10 @@

tensorboard_extra_require = ["tensorflow >=2.3.0, <=2.5.0"]
metadata_extra_require = ["pandas >= 1.0.0"]
full_extra_require = tensorboard_extra_require + metadata_extra_require
xai_extra_require = ["tensorflow-cpu>=2.3.0, <=2.5.0"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"tensorflow" is the CPU-only release: https://www.tensorflow.org/install/pip

This will also make it easier to manage with TB's TF dependency.

setup.py Outdated
@@ -31,7 +31,10 @@

tensorboard_extra_require = ["tensorflow >=2.3.0, <=2.5.0"]
metadata_extra_require = ["pandas >= 1.0.0"]
full_extra_require = tensorboard_extra_require + metadata_extra_require
xai_extra_require = ["tensorflow-cpu>=2.3.0, <=2.5.0"]
full_extra_require = (
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tensorboard and xai have the same dependencies so we may need to deduplicate this with list(set(...)).



class SavedModelMetadataBuilderTest(tf.test.TestCase):
def _set_up(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is only used by one test, it should perhaps not be a shared method.

@ji-yaqi ji-yaqi requested review from leahecole and removed request for a team June 30, 2021 22:59
@ji-yaqi ji-yaqi merged commit f6f9a97 into googleapis:master Jul 8, 2021
@ji-yaqi ji-yaqi deleted the xaitf2 branch July 8, 2021 19:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: aiplatform Issues related to the AI Platform API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants