feat: Added tabular forecasting samples #128

ivanmkc · 2020-12-10T16:49:09Z

Added samples and tests for AutoML forecasting

list_model_evaluations_tabular_forecasting_sample
create_training_pipeline_tabular_forecasting_sample
get_model_evaluation_tabular_forecasting
predict_tabular_forecasting <- Forecasting doesn't support predict so will do in a subsequent PR.

…les for AutoML Forecasting

snippet-bot · 2020-12-10T16:49:16Z

Here is the summary of changes.

You added 1 region tag.

samples/snippets/create_training_pipeline_tabular_forecasting_sample.py:15, tag aiplatform_create_training_pipeline_tabular_forecasting_sample

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

Refresh this comment

ivanmkc · 2020-12-10T16:57:18Z

.sample_configs/param_handlers/create_training_pipeline_tabular_forecasting_sample.py

+
+    training_task_inputs_dict = {
+        # required inputs
+        "targetColumn": target_column,


Curious as to if we should show comments for each of these values in the sample.

@ivanmkc do you mean comments like # display_name: YOUR_DISPLAY_NAME?

Thanks for reviewing, I was wondering if we should add comments for each param in the samples.

Seems like Yuhan is suggesting to just tell them to read the docstrings.

samples/snippets/create_training_pipeline_tabular_forecasting_sample_test.py

ivanmkc · 2020-12-10T16:59:50Z

samples/snippets/get_model_evaluation_tabular_forecasting_sample.py

@@ -0,0 +1,37 @@
+# Copyright 2020 Google LLC


Same as corresponding classification sample.

I think we don’t really need to have sample variants for get and list methods. I see that we have them for some other kinds of AutoML services and perhaps those need to be removed (in a separate PR).

ivanmkc · 2020-12-10T17:00:03Z

samples/snippets/predict_tabular_forecasting_sample.py

@@ -0,0 +1,44 @@
+# Copyright 2020 Google LLC


Same as corresponding classification sample.

Is it possible that all the predict tabular samples would be identical? Is so we should consolidate those too perhaps. On the other hand if the sample tests would be very different (Eg you might be looking for different specific fields in the predictions), then let’s keep this.

Predict cannot be used with 'tabular'. They only support batch_predict, I will think about consolidation when doing batch_predict next week.

…t, fixed get_model_evaluation_tabular_forecasting_sample, and fixed create_training_pipeline_tabular_forecasting_sample

ivanmkc · 2020-12-15T09:33:50Z

samples/snippets/predict_tabular_forecasting_sample_test.py

+
+import predict_tabular_classification_sample
+
+ENDPOINT_ID = "TODO"


Blocked by: https://b.corp.google.com/issues/175666946

ivanmkc · 2020-12-15T15:12:59Z

Fixing CI errors.

dizcology · 2020-12-17T19:54:39Z

.sample_configs/param_handlers/create_training_pipeline_tabular_forecasting_sample.py

+        "transformations": transformations,
+        "period": period,
+
+        # Objective function the model is to be optimized towards.


Maybe it's better to point to the doc pages instead of including the information below here.

Do you mean include a link in the comments?

right - if the information here is available on a particular documentation page.

dizcology · 2020-12-17T19:55:48Z

.sample_configs/variants.yaml

@@ -175,7 +177,7 @@ list_hyperparameter_tuning_jobs:
 list_model_evaluation_slices:
 - ''
 list_model_evaluations:
- ''
+- tabular_forecasting


this would replace the list_model_evaluation sample. Please keep the "default" variant keyed with the empty string.

Also - perhaps we don't need a special list_model_evaluations variant unless the call to list_model_evaluations is different in the tabular_forecasting use case.

Oh I see. Good catch.

dizcology · 2020-12-17T19:58:00Z

.sample_configs/process_configs.yaml

@@ -182,6 +183,7 @@ get_model_evaluation_sample:
  - model_explanation
 get_model_evaluation_slice_sample: {}
 get_model_evaluation_tabular_classification_sample: {}
+get_model_evaluation_tabular_forecasting_sample: {}


If the tabular_forcasting variant for get_model_evaluation ends up identical with any get_model_evaluation, please skip it. (and we should separately clean up the others - but perhaps i missed something here)

I got it, we want to consolidate if possible. Let me do another pass.

ivanmkc · 2020-12-22T16:34:30Z

@dizcology can I get another look?

Ivan Cheung added 4 commits December 8, 2020 23:48

Added predict, get_model_evaluation and create_training_pipeline samp…

d745036

…les for AutoML Forecasting

Added param handlers

0fee278

Added headers manually

57318ce

fix: Improved forecasting sample

a6ac96d

ivanmkc requested review from dizcology and a team as code owners December 10, 2020 16:49

ivanmkc requested review from kurtisvg and removed request for a team December 10, 2020 16:49

google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Dec 10, 2020

product-auto-label bot added the samples Issues that are directly related to samples. label Dec 10, 2020

ivanmkc commented Dec 10, 2020

View reviewed changes

samples/snippets/create_training_pipeline_tabular_forecasting_sample_test.py Outdated Show resolved Hide resolved

ivanmkc commented Dec 10, 2020

View reviewed changes

ivanmkc changed the title ~~feat: Added tabular forecasting samples~~ feat: Added tabular forecasting samples [WIP] Dec 10, 2020

Ivan Cheung added 2 commits December 11, 2020 20:51

Added forecasting test

42fcc42

Added tests for predict and get_model_evaluation

e75b880

ivanmkc force-pushed the imkc--tabular-forecasting-samples branch from a74d6c3 to e75b880 Compare December 11, 2020 11:52

Ivan Cheung added 3 commits December 15, 2020 01:36

fix: Fixed create_training_pipeline_sample

7e8f7ba

feat: Added list_model_evaluations_tabular_forecasting_sample and tes…

848ee55

…t, fixed get_model_evaluation_tabular_forecasting_sample, and fixed create_training_pipeline_tabular_forecasting_sample

fix: Reverted back to generated BUILD_SPECIFIC_GCLOUD_PROJECT

669a664

ivanmkc changed the title ~~feat: Added tabular forecasting samples [WIP]~~ feat: Added tabular forecasting samples Dec 15, 2020

ivanmkc requested review from sirtorry and aribray December 15, 2020 09:10

fix: Fixed name of test

a779c4f

ivanmkc commented Dec 15, 2020

View reviewed changes

fix: Fixed lint errors

b08ac1e

Ivan Cheung added 2 commits December 16, 2020 00:37

fix: Fixed assertion

fd87673

fix: Removed predict samples

2fc3f90

dizcology reviewed Dec 17, 2020

View reviewed changes

Consolidated samples

087ff39

ivanmkc force-pushed the imkc--tabular-forecasting-samples branch from 9f113a4 to 087ff39 Compare December 21, 2020 20:26

Ivan Cheung added 2 commits December 22, 2020 05:27

fix: Removed list_model_evaluations_tabular_forecasting

ddd443e

fix: tweaks

f12db5b

dizcology approved these changes Dec 22, 2020

View reviewed changes

ivanmkc merged commit 69fc7fd into googleapis:master Dec 22, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Added tabular forecasting samples #128

feat: Added tabular forecasting samples #128

ivanmkc commented Dec 10, 2020 •

edited

snippet-bot bot commented Dec 10, 2020 •

edited

ivanmkc Dec 10, 2020

aribray Dec 15, 2020

ivanmkc Dec 17, 2020

ivanmkc Dec 10, 2020

dizcology Dec 15, 2020

ivanmkc Dec 17, 2020

ivanmkc Dec 10, 2020

dizcology Dec 15, 2020

ivanmkc Dec 17, 2020

ivanmkc Dec 15, 2020

ivanmkc Dec 15, 2020

ivanmkc commented Dec 15, 2020

dizcology Dec 17, 2020

ivanmkc Dec 17, 2020

dizcology Dec 17, 2020

dizcology Dec 17, 2020

dizcology Dec 17, 2020

ivanmkc Dec 17, 2020

dizcology Dec 17, 2020

ivanmkc Dec 17, 2020

ivanmkc commented Dec 22, 2020


		import predict_tabular_classification_sample

		ENDPOINT_ID = "TODO"

feat: Added tabular forecasting samples #128

feat: Added tabular forecasting samples #128

Conversation

ivanmkc commented Dec 10, 2020 • edited

snippet-bot bot commented Dec 10, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ivanmkc commented Dec 15, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ivanmkc commented Dec 22, 2020

ivanmkc commented Dec 10, 2020 •

edited

snippet-bot bot commented Dec 10, 2020 •

edited