Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bigquery: max staleness cannot be retreived/updated #10026

Open
nonmanager opened this issue Apr 23, 2024 · 0 comments · May be fixed by #10066
Open

bigquery: max staleness cannot be retreived/updated #10026

nonmanager opened this issue Apr 23, 2024 · 0 comments · May be fixed by #10066
Assignees
Labels
api: bigquery Issues related to the BigQuery API. priority: p2 Moderately-important priority. Fix may not be included in next release.

Comments

@nonmanager
Copy link

Client

https://pkg.go.dev/cloud.google.com/go/bigquery@v1.60.0

Code

e.g.

package main

import (
	"context"
	"fmt"
	"os"
	"time"

	"cloud.google.com/go/bigquery"
)

func main() {
	// Part 1: read current MaxStaleness
	ctx := context.TODO()
	client, _ := bigquery.NewClient(ctx, "my_project")
	table := client.Dataset("my_dataset").Table("my_materialized_view_with_staleness")
	md, _ := table.Metadata(ctx)
	fmt.Printf("%v\n", md.MaterializedView.MaxStaleness)

	// Part 2: update MaxStaleness to 5 hours
	tm := bigquery.TableMetadataToUpdate{
		MaterializedView: md.MaterializedView,
	}
	tm.MaterializedView.MaxStaleness = bigquery.IntervalValueFromDuration(5 * time.Hour)
	_, err := table.Update(ctx, tm, md.ETag)
	if err != nil {
		fmt.Fprintf(os.Stderr, "oops: %v\n", err)
		os.Exit(1)
	}
}

Expected behavior

When run against a MV that has max_staleness set, the code should:

  • print the current max staleness interval (for example, 0-0 0 0:30:0)
  • update the interval to 0-0 0 5:0:0

Actual behavior

The code:

  • prints out <nil> as the current value of MaxStaleness (the actual value, set via SQL, is visible in BQ Studio)
  • executes the update, but the MV is unchaged

Additional context

By inspecting the client code, I've observed that the Table returned by the v2 API does return the current max staleness in the root Table.maxStaleness field. However, the client uses Table.materializedView.maxStaleness, which is always empty in the server response.

When it comes to writing, I couldn't get the BigQuery API to change the existing max staleness value. Setting Table.maxStaleness explicitly (instead of Table.materializedView.maxStaleness) in the client library did not work.

By the way, the CDC feature has its own max staleness setting (which applies to tables, not MVs), so I wonder if these two are conflated somewhere in the API/client.

Relevant PR by @alvarowolfx (introduces max staleness to the client).

@nonmanager nonmanager added the triage me I really want to be triaged. label Apr 23, 2024
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the BigQuery API. label Apr 23, 2024
@alvarowolfx alvarowolfx assigned alvarowolfx and unassigned shollyman Apr 23, 2024
@alvarowolfx alvarowolfx added priority: p2 Moderately-important priority. Fix may not be included in next release. and removed triage me I really want to be triaged. labels Apr 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the BigQuery API. priority: p2 Moderately-important priority. Fix may not be included in next release.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants