New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Spanner latency issue on streaming read #2757
Comments
A few queries to start investigating
|
it looks like for read queries fetching meta data using stream reading takes up to 50ms each time, reading elapsed time is from 0 to 1ms but fetching metadata takes more most of the time. I also tried native queries / derived / using jdbc and still I am facing same issue. Exmaple query
|
For every query stream seems to be in state not_ready what causes starting new stream for every request and it fetches metadata about the table using streaming and it takes more than 40ms which seems to be very long I don't know. |
The default session pool options have 100 min sessions and 400 max sessions. Can you share with me the code snippet how the client object is being created? |
@amb-sebastian-podgorski-pt I think that there is an underlying misunderstanding here. | Reading data is really fast, it is taking below 1ms but from logs I can see that establishing grcp connection to Spanner takes most of the time. This observation is where the misunderstanding is. The Java client does not execute the query directly when you call the
40ms is however a too high value for a very simple query, which indicates that there might be more going on here. It is not clear to me whether the
Using the above strategy should bring a test query like the above down to around 5ms. Note that the very first query might be slower, as the first time that Cloud Spanner sees a query, it needs to parse and plan it. Subsequent executions will re-use the plan that has already been compiled and cached. |
@amb-sebastian-podgorski-pt Is this still an issue? If so, could you try the steps that I described in the previous comment to verify the actual latency for your query? |
I found our service has problems with latency and every single read statements is taking more than 40ms.
I tried to use native/derived queries and it always takes more than 40ms.
Reading data is really fast, it is taking below 1ms but from logs I can see that establishing grcp connection to Spanner takes most of the time.
logs from reading
as you can see from timestamps of the last two logs it takes 39ms, I tried to use read only transactions, makes read without transaction, set rcp priority on SpannerTemplate to low but nothing helps and simple call to rest endpoint which is fetching object by id takes more than 40ms when reading from db takes less than 0ms.
Application details
Java 17
google-cloud-spanner - 6.55
spring-cloud-gcp-starter-data-spanner - 4.8.4
The text was updated successfully, but these errors were encountered: