New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to only read/write new logs from a specific pod using ReadNamespacedPodLogWithHttpMessagesAsync
#1491
Comments
check out example of https://github.com/kubernetes-client/csharp/blob/master/examples/logs/Logs.cs you will get the steam of log output, thus, everything from the stream is new |
Thanks @tg123! I tried giving this a shot by setting the follow flag to Edit : I'm not really receiving any logs if I set |
Thank you @tg123! I was able to get this to work. However, seems like the cancellation token is not respected? I have a scenario where I'm constantly writing to the pod and simultaneously reading from it, the reader continues to read logs till the end of stream if after requesting a cancellation token. |
linking ct to stream readline should work |
@tg123 Thanks! It looks like the stream with the pod logs is returned almost instantly but it has been very inefficient to read the pod logs line by line. I've experimented with I need to read about 2000 lines in a few seconds but takes almost 10 minutes currently. |
The Kubernetes project currently lacks enough contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale |
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten |
Describe the bug
A clear and concise description of what the bug is.
ReadNamespacedPodLogWithHttpMessagesAsync reads all the logs (including historical ones) from a specific pod. I cannot find a good way to only read the recent logs that have not been read/written before in the previous iterations.
I tried the following but none of them actually fix this.
1.Watch mechanism is not supported by ReadNamespacedPodLogWithHttpMessagesAsync so we are not able to detect any new logs using watch and only write them. Also added a watch mechanism to write only if there was an event detected, but this would work only if a pod was added/deleted/modified/errored in the namespace, any change in the log would not trigger an event, so this is not a viable solution.
2.Seek is not supported, so we cannot move the pointer to the end of the old logs and just write the new ones.
3.Experimented with the tailLines argument but it's not clear how to set the number of lines in this argument without doing extra read operations. Does not seem like a good solution to me.
Kubernetes C# SDK Client Version
e.g.
9.0.1
Server Kubernetes Version
e.g.
1.22.3
Dotnet Runtime Version
e.g. net6
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
Capability to only read the recent logs that have not already been read from the pod.
Example - the following code reads all the logs (including historical ones) every time. The expectation is to read only the logs that have not been read in the previous iterations without having to hard code
tailLines
argument because this can risk losing some logs.KubeConfig
If applicable, add a KubeConfig file with secrets redacted.
Where do you run your app with Kubernetes SDK (please complete the following information):
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: