-
Notifications
You must be signed in to change notification settings - Fork 18.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tarexport: Plumb ctx, add OTEL spans, handle cancellation #47629
Conversation
@@ -393,6 +423,11 @@ func (s *saveSession) saveImage(id image.ID) (map[layer.DiffID]distribution.Desc | |||
var layers []layer.DiffID | |||
var foreignSrcs map[layer.DiffID]distribution.Descriptor | |||
for i, diffID := range img.RootFS.DiffIDs { | |||
select { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we check this down at L581 as well? (sorry GH won't let me add a comment there)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure - it's calling system.Chtimes
on two paths. Doesn't look like it has a potential to take a noticeable amount of time.
pkg/ioutils/readers.go
Outdated
r io.Reader | ||
} | ||
|
||
func NewCtxReader(ctx context.Context, r io.Reader) io.Reader { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Need a doc string here.
Also I think something like an io.PipeReader
would be a better for the use case which you can call CloseWithError(ctx.Err())
on and handle cancellation a little more completely (rather than be potentially blocked on reading the underlying reader).
e.g.
pr, pw := io.Pipe()
go func() {
<-ctx.Done()
pr.CloseWithError(ctx.Err())
}()
// ...
Or we can forgo the pipe and just close the reader normally when the context is cancelled and check after the io.Copy
if it was cancelled.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure if the pipe actually solves anything though.
If the underlying reader Read
method blocks indefinitely, then there's actually no way of stopping it, other than calling Close
on the reader if it's actually also an io.Closer
and hoping that ends all pending Read
s.
I would prefer to avoid doing any Close
inside the wrapper, because of the defer closer.Close()
being actually the most common way of handling calling close. Doing that could potentially result in Close
being called twice (first in the wrapper Read
call, and then in the defer), which is implementation specific and can lead to subtle bugs like #46418.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a CopyCtx
as a replacement for the io.Copy
as a best-effort recovery in a situation where Read
blocks completely - in this case the calling goroutine won't be blocked, only the child goroutine that does the io.Copy
.
3b2bc73
to
2ca48aa
Compare
pkg/ioutils/copy.go
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the only part I have some questions about; not necessarily a blocker, but just cautious where we extend anything in pkg/
;
- Do we expect these functions to be used externally (i.e., will we need these in, e.g.
buildkit
or elsewhere)? - ☝️ if not (or at least not currently) perhaps we should consider moving this to an
internal/
package - ☝️ if we DO need those externally; is
pkg/ioutils
the correct package for this, or should it be a separate package? (e.g.contextreader
/contextreader.New()
(I'd like to be sure we don't end up makingioutils
too much of a "utils" package, and it to become a "grab bag" or only "somewhat" related functionality).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm right, I don't expect it to be used outside of moby, so I'm definitely +1 on making it internal.
We can always make it public if we need to, but going the reverse is always harder.
Let me move it to internal.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! Yes, trying to be more conservative a bit on expanding things in pkg/
❤️
pkg/ioutils/copy.go
Outdated
// NewCtxReader wraps the given reader with a reader that doesn't proceed with | ||
// reading if the context is done. | ||
// | ||
// Note: Read will still block if the underlying reader blocks. | ||
func NewCtxReader(ctx context.Context, r io.Reader) io.Reader { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was curious as well if we checked if the Go team is considering something along these lines (and/or if there's proposals), and if we in that case could align to whatever design may be worked on.
I did find this proposal, but haven't read in-depth if there's any progress / good proposal on design there;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Pass `context.Context` through `tarexport.Load` and `tarexport.Save`. Create OTEL spans for the most time consuming operations. Also, handle context cancellations to actually end saving/loading when the operation is cancelled - before this PR the daemon would still be performing the operation even though the user already cancelled it. Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2ca48aa
to
ad0f263
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Pass
context.Context
throughtarexport.Load
andtarexport.Save
. Create OTEL spans for the most time consuming operations.Also, handle context cancellations to actually end saving/loading when the operation is cancelled - before this PR the daemon would still be performing the operation even though the user already cancelled it.
- How to verify it
- Description for the changelog
- A picture of a cute animal (not mandatory but encouraged)