Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support go heapdumps from 1.7+ #6

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jbarciauskas
Copy link

@jbarciauskas jbarciauskas commented Apr 10, 2018

Update the reader to support the newest heapdump format, based on https://github.com/golang/go/wiki/heapdump15-through-heapdump17

Trying to figure out how to get the type information of objects.

Copy link
Owner

@randall77 randall77 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no way to reconstruct type information in a 1.7 heap, unfortunately.

log.Fatal(err)
}
return b
return []byte(x.Contents)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does a copy of the contents for every call. Probably you want to keep the Contents as a []byte.

offset int64 // position of object contents in dump file
Addr uint64
Addr uint64
Fields []Field
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will definitely lead to a larger object record. That's the limiting factor for how big a heap dump this code can process. Maybe there is a better way?

// map from itab address whether the data field of an iface
// with that itab contains a pointer.
ItabMap map[uint64]bool
// map from the address of an interface to the address of the type
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Substitute "an interface" with "an itab".
It's not the address of the interface itself.

@@ -536,34 +522,23 @@ func rawRead(filename string) *Dump {
if err != nil {
log.Fatal(err)
}
if prefix || string(hdr) != "go1.3 heap dump" {
log.Fatal("not a go1.3 heap dump file")
if prefix || string(hdr) != "go1.7 heap dump" {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not adding 1.7 support, it is removing 1.3 support and adding 1.7 support. I'd rather add functionality instead of replacing it (even if that means more complicated parsing here).

@jbarciauskas
Copy link
Author

Thanks for the feedback. I found the same thing digging through history re: type info. I definitely opened this prematurely. It did happen to parse everything so I just wanted to get it up.

I can go back and make this support multiple formats, though without the type information I don't know how useful 1.5+ heapdumps are, either to me or anyone else.

@randall77
Copy link
Owner

Yes, since 1.4 there just isn't enough type info recorded by the runtime during allocation. The heap dump contents reflect that.

We're moving to a new model where functionality like this is provided by standard core dumps. There we have DWARF information so we can do a better job reconstructing types. You might want to check out the new tool and libraries:
golang.org/x/debug/core
golang.org/x/debug/gocore
golang.org/x/debug/cmd/viewcore

@jbarciauskas
Copy link
Author

@randall77 following up on this, viewcore looks great, but where can I give feedback/ask qs?

@randall77
Copy link
Owner

The standard Go issue tracker can be used to report issues.
For just discussion (questions, etc.) you can use the golang-nuts mailing list. Or mail me directly.
See https://golang.org/project/ for details.

@ptabor
Copy link

ptabor commented Mar 5, 2021

@randall77 I think we a lot of people find this thread looking for an alternative to:

debug.WriteHeapDump(f.Fd())

that allows to dump the heap without stopping the process to inspect it later using viewcore.
Could you, please, point into the right direction ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants