Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Allow checking vmprof output in progress. #139

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

bawr
Copy link
Contributor

@bawr bawr commented May 2, 2017

This PR isn't quite ready yet, but I thought I'd start early, then we can talk about the preferred approach as I'm hacking away. Related to #127.

Two improvements are made so far:

  1. Make the reader interface a little bit more abstract.
  2. When writing out only needed symbols, don't write the same symbol multiple times.

Reasoning:

  1. As a next step, we can have a far simpler implementation when it comes to handling incomplete profile reads - which we'll need if we want to be able to query an ongoing profiling run.
  2. To have those in-progress reads actually useful, we'll need to be able to force a symbol write before the profile is actually closed. The function for this is already exposed to the Python layer, even though it's not mentioned in the main docs. I haven't selected a way of doing that yet, but once that is in place, we definitely don't want to be forced to repeatedly write out the same virtual ips over and over again.

The current implementation of (2) seems slightly iffy to me, is it okay to use a PySet here, or should this be ported over to the khash thing? Any other comments? ;)

#ifndef RPYTHON_VMPROF
PyObject *all_done_uids = NULL;
if (all_code_uids != NULL) {
all_done_uids = PySet_New(NULL);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in this function we cant use CPython API. It will not work on PyPy. I have used khash for the native symbols already (to avoid duplication). We can reuse khash for that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, here's my confusion - we already use CPython API further down the line, similarly if an #ifndef RPYTHON_VMPROF block. :(

...so, is the right fix to change both?

Or is the other one special enough to be kept around?

fileobj = self.fileobj

self.detect_file_sizes()
self.read_static_header()

function_lookup = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should have been done this way some time ago...

self.profile_memory = False
self.profile_lines = None
self.profile_memory = None
self.adr_size = None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we change adr_size or remove it? it is simply the len of state.virtual_ips... it should not be confused with the byte size of the addresses contained in the profile. We could turn that into a method with a longer name...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, that particular thing I added literally only for one test, I suppose we could rework that test instead to use a lower-level API?

@mattip mattip changed the title Allow checking vmprof output in progress. ENH: Allow checking vmprof output in progress. Oct 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants