Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Information on buffer size used after serialization #691

Open
mdorier opened this issue Jun 13, 2023 · 0 comments
Open

Information on buffer size used after serialization #691

mdorier opened this issue Jun 13, 2023 · 0 comments
Milestone

Comments

@mdorier
Copy link

mdorier commented Jun 13, 2023

Is your feature request related to a problem? Please describe.

For monitoring purpose, I would like to know RPC input and output sizes after serialization. The use-case would be to capture the patterns of arbitrarily complex Mercury/Margo-based applications and do a trace replayer that replaces the potentially complex datatypes used by the application with "dummy" datatypes that take up the same space when serialized. For this to be possible, we need to be able to monitor how much buffer space was used when the original application's datatypes were serialized by Mercury.

Describe the solution you'd like

On sender:

size_t size;
HG_Get_serialized_input_size(handle, &size); # size will be 0 because HG_Forward hasn't been called yet
HG_Get_serialized_output_size(handle, &size); # size will be 0 because the RPC hasn't completed

HG_Forward(handle, ...);

HG_Get_serialized_input_size(handle, &size); # size will be the size copied into an internal Mercury buffer when the input was serialized by HG_Forward
HG_Get_serialized_output_size(handle, &size); # size will be 0 because the RPC hasn't completed

// after the RPC has completed:
HG_Get_serialized_output_size(handle, &size); # size will be the size of the buffer internally used by Mercury to hold the serialized output

Similarly on receiver, HG_Get_serialized_input would have a value right away (because the RPC was received and should have its data in a buffer already), HG_Get_serialized_output would have a relevant value only after HG_Respond is called.

Describe alternatives you've considered

Alternatives would include being able to inject a callback into the serialization path so that this callback can see all the serialized data passing (e.g. a function that by default is a memcpy but could be overwritten by something else). Note that this solution would also allow to do cool things like transparent compression/decompression that is agnostic to the user-provided data types.

@soumagne soumagne added this to the future milestone Jun 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants