About alloc engine throttling mechanism

[yanglong1010]

hi,

async-profiler's alloc engine uses intervals to reduce the number of events and reduce overhead.
For many applications, interval=256k or 512k is good enough, but for many applications with large memory throughput, it may not be appropriate and cannot well control the upper limit of the number of events and overhead.

JFR provides a throttling mechanism in JDK16, which can limit the number of events per unit time and try to ensure that it is statistically representative.

I wonder if I can add such feature to async-profiler?

[pnf]

With jfrsync and an appropriate profile, do you pick up the new sampling algorithm? For html and collapsed output, overhead should be less of a problem, since individual sample events are not stored.

…

hi, async-profiler's alloc engine uses intervals to reduce the number of events and reduce overhead. For many applications, interval=256k or 512k is good enough, but for many applications with large memory throughput, it may not be appropriate and cannot well control the upper limit of the number of events and overhead. JFR provides a [throttling mechanism] (https://bugs.openjdk.org/browse/JDK-8257602) in JDK16, which can limit the number of events per unit time and try to ensure that it is statistically representative. I wonder if I can add such feature to async-profiler? — Reply to this email directly, view it on GitHub <#826>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJU73MKSURK2PZACUJLQ5LX7654NAVCNFSM6AAAAAA6FLYAASVHI2DSMVQWIX3LMV43ASLTON2WKOZRHE2DSNJTHA4TEMQ>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[yanglong1010]

With jfrsync and an appropriate profile, do you pick up the new sampling algorithm? For html and collapsed output, overhead should be less of a problem, since individual sample events are not stored.
…
hi, async-profiler's alloc engine uses intervals to reduce the number of events and reduce overhead. For many applications, interval=256k or 512k is good enough, but for many applications with large memory throughput, it may not be appropriate and cannot well control the upper limit of the number of events and overhead. JFR provides a [throttling mechanism] (https://bugs.openjdk.org/browse/JDK-8257602) in JDK16, which can limit the number of events per unit time and try to ensure that it is statistically representative. I wonder if I can add such feature to async-profiler? — Reply to this email directly, view it on GitHub <#826>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJU73MKSURK2PZACUJLQ5LX7654NAVCNFSM6AAAAAA6FLYAASVHI2DSMVQWIX3LMV43ASLTON2WKOZRHE2DSNJTHA4TEMQ. You are receiving this because you are subscribed to this thread.Message ID: @.***>

hi, Peter

For JDK 16 and above, jfrsync is a good choice.
However, there are still many applications running on JDK8 or JDK11, and jdk.ObjectAllocationSample is not backported to these versions.

html and collapsed formats can help control file size, but may not avoid the overhead associated with stack unwindling.
In the field of Continuous Profiling, most projects use the JFR format because it has stronger description capabilities and can save more detailed data.

[apangin]

I agree the default value is apparently too low, although it matches the default interval for JVM TI SampledObjectAlloc event.
In my experience, alloc=2m works well for the majority of applications - that's what we set for our continuous always-on profiler.

I'd likely accept an alternative mechanism if it's simple enough and provides quality results. The throttling algorithm for ObjectAllocationSample events in JFR is neither simple nor high-quality (the result profiles look quite different from the profiles collected with the traditional allocation events).