You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In streaming workload, high join amplification can lead to barrier latency spike.
So far our approaches to mitigate these have been more indirect, e.g. decoupling sink, adaptive rate limit. It can isolate the slowness to the specific stream job.
I think we should also explore optimizing join itself.
Come up with a high cardinality join scenario, with few columns.
Come up with a high cardinality join scenario, with many columns.
Observe the flamegraph to see where it can be optimized further.
Investigate other systems to see what optimizations can be made in similar situations, e.g. cross join.
The text was updated successfully, but these errors were encountered:
In streaming workload, high join amplification can lead to barrier latency spike.
So far our approaches to mitigate these have been more indirect, e.g. decoupling sink, adaptive rate limit. It can isolate the slowness to the specific stream job.
I think we should also explore optimizing join itself.
The text was updated successfully, but these errors were encountered: