[SDK] BatchSpanProcessor::ForceFlush
appears to be hanging forever
#2574
Labels
bug
Something isn't working
triage/accepted
Indicates an issue or PR is ready to be actively worked on.
Describe your environment
Platform: Linux/x86_64
Build system: CMake
Package manager: conda/conda-forge
OpenTelemetry version: 1.14.1
Steps to reproduce
Unfortunately, this is entwined into a large proprietary application and I've not been able to extricate it. We are using the OTLP exporter, BatchSpanProcessor, and we are calling Flush periodically with a timeout.
What is the expected behavior?
ForceFlush with a timeout should reliably complete within the timeout (roughly)
What is the actual behavior?
ForceFlush appears to get stuck inside the spinlock:
or
These traces appear to me to correspond with the spinlock portion. It appears to me that the spinlock portion does not respect the timeout:
opentelemetry-cpp/sdk/src/trace/batch_span_processor.cc
Lines 120 to 137 in 07f6cb5
Additional context
We think this only occurred once we upgraded from 1.13.0 to 1.14.1. Note that we did not used to have a timeout, since we did not feel a need to. When we upgraded, we noticed it was getting stuck exporting, and added the timeout to try to avoid this (and as general good practice), but it seems there's some deeper issue as to why the export seems to never complete.
I did set a breakpoint on the actual Export method and it does appear to be called regularly, but the main thread that is blocking in ForceFlush isn't getting unblocked.
The text was updated successfully, but these errors were encountered: