You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We have seen our system occasionally terminated the outbound connection for some traffic. It seems happen when the system is waiting for the payload from backend service.
In the system log, we see NPE errors thrown from Zuul. See stack trace below for details. LOG_1
message: Error with filter: -, path: /xxxx, method: POST
stack_trace:
j.l.NullPointerException: parameter 'runningFilterIndex' cannot be null
at c.n.s.i.Preconditions.checkNotNull(Preconditions.java:35)
at c.n.z.n.f.BaseZuulFilterRunner.getRunningFilterIndex(BaseZuulFilterRunner.java:105)
at c.n.z.n.f.ZuulFilterChainRunner.filter(ZuulFilterChainRunner.java:108)
at c.n.z.f.e.ProxyEndpoint.filterResponseChunk(ProxyEndpoint.java:340)
at c.n.z.f.e.ProxyEndpoint.lambda$invokeNext$1(ProxyEndpoint.java:326)
at c.n.z.n.s.MethodBinding.bind(MethodBinding.java:42)
at c.n.z.f.e.ProxyEndpoint.invokeNext(ProxyEndpoint.java:326)
at c.n.z.n.s.OriginResponseReceiver.channelRead(OriginResponseReceiver.java:80)
at i.n.c.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at i.n.c.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
... 51 frames truncated
LOG_2
message: ClientResponseWriter caught exception in client connection pipeline: Channel: [id: 0x2f0b1a18, L:/x.x.x.x:8080 - R:/y.y.y.y:41728], active=true, open=true, registered=true, writable=true, id=2f0b1a18,
Passport: CurrentPassport {start_ms=1661474271456,
[
+0=IN_REQ_HEADERS_RECEIVED,
+142857=FILTERS_INBOUND_START,
+746476=IN_REQ_LAST_CONTENT_RECEIVED,
+7517278=FILTERS_INBOUND_END,
+7593714=ORIGIN_CONN_ACQUIRE_START,
+7594596=ORIGIN_CONN_ACQUIRE_END,
+7629669=OUT_REQ_HEADERS_SENDING,
+7679089=OUT_REQ_LAST_CONTENT_SENDING,
+7740466=OUT_REQ_HEADERS_SENT,
+7743783=OUT_REQ_LAST_CONTENT_SENT,
+225603027=IN_RESP_HEADERS_RECEIVED,
+225634814=FILTERS_OUTBOUND_START,
+225780680=FILTERS_OUTBOUND_END,
+225800353=OUT_RESP_HEADERS_SENDING,
+225841816=OUT_RESP_HEADERS_SENT,
+227994402=NOW
]}
stack_trace:
j.l.NullPointerException: parameter 'runningFilterIndex' cannot be null
at c.n.s.i.Preconditions.checkNotNull(Preconditions.java:35)
at c.n.z.n.f.BaseZuulFilterRunner.getRunningFilterIndex(BaseZuulFilterRunner.java:105)
at c.n.z.n.f.ZuulFilterChainRunner.filter(ZuulFilterChainRunner.java:108)
at c.n.z.f.e.ProxyEndpoint.filterResponseChunk(ProxyEndpoint.java:340)
at c.n.z.f.e.ProxyEndpoint.lambda$invokeNext$1(ProxyEndpoint.java:326)
at c.n.z.n.s.MethodBinding.bind(MethodBinding.java:42)
at c.n.z.f.e.ProxyEndpoint.invokeNext(ProxyEndpoint.java:326)
at c.n.z.n.s.OriginResponseReceiver.channelRead(OriginResponseReceiver.java:80)
at i.n.c.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at i.n.c.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
... 53 frames truncated
LOG_3
message: ClientResponseWriter caught exception in client connection pipeline: Channel: [id: 0xaf8c22ac, L:/x.x.x.x:8080 - R:/y.y.y.y:50388], active=true, open=true, registered=true, writable=true, id=af8c22ac,
Passport: CurrentPassport {start_ms=1661474239671,
[
+0=IN_REQ_HEADERS_RECEIVED,
+140498=FILTERS_INBOUND_START,
+936973=IN_REQ_LAST_CONTENT_RECEIVED,
+6553469=FILTERS_INBOUND_END,
+6633562=ORIGIN_CONN_ACQUIRE_START,
+6635050=ORIGIN_CONN_ACQUIRE_END,
+6684741=OUT_REQ_HEADERS_SENDING,
+6726169=OUT_REQ_LAST_CONTENT_SENDING,
+6821852=OUT_REQ_HEADERS_SENT,
+6826904=OUT_REQ_LAST_CONTENT_SENT,
+276210601=IN_RESP_HEADERS_RECEIVED,
+276262474=FILTERS_OUTBOUND_START,
+276452723=FILTERS_OUTBOUND_END,
+276474704=OUT_RESP_HEADERS_SENDING,
+276530773=OUT_RESP_HEADERS_SENT,
+276540177=IN_RESP_LAST_CONTENT_RECEIVED,
+277482075=NOW
]}
stack_trace:
j.l.NullPointerException: channel handler context
at c.g.c.b.Preconditions.checkNotNull(Preconditions.java:907)
at c.n.z.n.f.BaseZuulFilterRunner.getChannelHandlerContext(BaseZuulFilterRunner.java:89)
at c.n.z.n.f.BaseZuulFilterRunner.handleException(BaseZuulFilterRunner.java:380)
at c.n.z.n.f.ZuulFilterChainRunner.filter(ZuulFilterChainRunner.java:161)
at c.n.z.f.e.ProxyEndpoint.filterResponseChunk(ProxyEndpoint.java:340)
at c.n.z.f.e.ProxyEndpoint.lambda$invokeNext$1(ProxyEndpoint.java:326)
at c.n.z.n.s.MethodBinding.bind(MethodBinding.java:42)
at c.n.z.f.e.ProxyEndpoint.invokeNext(ProxyEndpoint.java:326)
at c.n.z.n.s.OriginResponseReceiver.channelRead(OriginResponseReceiver.java:80)
at i.n.c.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
... 52 frames truncated
As a temp workaround, we buffer the response payload by overriding the needsBodyBuffered method in an outbound filter.
Any thoughts on why it happened? Could this be a Zuul bug? Would Zuul upgrade to a later version fix it?
Zuul: v2.1.7
netty-codec-http2:4.1.59.Final
The text was updated successfully, but these errors were encountered:
We have seen our system occasionally terminated the outbound connection for some traffic. It seems happen when the system is waiting for the payload from backend service.
In the system log, we see NPE errors thrown from Zuul. See stack trace below for details.
LOG_1
LOG_2
LOG_3
As a temp workaround, we buffer the response payload by overriding the
needsBodyBuffered
method in an outbound filter.Any thoughts on why it happened? Could this be a Zuul bug? Would Zuul upgrade to a later version fix it?
Zuul: v2.1.7
netty-codec-http2:4.1.59.Final
The text was updated successfully, but these errors were encountered: