You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I use Kafka as my message queue, and after the service runs for a period of time, the tb_core service often encounters exceptions.
I have adjusted the parameters of the following two kafka, but it did not work.
In kafka.yml:
KAFKA_MESSAGE_MAX_BYTES: 33554432
In queue-kafka.env:
TB_KAFKA_MAX_REQUEST_SIZE=33554432
However, the following errors still occur:
2024-05-16+13:25:23.043 [kafka-coordinator-heartbeat-thread | tb-core-node] ERROR org.apache.kafka.common.network.NetworkReceive - Allocating buffer of size 26211464 for source 2
2024-05-16+13:25:23.043 [kafka-coordinator-heartbeat-thread | tb-core-node] ERROR org.apache.kafka.common.network.NetworkReceive - Stack Trace: java.base/java.lang.Thread.getStackTrace(Thread.java:1602)|org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:125)|org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:452)|org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:402)|org.apache.kafka.common.network.Selector.attemptRead(Selector.java:674)|org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:576)|org.apache.kafka.common.network.Selector.poll(Selector.java:481)|org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:560)|org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:265)|org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.pollNoWakeup(ConsumerNetworkClient.java:306)|org.apache.kafka.clients.consumer.internals.AbstractCoordinator$HeartbeatThread.run(AbstractCoordinator.java:1433)
Some error log from zookeeper, ip 192.168.32.1 looks like gateway ip of tb core and tb rule engine:
2024-05-16 14:22:43,194 [myid:1] - INFO [NIOWorkerThread-16:ZooKeeperServer@1047] - Refusing session request for client /192.168.32.1:52408 as it has seen zxid 0x189c0000179f our last zxid is 0xc00000634 client must try another server
2024-05-16 14:22:43,352 [myid:1] - INFO [NIOWorkerThread-29:ZooKeeperServer@1047] - Refusing session request for client /192.168.32.1:38874 as it has seen zxid 0x189d00000001 our last zxid is 0xc00000634 client must try another server
2024-05-16 14:22:43,502 [myid:1] - INFO [NIOWorkerThread-11:ZooKeeperServer@1047] - Refusing session request for client /192.168.32.1:38876 as it has seen zxid 0x189d00000001 our last zxid is 0xc00000634 client must try another server
2024-05-16 14:22:43,518 [myid:1] - INFO [NIOWorkerThread-32:ZooKeeperServer@1047] - Refusing session request for client /192.168.32.1:46678 as it has seen zxid 0x189d00000003 our last zxid is 0xc00000634 client must try another server
2024-05-16 14:22:43,791 [myid:1] - INFO [NIOWorkerThread-13:ZooKeeperServer@1047] - Refusing session request for client /192.168.32.1:51388 as it has seen zxid 0x189c0000179f our last zxid is 0xc00000634 client must try another server
2024-05-16 14:22:43,893 [myid:1] - INFO [NIOWorkerThread-3:ZooKeeperServer@1047] - Refusing session request for client /192.168.32.1:46682 as it has seen zxid 0x189d00000003 our last zxid is 0xc00000634 client must try another server
The text was updated successfully, but these errors were encountered:
These logs are only indicating that the size of the records returned from single poll, exceeds certain threshold.
The level of the logs should be "WARN", as there is no error happening and the messages are processed anyway.
However, consider decreasing TB_QUEUE_KAFKA_MAX_POLL_RECORDS, to avoid getting too big results from poll.
I use Kafka as my message queue, and after the service runs for a period of time, the tb_core service often encounters exceptions.
I have adjusted the parameters of the following two kafka, but it did not work.
In kafka.yml:
KAFKA_MESSAGE_MAX_BYTES: 33554432
In queue-kafka.env:
TB_KAFKA_MAX_REQUEST_SIZE=33554432
However, the following errors still occur:
2024-05-16+13:25:23.043 [kafka-coordinator-heartbeat-thread | tb-core-node] ERROR org.apache.kafka.common.network.NetworkReceive - Allocating buffer of size 26211464 for source 2
2024-05-16+13:25:23.043 [kafka-coordinator-heartbeat-thread | tb-core-node] ERROR org.apache.kafka.common.network.NetworkReceive - Stack Trace: java.base/java.lang.Thread.getStackTrace(Thread.java:1602)|org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:125)|org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:452)|org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:402)|org.apache.kafka.common.network.Selector.attemptRead(Selector.java:674)|org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:576)|org.apache.kafka.common.network.Selector.poll(Selector.java:481)|org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:560)|org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:265)|org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.pollNoWakeup(ConsumerNetworkClient.java:306)|org.apache.kafka.clients.consumer.internals.AbstractCoordinator$HeartbeatThread.run(AbstractCoordinator.java:1433)
Some error log from zookeeper, ip 192.168.32.1 looks like gateway ip of tb core and tb rule engine:
2024-05-16 14:22:43,194 [myid:1] - INFO [NIOWorkerThread-16:ZooKeeperServer@1047] - Refusing session request for client /192.168.32.1:52408 as it has seen zxid 0x189c0000179f our last zxid is 0xc00000634 client must try another server
2024-05-16 14:22:43,352 [myid:1] - INFO [NIOWorkerThread-29:ZooKeeperServer@1047] - Refusing session request for client /192.168.32.1:38874 as it has seen zxid 0x189d00000001 our last zxid is 0xc00000634 client must try another server
2024-05-16 14:22:43,502 [myid:1] - INFO [NIOWorkerThread-11:ZooKeeperServer@1047] - Refusing session request for client /192.168.32.1:38876 as it has seen zxid 0x189d00000001 our last zxid is 0xc00000634 client must try another server
2024-05-16 14:22:43,518 [myid:1] - INFO [NIOWorkerThread-32:ZooKeeperServer@1047] - Refusing session request for client /192.168.32.1:46678 as it has seen zxid 0x189d00000003 our last zxid is 0xc00000634 client must try another server
2024-05-16 14:22:43,791 [myid:1] - INFO [NIOWorkerThread-13:ZooKeeperServer@1047] - Refusing session request for client /192.168.32.1:51388 as it has seen zxid 0x189c0000179f our last zxid is 0xc00000634 client must try another server
2024-05-16 14:22:43,893 [myid:1] - INFO [NIOWorkerThread-3:ZooKeeperServer@1047] - Refusing session request for client /192.168.32.1:46682 as it has seen zxid 0x189d00000003 our last zxid is 0xc00000634 client must try another server
The text was updated successfully, but these errors were encountered: