Exit code 1 for multiple containers, missing Dockerfile

[marcin2x4]

Below services end with Exit 1 code:

• broker
• ksql-datagen
• rest-proxy

I tried adding memory limit but no luck, should these be placed in specific line of docker-compose file?

deploy:
      resources:
        limits:
          cpus: xxx
          memory: xxx
        reservations:
          cpus: xxx
          memory: xxx

Also I would suggest to add to each docker-compose a Dockerfile as well.

[ybyzek]

@marcin2x4 what are the error messages? docker-compose logs <name of container>

[marcin2x4]

@ybyzek - very long ;) Shall I zip them?

[ybyzek]

@marcin2x4 can you view the log and pull out the ERROR messages?

[marcin2x4]

@ybyzek - as I ran docker-compose logs schema-registry I get below errors:

schema-registry | org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata, deadlineMs=1606854333491) timed out at 1606854333492 after 1 attempt(s)

schema-registry | Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call.

schema-registry | [main] ERROR io.confluent.admin.utils.ClusterStatus - Error while getting broker list.

[marcin2x4]

confluent_exit1_error.txt
@ybyzek - file attached for schema-registry

[ybyzek]

The last part of the Schema Registry log suggests an issue with the broker? What do broker logs say?

�[36mschema-registry    |�[0m [kafka-admin-client-thread | adminclient-1] INFO org.apache.kafka.clients.admin.internals.AdminMetadataManager - [AdminClient clientId=adminclient-1] Metadata update failed
�[36mschema-registry    |�[0m org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata, deadlineMs=1607371198629) timed out at 1607371198630 after 1 attempt(s)
�[36mschema-registry    |�[0m Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call.
�[36mschema-registry    |�[0m [main] ERROR io.confluent.admin.utils.ClusterStatus - Error while getting broker list.
�[36mschema-registry    |�[0m java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Call(callName=listNodes, deadlineMs=1607371208630) timed out at 1607371208631 after 1 attempt(s)
�[36mschema-registry    |�[0m 	at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
�[36mschema-registry    |�[0m 	at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
�[36mschema-registry    |�[0m 	at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
�[36mschema-registry    |�[0m 	at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
�[36mschema-registry    |�[0m 	at io.confluent.admin.utils.ClusterStatus.isKafkaReady(ClusterStatus.java:149)
�[36mschema-registry    |�[0m 	at io.confluent.admin.utils.cli.KafkaReadyCommand.main(KafkaReadyCommand.java:150)
�[36mschema-registry    |�[0m Caused by: org.apache.kafka.common.errors.TimeoutException: Call(callName=listNodes, deadlineMs=1607371208630) timed out at 1607371208631 after 1 attempt(s)
�[36mschema-registry    |�[0m Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
�[36mschema-registry    |�[0m [kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 (broker/172.28.0.3:29092) could not be established. Broker may not be available.
�[36mschema-registry    |�[0m [main] INFO io.confluent.admin.utils.ClusterStatus - Expected 1 brokers but found only 0. Trying to query Kafka for metadata again ...
�[36mschema-registry    |�[0m [main] ERROR io.confluent.admin.utils.ClusterStatus - Expected 1 brokers but found only 0. Brokers found [].

[marcin2x4]

Due to size I ziped the thing.
broker_error_exit1.zip

[ybyzek]

Some errors from the logs:

^[[36mbroker             |^[[0m [2020-09-19 23:45:41,196] ERROR [Controller id=1] Error while electing or becoming controller on broker 1. Trigger controller movement immediately (kafka.controller.KafkaController)
^[[36mbroker             |^[[0m org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired
^[[36mbroker             |^[[0m [2020-09-19 23:50:51,291] ERROR Error while creating ephemeral at /brokers/ids/1, node already exists and owner '72063151865331727' does not match current session '72057602203713540' (kafka.zk.KafkaZkClient$CheckedEphemeral)

Is there another instance of Kafka or ZooKeeper running locally? (either Docker or local install)

[marcin2x4]

@ybyzek - no, only the one I mentioned. Here is dockerfile and docker-compose I use to run this environment.
https://github.com/marcin2x4/confluent_kafka_docker_image

[tooptoop4]

@marcin2x4 did u solve?