Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

馃悰 [BUG] - valueSchema is always null #122

Open
1 task done
AishD3 opened this issue Dec 1, 2023 · 3 comments
Open
1 task done

馃悰 [BUG] - valueSchema is always null #122

AishD3 opened this issue Dec 1, 2023 · 3 comments
Labels

Comments

@AishD3
Copy link

AishD3 commented Dec 1, 2023

What happened?

I am passing below settings in my connector , but its always saying , valueSchema is null , Is my configs needs to be corrected.

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
name: mq-source-schema
labels:
strimzi.io/cluster: cdt-connect-shared-emp
spec:
class: com.ibm.eventstreams.connect.mqsource.MQSourceConnector
config:
key.converter: io.confluent.connect.avro.AvroConverter
mq.channel.name: DEV.APP.SVRCONN
mq.connection.name.list: secureapphelm-ibm-mq-qm(1414)
mq.password: XXXXXX
mq.queue: DEV.LOCAL.QUEUE.SOURCE.SCHEMA
mq.queue.manager: secureapphelm
mq.record.builder: com.ibm.eventstreams.connect.mqsource.builders.JsonRecordBuilder
mq.ssl.use.ibm.cipher.mappings: false
mq.user.name: admin
tasks.max: 1
topic: MSk.emp.retinatoolkit.schema.test.v1
value.converter: io.confluent.connect.avro.AvroConverter
mq.message.body.jms: false
value.converter.schema.registry.ssl.key.password: XXXXXXX
value.converter.schema.registry.ssl.keystore.location: XXXXXXX/keystore.jks
value.converter.schema.registry.ssl.keystore.password: XXXXXX
value.converter.schema.registry.ssl.truststore.location: /XXXXXXs/truststore.jks
value.converter.schema.registry.ssl.truststore.password: XXXXXXX
value.converter.schema.registry.url: https://XXXXXa-sr.maersk-digital.net
value.converter.schemas.enable: true
key.converter.schema.registry.ssl.key.password: XXXXXX
key.converter.schema.registry.ssl.keystore.location: /XXXXXX/keystore.jks
key.converter.schema.registry.ssl.keystore.password: XXXXXX
key.converter.schema.registry.ssl.truststore.location: /XXXXX-certs/truststore.jks
key.converter.schema.registry.ssl.truststore.password: XXXXXXX
key.converter.schema.registry.url: https://XXXXXXnet
key.converter.schemas.enable: false
errors.retry.timeout: 3
errors.retry.delay.max.ms: 10
errors.tolerance: all
errors.log.enable: true
errors.log.include.messages: true
auto.register.schemas: true
transforms: ExtractFieldAndOverrideSchema
transforms.ExtractFieldAndOverrideSchema.type: io.maersk.kafka.connect.ExtractFieldAndOverrideSchema
transforms.ExtractFieldAndOverrideSchema.field.name: text
transforms.ExtractFieldAndOverrideSchema.schema.registry.ssl.key.password: XXXXXXXX
transforms.ExtractFieldAndOverrideSchema.schema.registry.ssl.keystore.location: /XXXXX/sr-certs/keystore.jks
transforms.ExtractFieldAndOverrideSchema.schema.registry.ssl.keystore.password: XXXXXX
transforms.ExtractFieldAndOverrideSchema.schema.registry.ssl.protocol: TLS
transforms.ExtractFieldAndOverrideSchema.schema.registry.ssl.truststore.location: /opt/kafka/external-configuration/sr-certs/truststore.jks
transforms.ExtractFieldAndOverrideSchema.schema.registry.ssl.truststore.password: XXXXX
transforms.ExtractFieldAndOverrideSchema.schema.registry.url: https://XXXXX.digital.net
transforms.ExtractFieldAndOverrideSchema.field.bytes.encoding: UTF_8
transforms.ExtractFieldAndOverrideSchema.field.type: TEXT
transforms.ExtractFieldAndOverrideSchema.schema.cache.expiry.secs: 300
transforms.ExtractFieldAndOverrideSchema.schema.cache.max.size: 1000
transforms.ExtractFieldAndOverrideSchema.schema.key.enable: true
transforms.ExtractFieldAndOverrideSchema.schema.value.enable: true

Steps To Reproduce

Create Connector with above configs and get ValueSchema

Environment

- OS:
- Openshift:
- Kubernetes:
DEV

Anything else?

No response

Version

1.3.4 (Default)

Relevant log output

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct
@AishD3 AishD3 added the bug label Dec 1, 2023
@AishD3
Copy link
Author

AishD3 commented Dec 1, 2023

image

@AishD3
Copy link
Author

AishD3 commented Dec 1, 2023

ConnectRecord{topic='MSk.emp.retinatoolkit.schema.test.v1', kafkaPartition=null, key=null, keySchema=null, value=[B@29c96fef, valueSchema=Schema{BYTES}, timestamp=null, headers=ConnectHeaders(headers=)}. (org.apache.kafka.connect.runtime.errors.LogReporter) [task-thread-mq-source-schema-0]

Now ValueSchema is in Bytes , how to convert to String or Json or Avro

@jhughes24816
Copy link
Contributor

@AishD3 can you provide more detail as to what you are trying to do please?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants