unable to retrieve the schemes on on UI

[karthikvijayendra]

Hi @muralibasani/Team,

I'm trying to synchronize schemas associated with topics. From the backend logs I could see that klaw is able to sync the schems from confluent platfrom. However, when i try to see the same in UI i get below error and ui keeps loading and never retrives schema.

{"timestamp":"2024-01-02T07:28:24.175+00:00","status":500,"error":"Internal Server Error","path":"/getSchemaOfTopic"}

Attached are the debug logs. Request you to suggest on the issue.

Regards,
Karthik[
schema error debug logs.txt
](url)

[muralibasani]

Hey @karthikvijayendra , thank you for raising this issue. We will take a look and come back asap with a fix

[aindriu-aiven]

Hey @karthikvijayendra I was able to recreate this issue,

The root cause is that the tenantModel is not configured entirely. e.g. see below where the requestTopicsEnvironmentList is null

  "tenantModel" : {
    "tenantName" : "default",
    "baseSyncEnvironment" : "DEV",
    "orderOfTopicPromotionEnvsList" : [ "DEV", "TEST" ],
    "requestTopicsEnvironmentsList" : null
  }
}

This is a correctly configured one

{
  "tenantModel" : {
    "tenantName" : "default",
    "baseSyncEnvironment" : "DEV",
    "orderOfTopicPromotionEnvsList" : [ "DEV", "TEST" ],
    "requestTopicsEnvironmentsList" : [ "DEV" ],
    "baseSyncKafkaConnectCluster" : "DEV",
    "orderOfConnectorsPromotionEnvsList" : [ "DEV" ],
    "requestConnectorsEnvironmentsList" : [ "DEV" ]
  }
}

Please see the documentation on the tenant config for more details
https://www.klaw-project.io/docs/HowTo/kafka-cluster-migration/tenant-config

You can configure the above configurations under Dashboard -> Settings with a SUPERADMIN role or a user with the UPDATE_SERVERCONFIG permission.

Thanks,
Aindriú

[karthikvijayendra]

Thanks Aiven-Open/klaw and @Murali Basani ***@***.***> - With the above suggestion i was able to retrieve the schemas. Another one that i need to check is, as a workaround to confluent ACL application issue, we thought of capturing the CDC logs from mysql db and to run few scripts to provide the role bindings by filtering out the required requests. Do you think is it achievable? Also, need to change the current metadata store from H2 to mysql - Do we need to install mysql before making to changes to properties ? If not, could you please suggest the better way to change the metadata store from H2 to mysql. Regards, Karthik

…

On Tue, Jan 2, 2024 at 7:55 PM Aindriú Lavelle ***@***.***> wrote: Hey @karthikvijayendra <https://github.com/karthikvijayendra> I was able to recreate this issue, The root cause is that the tenantModel is not configured entirely. e.g. see below where the requestTopicsEnvironmentList is null "tenantModel" : { "tenantName" : "default", "baseSyncEnvironment" : "DEV", "orderOfTopicPromotionEnvsList" : [ "DEV", "TEST" ], "requestTopicsEnvironmentsList" : null } } This is a correctly configured one { "tenantModel" : { "tenantName" : "default", "baseSyncEnvironment" : "DEV", "orderOfTopicPromotionEnvsList" : [ "DEV", "TEST" ], "requestTopicsEnvironmentsList" : [ "DEV" ], "baseSyncKafkaConnectCluster" : "DEV", "orderOfConnectorsPromotionEnvsList" : [ "DEV" ], "requestConnectorsEnvironmentsList" : [ "DEV" ] } } Please see the documentation on the tenant config for more details https://www.klaw-project.io/docs/HowTo/kafka-cluster-migration/tenant-config *You can configure the above configurations under Dashboard -> Settings with a SUPERADMIN role or a user with the UPDATE_SERVERCONFIG permission.* Thanks, Aindriú — Reply to this email directly, view it on GitHub <#2156 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANNFNH4X46AUUMJGCSDVPVDYMQKGLAVCNFSM6AAAAABBJV3JUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZUGA4TCMZUGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[muralibasani]

@karthikvijayendra glad it worked. Thanks for mentioning.

Regarding moving to mysql or pg, just update the config ex like below (this example already exists in the repo and also for postgres)

#spring.datasource.url=jdbc:mysql://localhost:3306/kafkametadbpro?autoReconnect=true&useUnicode=true&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=UTC&cachePrepStmts=true&useServerPrepStmts=true&rewriteBatchedStatements=true&verifyServerCertificate=false&useSSL=false&requireSSL=false&allowPublicKeyRetrieval=true
#spring.datasource.username=kafkauser
#spring.datasource.password=kafkauser123
#spring.datasource.driver.class=com.mysql.cj.jdbc.Driver
#spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQLDialect
#spring.datasource.platform=mysql

[aindriu-aiven]

@karthikvijayendra just to add to what @muralibasani has mentioned there is an import export data function detailed here https://www.klaw-project.io/docs/HowTo/exportimport/

But I think it would be as quick and probably a bit easier to simply setup as you did your config for Klaw.

[karthikvijayendra]

Thanks Aindriu and Murali. Will check and get back. At this point, when i change the db parameters to point to mysql one, builds are failing. Will check on this. Thanks for yor support. Regards, Karthik

…

On Wed, Jan 3, 2024 at 2:23 PM Aindriú Lavelle ***@***.***> wrote: @karthikvijayendra <https://github.com/karthikvijayendra> just to add to what @muralibasani <https://github.com/muralibasani> has mentioned there is an import export data function detailed here https://www.klaw-project.io/docs/HowTo/exportimport/ But I think it would be as quick and probably a bit easier to simply setup as you did your config for Klaw. — Reply to this email directly, view it on GitHub <#2156 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANNFNHZRLXIJXI557CNCUJDYMUMAPAVCNFSM6AAAAABBJV3JUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZVGAZDMNBXHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[aindriu-aiven]

Hey @karthikvijayendra thats interesting, you can build without changing to mysql however if you wish and use external configuration to point to your mysql server after you deploy it, you can do this through environment variables or through specifying an external application.properties, or by creating a new application.properties and using spring profiles to use the mysql version in DEV,TEST,PRD etc.

What method for deployment are you using?

However when I get a chance I will check the mysql build as I actually tested this back on 2.5.1 and at that time it was good so it would be good to know and fix any potential issue that was added in since then

[aindriu-aiven]

Hey @karthikvijayendra did you have any success?