Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Optimization][CDCSOURCE] Optimize the way cdcsource parses options #3284

Open
2 of 3 tasks
Zzm0809 opened this issue Mar 13, 2024 · 2 comments
Open
2 of 3 tasks

[Optimization][CDCSOURCE] Optimize the way cdcsource parses options #3284

Zzm0809 opened this issue Mar 13, 2024 · 2 comments
Assignees
Labels
Invalid Invalid Optimization Optimization function
Milestone

Comments

@Zzm0809
Copy link
Contributor

Zzm0809 commented Mar 13, 2024

Search before asking

  • I had searched in the issues and found no similar optimization requirement.

Description

目前整库同步中连接器的参数在代码中只能解析固定的一些配置, 这样存在用户定义的参数不在处理列表内, 导致参数无法正常传递给连接器, 针对这个问题有以下建议:

  1. 利用反射获取每个连接器的 options 类的每个配置项的 key 形成列表
  2. 拿到用户定义的 配置项列表, 进行匹配
  3. 匹配到的获取用户定义的值
  4. 匹配不到的跳过
  5. 最终将匹配到的所有 参数传递给对应的连接器

基于以上方式可以支持所有连接器的参数解析,而无需手动新增代码解析

简单来说就是:
连接器的所有 options - 用户指定的(包含 dinky 内置的) = 差集 -> 获取用户指定的配置项对应值-> 传给对应连接器

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@Zzm0809 Zzm0809 added Waiting for reply Waiting for reply Optimization Optimization function and removed Waiting for reply Waiting for reply labels Mar 13, 2024
Copy link

Hello @Zzm0809, this issue is about CDC/CDCSOURCE, so I assign it to @aiwenmo. If you have any questions, you can comment and reply.

你好 @Zzm0809, 这个 issue 是关于 CDC/CDCSOURCE 的,所以我把它分配给了 @aiwenmo。如有任何问题,可以评论回复。

@Zzm0809 Zzm0809 added this to the Roadmap milestone Mar 13, 2024
Copy link

github-actions bot commented May 1, 2024

Hello @, this issue has not been active for more than 30 days. This issue will be closed in 7 days if there is no response. If you have any questions, you can comment and reply.

你好 @, 这个 issue 30 天内没有活跃,7 天后将关闭,如需回复,可以评论回复。

@github-actions github-actions bot added the Invalid Invalid label May 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Invalid Invalid Optimization Optimization function
Projects
Status: ToDo
Development

No branches or pull requests

2 participants