Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improve][Connector] Add split progress in jdbc/mongo #389

Open
1 of 2 tasks
lichang-bd opened this issue Feb 13, 2023 · 5 comments
Open
1 of 2 tasks

[Improve][Connector] Add split progress in jdbc/mongo #389

lichang-bd opened this issue Feb 13, 2023 · 5 comments
Assignees
Labels
difficulty-easy Easy difficulty to fix this issue good first issue Good for newcomers

Comments

@lichang-bd
Copy link
Member

lichang-bd commented Feb 13, 2023

Description

In the Batch scenario, task progress can be measured according to Complete Split/ Total Split, Especially in JDBC and Mongo scenarios, we can add split processing information

1000 can be changed to parameter configuration
image

JDBC lack of split progress information in log
image

image

BitSail Component or Code Module

BitSail Connector

Are you willing to submit PR?

  • Yes, I am willing to submit a PR!

Code of Conduct

@lichang-bd lichang-bd added difficulty-easy Easy difficulty to fix this issue good first issue Good for newcomers labels Feb 13, 2023
@CodingGPT
Copy link

please assign to me, thanks

@BlockLiu BlockLiu assigned lichang-bd and CodingGPT and unassigned lichang-bd Feb 15, 2023
@yucongcong654
Copy link
Collaborator

Hi CodingGPT, thanks for your contribution to the BitSail community. we build a group for all contributor.Here is the QR code. If it expires, you can add WeChat ID 13651994967. Look forward to your arrival! @CodingGPT
1321676443482_ pic

@lichang-bd
Copy link
Member Author

how about this issue, any problem about this, we can disscuss @CodingGPT

beyond-up added a commit to beyond-up/bitsail that referenced this issue Mar 3, 2023
@beyond-up
Copy link
Contributor

beyond-up commented Mar 3, 2023

Sorry I accidentally added the comment above and I can not delete it , please delete it for me, thx! @lichang-bd

@CodingGPT
Copy link

I am testing, found range and currentTaskId is null, the log like this
com.bytedance.bitsail.connector.legacy.mongodb.source.MongoDBInputFormat [] - Start to process range info: [null,null] [0/1]

How to debug this program in idea on my local machine?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
difficulty-easy Easy difficulty to fix this issue good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

4 participants