Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TASK][MEDIUM] Setup GitHub Actions for Kyuubi Python client (former PyHive) #6281

Closed
2 of 3 tasks
Tracked by #6325
pan3793 opened this issue Apr 9, 2024 · 6 comments · May be fixed by #6341
Closed
2 of 3 tasks
Tracked by #6325

[TASK][MEDIUM] Setup GitHub Actions for Kyuubi Python client (former PyHive) #6281

pan3793 opened this issue Apr 9, 2024 · 6 comments · May be fixed by #6341
Assignees

Comments

@pan3793
Copy link
Member

pan3793 commented Apr 9, 2024

What's the level of this task?

MEDIUM

Code of Conduct

Search before creating

  • I have searched in the task list and found no similar tasks.

Mentor

  • I have sufficient expertise on this task, and I volunteer to be a mentor of this task to guide contributors through the task.

Skill requirements

  • GitHub Actions
  • Python/Shell scripts
  • Hadoop components deployment

Background and Goals

PyHive has been donated to Apache Kyuubi, the codebase was imported into main repo under ./python, previously, it used Travis for CI, now, we should migrate the workflow to GitHub Actions.

https://incubator.apache.org/ip-clearance/kyuubi-pyhive.html

Implementation steps

  • Migrate existing CI workflow to GitHub Actions
  • Add Spark Thrift Server / Kyuubi to CI workflow

Additional context

Introduction of 2024H1 Kyuubi Code Contribution Program

@sudohainguyen
Copy link
Contributor

I can help, please assign this PR to me

@pan3793
Copy link
Member Author

pan3793 commented Apr 12, 2024

@sudohainguyen thanks, assigned. And you are welcome to improve this new Python module~

@pan3793
Copy link
Member Author

pan3793 commented Apr 19, 2024

@sudohainguyen hi, do you have any progress on this task?

@sudohainguyen
Copy link
Contributor

working on it @pan3793
will have a PR this weekend

@sudohainguyen
Copy link
Contributor

sudohainguyen commented Apr 21, 2024

update: since most of the tasks were running on outdated Python versions and components versions including trino, presto, etc.., think we should implement entirely different ones
targeting primarily on python 3.9 and 3.10, will extend afterwards

@pan3793
Copy link
Member Author

pan3793 commented Apr 21, 2024

@sudohainguyen thanks for updating, starting from Python 3.9 and 3.10 SGTM

@pan3793 pan3793 closed this as completed in 9075fbb May 7, 2024
sudohainguyen added a commit to sudohainguyen/kyuubi that referenced this issue May 7, 2024
sudohainguyen added a commit to sudohainguyen/kyuubi that referenced this issue May 7, 2024
pan3793 pushed a commit that referenced this issue May 8, 2024
# 🔍 Description
## Issue References 🔗

This pull request fixes #6281

Following up to #6343 , this PR adds `pytest-xdist` plugin to speed up python client unit testing

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️
Running tests usually takes ~10 mins

#### Behavior With This Pull Request 🎉
Running tests can take up to 5 mins

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6369 from sudohainguyen/ci/speed-up-test.

Closes #6281

f0ce557 [Harry] [KYUUBI #6281][PY] Speed up testing with xdist plugin

Authored-by: Harry <quanghai.ng1512@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
sudohainguyen added a commit to sudohainguyen/kyuubi that referenced this issue May 10, 2024
sudohainguyen added a commit to sudohainguyen/kyuubi that referenced this issue May 10, 2024
pan3793 pushed a commit that referenced this issue May 15, 2024
# 🔍 Description
## Issue References 🔗

This pull request enables running hive test cases in python client, however there's one trivial case not covered yet and two others require a proper container setup

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️
Hive test disabled in #6343

#### Behavior With This Pull Request 🎉
Can cover hive test cases

#### Related Unit Tests
No

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6381 from sudohainguyen/ci/hive.

Closes #6281

a861382 [Harry] [KYUUBI #6281][PY] Enable hive test in python client

Authored-by: Harry <quanghai.ng1512@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment