Skip to content

[WIP] Add support for spark-connect #56

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
+76 −15
Open

[WIP] Add support for spark-connect #56

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
05b8cf8
try to add spark-connect test
lu-wang-dl Jun 13, 2024
3a74ef5
revert
lu-wang-dl Jun 13, 2024
a4b8896
Update test-spark-connect.yml
lu-wang-dl Jul 18, 2024
a9d8ae0
Update test-spark-connect.yml
lu-wang-dl Jul 18, 2024
22cab48
Update test-spark-connect.yml
lu-wang-dl Jul 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Loading
Loading status checks…
Update test-spark-connect.yml
  • Loading branch information
@lu-wang-dl
lu-wang-dl authored Jul 18, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode
commit a4b8896e073f7042a2f421cef4ce1f2952f8a365
2 changes: 1 addition & 1 deletion .github/workflows/test-spark-connect.yml
View file
Original file line number Diff line number Diff line change
@@ -21,7 +21,7 @@ jobs:
- name: Install python packages
run: |
pip install joblib==${{ matrix.JOBLIB_VERSION }} scikit-learn>=0.23.1 pytest pylint
pip install 'numpy==1.25.1' 'pyarrow==12.0.1' 'pandas<=2.0.3'
pip install setuptools 'numpy==1.25.1' 'pyarrow==12.0.1' 'pandas<=2.0.3'
# Add Python deps for Spark Connect.
pip install 'grpcio>=1.48,<1.57' 'grpcio-status>=1.48,<1.57' 'protobuf==3.20.3' 'googleapis-common-protos==1.56.4'
pip install "pyspark[connect]==${{ matrix.PYSPARK_VERSION }}"