Skip to content

[BUG] Spark Connect smoke test uses a pip package pyspark[connect] with jars #13718

@gerashegalov

Description

@gerashegalov

Describe the bug
We ended up using a package with JVM dependencies due to a stale pre-4.0 doc apache/spark#52901. It should be replaced with pyspark-client . The point of the test to make sure it is a pure Python app.

Additional context
https://github.com/NVIDIA/spark-rapids-examples/blob/main/examples/spark-connect-gpu/spark-connect-client/requirements.txt#L21

apache/spark#52901

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions