Skip to content

Release v1.39.1 #3806

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Release v1.39.1 #3806

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
aabd25a
RELEASE ONLY disable hybrid mode
sfc-gh-joshi Sep 16, 2025
2f8342e
Prepare for release-v1.39.1
sfc-gh-jrose Sep 24, 2025
e984327
SNOW-2360274: Fix schema query sql generation for structured types (#…
sfc-gh-jrose Sep 25, 2025
465f7d2
NO SNOW: pin databricks-sql-connector version (#3802)
sfc-gh-yuwang Sep 24, 2025
7c015e3
Update recipe/meta
sfc-gh-jrose Sep 26, 2025
d3a5585
update version.py
sfc-gh-jrose Sep 26, 2025
47c7113
update ast
sfc-gh-jrose Sep 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
# Release History

## 1.40.0 (YYYY-MM-DD)
## 1.39.1 (2024-09-25)

### Snowpark Python API Updates

#### New Features
#### Bug Fixes

- Added an experimental fix for a bug in schema query generation that could cause invalid sql to be genrated when using nested structured types.
Copy link
Contributor

@graphite-app graphite-app bot Sep 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a typo in the changelog entry: "genrated" should be "generated".

Suggested change
- Added an experimental fix for a bug in schema query generation that could cause invalid sql to be genrated when using nested structured types.
- Added an experimental fix for a bug in schema query generation that could cause invalid sql to be generated when using nested structured types.

Spotted by Diamond

Fix in Graphite

Is this helpful? React 👍 or 👎 to let us know.


## 1.39.0 (YYYY-MM-DD)

Expand Down Expand Up @@ -73,7 +75,6 @@

#### Improvements

- Hybrid execution mode is now enabled by default. Certain operations on smaller data will now automatically execute in native pandas in-memory. Use `from modin.config import AutoSwitchBackend; AutoSwitchBackend.disable()` to turn this off and force all execution to occur in Snowflake.
- Downgraded to level `logging.DEBUG - 1` the log message saying that the
Snowpark `DataFrame` reference of an internal `DataFrameReference` object
has changed.
Expand Down
2 changes: 1 addition & 1 deletion recipe/meta.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{% set name = "snowflake-snowpark-python" %}
{% set version = "1.38.0" %}
{% set version = "1.39.1" %}
{% set noarch_build = (os.environ.get('SNOWFLAKE_SNOWPARK_PYTHON_NOARCH_BUILD', 'false')) == 'true' %}
{% set build_number = os.environ.get('SNOWFLAKE_SNOWPARK_PYTHON_BUILD_NUMBER', 0) %}

Expand Down
23 changes: 17 additions & 6 deletions src/snowflake/snowpark/_internal/analyzer/datatype_mapper.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
from decimal import Decimal
from typing import Any

import snowflake.snowpark.context as context
import snowflake.snowpark._internal.analyzer.analyzer_utils as analyzer_utils
from snowflake.snowpark._internal.type_utils import convert_sp_to_sf_type
from snowflake.snowpark._internal.utils import (
Expand Down Expand Up @@ -518,18 +519,26 @@ def schema_expression(data_type: DataType, is_nullable: bool) -> str:
if isinstance(data_type, ArrayType):
if data_type.structured:
assert data_type.element_type is not None
element = schema_expression(data_type.element_type, data_type.contains_null)
if context._enable_fix_2360274:
element = "NULL"
else:
element = schema_expression(
data_type.element_type, data_type.contains_null
)
return f"to_array({element}) :: {convert_sp_to_sf_type(data_type)}"
return "to_array(0)"
if isinstance(data_type, MapType):
if data_type.structured:
assert data_type.key_type is not None and data_type.value_type is not None
# Key values can never be null
key = schema_expression(data_type.key_type, False)
# Value nullability is variable. Defaults to True
value = schema_expression(
data_type.value_type, data_type.value_contains_null
)
if context._enable_fix_2360274:
value = "NULL"
else:
# Value nullability is variable. Defaults to True
value = schema_expression(
data_type.value_type, data_type.value_contains_null
)
return f"object_construct_keep_null({key}, {value}) :: {convert_sp_to_sf_type(data_type)}"
return "to_object(parse_json('0'))"
if isinstance(data_type, StructType):
Expand All @@ -539,7 +548,9 @@ def schema_expression(data_type: DataType, is_nullable: bool) -> str:
# Even if nulls are allowed the cast will fail due to schema mismatch when passed a null field.
schema_strings += [
f"'{field.name}'",
schema_expression(field.datatype, is_nullable=False),
"NULL"
if context._enable_fix_2360274
else schema_expression(field.datatype, is_nullable=False),
]
return f"object_construct_keep_null({', '.join(schema_strings)}) :: {convert_sp_to_sf_type(data_type)}"
return "to_object(parse_json('{}'))"
Expand Down
4 changes: 4 additions & 0 deletions src/snowflake/snowpark/context.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,10 @@
# This is an internal-only global flag, used to determine whether to enable query line tracking for tracing sql compilation errors.
_enable_trace_sql_errors_to_dataframe = False

# SNOW-2362050: Enable this fix by default.
# Global flag for fix 2360274. When enabled schema queries will use NULL as a place holder for any values inside structured objects
_enable_fix_2360274 = False


def configure_development_features(
*,
Expand Down
13 changes: 1 addition & 12 deletions src/snowflake/snowpark/modin/plugin/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
import inspect
import sys
from typing import Union, Callable, Any
import warnings

from packaging import version

Expand Down Expand Up @@ -174,19 +173,9 @@
)
from modin.config import AutoSwitchBackend # isort: skip # noqa: E402

HYBRID_WARNING = (
"Snowpark pandas now runs with hybrid execution enabled by default, and will perform certain operations "
+ "on smaller data using local, in-memory pandas. To disable this behavior and force all computations to occur in "
+ "Snowflake, run this line:\nfrom modin.config import AutoSwitchBackend; AutoSwitchBackend.disable()"
)

warnings.filterwarnings("once", message=HYBRID_WARNING)

if AutoSwitchBackend.get_value_source() is ValueSource.DEFAULT:
AutoSwitchBackend.enable()

if AutoSwitchBackend.get():
warnings.warn(HYBRID_WARNING, stacklevel=1)
AutoSwitchBackend.disable()

# Hybrid Mode Registration
# In hybrid execution mode, the client will automatically switch backends when a
Expand Down
2 changes: 1 addition & 1 deletion src/snowflake/snowpark/version.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@


# Update this for the versions
VERSION = (1, 38, 0)
VERSION = (1, 39, 1)
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.agg.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -508,6 +508,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.ai.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2146,6 +2146,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.col_ilike.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.collect.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -511,6 +511,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.count.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.count2.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -315,6 +315,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.create_or_replace.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -580,6 +580,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.cross_join.lsuffix.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.cross_join.rsuffix.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.cross_join.suffix.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.describe.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.flatten.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.indexers.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.join.inner.column.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.join.inner.column_list.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.join.inner.column_list_predicate.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -326,6 +326,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.join.inner.predicate.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -287,6 +287,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.join.inner.predicate_rsuffix.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -277,6 +277,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.join.left_outer.column.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.join.right_outer.predicate.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.join_table_function.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.natural_join.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.pivot.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -746,6 +746,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.select_expr.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -359,6 +359,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.stat.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1647,6 +1647,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.to_df.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.to_local_iterator.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -539,6 +539,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.to_pandas.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -298,6 +298,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.to_pandas_batch.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -298,6 +298,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
3 changes: 2 additions & 1 deletion tests/ast/data/DataFrame.unpivot.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,7 @@ client_language {
}
client_version {
major: 1
minor: 38
minor: 39
patch: 1
}
id: "\003U\"\366q\366P\346\260\261?\234\303\254\316\353"
Loading
Loading