Skip to content

Add ca_yaml_features macro: YAML-only features (time_dimensions, filters, data_type) in DDL - #31

Open
cletoigor wants to merge 1 commit into
Snowflake-Labs:mainfrom
cletoigor:feat/ddl-yaml-only-features
Open

Add ca_yaml_features macro: YAML-only features (time_dimensions, filters, data_type) in DDL#31
cletoigor wants to merge 1 commit into
Snowflake-Labs:mainfrom
cletoigor:feat/ddl-yaml-only-features

Conversation

@cletoigor

Copy link
Copy Markdown

Summary

Brings the semantic-view features that are expressible in the Cortex Analyst YAML spec but not in plain CREATE SEMANTIC VIEW DDL — time_dimensions, standalone filters, and data_type declarations (YAML vs. DDL) — into the DDL path.

  • Adds a ca_yaml_features Jinja macro (macros/ca_yaml_features.sql) that generates the WITH EXTENSION (CA=$$...$$) clause carrying these fields, grouping entries by table and emitting only populated keys.
  • Wires the same three inputs through config keys (time_dimensions, ca_filters, ca_dimensions) in macros/relations/semantic_view/create.sql, so both an inline-macro form and a config-driven form work. A guard raises a compiler error if config-driven usage is combined with a hand-written WITH EXTENSION clause (a single CA extension is allowed). Runs before the COPY GRANTS append so COPY GRANTS stays last.
  • Adds integration-test models for both forms (semantic_view_with_yaml_features.sql, semantic_view_with_yaml_features_config.sql) and a get_ddl-based data test (semantic_view_yaml_features_has_extension.sql).
  • Adds an offline Jinja render test (scripts/test_ca_yaml_features.py, only requires jinja2) that exercises the real macro with no Snowflake/dbt dependency.
  • Documents the feature in the README under a new "YAML-only features in DDL" section with a parameter table and both usage forms.

Motivation

Per Snowflake's YAML vs. DDL comparison, a few semantic-view features live only in the Cortex Analyst YAML spec: time_dimensions (a distinct category for date/timestamp columns), standalone filters (named table-level filter expressions), and explicit data_type declarations. In Snowflake these fields are carried in the CA extension, which a DDL-created view can attach via WITH EXTENSION (CA=$$ ...json... $$).

The existing semantic_view_with_ca_extension.sql integration test shows that users currently have to hand-write raw, escaped JSON in the with extension clause to reach these fields — error-prone and hard to maintain. This macro lets users declare them as structured Jinja dicts and relies on | tojson for correct serialization, bringing the YAML-only feature set into the package's DDL path.

Usage

Inline:

{{ config(materialized='semantic_view') }}
TABLES(orders AS {{ ref('orders') }})
DIMENSIONS(orders.status AS status)
METRICS(orders.total AS SUM(orders.amount))
{{ dbt_semantic_view.ca_yaml_features(
    time_dimensions=[{'table': 'orders', 'name': 'order_ts', 'expr': 'ORDER_TS', 'data_type': 'TIMESTAMP_NTZ'}],
    filters=[{'table': 'orders', 'name': 'recent', 'expr': 'order_ts > dateadd(day, -30, current_date)'}],
    dimensions=[{'table': 'orders', 'name': 'amount', 'expr': 'AMOUNT', 'data_type': 'NUMBER(38,2)'}]
) }}

Config-driven (equivalent result):

{{ config(
    materialized='semantic_view',
    time_dimensions=[{'table': 'orders', 'name': 'order_ts', 'expr': 'ORDER_TS', 'data_type': 'TIMESTAMP_NTZ'}],
    ca_filters=[{'table': 'orders', 'name': 'recent', 'expr': 'order_ts > dateadd(day, -30, current_date)'}],
    ca_dimensions=[{'table': 'orders', 'name': 'amount', 'expr': 'AMOUNT', 'data_type': 'NUMBER(38,2)'}]
) }}
TABLES(orders AS {{ ref('orders') }})
DIMENSIONS(orders.status AS status)
METRICS(orders.total AS SUM(orders.amount))

Testing

Offline render test (python scripts/test_ca_yaml_features.py, requires only jinja2) — 9/9 checks passed. Sample output:

WITH EXTENSION (CA=$${"tables": [{"name": "orders", "time_dimensions": [{"name": "order_ts", "expr": "ORDER_TS", "data_type": "TIMESTAMP_NTZ"}], "dimensions": [{"name": "amount", "expr": "AMOUNT", "data_type": "NUMBER(38,2)"}], "filters": [{"name": "recent", "expr": "order_ts > dateadd(day, -30, current_date)"}]}]}$$)

Checks cover: WITH EXTENSION (CA=$$...$$) wrapping, per-table grouping, time_dimensions with data_type, standalone filters, data_type on a regular dimension, stripping of the routing-only table key, empty-input → '', and the missing-table-key compiler error.

Live Snowflake — verified end-to-end on a Snowflake sandbox schema via dbt build:

  • semantic_view_with_yaml_features (inline form) — created successfully
  • semantic_view_with_yaml_features_config (config-driven form) — created successfully
  • semantic_view_yaml_features_has_extensionpassed, confirming via get_ddl that the live CREATE SEMANTIC VIEW DDL actually contains the CA extension with time_dimensions / filters / data_type (not just the offline-rendered string)

The live run selected the new models plus ancestors. A full dbt build --target snowflake (no --select) is still worth running in CI to confirm nothing else regressed; note the unrelated semantic_view_sum_matches_base_table test depends on semantic_view_basic and will error only if that ancestor is out of the selected scope.

Bring the semantic-view features that are expressible in the Cortex
Analyst YAML spec but not in plain CREATE SEMANTIC VIEW DDL
(time_dimensions, standalone filters, and data_type declarations) into
the DDL path by generating a WITH EXTENSION (CA=$$...$$) clause from
structured dicts instead of hand-written JSON.

- macros/ca_yaml_features.sql: core macro, groups entries by table and
  emits only populated keys.
- macros/relations/semantic_view/create.sql: config-driven wiring
  (time_dimensions / ca_filters / ca_dimensions) with a guard against
  combining with a hand-written WITH EXTENSION clause.
- integration_tests: inline and config-driven models plus a get_ddl
  data test asserting the extension is present.
- scripts/test_ca_yaml_features.py: offline Jinja render test (no
  Snowflake/dbt required) exercising the real macro.
- README: new "YAML-only features in DDL" section.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant