Add ca_yaml_features macro: YAML-only features (time_dimensions, filters, data_type) in DDL - #31
Open
cletoigor wants to merge 1 commit into
Open
Conversation
Bring the semantic-view features that are expressible in the Cortex Analyst YAML spec but not in plain CREATE SEMANTIC VIEW DDL (time_dimensions, standalone filters, and data_type declarations) into the DDL path by generating a WITH EXTENSION (CA=$$...$$) clause from structured dicts instead of hand-written JSON. - macros/ca_yaml_features.sql: core macro, groups entries by table and emits only populated keys. - macros/relations/semantic_view/create.sql: config-driven wiring (time_dimensions / ca_filters / ca_dimensions) with a guard against combining with a hand-written WITH EXTENSION clause. - integration_tests: inline and config-driven models plus a get_ddl data test asserting the extension is present. - scripts/test_ca_yaml_features.py: offline Jinja render test (no Snowflake/dbt required) exercising the real macro. - README: new "YAML-only features in DDL" section. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Brings the semantic-view features that are expressible in the Cortex Analyst YAML spec but not in plain
CREATE SEMANTIC VIEWDDL —time_dimensions, standalonefilters, anddata_typedeclarations (YAML vs. DDL) — into the DDL path.ca_yaml_featuresJinja macro (macros/ca_yaml_features.sql) that generates theWITH EXTENSION (CA=$$...$$)clause carrying these fields, grouping entries by table and emitting only populated keys.time_dimensions,ca_filters,ca_dimensions) inmacros/relations/semantic_view/create.sql, so both an inline-macro form and a config-driven form work. A guard raises a compiler error if config-driven usage is combined with a hand-writtenWITH EXTENSIONclause (a single CA extension is allowed). Runs before theCOPY GRANTSappend soCOPY GRANTSstays last.semantic_view_with_yaml_features.sql,semantic_view_with_yaml_features_config.sql) and aget_ddl-based data test (semantic_view_yaml_features_has_extension.sql).scripts/test_ca_yaml_features.py, only requiresjinja2) that exercises the real macro with no Snowflake/dbt dependency.Motivation
Per Snowflake's YAML vs. DDL comparison, a few semantic-view features live only in the Cortex Analyst YAML spec:
time_dimensions(a distinct category for date/timestamp columns), standalonefilters(named table-level filter expressions), and explicitdata_typedeclarations. In Snowflake these fields are carried in the CA extension, which a DDL-created view can attach viaWITH EXTENSION (CA=$$ ...json... $$).The existing
semantic_view_with_ca_extension.sqlintegration test shows that users currently have to hand-write raw, escaped JSON in thewith extensionclause to reach these fields — error-prone and hard to maintain. This macro lets users declare them as structured Jinja dicts and relies on| tojsonfor correct serialization, bringing the YAML-only feature set into the package's DDL path.Usage
Inline:
{{ config(materialized='semantic_view') }} TABLES(orders AS {{ ref('orders') }}) DIMENSIONS(orders.status AS status) METRICS(orders.total AS SUM(orders.amount)) {{ dbt_semantic_view.ca_yaml_features( time_dimensions=[{'table': 'orders', 'name': 'order_ts', 'expr': 'ORDER_TS', 'data_type': 'TIMESTAMP_NTZ'}], filters=[{'table': 'orders', 'name': 'recent', 'expr': 'order_ts > dateadd(day, -30, current_date)'}], dimensions=[{'table': 'orders', 'name': 'amount', 'expr': 'AMOUNT', 'data_type': 'NUMBER(38,2)'}] ) }}Config-driven (equivalent result):
{{ config( materialized='semantic_view', time_dimensions=[{'table': 'orders', 'name': 'order_ts', 'expr': 'ORDER_TS', 'data_type': 'TIMESTAMP_NTZ'}], ca_filters=[{'table': 'orders', 'name': 'recent', 'expr': 'order_ts > dateadd(day, -30, current_date)'}], ca_dimensions=[{'table': 'orders', 'name': 'amount', 'expr': 'AMOUNT', 'data_type': 'NUMBER(38,2)'}] ) }} TABLES(orders AS {{ ref('orders') }}) DIMENSIONS(orders.status AS status) METRICS(orders.total AS SUM(orders.amount))Testing
Offline render test (
python scripts/test_ca_yaml_features.py, requires onlyjinja2) — 9/9 checks passed. Sample output:Checks cover:
WITH EXTENSION (CA=$$...$$)wrapping, per-table grouping,time_dimensionswithdata_type, standalonefilters,data_typeon a regular dimension, stripping of the routing-onlytablekey, empty-input →'', and the missing-table-key compiler error.Live Snowflake — verified end-to-end on a Snowflake sandbox schema via
dbt build:semantic_view_with_yaml_features(inline form) — created successfullysemantic_view_with_yaml_features_config(config-driven form) — created successfullysemantic_view_yaml_features_has_extension— passed, confirming viaget_ddlthat the liveCREATE SEMANTIC VIEWDDL actually contains the CA extension withtime_dimensions/filters/data_type(not just the offline-rendered string)The live run selected the new models plus ancestors. A full
dbt build --target snowflake(no--select) is still worth running in CI to confirm nothing else regressed; note the unrelatedsemantic_view_sum_matches_base_tabletest depends onsemantic_view_basicand will error only if that ancestor is out of the selected scope.