Skip to content

transaction: false model/seed hooks are not honored (silently skipped on default path, error on v2/seed/metric_view) #1549

Description

@sd-db

Describe the bug

A model or seed hook configured with transaction: false (e.g. pre_hook={'sql': '...', 'transaction': false}) is not run the way the user intends on dbt-databricks. The failure mode depends on the materialization path:

  • Default path (use_materialization_v2 = false, the default — table / view / incremental): the transaction: false hook is silently skipped. No error, no log line — it simply never executes.
  • use_materialization_v2 = true, and seeds, and metric views: the run fails with:
    [NO_ACTIVE_TRANSACTION] There is no active transaction to commit.
    

Default-transaction hooks (transaction: true or unset) work correctly in both cases. The model/seed config itself is parsed correctly — the issue is in how the materializations execute hooks.

Steps To Reproduce

A model with a transaction: false hook is enough. Minimal repro:

-- models/hook_demo.sql
{{ config(
    materialized='table',
    pre_hook=[
        "create table if not exists {{ this.schema }}.hook_audit (marker string)",
        {"sql": "insert into {{ this.schema }}.hook_audit values ('txn_true')",  "transaction": true},
        {"sql": "insert into {{ this.schema }}.hook_audit values ('txn_false')", "transaction": false}
    ]
) }}
select 1 as id
dbt run -s hook_demo
# then inspect:
#   select * from <schema>.hook_audit;
  • Default config: dbt run succeeds, but hook_audit contains only txn_true — the txn_false row is missing. The transaction: false hook never ran.
  • With flags: {use_materialization_v2: true} in dbt_project.yml (or for a seed / metric view): dbt run fails with [NO_ACTIVE_TRANSACTION] There is no active transaction to commit.

Expected behavior

Because Databricks has no transactions, a model's pre/post hooks should run regardless of their transaction setting — a transaction: false hook should execute exactly once, like a default hook. (This is the stated intent of dbt-databricks's run_pre_hooks / run_post_hooks override, which calls run_hooks for both inside_transaction passes.)

Screenshots and log output

use_materialization_v2 = true (or seed / metric view) path — the run errors on the commit; that dbt-core's run_hooks emits for the outside-transaction pass:

Databricks adapter: Exception while trying to execute query
        commit;
: [NO_ACTIVE_TRANSACTION] There is no active transaction to commit.
  ... com.databricks.sql.transaction.tahoe.DeltaUnsupportedOperationException:
      [NO_ACTIVE_TRANSACTION] There is no active transaction to commit.
  ... com.databricks.sql.transaction.tahoe.commands.CommitTransactionCommand.run(CommitTransactionCommand.scala:76)

Database Error in model hook_demo (models/hook_demo.sql)
  [NO_ACTIVE_TRANSACTION] There is no active transaction to commit.

Default path: no error; the transaction: false insert is simply absent from the generated SQL.

System information

The output of dbt --version:

Core:
  - installed: 1.11.11
Plugins:
  - databricks: 1.12.1
  - spark: 1.10.1

The operating system you're using: macOS (reproduced on a serverless SQL warehouse; the cause is SQL/macro-level, so any Databricks compute is affected)

The output of python --version: Python 3.10.20

Additional context

Root cause (with file references in this repo):

dbt-core's run_hooks(hooks, inside_transaction) splits hooks by their transaction flag (selectattr('transaction', 'equalto', inside_transaction); default True) and, for the inside_transaction=False pass, emits a literal commit; before the first hook. Databricks/Delta has no transactions and rejects a bare commit; ([NO_ACTIVE_TRANSACTION]).

dbt-databricks calls hooks two inconsistent ways:

Materialization Path Hook call transaction: false result
table.sql v2 (run_pre_hooks/run_post_hooks, lines 17/43) calls inside_transaction=False pass error (commit;)
table.sql v1 (default) run_hooks(pre_hooks) (45/77) default inside_transaction=True silently skipped
view.sql v2 (10/42) / v1 (45/69) same error / silent skip
incremental.sql v2 (30/92) / v1 (102/219) same error / silent skip
seeds.sql v1 (71/74,104/109) & v2 (29/51) both reach inside_transaction=False error
metric_view.sql always run_pre_hooks/run_post_hooks (8/38) calls inside_transaction=False pass error

use_materialization_v2 defaults to False (dbt/adapters/databricks/impl.py), so the silent-skip path is the default user experience. The commit; reaches the server as raw SQL via statementadd_queryhandle.execute("commit;"), bypassing the adapter's already-no-op commit().

Suggested fix: override the global run_hooks macro in dbt-databricks to run all hooks in a single pass with no transaction split and no commit; preamble (Databricks has no transaction to commit). This is the only single change that covers both failure modes everywhere, since the v1 else branches call run_hooks directly (not the run_pre_hooks/run_post_hooks wrappers). Overriding only the wrappers fixes the error paths but not the v1 silent-skip; a connection-layer commit; no-op fixes the error paths but not the silent-skip. The v1/v2 hook paths should also be unified for consistency.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions