Skip to content

[BUG]: Sample deployment: sed -i '' substitution command not compatible with Linux and Windows Subsystem Linux #43

@Ben-Dowling

Description

@Ben-Dowling

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

In samples/common.sh, the use of sed -i '' (the macOS syntax for in-place edits) is not compatible with GNU sed on Linux and Windows Subsystem for Linux (WSL). This causes failures when running the substitution commands on those platforms.

As a result, the substitutions in the pipeline configuration files (e.g. dev_substitutions.json) are not applied, causing the sample pipeline to fail.

Expected Behavior

The substitution script works cross-platform (macOS, Linux, WSL).

Steps To Reproduce

  1. Clone or fork the repo on a Linux or WSL environment
  2. Run the samples/deploy.sh (I used Databricks CLI v0.297.2, on a MinGW terminal on Windows)
  3. Check the resultant pipeline config files (e.g. pipeline_configs/dev_substitutions.json), they have not received the correct substitutions.
  4. Run a sample pipeline (Lakeflow Framework - Run 1 - Load and Schema Initialization), dependant tasks (e.g. bronze_base_pipeline) fail.

Channel

CURRENT

Relevant log output

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions