From ac309c723e260e8bfd09cac9497f7c7e9ed31c0f Mon Sep 17 00:00:00 2001 From: Joe Clark Date: Tue, 14 Oct 2025 12:16:18 +0100 Subject: [PATCH 1/5] draft sandbox docs --- docs/build/workflows.md | 272 +++++++++++++++++++--------------------- 1 file changed, 130 insertions(+), 142 deletions(-) diff --git a/docs/build/workflows.md b/docs/build/workflows.md index f22bd14d61dd..fbf24985f3ae 100644 --- a/docs/build/workflows.md +++ b/docs/build/workflows.md @@ -1,203 +1,191 @@ --- -title: Workflows -sidebar_label: Workflows +title: sandboxes +sidebar_label: sandboxes --- -**Workflows** are automated processes or sets of instructions that accomplish a -task. In OpenFn configuration, a Workflow consists of a Trigger, Steps, and -Paths that define automation logic. Read on to learn how to configure Workflows. +sandboxes are a way to develop fixes and new features on workflows without +affecting live, or "in production", runs. -## Create Workflows +:::info sandboxes are new to OpenFn since October 2025. -To create a new Workflow in your Project: +At the time of writing sandboxes are under active development and testing. We +expect to be in full working order by the end of November 2025, but until then +we recommend not using them with live workflows. -1. Go to the **Workflows** page. -2. Click the **Create new workflow** button. -3. Give your Workflow a descriptive `Name` (e.g., `Register patients`, - `Refer cases`, `Monthly payroll`). -4. Choose your [Trigger](../build/triggers.md) -5. Edit your first [Step](../build/steps/steps.md) -6. Modify the [Path Condition](../build/paths.md), if needed, to define _when_ - the Workflow should proceed to the next Step. -7. Configure more Steps as needed +To access sandboxes, you'll need to enable the Experimental Features option in +your user settings. -Check out the video overview below to learn how to create a Workflow. - - - -### Merging branches and Skipping Steps +::: -The workflow builder allows branch merging and skipping steps. To merge two or -more steps into one step or to skip some steps: +A sandbox is essentially a clone of a project, with its own private history, +webhooks, cron triggers credentials and access rights. It also has its own +billing rules - so sandbox runs and AI tokens don't affect your main project. +Unlike most sandbox settings, the billing rules derive from the original +project, rather than duplicate them. -1. Hover on the step you want to merge or initiate a skip -2. You will see a plus icon -3. Click on the plus icon and drag to create a path -4. Drop the new path on the desired step in your workflow +The idea is that you can develop the workflow in total isolation from the main +project, and once you're done, merge (read as "push" or "promote") changes back. - +with a copy of each workflow, its own webhooks, cron triggers and -:::note Looping is not supported +:::tip Short-lived sandboxes -Looping workflows are not supported so you have to connect paths to downstream -steps. When using branching and skipping paths, you can use edge conditions like -with any other step. +sandboxes work best when they're short-lived, so right now they are destroyed +after merging. While you can create as many sandboxes as you like (subject to +your usage allowance), we recommend keeping the number low to reduce the risk of +merge conflicts. ::: -## Run Workflows +## Isolated Context -Workflows will run automatically when they are "enabled"—i.e., when their -trigger is turned on. A webhook trigger will run your workflow whenever a -request is received at that trigger's URL, and a cron trigger will run a -workflow whenever its cron schedule matches the current time. +A sandbox is an isolated copy of your original project with its own context. As +such, your sandbox has its own "private" copies of the following artefacts and +features: -:::info +- Workflows +- Dataclips +- Run History +- Subscription (run limits, AI credits and collection memory) +- Collections +- Settings -Please note that workflows are disabled by default. When you are ready to have -your workflow running, you have to manually enable the workflow. +By "private copy", what we mean is that changing a Workflow in a sandbox will +not affect the same-named Workflow in any other project or sandbox, or that a +Run in a sandbox will contribute to your usage in any other sandbox or project, +and so on. -::: +## Creating sandboxes -### Enabling or Disabling a Workflow +When you create a new sandbox, we basically create a total copy of your project. +Any changes made to the sandbox will not affect your main project workflows - so +you can experiment freely without breaking anything. -There are two ways to disable or enable a workflow in your OpenFn project: +To create a sandbox, enter a Project, click on sandboxes, and click on the +Create sandbox button. -1. via the workflow state toggle -2. via the workflow trigger +You'll need to set a name for the sandbox. This is unique within your project +and its sandboxes. If you're familiar with git, treat it like a branch name. +Otherwise, you can either give it a general name like `testing`, or name it for +a specific feature, like `new-patient-workflow`. -#### Via the workflow state toggle +You'll also need to set an Environment (see below). This configures all +credentials within the sandbox to use that environment variant. If you're not +sure, set the environment to `dev`. -You can enable or disable your workflow by using the toggle button located on -the corresponding row in the project workflows list or the toggle on the -navigation bar in the workflow canvas. +A color will be randomly selected to associate with the sandbox. You'll see this +color in the app UI while you're using the sandbox. You can select a different +color if you like. -The screenshot below shows an enabled workflow in the workflow list. +Click Create sandbox when you're ready. You'll automatically enter the sandbox. -![Via the workflow list](/img/workflow_list_toggle.webp) +## Viewing a sandbox -The screenshot below shows a disabled workflow in the workflow canvas. +To develop and test a sandbox, you need to enter it in the app from the +sandboxes menu. -![Via the workflow canvas](/img/workflow_canvas_toggle.webp) +When sandbox is active, the app will change color to help you understand what +you're looking at. [TODO] We also list the active sandbox in the breadcrumbs at +the top of the page, and in a banner on the Inspector. -#### Via the workflow trigger +Each sandbox has its own isolated Workflows, Subscription, History and settings. +As you click through the pages, you'll notice that your original project's +details are excluded. This is because your sandbox is an independent clone of +the original project. -To enable or disable a workflow via the workflow trigger, select the trigger -icon on the canvas and use the toggle in the configuration panel to toggle the -workflow state. +## Environments -![Enabled workflow in the trigger panel](/img/via-trigger-panel.webp) +Environments allow you to run a Workflow with a special set of credentials. This +lets you use development servers, modes and databases while building your +sandbox, without interfering with live production services. -### Manual Runs +Each sandbox has an associated environment. By default it's `main`, which +implies that this is your live production environment. But you can create an +environment like `dev` or `staging`. -Check out the video for a quick overview. +The environment is just a label. Each credential used in your workflow with have +a set of values associated with that environment. For example, when connecting +to DHIS2, your main credential will contain private login details. But your +`dev` environment might use the public sandbox and so contain a different +username and password. - +All environments are securely stored and encrypted within our database, so it's +perfectly safe to duplicate production credentials across multiple environments. -You can run a workflow manually in three ways: +For each Credential used in your workflow, you must ensure there is a value set +to match your sandbox environment. If you do not configure your credentials, the +Workflow will fail with clear instructions on how to correct it. -#### With an empty input +## Merging sandboxes -This is the default behavior and the input dataclip for your run will be `{}`. +Once you've finished making changes to your workflows, it's time to merge them +back into your main project. - +This is easy in the app: simply head to the Sandboxes page, find the Sandbox you +want to merge in the list, and click the Merge icon on the right-hand side. -#### With a custom input +You'll be prompted to select the target project or sandbox to merge into: pick +from the list and click Merge. Usually you'll want to merge into the original +project, which is selected by default. -You can type, copy/paste, or import (browse your file system or drag & drop) any -file with valid JSON. +When merging, we replace the contents of workflows in your project with those in +your sandbox. Any workflows which are not in the sandbox will be ignored. If you +rename a workflow in the sandbox, you'll see the new workflow appear in your +main project, and the original workflow will be left alone (you'll probably want +to delete that manually). - +Note that settings and options, like concurrency and data retention rules, are +not transferred in the merge, nor are historical runs or dataclips. Just the +Workflow contents. -#### With an existing input +After merging, the sandbox will be destroyed, along with its history and +dataclips. -You can pick from a list of previous inputs that were used to run this step. +:::tip - +You can also use the CLI to merge your changes locally, give them one final +test, and then deploy them to your main project. -### Named Dataclips +:::: -Dataclips (custom inputs, step results, webhook requests) can be named to make -them easier to find and use for testing. +## Conflicts -:::info Named dataclips aren't erased +If you've ever worked with a source version control system - like git or +Subversion - you'll be familiar with the idea of conflicts. -Named dataclips will not be removed alongside other project history when your -retention period is reached. They will be stored indefinitely. +A conflict occurs when you're trying to merge a Sandbox into your original +project (or another sandbox), and there are incompatible changes between them. -::: +Say you create a Sandbox from your main project and in the sandbox, you change +the adaptor of one step from `common` to `http`. And while you're making this +change, a colleague goes to the main project and sets the adaptor of the same +step to `salesforce`. -Assign your dataclip a name by clicking the label field. +What happens when you try and merge the sandbox? Should we preserve the original +change? Or accept the change in the sandbox? Or something else? - +Occasionally these conflicts are trivial to resolve and you might wonder what +all the fuss is about. But often they are complex, and it can be difficult or +impossible to automate a solution. -After assigning names to your inputs you can search for them by their name on -the search bar. +[TODO not implemented yet] When we detect a conflict like this, we'll show a +warning when you try and merge the Sandbox. You can choose to "force push" the +Sandbox and overwrite whatever changes happened on the target Project, or you +can cancel and resolve the conflict yourself. - +For now, the only way to resolve conflicts manually is to use the CLI to edit +your project locally, and pushed the resolved, final version up to the app. -Filter only named inputs by clicking the tag button. - - - -## Limit Concurrency - -Workflow **concurrency** is the number of runs that will be allowed for a given -workflow **_at the same time_**. In OpenFn, project owners and administrators -are able to limit the maximum number of the runs that can be executed at the -same time for a workflow. You might do this to ensure "one at a time" serial -processing or to keep a fast OpenFn workflow from overwhelming the API rate -limit of some other connected system. - -:::info +:::tip -Please check to make sure that your parallel execution is not disabled for your -project as it will override the workflow level concurrency limit. +We'll be adding better support for resolving conflicts soon. ::: -### What happens when concurrency limit is set on a workflow? +## Editing sandboxes Locally -When concurrency limit is configured for a workflow, the maximum number of runs -that are executed concurrently will not exceed the number that has been set for -the workflow. For example: - -- **Concurrency not set (or = 0)**: There's no artificial limit applied, and - this workflow will only be limited by the total computing power available - across your OpenFn installation. -- **Concurrency = 1**: Runs for this workflow will only take place 1-at-a-time. - Each run must _finish_ before the next run can start. -- **Concurrency = 2**: No more than 2 runs for this workflow can be executed at - a time and other runs will have stay `enqueued`. If runs "A", "B", and "C" are - all enqueued, "A" and "B" will start executing. Once "A" finishes, "C" will - start. (No more than 2-at-a-time.) - -### Setting Concurrency for a workflow - -Concurrency limits can be set via the workflow settings modal on the workflow -canvas. - -1. Click on the settings icon beside the save button on your workflow to open - the workflow settings -2. In the modal, enter the maximum concurrency limit -3. Click save. - -![Configuring Concurrency](/img/configuring-concurrency.webp) - -### Log Outputs - -For data security and compliance purposes, the log output of a workflow run can -be configured to disable logging `console.log()` statements. This can be done -via the workflow configuration modal by a project owner or administrator. - -1. Click on the settings icon. -2. In the modal, toggle the **Allow `console.log()` usage** switch to disable - logging `console.log()` statements. By default, this is enabled. - -![Configuring Log Outputs](/img/configuring-log-outputs.webp) +:::info -## Keyboard Shortcuts +Sandboxes are fully compatible with the CLI. -From the canvas you can perform certain common actions (e.g., save) using -keystrokes. Check out the full list of keyboard shortcuts -[here](/documentation/keyboard-shortcuts). +For more details, see these CLI docs (link to follow) From d56af975996e0b5a66f832e026a02af1e1d5d7da Mon Sep 17 00:00:00 2001 From: Joe Clark Date: Tue, 14 Oct 2025 12:18:31 +0100 Subject: [PATCH 2/5] fix nav --- docs/build/sandboxes.md | 191 ++++++++++++++++++++++++++++ docs/build/workflows.md | 272 +++++++++++++++++++++------------------- sidebars-main.js | 1 + 3 files changed, 334 insertions(+), 130 deletions(-) create mode 100644 docs/build/sandboxes.md diff --git a/docs/build/sandboxes.md b/docs/build/sandboxes.md new file mode 100644 index 000000000000..2875442e9d7d --- /dev/null +++ b/docs/build/sandboxes.md @@ -0,0 +1,191 @@ +--- +title: Sandboxes +sidebar_label: Sandboxes +--- + +sandboxes are a way to develop fixes and new features on workflows without +affecting live, or "in production", runs. + +:::info sandboxes are new to OpenFn since October 2025. + +At the time of writing sandboxes are under active development and testing. We +expect to be in full working order by the end of November 2025, but until then +we recommend not using them with live workflows. + +To access sandboxes, you'll need to enable the Experimental Features option in +your user settings. + +::: + +A sandbox is essentially a clone of a project, with its own private history, +webhooks, cron triggers credentials and access rights. It also has its own +billing rules - so sandbox runs and AI tokens don't affect your main project. +Unlike most sandbox settings, the billing rules derive from the original +project, rather than duplicate them. + +The idea is that you can develop the workflow in total isolation from the main +project, and once you're done, merge (read as "push" or "promote") changes back. + +with a copy of each workflow, its own webhooks, cron triggers and + +:::tip Short-lived sandboxes + +sandboxes work best when they're short-lived, so right now they are destroyed +after merging. While you can create as many sandboxes as you like (subject to +your usage allowance), we recommend keeping the number low to reduce the risk of +merge conflicts. + +::: + +## Isolated Context + +A sandbox is an isolated copy of your original project with its own context. As +such, your sandbox has its own "private" copies of the following artefacts and +features: + +- Workflows +- Dataclips +- Run History +- Subscription (run limits, AI credits and collection memory) +- Collections +- Settings + +By "private copy", what we mean is that changing a Workflow in a sandbox will +not affect the same-named Workflow in any other project or sandbox, or that a +Run in a sandbox will contribute to your usage in any other sandbox or project, +and so on. + +## Creating sandboxes + +When you create a new sandbox, we basically create a total copy of your project. +Any changes made to the sandbox will not affect your main project workflows - so +you can experiment freely without breaking anything. + +To create a sandbox, enter a Project, click on sandboxes, and click on the +Create sandbox button. + +You'll need to set a name for the sandbox. This is unique within your project +and its sandboxes. If you're familiar with git, treat it like a branch name. +Otherwise, you can either give it a general name like `testing`, or name it for +a specific feature, like `new-patient-workflow`. + +You'll also need to set an Environment (see below). This configures all +credentials within the sandbox to use that environment variant. If you're not +sure, set the environment to `dev`. + +A color will be randomly selected to associate with the sandbox. You'll see this +color in the app UI while you're using the sandbox. You can select a different +color if you like. + +Click Create sandbox when you're ready. You'll automatically enter the sandbox. + +## Viewing a sandbox + +To develop and test a sandbox, you need to enter it in the app from the +sandboxes menu. + +When sandbox is active, the app will change color to help you understand what +you're looking at. [TODO] We also list the active sandbox in the breadcrumbs at +the top of the page, and in a banner on the Inspector. + +Each sandbox has its own isolated Workflows, Subscription, History and settings. +As you click through the pages, you'll notice that your original project's +details are excluded. This is because your sandbox is an independent clone of +the original project. + +## Environments + +Environments allow you to run a Workflow with a special set of credentials. This +lets you use development servers, modes and databases while building your +sandbox, without interfering with live production services. + +Each sandbox has an associated environment. By default it's `main`, which +implies that this is your live production environment. But you can create an +environment like `dev` or `staging`. + +The environment is just a label. Each credential used in your workflow with have +a set of values associated with that environment. For example, when connecting +to DHIS2, your main credential will contain private login details. But your +`dev` environment might use the public sandbox and so contain a different +username and password. + +All environments are securely stored and encrypted within our database, so it's +perfectly safe to duplicate production credentials across multiple environments. + +For each Credential used in your workflow, you must ensure there is a value set +to match your sandbox environment. If you do not configure your credentials, the +Workflow will fail with clear instructions on how to correct it. + +## Merging sandboxes + +Once you've finished making changes to your workflows, it's time to merge them +back into your main project. + +This is easy in the app: simply head to the Sandboxes page, find the Sandbox you +want to merge in the list, and click the Merge icon on the right-hand side. + +You'll be prompted to select the target project or sandbox to merge into: pick +from the list and click Merge. Usually you'll want to merge into the original +project, which is selected by default. + +When merging, we replace the contents of workflows in your project with those in +your sandbox. Any workflows which are not in the sandbox will be ignored. If you +rename a workflow in the sandbox, you'll see the new workflow appear in your +main project, and the original workflow will be left alone (you'll probably want +to delete that manually). + +Note that settings and options, like concurrency and data retention rules, are +not transferred in the merge, nor are historical runs or dataclips. Just the +Workflow contents. + +After merging, the sandbox will be destroyed, along with its history and +dataclips. + +:::tip + +You can also use the CLI to merge your changes locally, give them one final +test, and then deploy them to your main project. + +:::: + +## Conflicts + +If you've ever worked with a source version control system - like git or +Subversion - you'll be familiar with the idea of conflicts. + +A conflict occurs when you're trying to merge a Sandbox into your original +project (or another sandbox), and there are incompatible changes between them. + +Say you create a Sandbox from your main project and in the sandbox, you change +the adaptor of one step from `common` to `http`. And while you're making this +change, a colleague goes to the main project and sets the adaptor of the same +step to `salesforce`. + +What happens when you try and merge the sandbox? Should we preserve the original +change? Or accept the change in the sandbox? Or something else? + +Occasionally these conflicts are trivial to resolve and you might wonder what +all the fuss is about. But often they are complex, and it can be difficult or +impossible to automate a solution. + +[TODO not implemented yet] When we detect a conflict like this, we'll show a +warning when you try and merge the Sandbox. You can choose to "force push" the +Sandbox and overwrite whatever changes happened on the target Project, or you +can cancel and resolve the conflict yourself. + +For now, the only way to resolve conflicts manually is to use the CLI to edit +your project locally, and pushed the resolved, final version up to the app. + +:::tip + +We'll be adding better support for resolving conflicts soon. + +::: + +## Editing sandboxes Locally + +:::info + +Sandboxes are fully compatible with the CLI. + +For more details, see these CLI docs (link to follow) diff --git a/docs/build/workflows.md b/docs/build/workflows.md index fbf24985f3ae..f22bd14d61dd 100644 --- a/docs/build/workflows.md +++ b/docs/build/workflows.md @@ -1,191 +1,203 @@ --- -title: sandboxes -sidebar_label: sandboxes +title: Workflows +sidebar_label: Workflows --- -sandboxes are a way to develop fixes and new features on workflows without -affecting live, or "in production", runs. +**Workflows** are automated processes or sets of instructions that accomplish a +task. In OpenFn configuration, a Workflow consists of a Trigger, Steps, and +Paths that define automation logic. Read on to learn how to configure Workflows. -:::info sandboxes are new to OpenFn since October 2025. +## Create Workflows -At the time of writing sandboxes are under active development and testing. We -expect to be in full working order by the end of November 2025, but until then -we recommend not using them with live workflows. +To create a new Workflow in your Project: -To access sandboxes, you'll need to enable the Experimental Features option in -your user settings. +1. Go to the **Workflows** page. +2. Click the **Create new workflow** button. +3. Give your Workflow a descriptive `Name` (e.g., `Register patients`, + `Refer cases`, `Monthly payroll`). +4. Choose your [Trigger](../build/triggers.md) +5. Edit your first [Step](../build/steps/steps.md) +6. Modify the [Path Condition](../build/paths.md), if needed, to define _when_ + the Workflow should proceed to the next Step. +7. Configure more Steps as needed -::: +Check out the video overview below to learn how to create a Workflow. + + + +### Merging branches and Skipping Steps -A sandbox is essentially a clone of a project, with its own private history, -webhooks, cron triggers credentials and access rights. It also has its own -billing rules - so sandbox runs and AI tokens don't affect your main project. -Unlike most sandbox settings, the billing rules derive from the original -project, rather than duplicate them. +The workflow builder allows branch merging and skipping steps. To merge two or +more steps into one step or to skip some steps: -The idea is that you can develop the workflow in total isolation from the main -project, and once you're done, merge (read as "push" or "promote") changes back. +1. Hover on the step you want to merge or initiate a skip +2. You will see a plus icon +3. Click on the plus icon and drag to create a path +4. Drop the new path on the desired step in your workflow -with a copy of each workflow, its own webhooks, cron triggers and + -:::tip Short-lived sandboxes +:::note Looping is not supported -sandboxes work best when they're short-lived, so right now they are destroyed -after merging. While you can create as many sandboxes as you like (subject to -your usage allowance), we recommend keeping the number low to reduce the risk of -merge conflicts. +Looping workflows are not supported so you have to connect paths to downstream +steps. When using branching and skipping paths, you can use edge conditions like +with any other step. ::: -## Isolated Context +## Run Workflows -A sandbox is an isolated copy of your original project with its own context. As -such, your sandbox has its own "private" copies of the following artefacts and -features: +Workflows will run automatically when they are "enabled"—i.e., when their +trigger is turned on. A webhook trigger will run your workflow whenever a +request is received at that trigger's URL, and a cron trigger will run a +workflow whenever its cron schedule matches the current time. -- Workflows -- Dataclips -- Run History -- Subscription (run limits, AI credits and collection memory) -- Collections -- Settings +:::info -By "private copy", what we mean is that changing a Workflow in a sandbox will -not affect the same-named Workflow in any other project or sandbox, or that a -Run in a sandbox will contribute to your usage in any other sandbox or project, -and so on. +Please note that workflows are disabled by default. When you are ready to have +your workflow running, you have to manually enable the workflow. -## Creating sandboxes +::: -When you create a new sandbox, we basically create a total copy of your project. -Any changes made to the sandbox will not affect your main project workflows - so -you can experiment freely without breaking anything. +### Enabling or Disabling a Workflow -To create a sandbox, enter a Project, click on sandboxes, and click on the -Create sandbox button. +There are two ways to disable or enable a workflow in your OpenFn project: -You'll need to set a name for the sandbox. This is unique within your project -and its sandboxes. If you're familiar with git, treat it like a branch name. -Otherwise, you can either give it a general name like `testing`, or name it for -a specific feature, like `new-patient-workflow`. +1. via the workflow state toggle +2. via the workflow trigger -You'll also need to set an Environment (see below). This configures all -credentials within the sandbox to use that environment variant. If you're not -sure, set the environment to `dev`. +#### Via the workflow state toggle -A color will be randomly selected to associate with the sandbox. You'll see this -color in the app UI while you're using the sandbox. You can select a different -color if you like. +You can enable or disable your workflow by using the toggle button located on +the corresponding row in the project workflows list or the toggle on the +navigation bar in the workflow canvas. -Click Create sandbox when you're ready. You'll automatically enter the sandbox. +The screenshot below shows an enabled workflow in the workflow list. -## Viewing a sandbox +![Via the workflow list](/img/workflow_list_toggle.webp) -To develop and test a sandbox, you need to enter it in the app from the -sandboxes menu. +The screenshot below shows a disabled workflow in the workflow canvas. -When sandbox is active, the app will change color to help you understand what -you're looking at. [TODO] We also list the active sandbox in the breadcrumbs at -the top of the page, and in a banner on the Inspector. +![Via the workflow canvas](/img/workflow_canvas_toggle.webp) -Each sandbox has its own isolated Workflows, Subscription, History and settings. -As you click through the pages, you'll notice that your original project's -details are excluded. This is because your sandbox is an independent clone of -the original project. +#### Via the workflow trigger -## Environments +To enable or disable a workflow via the workflow trigger, select the trigger +icon on the canvas and use the toggle in the configuration panel to toggle the +workflow state. -Environments allow you to run a Workflow with a special set of credentials. This -lets you use development servers, modes and databases while building your -sandbox, without interfering with live production services. +![Enabled workflow in the trigger panel](/img/via-trigger-panel.webp) -Each sandbox has an associated environment. By default it's `main`, which -implies that this is your live production environment. But you can create an -environment like `dev` or `staging`. +### Manual Runs -The environment is just a label. Each credential used in your workflow with have -a set of values associated with that environment. For example, when connecting -to DHIS2, your main credential will contain private login details. But your -`dev` environment might use the public sandbox and so contain a different -username and password. +Check out the video for a quick overview. -All environments are securely stored and encrypted within our database, so it's -perfectly safe to duplicate production credentials across multiple environments. + -For each Credential used in your workflow, you must ensure there is a value set -to match your sandbox environment. If you do not configure your credentials, the -Workflow will fail with clear instructions on how to correct it. +You can run a workflow manually in three ways: -## Merging sandboxes +#### With an empty input -Once you've finished making changes to your workflows, it's time to merge them -back into your main project. +This is the default behavior and the input dataclip for your run will be `{}`. -This is easy in the app: simply head to the Sandboxes page, find the Sandbox you -want to merge in the list, and click the Merge icon on the right-hand side. + -You'll be prompted to select the target project or sandbox to merge into: pick -from the list and click Merge. Usually you'll want to merge into the original -project, which is selected by default. +#### With a custom input -When merging, we replace the contents of workflows in your project with those in -your sandbox. Any workflows which are not in the sandbox will be ignored. If you -rename a workflow in the sandbox, you'll see the new workflow appear in your -main project, and the original workflow will be left alone (you'll probably want -to delete that manually). +You can type, copy/paste, or import (browse your file system or drag & drop) any +file with valid JSON. -Note that settings and options, like concurrency and data retention rules, are -not transferred in the merge, nor are historical runs or dataclips. Just the -Workflow contents. + -After merging, the sandbox will be destroyed, along with its history and -dataclips. +#### With an existing input -:::tip +You can pick from a list of previous inputs that were used to run this step. -You can also use the CLI to merge your changes locally, give them one final -test, and then deploy them to your main project. + -:::: +### Named Dataclips -## Conflicts +Dataclips (custom inputs, step results, webhook requests) can be named to make +them easier to find and use for testing. -If you've ever worked with a source version control system - like git or -Subversion - you'll be familiar with the idea of conflicts. +:::info Named dataclips aren't erased -A conflict occurs when you're trying to merge a Sandbox into your original -project (or another sandbox), and there are incompatible changes between them. +Named dataclips will not be removed alongside other project history when your +retention period is reached. They will be stored indefinitely. -Say you create a Sandbox from your main project and in the sandbox, you change -the adaptor of one step from `common` to `http`. And while you're making this -change, a colleague goes to the main project and sets the adaptor of the same -step to `salesforce`. +::: -What happens when you try and merge the sandbox? Should we preserve the original -change? Or accept the change in the sandbox? Or something else? +Assign your dataclip a name by clicking the label field. -Occasionally these conflicts are trivial to resolve and you might wonder what -all the fuss is about. But often they are complex, and it can be difficult or -impossible to automate a solution. + -[TODO not implemented yet] When we detect a conflict like this, we'll show a -warning when you try and merge the Sandbox. You can choose to "force push" the -Sandbox and overwrite whatever changes happened on the target Project, or you -can cancel and resolve the conflict yourself. +After assigning names to your inputs you can search for them by their name on +the search bar. -For now, the only way to resolve conflicts manually is to use the CLI to edit -your project locally, and pushed the resolved, final version up to the app. + -:::tip +Filter only named inputs by clicking the tag button. -We'll be adding better support for resolving conflicts soon. + -::: +## Limit Concurrency -## Editing sandboxes Locally +Workflow **concurrency** is the number of runs that will be allowed for a given +workflow **_at the same time_**. In OpenFn, project owners and administrators +are able to limit the maximum number of the runs that can be executed at the +same time for a workflow. You might do this to ensure "one at a time" serial +processing or to keep a fast OpenFn workflow from overwhelming the API rate +limit of some other connected system. :::info -Sandboxes are fully compatible with the CLI. +Please check to make sure that your parallel execution is not disabled for your +project as it will override the workflow level concurrency limit. + +::: + +### What happens when concurrency limit is set on a workflow? + +When concurrency limit is configured for a workflow, the maximum number of runs +that are executed concurrently will not exceed the number that has been set for +the workflow. For example: + +- **Concurrency not set (or = 0)**: There's no artificial limit applied, and + this workflow will only be limited by the total computing power available + across your OpenFn installation. +- **Concurrency = 1**: Runs for this workflow will only take place 1-at-a-time. + Each run must _finish_ before the next run can start. +- **Concurrency = 2**: No more than 2 runs for this workflow can be executed at + a time and other runs will have stay `enqueued`. If runs "A", "B", and "C" are + all enqueued, "A" and "B" will start executing. Once "A" finishes, "C" will + start. (No more than 2-at-a-time.) + +### Setting Concurrency for a workflow + +Concurrency limits can be set via the workflow settings modal on the workflow +canvas. + +1. Click on the settings icon beside the save button on your workflow to open + the workflow settings +2. In the modal, enter the maximum concurrency limit +3. Click save. + +![Configuring Concurrency](/img/configuring-concurrency.webp) + +### Log Outputs + +For data security and compliance purposes, the log output of a workflow run can +be configured to disable logging `console.log()` statements. This can be done +via the workflow configuration modal by a project owner or administrator. + +1. Click on the settings icon. +2. In the modal, toggle the **Allow `console.log()` usage** switch to disable + logging `console.log()` statements. By default, this is enabled. + +![Configuring Log Outputs](/img/configuring-log-outputs.webp) + +## Keyboard Shortcuts -For more details, see these CLI docs (link to follow) +From the canvas you can perform certain common actions (e.g., save) using +keystrokes. Check out the full list of keyboard shortcuts +[here](/documentation/keyboard-shortcuts). diff --git a/sidebars-main.js b/sidebars-main.js index 57698b0ee577..0b3a2ba99e81 100644 --- a/sidebars-main.js +++ b/sidebars-main.js @@ -66,6 +66,7 @@ module.exports = { 'build/credentials', 'build/collections', 'build/limits', + 'build/sandboxes', 'build/editing-locally', 'build/working-with-branches', 'build/troubleshooting', From 6bf9554302dd8cd59c2745d853fdadbd188934b1 Mon Sep 17 00:00:00 2001 From: Joe Clark Date: Tue, 14 Oct 2025 15:45:03 +0100 Subject: [PATCH 3/5] update changing env --- docs/build/sandboxes.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/docs/build/sandboxes.md b/docs/build/sandboxes.md index 2875442e9d7d..a93d7f55140a 100644 --- a/docs/build/sandboxes.md +++ b/docs/build/sandboxes.md @@ -71,7 +71,8 @@ a specific feature, like `new-patient-workflow`. You'll also need to set an Environment (see below). This configures all credentials within the sandbox to use that environment variant. If you're not -sure, set the environment to `dev`. +sure, set the environment to `dev` (you can change this at any time from the +Setup tab of the sandbox Settings page). A color will be randomly selected to associate with the sandbox. You'll see this color in the app UI while you're using the sandbox. You can select a different @@ -116,6 +117,10 @@ For each Credential used in your workflow, you must ensure there is a value set to match your sandbox environment. If you do not configure your credentials, the Workflow will fail with clear instructions on how to correct it. +To change a the environment used by a Sandbox, first enter the Sandbox from the +Sandboxes page, then go to the Settings page. The environment can be edited +under the Identity section of the Setup tab (right at the top of the page). + ## Merging sandboxes Once you've finished making changes to your workflows, it's time to merge them From de19b6a5596df6a28b5ab611344e6fd63b202bc1 Mon Sep 17 00:00:00 2001 From: Joe Clark Date: Wed, 15 Oct 2025 12:00:07 +0100 Subject: [PATCH 4/5] feedback --- docs/build/sandboxes.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/build/sandboxes.md b/docs/build/sandboxes.md index a93d7f55140a..730a17ce02c8 100644 --- a/docs/build/sandboxes.md +++ b/docs/build/sandboxes.md @@ -104,11 +104,11 @@ Each sandbox has an associated environment. By default it's `main`, which implies that this is your live production environment. But you can create an environment like `dev` or `staging`. -The environment is just a label. Each credential used in your workflow with have -a set of values associated with that environment. For example, when connecting -to DHIS2, your main credential will contain private login details. But your -`dev` environment might use the public sandbox and so contain a different -username and password. +The environment is just a label, and each credential used in your workflow has a +set of values associated with that label. For example, when connecting to DHIS2, +your main credential will contain private login details. But your `dev` +environment might use the public sandbox and so contain a different username and +password. All environments are securely stored and encrypted within our database, so it's perfectly safe to duplicate production credentials across multiple environments. @@ -144,7 +144,8 @@ not transferred in the merge, nor are historical runs or dataclips. Just the Workflow contents. After merging, the sandbox will be destroyed, along with its history and -dataclips. +dataclips. Any environments and credentials assocaited with the project will be +unaffected. :::tip @@ -189,8 +190,7 @@ We'll be adding better support for resolving conflicts soon. ## Editing sandboxes Locally -:::info - Sandboxes are fully compatible with the CLI. -For more details, see these CLI docs (link to follow) +We're still working on docs and features for that - check back soon for more +details! From 6c06123cddd5ae9fa9fbd6565f86c391bf19c95d Mon Sep 17 00:00:00 2001 From: Joe Clark Date: Wed, 15 Oct 2025 12:00:40 +0100 Subject: [PATCH 5/5] fixes --- docs/build/sandboxes.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/docs/build/sandboxes.md b/docs/build/sandboxes.md index 730a17ce02c8..bcfa028bec5b 100644 --- a/docs/build/sandboxes.md +++ b/docs/build/sandboxes.md @@ -3,10 +3,10 @@ title: Sandboxes sidebar_label: Sandboxes --- -sandboxes are a way to develop fixes and new features on workflows without +Sandboxes are a way to develop fixes and new features on workflows without affecting live, or "in production", runs. -:::info sandboxes are new to OpenFn since October 2025. +:::info Sandboxes are new to OpenFn since October 2025. At the time of writing sandboxes are under active development and testing. We expect to be in full working order by the end of November 2025, but until then @@ -26,8 +26,6 @@ project, rather than duplicate them. The idea is that you can develop the workflow in total isolation from the main project, and once you're done, merge (read as "push" or "promote") changes back. -with a copy of each workflow, its own webhooks, cron triggers and - :::tip Short-lived sandboxes sandboxes work best when they're short-lived, so right now they are destroyed