Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .bumpy/set-values-bulk-create-missing-sensitive.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
varlock: minor
---

@setValuesBulk createMissing option now accepts "sensitive" to create missing items as sensitive
Original file line number Diff line number Diff line change
Expand Up @@ -169,15 +169,15 @@ IMPORTED_ITEM=overridden-value
<div>
### `@setValuesBulk()`
**Arg types:** `[ data: string ]`
**Named args:** `format?: "json" | "env"`, `createMissing?: boolean`, `enabled?: boolean`, `pick?: string[]`, `omit?: string[]`
**Named args:** `format?: "json" | "env"`, `createMissing?: boolean | "sensitive"`, `enabled?: boolean`, `pick?: string[]`, `omit?: string[]`

Injects multiple config values at once from an external data source. The first argument is a resolver that produces a string, typically a bulk resolver from a [secrets provider plugin](/plugins/overview/) such as [`opLoadEnvironment()`](/plugins/1password/#oploadenvironment) (1Password), [`infisicalBulk()`](/plugins/infisical/), or [`vaultSecret(…, raw=true)`](/plugins/hashicorp-vault/) (HashiCorp Vault). The string is parsed and injected as definitions within the file containing the decorator.

Bulk values participate in the normal file override chain: `process.env` still overrides everything, higher-precedence files (`.env.local`, `.env.production`, etc.) override bulk values, and bulk values override schema-defined defaults in the same file. You control precedence by choosing which file to put `@setValuesBulk` in.

**Options:**
- `format`: How to parse the data string. `json` expects a flat JSON object, `env` expects `.env` file format. If not specified, auto-detected by checking if the string starts with `{`.
- `createMissing`: If `true`, keys in the bulk data that don't already exist in your schema will be created as new config items. Defaults to `false` (unknown keys are silently skipped).
- `createMissing`: If `true`, keys in the bulk data that don't already exist in your schema will be created as new config items. Set to `"sensitive"` to also mark every created item as sensitive, so their values are redacted in `varlock load` output regardless of any `@defaultSensitive` setting. Defaults to `false` (unknown keys are silently skipped).
- `enabled`: If `false`, the bulk data resolver is skipped entirely. Accepts any boolean expression, including dynamic references to other config items. Defaults to `true`.
- `pick`: An array of key names to inject, an **allowlist**. Only matching keys are injected; everything else from the source is ignored.
- `omit`: An array of key names to skip, a **denylist**. Every key _except_ the matches is injected.
Expand Down
8 changes: 8 additions & 0 deletions packages/varlock/src/env-graph/lib/config-item.ts
Original file line number Diff line number Diff line change
Expand Up @@ -457,6 +457,8 @@ export class ConfigItem {

_isSensitive: boolean = true;
_sensitiveExplicitlySet = false;
/** externally-forced sensitivity (e.g. @setValuesBulk createMissing="sensitive") — wins over all inference */
_forceSensitive?: boolean;
/** how sensitivity was determined (undefined = the global default that items are sensitive) */
_sensitiveSource?: 'explicit' | 'data-type' | 'resolver' | 'default-decorator' | 'prefix';
get isSensitive(): boolean {
Expand Down Expand Up @@ -486,6 +488,12 @@ export class ConfigItem {
return this._preventLeaks;
}
private async processSensitive() {
if (this._forceSensitive !== undefined) {
this._isSensitive = this._forceSensitive;
this._sensitiveExplicitlySet = true;
this._sensitiveSource = 'explicit';
return;
}
const sensitiveFromDataType = this.dataType?.isSensitive;

// Pass 1: explicit per-item @sensitive / @public decorators take highest priority
Expand Down
12 changes: 9 additions & 3 deletions packages/varlock/src/env-graph/lib/decorators.ts
Original file line number Diff line number Diff line change
Expand Up @@ -404,8 +404,8 @@ export const builtInRootDecorators: Array<RootDecoratorDef<any>> = [
const createMissingResolver = argsVal.objArgs.createMissing;
if (createMissingResolver?.isStatic) {
const cmVal = createMissingResolver.staticValue;
if (cmVal !== true && cmVal !== false) {
throw new SchemaError('@setValuesBulk: createMissing must be true or false');
if (cmVal !== true && cmVal !== false && cmVal !== 'sensitive') {
throw new SchemaError('@setValuesBulk: createMissing must be true, false, or "sensitive"');
}
}
}
Expand Down Expand Up @@ -445,6 +445,9 @@ export const builtInRootDecorators: Array<RootDecoratorDef<any>> = [
const dataString = resolved.arr[0];
const format = resolved.obj?.format as string | undefined;
const createMissing = resolved.obj?.createMissing ?? false;
if (createMissing !== true && createMissing !== false && createMissing !== 'sensitive') {
throw new SchemaError('@setValuesBulk: createMissing must be true, false, or "sensitive"');
}

if (dataString === undefined || dataString === null || dataString === '') {
return; // empty data is a no-op
Expand Down Expand Up @@ -491,11 +494,14 @@ export const builtInRootDecorators: Array<RootDecoratorDef<any>> = [
};
}

// if key doesn't exist in configSchema and createMissing is true, create a new ConfigItem
// if key doesn't exist in configSchema and createMissing is truthy, create a new ConfigItem
if (!existsInSchema && createMissing) {
const newItem = new ConfigItem(graph, key);
graph.configSchema[key] = newItem;
await newItem.process();
// createMissing="sensitive" forces created items to be sensitive, overriding
// whatever @defaultSensitive / prefix rules would otherwise infer
if (createMissing === 'sensitive') newItem._forceSensitive = true;
}
}
},
Expand Down
24 changes: 24 additions & 0 deletions packages/varlock/src/env-graph/test/set-values-bulk.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,30 @@ describe('@setValuesBulk() root decorator', () => {
NEW_KEY: 'new-val',
},
}));

test('createMissing=sensitive creates new items as sensitive', envFilesTest({
envFile: outdent`
# @defaultSensitive=false
# @setValuesBulk('{"NEW_KEY":"new-val"}', format=json, createMissing="sensitive")
# ---
API_KEY=val
`,
expectValues: {
NEW_KEY: 'new-val',
},
expectSensitive: {
NEW_KEY: true,
API_KEY: false,
},
}));

test('invalid createMissing value is an error', envFilesTest({
envFile: outdent`
# @setValuesBulk('{"A":"val"}', format=json, createMissing="banana")
# ---
`,
expectError: true,
}));
});

describe('key filters', () => {
Expand Down