Skip to content

Commit e4f9939

Browse files
Keep PEP 723 inline script CodeLens live and persistent (#1906)
## Summary - Keep the setup CodeLens available whenever a recognizable PEP 723 script block exists, including incomplete or malformed live edits. - Keep `Script environment ready (Python X.Y.Z)` persistent for validated, available script environments. - Re-evaluate unsaved metadata edits immediately while preserving Ready and exact-file routing for body-only edits. - Prevent pending saved-metadata reads from restoring stale routing after a live metadata edit. - Validate malformed metadata when setup is invoked, preserving diagnostics and avoiding invalid environment associations. - Harden setup, save, cache-repair, and stale-operation behavior and document the updated user flow. - Add focused parser, routing, CodeLens, detector, setup, manager, and real VS Code integration coverage. ## Validation - `npm run lint` - `npm run compile-tests` - `npm run unittest` — 2,812 passing, 7 pending after rebasing onto the latest `main` - Production webpack build - Inline VS Code integration — 8 passing - Real-environment PEP 723 workflows — 21 passing - Smoke suite — 31 passing - Environment discovery E2E — 4 passing - Manual Windows VS Code Insiders testbed matrix covering setup, persistent Ready, live edits, Undo/Revert, malformed metadata, diagnostics, quick fixes, Run File, Pylance routing, cache repair, restart persistence, and feature-off behavior - Static core, additional-use-case, quick-fix, and 8-KiB boundary verification ## Notes The tolerant block detector is presentation-only. Semantic parsing remains authoritative for diagnostics, setup eligibility, environment identity, persistence, and exact-file routing. A separate full integration run completed with 57 passing and 8 pending tests plus one existing environment-creation timeout. The timed-out fixture selects the system manager, which cannot quick-create headlessly; instrumentation confirmed it does not enter the PEP 723 path. --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
1 parent 803879b commit e4f9939

19 files changed

Lines changed: 1251 additions & 279 deletions

File tree

‎.github/instructions/testing-workflow.instructions.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ This guide covers the full testing lifecycle:
2121
- Pip commands that return JSON must pass `--disable-pip-version-check`; the process helper combines stderr with stdout, so update notices can otherwise make valid JSON unparseable (1).
2222
- When a view subscribes to a newly added provider event, TypeMoq-based view tests must return a real `EventEmitter.event`; an unstubbed event yields an undefined disposable and fails during teardown (1).
2323
- Test agent selections across a full reload with `python.defaultInterpreterPath` set. An effective manager value equal to the extension default does not prove a workspace value was saved; tool-owned persistence must inspect `workspaceValue` or startup can restore the global interpreter (1).
24+
- `vscode.executeCodeLensProvider` verifies provider output, not visible editor refresh. VS Code cancels its debounced CodeLens refresh when the editor loses focus; account for background UI automation restoring focus to another app before diagnosing a stale rendered lens (1).
2425

2526
### When to Use This Guide
2627

‎docs/managing-python-projects.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -101,7 +101,9 @@ When you create a script, the extension generates a single `.py` file with PEP 7
101101

102102
An inline-script environment is built from the script's `# /// script` block and stored in the extension's cache, where it is shared by every script with the same dependencies and base interpreter. Because editing one would silently change the others, these environments are not user-managed: the Python Environments views do not offer install, uninstall, or version-change actions for them. Their package list remains visible.
103103

104-
A CodeLens above the `# /// script` block offers **Set up environment for this script**, and the same action is available as a quick fix on an unresolved import. For a few seconds after setup succeeds it is replaced by a **Script environment ready (Python X.Y.Z)** confirmation naming the Python that was selected — useful when `requires-python` matches several installed versions, or when one was installed on demand. The confirmation is plain text rather than a clickable action, and it expires on its own; at every other time the setup CodeLens behaves exactly as before.
104+
A CodeLens stays above the `# /// script` block, including while its metadata is incomplete, malformed, or unsaved. It offers **Set up environment for this script** until the block matches a validated environment, then displays **Script environment ready (Python X.Y.Z)** as persistent, non-clickable text. The ready label also appears for environments restored after reopening VS Code.
105+
106+
The CodeLens follows the live block text: editing the block offers setup immediately without requiring a save, while editing Python code outside it does not change its fingerprint. Diagnostics continue to explain malformed metadata as you type; a warning popup appears only if you try to set up an invalid block. Setup saves valid unsaved changes before creating the environment. Execution and interpreter routing still use validated, saved metadata, and saving can restore a matching existing association without rebuilding it.
105107

106108
Setup records which distributions it installed. If that record and the environment's contents later disagree — for example after installing a package into it from a terminal — every script sharing the environment needs setup again. Saving or reopening a script does not repair it; use the script's setup action to rebuild from its declared dependencies.
107109

‎src/common/inlineScript/block.ts‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
// Copyright (c) Microsoft Corporation. All rights reserved.
2+
// Licensed under the MIT License.
3+
4+
export interface InlineScriptBlock {
5+
readonly start: number;
6+
readonly end: number;
7+
}
8+
9+
/**
10+
* Locate the first recognizable script opener and its next closing marker, without validation.
11+
* Incomplete blocks are included so editor actions remain available while typing.
12+
* This approximate presentation range must not be used for metadata validation or fingerprints.
13+
*/
14+
export function findInlineScriptBlock(text: string): InlineScriptBlock | undefined {
15+
const opener = /^[\t \uFEFF]*#[\t ]*\/\/\/[\t ]+script\b[^\r\n]*/gm.exec(text);
16+
if (!opener) {
17+
return undefined;
18+
}
19+
const closerPattern = /^[\t ]*#[\t ]*\/\/\/[\t ]*$/gm;
20+
closerPattern.lastIndex = opener.index + opener[0].length;
21+
const closer = closerPattern.exec(text);
22+
return {
23+
start: opener.index + (opener.index === 0 && text.startsWith('\uFEFF') ? 1 : 0),
24+
end: closer ? closer.index + closer[0].length : text.length,
25+
};
26+
}

‎src/common/inlineScript/metadata.ts‎

Lines changed: 126 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22
// Licensed under the MIT License.
33

44
import * as tomljs from '@iarna/toml';
5+
import { createHash } from 'crypto';
56
import * as fs from 'fs/promises';
67
import { l10n, Uri } from 'vscode';
78
import { traceVerbose, traceWarn } from '../logging';
@@ -29,6 +30,8 @@ export interface InlineScriptMetadata {
2930
*/
3031
readonly range: { readonly start: number; readonly end: number };
3132
readonly sourceRange?: { readonly start: number; readonly end: number };
33+
/** Saved block fingerprint for edit detection and presentation, separate from semantic/cache identity. */
34+
readonly sourceHash?: string;
3235
}
3336

3437
/**
@@ -70,6 +73,24 @@ export type InlineScriptMetadataParseResult =
7073
| { readonly kind: 'none' }
7174
| { readonly kind: 'invalid'; readonly problems: readonly InlineScriptMetadataProblem[] };
7275

76+
/**
77+
* Explain why parsed metadata cannot be used by the interactive setup action.
78+
* This presentation check does not change the parser's tolerant metadata or routing contract.
79+
*/
80+
export function getInlineScriptSetupProblem(
81+
result: InlineScriptMetadataParseResult,
82+
): 'metadata' | 'requires-python' | undefined {
83+
if (
84+
result.kind !== 'parsed' ||
85+
result.problems.length > 0 ||
86+
result.metadata.dependencies?.some((dependency) => dependency.trim().length === 0)
87+
) {
88+
return 'metadata';
89+
}
90+
const requirement = result.metadata.requiresPython?.trim();
91+
return requirement && !PythonVersionSpecifier.tryParse(requirement) ? 'requires-python' : undefined;
92+
}
93+
7394
/**
7495
* Canonical block regex from the PEP 723 spec, translated to JavaScript
7596
* (Python's `(?P<name>...)` becomes `(?<name>...)` in JS). The flag
@@ -122,6 +143,57 @@ const NO_METADATA: InlineScriptMetadataParseResult = { kind: 'none' };
122143

123144
const OPENER_PREFIX = '# /// ';
124145

146+
/**
147+
* Fingerprint the same structural script block selected by the metadata parser, without parsing TOML,
148+
* logging, or I/O. Malformed markers return undefined; ignored body examples do not affect the hash.
149+
* Normalize BOM/line endings as the parser does so disk and editor representations compare equally.
150+
*/
151+
export function getInlineScriptSourceHash(scriptText: string): string | undefined {
152+
const text = scriptText.replace(/^\uFEFF/, '').replace(/\r\n?/g, '\n');
153+
const { scriptMatches, problems } = scanInlineScriptBlocks(
154+
text,
155+
(start, end) => ({ start, end }),
156+
'',
157+
false,
158+
);
159+
return scriptMatches.length === 1 && problems.length === 0
160+
? hashScriptBlock(scriptMatches[0][0])
161+
: undefined;
162+
}
163+
164+
function hashScriptBlock(block: string): string {
165+
return createHash('sha256').update(block, 'utf8').digest('hex');
166+
}
167+
168+
function scanInlineScriptBlocks(
169+
text: string,
170+
toSourceRange: (start: number, end: number) => { start: number; end: number },
171+
where: string,
172+
logProblems = true,
173+
): { scriptMatches: RegExpMatchArray[]; problems: InlineScriptMetadataProblem[] } {
174+
const scriptMatches: RegExpMatchArray[] = [];
175+
// matchAll leaves the shared regex's lastIndex untouched.
176+
for (const match of text.matchAll(BLOCK_RE)) {
177+
if (match.groups?.type === 'script') {
178+
scriptMatches.push(match);
179+
}
180+
}
181+
const matchedRanges = scriptMatches.map((match) => ({ start: match.index!, end: match.index! + match[0].length }));
182+
const problems: InlineScriptMetadataProblem[] = [];
183+
const headerEnd = headerRegionEnd(text);
184+
for (const opener of findScriptOpeners(text)) {
185+
if (matchedRanges.some((range) => opener.offset >= range.start && opener.offset < range.end)) {
186+
continue;
187+
}
188+
const { problem, ignorableBelowHeader } = diagnoseMalformedBlock(text, opener, toSourceRange, where, logProblems);
189+
if (ignorableBelowHeader && opener.offset >= headerEnd) {
190+
continue;
191+
}
192+
problems.push(problem);
193+
}
194+
return { scriptMatches, problems };
195+
}
196+
125197
/** As `readInlineScriptMetadata`, but reports why and where parsing failed. Offsets index the original `scriptText`. */
126198
export function parseInlineScriptMetadata(scriptText: string, source?: string): InlineScriptMetadataParseResult {
127199
const where = source ? ` in ${source}` : '';
@@ -147,37 +219,7 @@ export function parseInlineScriptMetadata(scriptText: string, source?: string):
147219
end: bomOffset + sourceOffsetForNormalizedOffset(sourceText, end),
148220
});
149221

150-
// Collect ALL matches first so we can detect the "multiple script
151-
// blocks" error case the spec requires us to surface.
152-
//
153-
// `matchAll` constructs a fresh iterator and does not mutate the
154-
// shared `BLOCK_RE.lastIndex`, so this loop is re-entrant and safe
155-
// even if a caller (or an exception) ever interrupts a previous
156-
// pass.
157-
const scriptMatches: RegExpMatchArray[] = [];
158-
for (const m of text.matchAll(BLOCK_RE)) {
159-
// Per spec, tools MUST NOT read non-standardized block types.
160-
// The only standardized type today is `script`.
161-
if (m.groups?.type === 'script') {
162-
scriptMatches.push(m);
163-
}
164-
}
165-
166-
const matchedRanges = scriptMatches.map((m) => ({ start: m.index!, end: m.index! + m[0].length }));
167-
const problems: InlineScriptMetadataProblem[] = [];
168-
const headerEnd = headerRegionEnd(text);
169-
for (const opener of findScriptOpeners(text)) {
170-
if (matchedRanges.some((r) => opener.offset >= r.start && opener.offset < r.end)) {
171-
continue;
172-
}
173-
const { problem, ignorableBelowHeader } = diagnoseMalformedBlock(text, opener, toSourceRange, where);
174-
// Blocks the spec tells us to ignore are only worth flagging in the leading comment
175-
// region, where they are a header being typed rather than a documentation example.
176-
if (ignorableBelowHeader && opener.offset >= headerEnd) {
177-
continue;
178-
}
179-
problems.push(problem);
180-
}
222+
const { scriptMatches, problems } = scanInlineScriptBlocks(text, toSourceRange, where);
181223

182224
if (scriptMatches.length === 0) {
183225
if (problems.length === 0) {
@@ -368,7 +410,7 @@ export function parseInlineScriptMetadata(scriptText: string, source?: string):
368410
end += 1;
369411
}
370412

371-
return {
413+
const result: InlineScriptMetadataParseResult = {
372414
kind: 'parsed',
373415
problems,
374416
metadata: {
@@ -379,6 +421,13 @@ export function parseInlineScriptMetadata(scriptText: string, source?: string):
379421
sourceRange: toSourceRange(matchStart, end),
380422
},
381423
};
424+
return {
425+
...result,
426+
metadata: {
427+
...result.metadata,
428+
sourceHash: getInlineScriptSetupProblem(result) === undefined ? hashScriptBlock(match[0]) : undefined,
429+
},
430+
};
382431
}
383432

384433
/** 1-based line number of `offset` within LF-normalized `text`. */
@@ -447,13 +496,16 @@ function diagnoseMalformedBlock(
447496
opener: ScriptOpener,
448497
toSourceRange: (start: number, end: number) => { start: number; end: number },
449498
where: string,
499+
logProblems: boolean,
450500
): MalformedBlockDiagnosis {
451501
const openerRange = toSourceRange(opener.offset, opener.lineEnd);
452502

453503
if (opener.trailing.length > 0) {
454-
traceWarn(
455-
`inline script metadata${where}: the \`# /// script\` marker on line ${countLines(text, opener.offset)} has trailing whitespace`,
456-
);
504+
if (logProblems) {
505+
traceWarn(
506+
`inline script metadata${where}: the \`# /// script\` marker on line ${countLines(text, opener.offset)} has trailing whitespace`,
507+
);
508+
}
457509
return {
458510
ignorableBelowHeader: false,
459511
problem: {
@@ -475,9 +527,11 @@ function diagnoseMalformedBlock(
475527
}
476528

477529
if (line !== line.trimEnd() && line.trimEnd() === CLOSER_LINE) {
478-
traceWarn(
479-
`inline script metadata${where}: the closing \`# ///\` marker on line ${countLines(text, offset)} has trailing whitespace`,
480-
);
530+
if (logProblems) {
531+
traceWarn(
532+
`inline script metadata${where}: the closing \`# ///\` marker on line ${countLines(text, offset)} has trailing whitespace`,
533+
);
534+
}
481535
return {
482536
ignorableBelowHeader: false,
483537
problem: {
@@ -494,10 +548,12 @@ function diagnoseMalformedBlock(
494548
// closing marker is still ahead the author wrote a real block around a bad line.
495549
const isComment = line.startsWith('#');
496550
if (isComment || hasCloserAhead(text, lineEnd)) {
497-
traceWarn(
498-
`inline script metadata${where}: invalid content line ${countLines(text, offset)} ` +
499-
`(expected '#' or '# '): ${JSON.stringify(line)}`,
500-
);
551+
if (logProblems) {
552+
traceWarn(
553+
`inline script metadata${where}: invalid content line ${countLines(text, offset)} ` +
554+
`(expected '#' or '# '): ${JSON.stringify(line)}`,
555+
);
556+
}
501557
return {
502558
ignorableBelowHeader: !isComment,
503559
problem: {
@@ -520,9 +576,11 @@ function diagnoseMalformedBlock(
520576
offset = lineEnd + 1;
521577
}
522578

523-
traceWarn(
524-
`inline script metadata${where}: the \`# /// script\` block on line ${countLines(text, opener.offset)} is missing its closing \`# ///\` marker`,
525-
);
579+
if (logProblems) {
580+
traceWarn(
581+
`inline script metadata${where}: the \`# /// script\` block on line ${countLines(text, opener.offset)} is missing its closing \`# ///\` marker`,
582+
);
583+
}
526584
return {
527585
ignorableBelowHeader: true,
528586
problem: { code: 'unterminated-block', severity: 'warning', sourceRange: openerRange },
@@ -661,6 +719,28 @@ export async function readInlineScriptMetadataFromFile(
661719
uri: Uri,
662720
strict = false,
663721
): Promise<InlineScriptMetadata | undefined> {
722+
const text = await readInlineScriptHeaderFromFile(uri, strict);
723+
if (text === undefined) {
724+
return undefined;
725+
}
726+
const result = parseInlineScriptMetadata(text, uri.fsPath);
727+
if (
728+
strict &&
729+
(result.kind === 'invalid' ||
730+
(result.kind === 'parsed' && result.problems.some((problem) => problem.severity === 'error')))
731+
) {
732+
throw new Error(l10n.t('Fix the PEP 723 metadata in {0} before configuring its environment.', uri.fsPath));
733+
}
734+
return result.kind === 'parsed' ? result.metadata : undefined;
735+
}
736+
737+
/**
738+
* Read at most MAX_HEADER_BYTES of a local script as UTF-8, without parsing its metadata.
739+
* Unsupported URI schemes and I/O failures return undefined and are logged.
740+
* @param uri The local script to read.
741+
* @param strict Throw on I/O errors instead of treating them as absent.
742+
*/
743+
export async function readInlineScriptHeaderFromFile(uri: Uri, strict = false): Promise<string | undefined> {
664744
if (uri.scheme !== 'file') {
665745
traceVerbose(`inline script metadata: skipping non-file URI scheme '${uri.scheme}'`);
666746
return undefined;
@@ -683,15 +763,7 @@ export async function readInlineScriptMetadataFromFile(
683763
return undefined;
684764
}
685765

686-
const result = parseInlineScriptMetadata(text, uri.fsPath);
687-
if (
688-
strict &&
689-
(result.kind === 'invalid' ||
690-
(result.kind === 'parsed' && result.problems.some((problem) => problem.severity === 'error')))
691-
) {
692-
throw new Error(l10n.t('Fix the PEP 723 metadata in {0} before configuring its environment.', uri.fsPath));
693-
}
694-
return result.kind === 'parsed' ? result.metadata : undefined;
766+
return text;
695767
}
696768

697769
/**

0 commit comments

Comments
 (0)