Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .beads/interactions.jsonl
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{"id":"int-32efd6fd","kind":"field_change","created_at":"2026-04-07T12:34:26.075149Z","actor":"Ken Judy","issue_id":"code-quality-metrics-zbj","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Failing test written and confirmed red: net_additions_ratio_median is undefined (field doesn't exist yet)"}}
{"id":"int-5af8e10c","kind":"field_change","created_at":"2026-04-07T12:35:01.462055Z","actor":"Ken Judy","issue_id":"code-quality-metrics-bko","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Formula replaced: bounded (additions-deletions)/(additions+deletions). Zero-total guard added."}}
{"id":"int-b4b9bbe1","kind":"field_change","created_at":"2026-04-07T12:35:01.996044Z","actor":"Ken Judy","issue_id":"code-quality-metrics-at3","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Fields renamed: additions_ratio_median -> net_additions_ratio_median, additions_ratio_p90 -> net_additions_ratio_p90 in local-code-metrics.js:251-252"}}
{"id":"int-9a5597ae","kind":"field_change","created_at":"2026-04-07T12:35:46.381394Z","actor":"Ken Judy","issue_id":"code-quality-metrics-fii","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Test assertions updated to net_additions_ratio_median/p90. Confirmed no stale additions_ratio_* in source code."}}
{"id":"int-f2069a5f","kind":"field_change","created_at":"2026-04-07T12:36:05.30815Z","actor":"Ken Judy","issue_id":"code-quality-metrics-c77","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"117/117 tests pass. New regression test green. No additions_ratio_* in source. Console warns are pre-existing intentional error-handling tests in claudeAnalysis.test.js."}}
{"id":"int-5b7581d8","kind":"field_change","created_at":"2026-04-07T12:37:54.936752Z","actor":"Ken Judy","issue_id":"code-quality-metrics-prz","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"metrics-specification.md updated: formula, field names, thresholds (0.33/0.50), interpretation, formula rejection rationale. Output format reference section updated."}}
{"id":"int-3e1630bd","kind":"field_change","created_at":"2026-04-07T12:38:16.042213Z","actor":"Ken Judy","issue_id":"code-quality-metrics-h22","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"lint clean, typecheck clean, 117/117 tests, 96.21% lines / 96.42% functions — all thresholds met"}}
{"id":"int-cee835a3","kind":"field_change","created_at":"2026-04-07T12:38:16.543793Z","actor":"Ken Judy","issue_id":"code-quality-metrics-oga","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"All 7 implementation steps complete. Formula fixed, fields renamed, spec updated, tests green."}}
{"id":"int-d7bd06be","kind":"field_change","created_at":"2026-04-07T12:41:46.049872Z","actor":"Ken Judy","issue_id":"code-quality-metrics-zx6","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Both workflow files updated: formula fixed (bounded), fields renamed to net_additions_ratio_*, thresholds updated to <0.50, display labels updated. Also cleaned up redundant sort+map in code-metrics.yml ratioStats computation."}}
17 changes: 10 additions & 7 deletions .github/workflows/code-metrics.yml
Original file line number Diff line number Diff line change
Expand Up @@ -166,8 +166,8 @@ jobs:
p95_lines_changed: 0,
stddev_lines_changed: 0,
velocity_trend: "stable",
additions_ratio_median: 0,
additions_ratio_p90: 0,
net_additions_ratio_median: 0,
net_additions_ratio_p90: 0,
dora_archetype: "mixed-signals",
note: "Feature branches only - main/master excluded. Large commit threshold applied to production code only."
};
Expand Down Expand Up @@ -231,8 +231,11 @@ jobs:
const lineSizes = metrics.map(m => m.total_additions + m.total_deletions);
const lineStats = computeStats(lineSizes);
const fileStats = computeStats(metrics.map(m => m.files_changed));
const ratios = metrics.map(m => m.total_additions / (m.total_deletions || 1));
const ratioStats = computeStats([...ratios].sort((a, b) => a - b).map((_, i, a) => a[i]));
const ratios = metrics.map(m => {
const t = m.total_additions + m.total_deletions;
return t === 0 ? 0 : (m.total_additions - m.total_deletions) / t;
});
const ratioStats = computeStats(ratios);
const velocityTrend = computeVelocityTrend(metrics.map(m => m.date));

const qualityCount = metrics.filter(m => m.message_quality).length;
Expand Down Expand Up @@ -262,8 +265,8 @@ jobs:
p50_files_changed: Math.round(fileStats.p50),
p90_files_changed: Math.round(fileStats.p90),
velocity_trend: velocityTrend,
additions_ratio_median: parseFloat(ratioStats.p50.toFixed(2)),
additions_ratio_p90: parseFloat(ratioStats.p90.toFixed(2)),
net_additions_ratio_median: parseFloat(ratioStats.p50.toFixed(2)),
net_additions_ratio_p90: parseFloat(ratioStats.p90.toFixed(2)),
message_quality_pct: msgQualityPct,
dora_archetype: classifyDoraArchetype(largePct, sprawlingPct, testFirstPct, parseFloat(msgQualityPct)),
note: "Feature branches only - main/master excluded. Large commit threshold applied to production code only."
Expand Down Expand Up @@ -332,7 +335,7 @@ jobs:
`| Sprawling Commits (>5 files) | ${summary.sprawling_commits_pct}% | <10% | ${statusMark(summary.sprawling_commits_pct, 10)} |`,
`| Test-First Discipline | ${summary.test_first_pct}% | >50% | ${statusMark(summary.test_first_pct, 50, 'above')} |`,
`| Message Quality | ${summary.message_quality_pct}% | >60% | ${statusMark(summary.message_quality_pct, 60, 'above')} |`,
`| Additions Ratio (median) | ${summary.additions_ratio_median} | <3.0 | ${parseFloat(summary.additions_ratio_median) < 3.0 ? 'OK' : 'Warning'} |`,
`| Net Additions Ratio (median) | ${summary.net_additions_ratio_median} | <0.50 | ${parseFloat(summary.net_additions_ratio_median) < 0.50 ? 'OK' : 'Warning'} |`,
'',
'### Commit Size Distribution',
'',
Expand Down
15 changes: 8 additions & 7 deletions .github/workflows/pr-metrics.yml
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,8 @@ jobs:
const prodChanges = commitProdFiles.reduce((sum, f) => sum + f.additions + f.deletions, 0);
const testChanges = commitTestFiles.reduce((sum, f) => sum + f.additions + f.deletions, 0);
const additions = detail.data.stats?.additions || 0;
const deletions = detail.data.stats?.deletions || 1;
const deletions = detail.data.stats?.deletions || 0;
const totalChurn = additions + deletions;
const message = commit.commit.message.split('\n')[0];

commitMetrics.push({
Expand All @@ -107,7 +108,7 @@ jobs:
test_only: commitTestFiles.length > 0 && commitProdFiles.length === 0,
prod_only: commitProdFiles.length > 0 && commitTestFiles.length === 0,
test_ratio: prodChanges > 0 ? (testChanges / prodChanges) : 0,
additions_ratio: (additions / (deletions || 1)).toFixed(2),
net_additions_ratio: (totalChurn === 0 ? 0 : (additions - deletions) / totalChurn).toFixed(2),
message_quality: scoreMessageQuality(message)
});

Expand Down Expand Up @@ -157,7 +158,7 @@ jobs:
const msgQualityPct = commitMetrics.length > 0 ? (qualityCommits / commitMetrics.length * 100) : 0;

const medianAdditionsRatio = (() => {
const ratios = commitMetrics.map(c => parseFloat(c.additions_ratio)).sort((a, b) => a - b);
const ratios = commitMetrics.map(c => parseFloat(c.net_additions_ratio)).sort((a, b) => a - b);
if (ratios.length === 0) return 0;
const mid = Math.floor(ratios.length / 2);
return ratios.length % 2 !== 0 ? ratios[mid] : (ratios[mid - 1] + ratios[mid]) / 2;
Expand Down Expand Up @@ -207,8 +208,8 @@ jobs:
concerns.push(`${sprawlingCommits}/${commitMetrics.length} commits touch more than 5 files`);
}

if (medianAdditionsRatio > 3.0) {
concerns.push(`Median additions ratio ${medianAdditionsRatio.toFixed(2)} exceeds 3.0 - possible batch-acceptance pattern`);
if (medianAdditionsRatio > 0.50) {
concerns.push(`Median net additions ratio ${medianAdditionsRatio.toFixed(2)} exceeds 0.50 - possible batch-acceptance pattern`);
}

if (msgQualityPct < 40) {
Expand Down Expand Up @@ -316,7 +317,7 @@ jobs:
`| Small Batches | Sprawling commit % | ${sprawlingPct.toFixed(0)}% | <10% |`,
`| Version Control | Test-first discipline | ${testFirstPct.toFixed(0)}% | >50% |`,
`| Version Control | Message quality | ${msgQualityPct.toFixed(0)}% | >60% |`,
`| AI Risk Signal | Additions ratio (median) | ${medianAdditionsRatio.toFixed(2)} | <3.0 |`
`| AI Risk Signal | Net additions ratio (median) | ${medianAdditionsRatio.toFixed(2)} | <0.50 |`
].join('\n') : '';

const commentParts = [
Expand Down Expand Up @@ -346,7 +347,7 @@ jobs:
`| Sprawling commits (>5 files) | ${sprawlingCommits}/${commitMetrics.length} (${sprawlingPct.toFixed(0)}%) |`,
`| Test-first discipline | ${testFirstCommits}/${commitMetrics.length} (${testFirstPct.toFixed(0)}%) |`,
`| Message quality | ${qualityCommits}/${commitMetrics.length} (${msgQualityPct.toFixed(0)}%) |`,
`| Median additions ratio | ${medianAdditionsRatio.toFixed(2)} |`,
`| Median net additions ratio | ${medianAdditionsRatio.toFixed(2)} |`,
`| Test-only commits | ${testOnlyCommits} |`,
`| Production-only commits | ${prodOnlyCommits} |`,
'',
Expand Down
38 changes: 36 additions & 2 deletions __tests__/collectLocalMetrics.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -141,8 +141,8 @@ describe('collectLocalMetrics — successful run', () => {
const summary = JSON.parse(summaryCall[1]);
expect(typeof summary.velocity_commits_per_day).toBe('number');
expect(['accelerating', 'stable', 'decelerating']).toContain(summary.velocity_trend);
expect(typeof summary.additions_ratio_median).toBe('number');
expect(typeof summary.additions_ratio_p90).toBe('number');
expect(typeof summary.net_additions_ratio_median).toBe('number');
expect(typeof summary.net_additions_ratio_p90).toBe('number');
expect(typeof summary.message_quality_pct).toBe('string');
expect(['harmonious-high-achiever', 'foundational-challenges', 'legacy-bottleneck', 'mixed-signals'])
.toContain(summary.dora_archetype);
Expand Down Expand Up @@ -213,6 +213,40 @@ describe('collectLocalMetrics — successful run', () => {
expect(allLogs).toMatch(/Claude analysis skipped/);
});

test('net_additions_ratio_median is bounded to 1.0 for all-new-file commits', async () => {
// Regression test for formula bug: additions / max(deletions, 1) inflates ratio to ~500
// for commits with zero deletions (net-new files). The correct bounded formula is:
// (additions - deletions) / (additions + deletions) = (500 - 0) / (500 + 0) = 1.0
const SHA2 = 'b'.repeat(40);
const SHA3 = 'c'.repeat(40);
const newFileNumstat = `500\t0\tsrc/newfile.js`;
mockExecSequence(
FAKE_ROOT,
FAKE_REMOTE,
' feature/x',
[
`${SHA}|2024-01-14T10:00:00Z|Dev|feat: new file one`,
`${SHA2}|2024-01-15T10:00:00Z|Dev|feat: new file two`,
`${SHA3}|2024-01-16T10:00:00Z|Dev|feat: new file three`
].join('\n'),
newFileNumstat,
newFileNumstat,
newFileNumstat
);

await collectLocalMetrics();

const summaryCall = fs.writeFileSync.mock.calls.find(c => c[0].includes('local_metrics_summary'));
const summary = JSON.parse(summaryCall[1]);

// New field name: net_additions_ratio_median
expect(summary.net_additions_ratio_median).toBeDefined();
// Must be bounded: 1.0 means 100% net-new code, not 500 (the broken formula's result)
expect(summary.net_additions_ratio_median).toBeCloseTo(1.0, 5);
expect(summary.net_additions_ratio_median).toBeLessThanOrEqual(1.0);
expect(summary.net_additions_ratio_p90).toBeLessThanOrEqual(1.0);
});

test('deduplicates commits with the same SHA across branches', async () => {
// Two branches both surface the same commit SHA
mockExecSequence(
Expand Down
14 changes: 10 additions & 4 deletions local-code-metrics.js
Original file line number Diff line number Diff line change
Expand Up @@ -177,8 +177,14 @@ async function collectLocalMetrics() {
const dates = metrics.map(m => m.date);
const velocity = computeVelocity(dates);

// Additions ratio distribution
const ratios = metrics.map(m => m.total_additions / (m.total_deletions || 1));
// Net additions ratio distribution: (additions - deletions) / (additions + deletions)
// Bounded [-1, +1]: 1.0 = entirely net-new code, 0.0 = balanced, negative = net deletion (cleanup)
// Replaces the unbounded additions / max(deletions, 1) formula, which inflated ratios to ~500
// for net-new-file commits (zero deletions), distorting both median and p90.
const ratios = metrics.map(m => {
const total = m.total_additions + m.total_deletions;
return total === 0 ? 0 : (m.total_additions - m.total_deletions) / total;
});
const ratioStats = computeStatistics(ratios, timestamps);

// Message quality
Expand Down Expand Up @@ -242,8 +248,8 @@ async function collectLocalMetrics() {
commit_size_trend: lineStats.trend,
velocity_commits_per_day: velocity.commits_per_day,
velocity_trend: velocity.trend,
additions_ratio_median: ratioStats.p50,
additions_ratio_p90: ratioStats.p90,
net_additions_ratio_median: ratioStats.p50,
net_additions_ratio_p90: ratioStats.p90,
message_quality_pct,
dora_archetype: classifyDoraArchetype({ large_commits_pct, sprawling_commits_pct, test_first_pct, message_quality_pct }),
config: CONFIG,
Expand Down
37 changes: 24 additions & 13 deletions metrics-specification.md
Original file line number Diff line number Diff line change
Expand Up @@ -211,34 +211,45 @@ velocity_trend : "accelerating" | "stable" | "decelerating"

---

### Metric 7: Additions-to-Deletions Ratio Distribution
### Metric 7: Net Additions Ratio Distribution

**What it measures**: The median and 90th percentile of the per-commit ratio of lines added to lines deleted. High ratios indicate that new code is being added without commensurate refactoring or removal of replaced code. This is the systematic batch-acceptance pattern DORA associates with architectural debt accumulation.
**What it measures**: The median and 90th percentile of the per-commit net additions ratio — what fraction of all lines changed in a commit are net-new code. A high ratio indicates new code is being added without commensurate refactoring or removal of replaced code. This is the systematic batch-acceptance pattern DORA associates with architectural debt accumulation.

**Formula**:
```
per-commit ratio = total_additions / max(total_deletions, 1)
additions_ratio_median = quantile(all_ratios, 0.5)
additions_ratio_p90 = quantile(all_ratios, 0.9)
per-commit ratio = (total_additions - total_deletions) / (total_additions + total_deletions)
net_additions_ratio_median = quantile(all_ratios, 0.5)
net_additions_ratio_p90 = quantile(all_ratios, 0.9)
```

The formula is bounded **[-1.0, +1.0]**:
- `1.0` — commit is entirely net-new code (all additions, zero deletions)
- `0.0` — perfectly balanced additions and deletions
- negative — net deletion (cleanup or refactoring that removes more than it adds)

Zero-churn commits (no additions and no deletions) are assigned `0.0`.

**Fields**:
```
additions_ratio_median : float (median ratio across all commits)
additions_ratio_p90 : float (90th percentile ratio)
net_additions_ratio_median : float (median ratio across all commits; bounded [-1, +1])
net_additions_ratio_p90 : float (90th percentile ratio; bounded [-1, +1])
```

**Data source**: `total_additions` and `total_deletions` already collected per commit; no new git calls required

**Thresholds**:
| Range (median) | Signal |
|----------------|--------|
| < 2.0 | Healthy: balanced additions and deletions |
| 2.0–3.0 | Monitor: additions outpacing deletions |
| > 3.0 | Warning: systematic batch-acceptance pattern |
| < 0.33 | Healthy: balanced additions and deletions |
| 0.33–0.50 | Monitor: additions outpacing deletions |
| > 0.50 | Warning: >50% of all churn is net-new code; systematic batch-acceptance pattern |

The 0.50 threshold maps exactly to the prior threshold of 3.0: when `additions = 3 × deletions`, `(3d - d) / (3d + d) = 0.50`.

**Relationship to existing heuristic**: The existing `generateInsights()` function counts commits where `large_commit AND additions > deletions × 3` as "possible AI commits." This metric expresses the same pattern at the aggregate level with a distribution, so outlier commits don't distort the reading.

**Why not `additions / max(deletions, 1)`**: The prior formula yielded ratios approaching `total_additions` for net-new-file commits (zero deletions). A commit adding 500 lines to a new file produced ratio=500, collapsing median and p90 toward the maximum addition count rather than the signal. The bounded formula eliminates this distortion: the same commit correctly produces `(500-0)/(500+0) = 1.0`.

---

### Metric 8: Commit Message Quality Score
Expand Down Expand Up @@ -567,9 +578,9 @@ Single summary object for the analysis run:
velocity_commits_per_day: number,
velocity_trend: "accelerating" | "stable" | "decelerating",

// Additions ratio distribution (new)
additions_ratio_median: number,
additions_ratio_p90: number,
// Net additions ratio distribution (bounded [-1, +1])
net_additions_ratio_median: number,
net_additions_ratio_p90: number,

// Message quality (new)
message_quality_pct: string, // "XX.XX"
Expand Down
Loading