github-credit-score

Turn a public GitHub profile into a 300–850 developer credit score.

This is the scoring model behind Vaaya's developer credit line, open-sourced so the people it scores can see exactly how it works — and help make it better. It reads only public GitHub signals (with the developer's own read:user token), weighs what's hard to fake, and uses an LLM to judge the quality of real code, not just the count of it.

We're publishing it because a credit score you can't inspect isn't one you should trust. If you think a factor is wrong, a weight is off, or the code judge is too harsh or too soft — open an issue or a PR. This repo is an open invitation to argue with our math.

Scope — what this repo is and isn't. This repo is the score: the factors that describe a developer and how they combine into 300–850. It is not the money. How a score becomes a dollar credit line, an exposure limit, or an approval decision is a lender-side risk choice and lives on Vaaya's side, not here. You can reproduce the score; you can't read our risk appetite off it.

The model

The score is built from three components, each normalized to [0,1] and then weighted. The design principle: weight what a third party had to accept.

Component	Weight	What it measures	Why
C — Contribution	0.40	External merged PRs, distinct external repos, reviews for others	Hardest to fake — someone else had to merge your work into their repo.
W — Work quality	0.30	An LLM reads real code diffs, authored issues, and commit messages	Catches craft that counts miss. Self-authored, so it's weighted below C.
P — Provenance	0.30	Account age, active-month density, 2FA + verified email	The sybil floor: cheap to earn honestly over time, expensive to fake at scale.

Each countable signal is passed through a saturating curve sat(x, k) = 1 − e^(−x/k), so early progress counts and grinding the long tail doesn't. Then:

1. Adverse gate (multiplicative, applied last): fresh accounts (< 3 months) and accounts with zero externally accepted work are heavily discounted, each anomaly flag compounds, and the whole gate is floored so recovery is always possible.
2. Response curve + top-end compression: a mild concave lift restores resolution in the crowded low-mid range; above the top threshold the score approaches 850 only asymptotically, so 800+ takes near-perfection on every axis at once. A literally maxed profile lands ≈820; 850 is an asymptote.

Everything is transparent and in two small files:

• src/score.ts — the pure math (computeScore, scoreDrivers).
• src/config.ts — every weight, saturation constant, and gate.

The work-quality judge

The most interesting — and most improvable — surface is the LLM judge in src/quality.ts. It's given real samples (actual diffs, commit messages, authored issues/PRs) and returns three 0–100 subscores under a strict JSON schema, with a calibration bar written into the prompt ("50–60 is a competent working developer; 80+ is rare craftsmanship; polished prose over shallow diffs is not high quality"). If the judge can't run — no API key, thin data, any failure — it returns null and the score renormalizes over C and P. The credit path never breaks because the judge did.

The full prompt is right there in the source. Think it's biased or gameable? That's exactly the kind of PR we want.

Install

npm install @vaaya/github-credit-score

Usage

import {
  fetchScoreInputs,
  fetchWorkSamples,
  judgeWorkQuality,
  computeScore,
  scoreDrivers,
} from '@vaaya/github-credit-score'

// A GitHub OAuth token for the user being scored (scopes: read:user, user:email).
const token = process.env.GITHUB_TOKEN!

// 1. Countable signals (P + C) — a few GitHub API round trips.
const base = await fetchScoreInputs(token)

// 2. Optional LLM work-quality judge (W). Omit the key to score on C/P alone.
const quality = await judgeWorkQuality(await fetchWorkSamples(token), {
  apiKey: process.env.OPENAI_API_KEY,
  model: 'gpt-4o-mini',
})

// 3. Score.
const breakdown = computeScore({ ...base, quality })
console.log(breakdown.score)               // e.g. 712
console.log(scoreDrivers(breakdown)[0])    // strongest factor first

computeScore is pure and synchronous — you can feed it hand-built ScoreInputs with no network at all, which is how the test suite exercises the whole model.

Persistence & the credit line (not included, by design)

Vaaya wraps this score with a persistence + state layer (enrollment, a slow "score" vs. a fast trust ratio that grows on clean use and drops hard on bad events, decay, and the mapping to an actual line). That layer is not in this repo — it's coupled to our database, payments, and risk policy, and the dollar mapping is intentionally private. What you get here is everything needed to compute and reproduce the score itself. To wire it into your own product, provide two things: a source of the user's GitHub token, and somewhere to store the resulting ScoreBreakdown.

Develop

npm install
npm test          # vitest — the pure scoring math
npm run typecheck

Why we built this

Vaaya gives developers' coding agents a prepaid balance and a credit line to spend on real tools. The credit line is earned from your GitHub track record — this score is how. Curious? Try it.

License

MIT © Vaaya