Skip to content

Add CLI integration tests — cli.ts and fetch.ts have 0% coverage, including the exit-code CI-gate feature #65

@dmchaledev

Description

@dmchaledev

Problem

The project's primary selling point is its use as a CI gate — runs like:

security-headers https://staging.example.com || echo "Gate failed"

rely on process.exit(1) being called for D/F grades (cli.ts:95). That exit-code path is completely untested today. So are argument parsing, --json output, --timeout, error handling, and the --version flag.

Coverage today:

Module Statement coverage
rules.ts ~100%
analyzer.ts ~100%
index.ts ~50% (only analyzeHeaders(object) path; analyze(string) with real fetch is never called)
fetch.ts 0%
cli.ts 0%

Proposed fix

Add a test/cli.test.ts (and optionally test/fetch.test.ts) using Vitest's built-in module mocking — no extra dependencies needed.

Sketch of test/cli.test.ts

import { vi, describe, it, expect, beforeEach, afterEach } from 'vitest';

// Mock fetch so tests never hit the network
vi.mock('../src/fetch.js', () => ({
  fetchHeaders: vi.fn(),
}));

import { fetchHeaders } from '../src/fetch.js';

// Helper: run main() with given argv, capture stdout/stderr, and catch process.exit
async function runCli(args: string[]): Promise<{ stdout: string; stderr: string; exitCode: number }> {
  const originalArgv = process.argv;
  const originalExit = process.exit;
  const stdoutLines: string[] = [];
  const stderrLines: string[] = [];
  let exitCode = 0;

  process.argv = ['node', 'cli.js', ...args];
  vi.spyOn(process, 'exit').mockImplementation((code?: number) => { exitCode = code ?? 0; throw new Error('process.exit'); });
  vi.spyOn(console, 'log').mockImplementation((...a) => stdoutLines.push(a.join(' ')));
  vi.spyOn(console, 'error').mockImplementation((...a) => stderrLines.push(a.join(' ')));

  try {
    // dynamically import to re-execute main() each test
    await import('../src/cli.js?t=' + Date.now());
  } catch { /* swallow process.exit throw */ }

  process.argv = originalArgv;
  process.exit = originalExit;
  vi.restoreAllMocks();

  return { stdout: stdoutLines.join('\n'), stderr: stderrLines.join('\n'), exitCode };
}

// Good-grade headers fixture
const A_HEADERS = { /* full set of strong headers */ };
// Bad-grade headers fixture (empty → F)
const F_HEADERS = {};

describe('CLI', () => {
  it('exits 0 for an A-grade site', async () => {
    vi.mocked(fetchHeaders).mockResolvedValue(A_HEADERS);
    const { exitCode } = await runCli(['https://example.com']);
    expect(exitCode).toBe(0);
  });

  it('exits 1 for an F-grade site (CI gate)', async () => {
    vi.mocked(fetchHeaders).mockResolvedValue(F_HEADERS);
    const { exitCode } = await runCli(['https://bad.example.com']);
    expect(exitCode).toBe(1);
  });

  it('--json outputs valid JSON containing grade and score', async () => {
    vi.mocked(fetchHeaders).mockResolvedValue(A_HEADERS);
    const { stdout } = await runCli(['https://example.com', '--json']);
    const report = JSON.parse(stdout);
    expect(report).toHaveProperty('grade');
    expect(report).toHaveProperty('score');
  });

  it('--version prints a semver string', async () => {
    const { stdout, exitCode } = await runCli(['--version']);
    expect(stdout).toMatch(/^\d+\.\d+\.\d+/);
    expect(exitCode).toBe(0);
  });

  it('--help prints usage and exits 0', async () => {
    const { stdout, exitCode } = await runCli(['--help']);
    expect(stdout).toContain('Usage');
    expect(exitCode).toBe(0);
  });

  it('missing URL exits 1 with usage message', async () => {
    const { stderr, exitCode } = await runCli([]);
    expect(exitCode).toBe(1);
    expect(stderr).toContain('Usage');
  });

  it('network error exits 1 with an error message', async () => {
    vi.mocked(fetchHeaders).mockRejectedValue(new Error('ECONNREFUSED'));
    const { stderr, exitCode } = await runCli(['https://unreachable.example.com']);
    expect(exitCode).toBe(1);
    expect(stderr).toContain('ECONNREFUSED');
  });

  it('--timeout passes the value through to fetchHeaders', async () => {
    vi.mocked(fetchHeaders).mockResolvedValue(A_HEADERS);
    await runCli(['https://example.com', '--timeout', '3000']);
    expect(fetchHeaders).toHaveBeenCalledWith('https://example.com', { timeoutMs: 3000 });
  });
});

test/fetch.test.ts (network layer)

Use Vitest's vi.stubGlobal('fetch', ...) to mock the global fetch, then verify that:

  • headers are collected and lower-cased
  • AbortController fires after timeoutMs elapses (use fake timers)
  • Response body is cancelled after headers are read

Why this is worth doing

  1. The CI-gate exit code is the Add Dev.to launch post for security-headers CLI tool #1 feature — one line of untested code (if (report.grade === 'D' || report.grade === 'F') process.exit(1)) is what every CI user depends on.
  2. --json correctness matters for downstream tooling — teams pipe the JSON into dashboards, scripts, and SIEM tools; a regression here would be silent.
  3. analyze(string) in index.ts is also uncovered — the end-to-end path from URL → fetch → analyze → report is never exercised, so a future refactor could break the library's primary API silently.
  4. Zero new runtime dependencies — Vitest already has vi.mock, vi.spyOn, and vi.stubGlobal; no msw or other packages needed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions