Skip to content

Conversation

@mattpocock
Copy link
Owner

Fixes #234

This PR adds automatic .env file support to Evalite. Users no longer need to manually configure dotenv in their evalite.config.ts.

Changes

  • Add dotenv as a dependency
  • Create env-setup-file module that imports dotenv/config
  • Export env-setup-file as 'evalite/env-setup-file'
  • Automatically prepend env-setup-file to setupFiles array
  • Update documentation to reflect automatic .env loading
  • Update example config to remove manual dotenv setup

Testing

After merging, users can simply:

  1. Create a .env file in their project root
  2. Add their API keys (e.g., OPENAI_API_KEY=your-key)
  3. Run evalite - no configuration needed!

🤖 Generated with Claude Code

- Add dotenv as a dependency
- Create env-setup-file module that imports dotenv/config
- Export env-setup-file as 'evalite/env-setup-file'
- Automatically prepend env-setup-file to setupFiles array
- Update documentation to reflect automatic .env loading
- Update example config to remove manual dotenv setup

Fixes #234

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Matt Pocock <[email protected]>
@changeset-bot
Copy link

changeset-bot bot commented Oct 19, 2025

🦋 Changeset detected

Latest commit: 1ba5a23

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
evalite Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel
Copy link

vercel bot commented Oct 19, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Preview Comments Updated (UTC)
evalite Ignored Ignored Oct 19, 2025 7:01pm

💡 Enable Vercel Agent with $100 free credit for automated AI reviews

@mattpocock
Copy link
Owner Author

@claude Setup files from vitest.config.ts should also be supported, and added to the array. However - evalite.config.ts setupFiles should take precedence. Add tests to this effect.

@claude
Copy link

claude bot commented Oct 19, 2025

Claude finished @mattpocock's task —— View job


Setup files support implementation complete!

I've successfully added support for setupFiles from vitest.config.ts with evalite.config.ts taking precedence.

Changes Made

Core Implementation:

  • Added loadVitestSetupFiles() function in config.ts:92-129 to extract setupFiles from vitest.config.ts
  • Modified run-evalite.ts:211-241 to merge setupFiles from both configs
  • setupFiles execution order ensures proper precedence:
    1. evalite/env-setup-file (always first for .env support)
    2. setupFiles from vitest.config.ts
    3. setupFiles from evalite.config.ts (runs last, takes precedence)

Test Coverage:

  • Created config-setupfiles-vitest fixture to test vitest.config.ts setupFiles support
  • Created config-setupfiles-precedence fixture to test that evalite.config.ts overrides vitest.config.ts
  • Added two comprehensive test cases in config.test.ts:54-84

The implementation ensures that when both configs define setupFiles, they all run in order, with evalite.config.ts files running after vitest.config.ts files. This means if both set the same environment variable, the evalite.config.ts value wins (as demonstrated by the precedence test).

Files Changed

  • packages/evalite/src/config.ts - Added vitest config loading
  • packages/evalite/src/run-evalite.ts - Merged setupFiles logic
  • packages/evalite-tests/tests/config.test.ts - Added 2 new tests
  • Created 2 new test fixtures with 9 files

Committed and pushed to the PR branch!


Job RunBranch

@pkg-pr-new
Copy link

pkg-pr-new bot commented Oct 19, 2025

Open in StackBlitz

npm i https://pkg.pr.new/mattpocock/evalite@243

commit: 1ba5a23

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the final PR Bugbot will review for you during this billing cycle

Your free Bugbot reviews will reset on November 18

Details

You are on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

const setupFiles = [
"evalite/env-setup-file",
...(config?.setupFiles || []),
];
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Vitest Setup Files Override Vite Configuration

The setupFiles array is now always initialized with "evalite/env-setup-file", even when not specified in evalite.config.ts. This unconditionally sets Vitest's setupFiles, overriding any values configured in vite.config.ts that would otherwise apply.

Fix in Cursor Fix in Web

… precedence

- Add loadVitestSetupFiles() to load setupFiles from vitest.config.ts
- Merge setupFiles from both configs with evalite.config.ts taking precedence
- Add tests for vitest.config.ts setupFiles support and precedence
- setupFiles execution order: env-setup-file -> vitest -> evalite

Co-authored-by: Matt Pocock <[email protected]>
@mattpocock
Copy link
Owner Author

Closed, cherrypicked into #240

@mattpocock mattpocock closed this Oct 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support .env files by default via dotenv/config

2 participants