Skip to content

Fix non-interactive (rake / rails runner) audit on Evergreen - #9

Merged
grantcox merged 3 commits into
mainfrom
stagility/runner-detection-partial-rake
Sep 29, 2026
Merged

grantcox merged 3 commits into
mainfrom
stagility/runner-detection-partial-rake

Conversation

@grantcox

@grantcox grantcox commented Sep 28, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

On Evergreen, rails runner and rake commands produce no noninteractive_command record. Interactive consoles and the same commands on datadog-proxy-staging work. Both failures below are swallowed by the gem's fail-open rescues, so the only symptom is a warning on the one-off dyno's stderr.

1. Rake is only partially loaded

[console-audit] noninteractive audit failed (ignored): undefined method 'application' for module Rake

NoninteractiveAudit.detect checked defined?(Rake) before calling Rake.application. Evergreen requires rails/test_unit/railtie, and in Rails 8.1 that chain ends in require "rake/file_list". That defines Rake without Rake.application.

Fix: check defined?(Rake.application) instead.

2. The enqueue runs before rails_semantic_logger has patched ActiveJob

[console-audit] failed to queue noninteractive_command (ignored): undefined method 'current_tags' for nil

The Railtie's after_initialize is registered when Bundler requires the gem, so it runs before the hooks of gems listed later in the Gemfile. rails_semantic_logger swaps Rails.logger for a SemanticLogger during initializers, but only patches ActiveJob's logging in its own after_initialize. Until that patch runs, ActiveJob's logger_tagged_by_active_job? calls logger.formatter.current_tags, and a SemanticLogger has no formatter. In Evergreen, rails_semantic_logger sits in the production group below console_audit. In datadog-proxy it comes first, which is why datadog-proxy worked. Interactive consoles enqueue long after boot, so they're unaffected.

Fix: the non-interactive audit's after_initialize is now registered from an initializer. Initializers run after Bundler.require, so the hook runs after every gem's after_initialize, whatever the Gemfile order.

Tests

  • noninteractive_audit_spec: a bare Rake module still detects rails runner.
  • railtie_spec: boots a forked app with an after_initialize hook registered after console_audit loads, standing in for a later gem. The spec checks that the audit runs after that hook, and not at all when auditing is disabled.

Both new specs fail without their fix. The full suite (72 examples) and standardrb pass.

Manual testing

I have tested these changes on Evergreen PR https://github.com/ynab/evergreen/pull/30446 , working through our full console auditing test script.

Release

This bumps VERSION to 1.0.1 and moves the changelog entries under 1.0.1. Tag v1.0.1 on main after merge. Evergreen then needs to pin console_audit to that tag.

🤖 Generated with Claude Code

@grantcox grantcox changed the title Detect rails runner when Rake is only partially loaded Fix non-interactive (rake / rails runner) audit on Evergreen Sep 28, 2026
grantcox and others added 3 commits September 29, 2026 10:51
`require "rake/file_list"` (pulled in by rails/test_unit/railtie) defines
the Rake module without Rake.application, so detection raised NoMethodError
and the fail-open rescue silently skipped the noninteractive_command record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Railtie's after_initialize is registered at Bundler.require, so it ran
before hooks from gems later in the Gemfile. rails_semantic_logger patches
ActiveJob's logging in its after_initialize; until then ActiveJob calls
`logger.formatter.current_tags` on a SemanticLogger logger with no formatter,
so every rake / rails runner enqueue raised and failed open.

Registering the hook from an initializer places it after all gem hooks,
regardless of Gemfile order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@grantcox
grantcox force-pushed the stagility/runner-detection-partial-rake branch from 815a9f8 to bc74dc1 Compare September 29, 2026 00:52
@grantcox
grantcox marked this pull request as ready for review September 29, 2026 00:56
@grantcox
grantcox requested a review from becky-ynab September 29, 2026 00:56

@becky-ynab becky-ynab left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great

@grantcox
grantcox merged commit 6a28b86 into main Sep 29, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants