Skip to content

Fix test suite stability and extended test configuration - #308

Open
MasumRab wants to merge 2 commits into
mainfrom
fix-test-suite-stability-3968941239566233894
Open

Fix test suite stability and extended test configuration#308
MasumRab wants to merge 2 commits into
mainfrom
fix-test-suite-stability-3968941239566233894

Conversation

@MasumRab

@MasumRab MasumRab commented Feb 1, 2026

Copy link
Copy Markdown
Owner

This PR addresses test stability and configuration issues in the backend test suite.

  1. Fixed RateLimiter Flakiness: The test_rate_limiter.py tests were relying on the system clock (datetime.now()), which could lead to failures if tests ran across midnight Pacific Time. This was fixed by using unittest.mock.patch to freeze time during tests.
  2. Fixed make extended_tests: The Makefile target extended_tests used the --only-extended flag, which was not recognized by pytest. I added the necessary hooks in backend/tests/conftest.py to register this option and correctly filter tests marked with @pytest.mark.extended.
  3. Corrected Test Marking: backend/tests/test_utils_hypothesis.py contains property-based tests that should be part of the extended suite but were missing the marker. I added the marker to ensure they are excluded from the standard test run and included in the extended run.

These changes ensure that make test and make extended_tests work as intended and that the test suite is deterministic.


PR created automatically by Jules for task 3968941239566233894 started by @MasumRab

Summary by CodeRabbit

  • Tests
    • Enhanced test isolation by implementing datetime mocking for rate limiter tests.
    • Introduced CLI option to selectively run extended test suites.
    • Categorized hypothesis-based tests for organized test execution.

✏️ Tip: You can customize this high-level summary in your review settings.

- Refactor `backend/tests/agent/test_rate_limiter.py` to use `unittest.mock` for `datetime`, ensuring deterministic behavior at daily boundaries.
- Update `backend/tests/conftest.py` to implement `pytest_addoption` and `pytest_collection_modifyitems` for the `--only-extended` flag.
- Add `pytestmark = pytest.mark.extended` to `backend/tests/test_utils_hypothesis.py` to correctly exclude these property-based tests from standard runs.
- Verify full test suite (backend standard, extended, and frontend) passes.

Co-authored-by: MasumRab <8943353+MasumRab@users.noreply.github.com>
@google-labs-jules

Copy link
Copy Markdown

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

@coderabbitai

coderabbitai Bot commented Feb 1, 2026

Copy link
Copy Markdown

Walkthrough

This PR introduces a testing infrastructure enhancement that allows for extended test filtering via a new --only-extended CLI option. Additionally, datetime mocking is added to specific rate limiter tests for deterministic time-based testing, and extended tests are marked to participate in the new filtering mechanism.

Changes

Cohort / File(s) Summary
Rate Limiter Test Mocking
backend/tests/agent/test_rate_limiter.py
Introduces @patch decorator to mock datetime in test_daily_reset_logic and test_no_reset_same_day methods for deterministic time-based testing and fixed timestamp assertions.
Extended Test Infrastructure
backend/tests/conftest.py
Adds three pytest hooks (pytest_addoption, pytest_configure, pytest_collection_modifyitems) to implement --only-extended CLI option for selective test execution with custom marker registration.
Extended Test Marking
backend/tests/test_utils_hypothesis.py
Applies pytestmark = pytest.mark.extended to mark all tests in module for participation in extended test filtering.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 Datetime dances through the tests,
Extended filters pass the quests,
Pytest hooks hop into the fold,
Testing stories waiting to be told! 📖

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically summarizes the main changes: fixing test stability issues in the rate limiter through mocking and implementing extended test configuration support via pytest hooks.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch fix-test-suite-stability-3968941239566233894

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@MasumRab

Copy link
Copy Markdown
Owner Author

Closing as redundant; test suite stability and extended config already fixed in PR #326.

@MasumRab MasumRab closed this Feb 17, 2026
@google-labs-jules

Copy link
Copy Markdown

Closing as redundant; test suite stability and extended config already fixed in PR #326.

Understood. Acknowledging that this work is now obsolete and stopping work on this task.

- Refactor `backend/tests/agent/test_rate_limiter.py` to use `unittest.mock` for `datetime`, ensuring deterministic behavior at daily boundaries.
- Update `backend/tests/conftest.py` to implement `pytest_addoption` and `pytest_collection_modifyitems` for the `--only-extended` flag.
- Add `pytestmark = pytest.mark.extended` to `backend/tests/test_utils_hypothesis.py` to correctly exclude these property-based tests from standard runs.
- Verify full test suite (backend standard, extended, and frontend) passes.

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@MasumRab

MasumRab commented Jun 9, 2026

Copy link
Copy Markdown
Owner Author

@jules Conflicts detected. Please rebase branch on main using:
git fetch origin && git rebase origin/main
Then push: git push --force-with-lease
Focus on your changed files only - avoid large diff processing.
Report when resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant