test(mongo): raise container nofile limit; tolerate neighbor noise in pool test#245
Closed
angela-helios wants to merge 1 commit into
Closed
test(mongo): raise container nofile limit; tolerate neighbor noise in pool test#245angela-helios wants to merge 1 commit into
angela-helios wants to merge 1 commit into
Conversation
… pool test Two stability fixes for the shared-container MongoDB suite, both observed as one-off failures on real runs: - The coverage job died with TooManyFilesOpen (error 264) during an index build: every test creates its own uniquely-named database and WiredTiger keeps file handles open per collection/index across all of them, so 50+ test databases exhaust the stock container nofile limit late in the run. Start the shared container with ulimit nofile=64000 (mongod own recommended minimum). - mongodb_integration_reuses_client_pool_under_concurrent_read_search asserts on serverStatus totalCreated, a server-global counter on the shared mongo, so concurrently running neighbor tests inflate it past the old 50 ceiling on wide runners. The regression it guards against (a fresh client per operation) creates at least 160 connections (8 tasks x 20 ops); raise the ceiling to 120 to keep that detectable while tolerating neighbor noise.
Contributor
Author
|
Closing: per review feedback this fix rides directly on the failing PR (#230) instead of a separate PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two stability fixes for the shared-container MongoDB integration suite, each observed as a real one-off failure:
TooManyFilesOpen(error 264) in the coverage job (seen on #230's run —mongodb_integration_settings_optimistic_lockdied during an index build). Every test creates its own uniquely-named database, and WiredTiger keeps file handles open per collection/index across all of them — 50+ test databases exhaust the stock containernofilelimit late in the run. The shared testcontainer now starts with--ulimit nofile=64000(mongod's own recommended minimum). Same tuning spot as the earlier WiredTiger cache cap.mongodb_integration_reuses_client_pool_under_concurrent_read_searchflaking on wide runners. It asserts onserverStatus.connections.totalCreated, a server-global counter on the shared mongo, so concurrently running neighbor tests (each with its own client pool) inflate it past the old ceiling of 50. The regression it guards against — a fresh client per operation — creates ≥160 connections (8 tasks × 20 ops), so a 120 ceiling keeps the signal while tolerating the noise.Full suite green locally against the ulimit-configured container: 51/51.
Note for in-flight PRs (#229/#230/#241/#242/#244): the coverage job runs each branch's own tree, so they'll want a
mainmerge after this lands to pick the fix up.