feat: launch ingesters per enabled network in the bootstrap (#117) - #155
Open
gideonpius7-design wants to merge 1 commit into
Open
Conversation
…56#117) src/index.ts previously started exactly one instance of each venue ingester (SDEX, AMM, Soroswap, Snapshot, Aquarius), always bound to whatever network was active at startup. To index more than one network in the same process, each ingester now runs once per enabled network. - src/network/enabledNetworks.ts (new): getEnabledNetworks() reads the comma-separated ENABLED_NETWORKS env var (e.g. "testnet,mainnet"), trims, lowercases, dedupes and filters to known network names. When unset — or nothing valid remains — it falls back to [activeNetwork], so a single-network deployment starts exactly one instance of each ingester. - src/index.ts: loops over the enabled networks and starts each ingester with its network context. restartIngester now captures the specific (venue, network) pair, so a crash in one instance restarts only that instance and cannot affect the others. - src/ingesters/snapshot.ts: appendSnapshots() / startSnapshotIngester() take a network arg. price_snapshots rows are network-scoped via the (network, pair, ts) PK, so a single instance could only ever snapshot the active network — each enabled network now gets its own loop. - src/ingest/venues/aquarius.ts: startAquariusIngester() / ingestAquariusPair() take a network arg and resolve the per-network Aquarius block via getNetworkConfig(network). - .env.example: documents ENABLED_NETWORKS. - src/__tests__/enabledNetworks.test.ts (new): covers the fallback, explicit multi-network lists, whitespace/casing normalization, dedupe and filtering of unrecognized names. closes Miracle656#117 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019ky4A59maw6dB2CptXyhdW
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
src/index.tspreviously started exactly one instance of each of the five venue ingesters (SDEX, AMM, Soroswap, Snapshot, Aquarius), always bound to whatever network was active at startup. To index more than one network in the same process, each ingester now runs once per enabled network.The per-network client factories (#139) and per-network venue config (#138) this builds on are already on
main, so this diff is just the orchestration change.Changes
src/network/enabledNetworks.ts(new):getEnabledNetworks()reads the comma-separatedENABLED_NETWORKSenv var (e.g.testnet,mainnet), trims, lowercases, dedupes, and filters to known network names. When unset — or when nothing valid remains after filtering — it falls back to just the currently active network (STELLAR_NETWORK), so a single-network deployment starts exactly one instance of each ingester rather than one per network.src/index.ts: loops over the enabled networks and starts SDEX, AMM, Soroswap, Snapshot, and Aquarius once per network.restartIngester's retry closure now captures the specific(venue, network)pair, so a crash in, say, Soroswap on mainnet only restarts that one instance and cannot affect the ingesters running on testnet.src/ingesters/snapshot.ts:appendSnapshots()/startSnapshotIngester()now take anetworkargument.price_snapshotsrows are network-scoped via the(network, pair, ts)primary key, so a single shared instance could only ever snapshot the active network — each enabled network now gets its own loop.src/ingest/venues/aquarius.ts:startAquariusIngester()/ingestAquariusPair()take anetworkargument and resolve the per-network Aquarius block viagetNetworkConfig(network)(enabled flag + API URL)..env.example: documentsENABLED_NETWORKS.src/__tests__/enabledNetworks.test.ts(new): covers the default fallback, explicit multi-network lists, whitespace/casing normalization, deduplication, and filtering out unrecognized network names.Acceptance criteria (#117)
restartIngesterkeyed on(venue, network))ENABLED_NETWORKSunset →[activeNetwork])Testing
npx tsc --noEmitpassesnpx vitest runpasses — 312 passed, 1 skipped (7 new tests inenabledNetworks.test.ts)closes #117
🤖 Generated with Claude Code