Migrate IpfsPinningService's pin registry off in-memory storage

[balisdev]

Owner: whoever owns IPFS/storage (unassigned) · Priority: Medium — does not block rollout of the Redis approach elsewhere, but the raw-content sub-decision below should land before this migrates, not after.

Description

IpfsPinningService (src/ipfs-pinning/ipfs-pinning.service.ts) keeps two process-local stores: Map<cid, PinRecord> (pin metadata/status, read by RepinWorkerService to find under-replicated pins) and Map<cid, Buffer> (the original uploaded bytes, retained so a re-pin retry doesn't need the original caller to re-upload). Follow-up from the persistence spike (#181) — see backend/PERSISTENT_STORAGE_SPIKE.md (§2, §7).

These two stores don't have the same shape and shouldn't be migrated the same way:

• PinRecord metadata is a good Redis fit — small JSON per CID, point lookups, filter by replication status. Same pattern as GigService (Spike: Persistent, multi-instance-safe storage strategy to replace in-memory service state #181): SET pin:{cid} <json>, plus a set/sorted-set index for "needs re-pin" queries RepinWorkerService runs.
• Raw content Buffers are a poor fit for Redis as general-purpose KV storage — binary blobs bloat Redis memory fast, and this isn't what Redis is for. Needs its own decision: either move raw content to object storage (S3-compatible) with only a reference stored alongside the pin metadata, or drop the retry-without-refetch behavior and require the caller to resupply content on a failed re-pin. Get a rough per-record size × expected volume estimate before committing either way (see the spike's §7 sizing note).

Tasks

• Migrate PinRecord metadata to Redis following the GigService pattern (Spike: Persistent, multi-instance-safe storage strategy to replace in-memory service state #181) — including its transaction-result verification, fallback metric, and production fail-fast behavior
• Decide the raw-content Buffer map's fate: object storage vs. dropping retry-without-refetch (flag any new infra/cost implications — this may need its own follow-up rather than landing here)
• Update RepinWorkerService's under-replicated-pin query to use the new store (note: the distributed-lock work for this worker is tracked separately in Add distributed lock so background sweep workers run as a single active instance #182, not here)
• Add the same graceful-fallback-to-memory behavior on Redis errors, logged at error level, matching GigService/NonceStoreService

Acceptance Criteria

• Pin metadata survives a process restart
• Multiple backend instances observe the same pin state
• A decision on raw-content storage is documented, with a follow-up filed if it needs its own implementation work
• Existing ipfs-pinning.service.spec.ts / repin-worker.service.spec.ts tests still pass, updated for the new store

Estimated Time

2-3 days

[Peculiarr7]

Hi! I came across this issue and would love to work on it. I have experience with similar tasks and I'm confident I can investigate, implement a clean solution, and thoroughly test it before submitting a PR. If the issue is still open, I would appreciate it if you could assign it to me. I’ll prioritize this immediately and ensure a reliable solution is delivered as soon as possible. Thank you!

[blessedcodey-boy]

Hey Maintainer !!!!
I just went gone through the details of the issue and i will be capable of resolving it, I would love to be assigned to work on it. Thanks