[Performance] Eliminate N+1 query patterns in S3 scheduler and email queue flushing

[chromonav]

Branch reviewed: s3

Problem

Multiple loops fetch docs/rows individually (frappe.get_doc / SQL inside loop), producing N+1 query amplification for larger workloads.

Evidence (file + line)

• aws_integration/aws_integration/s3/scheduler.py#L76 (frappe.get_doc in loop)
• aws_integration/aws_integration/s3/scheduler.py#L233 (frappe.get_doc in loop)
• aws_integration/aws_integration/s3/scheduler.py#L235 (frappe.db.sql in loop)
• aws_integration/aws_integration/s3/scheduler.py#L338 (frappe.get_all in looped batches)
• aws_integration/aws_integration/s3/scheduler.py#L351 (frappe.get_doc in loop)
• aws_integration/aws_integration/utils/email.py#L132
• aws_integration/aws_integration/utils/email.py#L135

Risk / impact

• Throughput drops as file/email volume increases.
• Scheduler jobs can exceed timeout windows on busy sites.
• Unnecessary DB load impacts unrelated request latency.

Replication suggestion

1. Seed large pending queues (e.g., 5k+ files / email queue rows).
2. Run scheduler/email jobs with SQL query logging enabled.
3. Compare query count and wall time before/after batching/prefetch.

Resolution suggestion

• Prefetch records with frappe.get_all(..., fields=...) in chunks and materialize maps.
• Avoid repeat get_doc for error logging paths.
• Replace per-item SQL lock queries with grouped/claim-based selection where possible.
• Add explicit pagination strategy and bounded chunk size.

Acceptance criteria

• Query count scales ~linearly with chunk count, not item count.
• Large-run benchmark shows measurable drop in total queries and execution time.
• No behavior regression in retry/error handling.