Automatically discover package requirements from extra-requirements.txt files

[LestlinRobins]

Summary

Fixes #451

Automatically discovers package-level extra-requirements.txt files in both dev_tools/write-ci-requirements.py and setup.py, removing the need to maintain hard-coded package lists.

Changes

• Uses pathlib.Path.glob() to discover requirements files.
• Derives each extra's name from its parent directory.
• Preserves the existing QAOA-specific handling for the next Cirq version.

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[LestlinRobins]

I looked into the CI failure. It appears to be crashing during pytest collection, with workers terminating due to a Fatal Python error: Illegal instruction in fqe.lib.fqe_data, before any tests run.

This seems separate from the requirements-discovery changes in this PR. Could you confirm whether this is a known CI/environment issue?

[mhucka]

I looked into the CI failure. It appears to be crashing during pytest collection, with workers terminating due to a Fatal Python error: Illegal instruction in fqe.lib.fqe_data, before any tests run.

This seems separate from the requirements-discovery changes in this PR. Could you confirm whether this is a known CI/environment issue?

You're right. Sometimes the GitHub runners flake out, possibly due to the worsening strain on their infrastructure. Rerunning the CI jobs is necessary in those cases.

If you suspect this in the future, you should be able to cause the CI to rerun by pushing an empty commit:

git commit --allow-empty -m "Trigger CI"
git push origin <your-branch-name>

[mhucka]

This PR is close to being ready. A couple of change requests (plus see the inline comment):

1. Add the following to MANIFEST.in:

   recursive-include recirq extra-requirements.txt

2. Reduce the PR description. The obviously LLM-generated stuff is mostly useless. The first 2 sentences are enough.

[LestlinRobins]

Hi @mhucka, all CI checks are now passing, and I've also addressed the changes you suggested. Whenever you're available, could you please review and merge the PR? Thanks!

[mhucka]

Thanks again.

[LestlinRobins]

Thanks for approving, @mhucka, could you please merge this as well?

[LestlinRobins]

Thank you @mhucka :)