fix: Header name Zyte-Error → Zyte-Error-Type

[Laerte]

fix: #132
fix: #131

[Gallaecio]

Besides addressing the CI issues (assuming they are valid and related to this change), before we merge, have you tried this branch on your code, made sure it works as expected? Just to be sure there is no other issue in that scenario, since this is not an issue where tests can help much.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (8b16f37) to head (c76ba2d).
Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff            @@
##            master      #133   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            3         3           
  Lines          323       323           
=========================================
  Hits           323       323           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Gallaecio]

I just experienced the twine error myself in a different repo, upgrading the version of twine in tox.ini will solve that.

[Gallaecio]

The pre-commit issue is easy to address, tox -e pre-commit. I wonder why we have 2 CI jobs for pre-commit…

The min job issue appears a bit more involved, and CI-specific. Shall I take over to address it?

[Laerte]

Install package from my code:

pip install -e git+http://github.com/Laerte/scrapy-zyte-smartproxy.git@master#egg=scrapy-zyte-smartproxy

Run the spider:

scrapy runspider example.py

from uuid import uuid4

from scrapy import Spider
from scrapy.http import Request


class ExampleSpider(Spider):
    name = "example_spider"

    custom_settings = {
        "ZYTE_SMARTPROXY_ENABLED": True,
    }

    def start_requests(self):
        yield Request(
            "https://httpbin.org/status/429",
        )

    def parse(self, response):
        pass

With this I see the zyte_api_proxy/response/banned=1 stats.

[Gallaecio]

Shall I take over, to try and solve the CI issue?

[Laerte]

Shall I take over, to try and solve the CI issue?

Now the error is not related to CI but with one failing test: test_is_banned because stats is not incremented (because short-circuit on if not self._is_zyte_smartproxy_or_zapi_response(response)) but tbh I confused with this function, the idea here is check if smartproxy (crawlera) or Zyte API response?

[Laerte]

Shall I take over, to try and solve the CI issue?

Now the error is not related to CI but with one failing test: test_is_banned because stats is not incremented (because short-circuit on if not self._is_zyte_smartproxy_or_zapi_response(response)) but tbh I confused with this function, the idea here is check if smartproxy (crawlera) or Zyte API response?

Just to clarify and add this to docstring:
_is_zyte_smartproxy_or_zapi_response -> Checks if is a response from Crawlera or Zyte Smart Proxy? Because the current name it feels like Zyte API (which don't make sense since Zyte API is another thing).

[Gallaecio]

scrapy-zyte-smartproxy is a package that allows using 2 different proxy APIs from Zyte: Smart Proxy Manager and Zyte API proxy mode. In the context of this package, Zyte API always refers specifically to the proxy mode, i.e. the proxy API of Zyte API (which also has an HTTP API).

[Laerte]

scrapy-zyte-smartproxy is a package that allows using 2 different proxy APIs from Zyte: Smart Proxy Manager and Zyte API proxy mode. In the context of this package, Zyte API always refers specifically to the proxy mode, i.e. the proxy API of Zyte API (which also has an HTTP API).

Cool, added docstring to be clear for everyone else as well.

[Laerte]

@Gallaecio seems we tried the same thing at same time, I tested with lower-version OS did not work but explicit installing gcc did. 😆

[Laerte]

@Gallaecio @wRAR May I ask, when we will have a new version of the lib with this change? Thank in advance.

[Gallaecio]

Early next week, hopefully before Monday end-of-day.