SSL verification for HTTPX client #95
Description
Contex
We are deploying FirecREST on GKE, using Keycloak as IdP. We've enabled TLS on keycloak. We need to provide our CA cert to F7t so it can access Keycloak's endpoints using HTTPS.
Issue
We are installing the the CA certificate in F7T's host and setting env variable CURL_CA_BUNDLE with the appropiate value. This allows F7T to get public certs from Keycloak, but we get the following failure in the health checker:
Traceback (most recent call last):
File "/usr/local/lib/python3.12/site-packages/apscheduler/_schedulers/async_.py", line 1201, in _run_job
retval = await job_executor.run_job(func, job)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/apscheduler/executors/async_.py", line 22, in run_job
retval = await retval
^^^^^^^^^^^^
File "/app/firecrest/status/health_check/health_checker_cluster.py", line 93, in check
raise ex
File "/app/firecrest/status/health_check/health_checker_cluster.py", line 49, in check
token = await client.fetch_token(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/authlib/integrations/httpx_client/oauth2_client.py", line 163, in _fetch_token
resp = await self.post(
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1859, in post
return await self.request(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/authlib/integrations/httpx_client/oauth2_client.py", line 119, in request
return await super().request(method, url, auth=auth, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1540, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1629, in send
response = await self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1657, in _send_handling_auth
response = await self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1694, in _send_handling_redirects
response = await self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1730, in _send_single_request
response = await transport.handle_async_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_transports/default.py", line 393, in handle_async_request
with map_httpcore_exceptions():
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/contextlib.py", line 158, in __exit__
self.gen.throw(value)
File "/usr/local/lib/python3.12/site-packages/httpx/_transports/default.py", line 118, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1010)
Explanation
F7T uses 2 different HTTP clients to interact with IdP:
requestsin OIDC_token_auth.pyhttpxin health_checker_cluster.py
While requests uses environment variables (like CURL_CA_BUNDLE) to check for CA certificates, httpx requires the explicit creation of an SSL context in order to verify the identity of the host.
Solution
Create an ssl context using the certs used by requests. This can be done with CURL_CA_BUNDLE or REQUESTS_CA_BUNDLE:
ctx = ssl.create_default_context(
cafile=os.environ.get("CURL_CA_BUNDLE", certifi.where()),
)That way both clients can be configured using the same env variable.