Skip to content

[ISSUE] get_api_error fails with TypeError: object of type '_io.BufferedReader' has no len()¨ #1489

Description

@smarie

Description

When the HTTP PUT request body is streamed from a local file, request.body is not a string but a _io.BufferedReader.

In case of API error, _BaseClient._error_parser.get_api_error(response) will try to return the error and the associated query.

return _error_mapper(
response,
{"message": "unable to parse response. " + _unknown_error(response, self._debug_headers)},
)

Unfortunately, response.request.body is still a BufferedReader, not necessarily a string. When this body is passed to the logger, it fails

if request.body:
sb.append("> [raw stream]" if self._raw else self._redacted_dump("> ", request.body))

eventually calls self._redacted_dump("> ", request.body) but the latter is not meant to handle streams

Full strack trace :

Traceback (most recent call last):
  File "C:\_dev\my_proj\.venv\Lib\site-packages\dvc_objects\fs\generic.py", line 350, in transfer
    _try_links(
  File "C:\_dev\my_proj\.venv\Lib\site-packages\dvc_objects\fs\generic.py", line 281, in _try_links
    return copy(
           ^^^^^
  File "C:\_dev\my_proj\.venv\Lib\site-packages\dvc_objects\fs\generic.py", line 88, in copy
    return _put(
           ^^^^^
  File "C:\_dev\my_proj\.venv\Lib\site-packages\dvc_objects\fs\generic.py", line 161, in _put
    _put_one(from_paths[0], to_paths[0])
  File "C:\_dev\my_proj\.venv\Lib\site-packages\dvc_objects\fs\generic.py", line 151, in _put_one
    return to_fs.put_file(
           ^^^^^^^^^^^^^^^
  File "C:\_dev\my_proj\.venv\Lib\site-packages\dvc_objects\fs\base.py", line 673, in put_file
    self.fs.put_file(os.fspath(from_file), to_info, callback=callback, **kwargs)
  File "C:\_dev\my_proj\.venv\Lib\site-packages\dvc_databricks\filesystem.py", line 308, in put_file
    self._client.files.upload(rpath, fh, overwrite=True)
  File "C:\_dev\my_proj\.venv\Lib\site-packages\databricks\sdk\mixins\files.py", line 1430, in upload
    self._upload_single_thread_with_known_size(ctx, contents)
  File "C:\_dev\my_proj\.venv\Lib\site-packages\databricks\sdk\mixins\files.py", line 1513, in _upload_single_thread_with_known_size
    return self._single_thread_single_shot_upload(ctx, contents)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\_dev\my_proj\.venv\Lib\site-packages\databricks\sdk\mixins\files.py", line 1521, in _single_thread_single_shot_upload
    return super().upload(file_path=ctx.target_path, contents=contents, overwrite=ctx.overwrite)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\_dev\my_proj\.venv\Lib\site-packages\databricks\sdk\service\files.py", line 1027, in upload
    self._api.do(
  File "C:\_dev\my_proj\.venv\Lib\site-packages\databricks\sdk\core.py", line 95, in do
    return self._api_client.do(
           ^^^^^^^^^^^^^^^^^^^^
  File "C:\_dev\my_proj\.venv\Lib\site-packages\databricks\sdk\_base_client.py", line 197, in do
    response = call(
               ^^^^^
  File "C:\_dev\my_proj\.venv\Lib\site-packages\databricks\sdk\retries.py", line 59, in wrapper
    raise err
  File "C:\_dev\my_proj\.venv\Lib\site-packages\databricks\sdk\retries.py", line 38, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\_dev\my_proj\.venv\Lib\site-packages\databricks\sdk\_base_client.py", line 297, in _perform
    error = self._error_parser.get_api_error(response)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\_dev\my_proj\.venv\Lib\site-packages\databricks\sdk\errors\parser.py", line 95, in get_api_error
    {"message": "unable to parse response. " + _unknown_error(response, self._debug_headers)},
                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\_dev\my_proj\.venv\Lib\site-packages\databricks\sdk\errors\parser.py", line 45, in _unknown_error
    request_log = RoundTrip(response, debug_headers=debug_headers, debug_truncate_bytes=10 * 1024).generate()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\_dev\my_proj\.venv\Lib\site-packages\databricks\sdk\logger\round_trip_logger.py", line 47, in generate
    sb.append("> [raw stream]" if self._raw else self._redacted_dump("> ", request.body))
                                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\_dev\my_proj\.venv\Lib\site-packages\databricks\sdk\logger\round_trip_logger.py", line 113, in _redacted_dump
    if len(body) == 0:
       ^^^^^^^^^
TypeError: object of type '_io.BufferedReader' has no len()

A suggested fix could be :

    def _redacted_dump(self, prefix: str, body: str) -> str:
        if isinstance(body, io.IOBase):
            # Read the stream into a string-like
            body.seek(0)
            body = body.read().decode("utf-8")
        
        if len(body) == 0:
            return ""

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions