Hi, me again, and apologies if I've missed something again.
As the title says, the following code shows that when I call ls on a subclass of AsyncFileSystem it goes to the async version _ls and raises, as expected:
from fsspec.asyn import AsyncFileSystem
class FooFS(AsyncFileSystem):
pass
if __name__ == "__main__":
foo_fs = FooFS()
foo_fs.ls("foo")
for x in foo_fs.walk("foo", on_error="raise"):
print(x)
Traceback (most recent call last):
File "/tmp/foo/test.py", line 10, in <module>
foo_fs.ls("foo")
~~~~~~~~~^^^^^^^
File "/tmp/foo/.venv/lib64/python3.13/site-packages/fsspec/asyn.py", line 118, in wrapper
return sync(self.loop, func, *args, **kwargs)
File "/tmp/foo/.venv/lib64/python3.13/site-packages/fsspec/asyn.py", line 103, in sync
raise return_result
File "/tmp/foo/.venv/lib64/python3.13/site-packages/fsspec/asyn.py", line 56, in _runner
result[0] = await coro
^^^^^^^^^^
File "/tmp/foo/.venv/lib64/python3.13/site-packages/fsspec/asyn.py", line 709, in _ls
raise NotImplementedError
NotImplementedError
However, if I let it call walk then it goes to AbstractFileSystem.walk (which itself then calls the async'ed ls) instead of AsyncFileSystem._walk:
from fsspec.asyn import AsyncFileSystem
class FooFS(AsyncFileSystem):
pass
if __name__ == "__main__":
foo_fs = FooFS()
# foo_fs.ls("foo")
for x in foo_fs.walk("foo", on_error="raise"):
print(x)
Traceback (most recent call last):
File "/tmp/foo/test.py", line 11, in <module>
for x in foo_fs.walk("foo", on_error="raise"):
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/foo/.venv/lib64/python3.13/site-packages/fsspec/spec.py", line 432, in walk
listing = self.ls(path, detail=True, **kwargs)
File "/tmp/foo/.venv/lib64/python3.13/site-packages/fsspec/asyn.py", line 118, in wrapper
return sync(self.loop, func, *args, **kwargs)
File "/tmp/foo/.venv/lib64/python3.13/site-packages/fsspec/asyn.py", line 103, in sync
raise return_result
File "/tmp/foo/.venv/lib64/python3.13/site-packages/fsspec/asyn.py", line 56, in _runner
result[0] = await coro
^^^^^^^^^^
File "/tmp/foo/.venv/lib64/python3.13/site-packages/fsspec/asyn.py", line 709, in _ls
raise NotImplementedError
NotImplementedError
Is there something I should do for it to use the async walk?
🙏
Hi, me again, and apologies if I've missed something again.
As the title says, the following code shows that when I call
lson a subclass ofAsyncFileSystemit goes to the async version_lsand raises, as expected:However, if I let it call
walkthen it goes toAbstractFileSystem.walk(which itself then calls the async'edls) instead ofAsyncFileSystem._walk:Is there something I should do for it to use the async walk?
🙏