This repository was archived by the owner on Dec 7, 2019. It is now read-only.
This repository was archived by the owner on Dec 7, 2019. It is now read-only.
readAll() is not resilient to exceptions reading a single file #293
Description
The current readAll() implementation in FSAllReader fails if there is an exception reading a single file in the path. In this code:
bufferedSourceObservable = Observable
.fromIterable(fileSystem.list(path))
.map(s -> {
try {
return fileSystem.read(s);
} catch (FileNotFoundException e) {
throw Exceptions.propagate(e);
}
});
If an exception occurs on a fileSystem.read(s), then the whole chain will stop with an onError emission preventing us from getting subsequent files on the path.
One way around it, I think, would be to wrap the read in a flatMap() and emit an empty or special type empty buffer. Something like:
@Nonnull
@Override
public Observable<BufferedSource> readAll(@Nonnull final String path) throws FileNotFoundException {
return Observable.defer(() -> {
Observable<BufferedSource> bufferedSourceObservable;
try {
bufferedSourceObservable = Observable.fromIterable(fileSystem.list(path))
.flatMap(s -> Observable.just(fileSystem.read(s))
.onErrorReturn(throwable -> Okio.buffer((Source) new ExceptionBuffer(throwable))));
} catch (FileNotFoundException e) {
throw Exceptions.propagate(e);
}
return bufferedSourceObservable;
});
}
Then we can let the caller decide what to do with the ExceptionBuffer. Would something in this direction work?