readAll() is not resilient to exceptions reading a single file

[riclage]

The current readAll() implementation in FSAllReader fails if there is an exception reading a single file in the path. In this code:

        bufferedSourceObservable = Observable
                    .fromIterable(fileSystem.list(path))
                    .map(s -> {
                           try {
                                return fileSystem.read(s);
                            } catch (FileNotFoundException e) {
                                throw Exceptions.propagate(e);
                            }
                    });

If an exception occurs on a fileSystem.read(s), then the whole chain will stop with an onError emission preventing us from getting subsequent files on the path.

One way around it, I think, would be to wrap the read in a flatMap() and emit an empty or special type empty buffer. Something like:

    @Nonnull
    @Override
    public Observable<BufferedSource> readAll(@Nonnull final String path) throws FileNotFoundException {
        return Observable.defer(() -> {
            Observable<BufferedSource> bufferedSourceObservable;
            try {
                bufferedSourceObservable = Observable.fromIterable(fileSystem.list(path))
                    .flatMap(s -> Observable.just(fileSystem.read(s))
                        .onErrorReturn(throwable -> Okio.buffer((Source) new ExceptionBuffer(throwable))));
            } catch (FileNotFoundException e) {
                throw Exceptions.propagate(e);
            }
            return bufferedSourceObservable;
        });
    }

Then we can let the caller decide what to do with the ExceptionBuffer. Would something in this direction work?

[digitalbuddha]

Yup sounds like a great solution. Want to pr it in with some tests?

[riclage]

Cool, thanks. Yes, I will try to do one.

[riclage]

@digitalbuddha Did you have a chance to look at my PR?

[digitalbuddha]

Was moving. Will look today

…

On Dec 4, 2017 4:41 AM, "Ricardo" ***@***.***> wrote: @digitalbuddha <https://github.com/digitalbuddha> Did you have a chance to look at my PR? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#293 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEUX3EeUvNCrYPELk-xfHL_Ih9x1jbAeks5s874sgaJpZM4QohUK> .