research in error handling

[jlongster]

I'm not quite happy with this yet, but it shows off one implementation. This is from discussions in #14.

With this you can do:

var ch = chan();

go(function*() {
  yield timeout(1000);
  yield put(ch, new Failure("fail!"));
});

go(function*() {
  console.log(yield take(ch));
});

You would see an error "fail!" pointing at the take in the second go block, and the thrown line number as put in the first one.

I also experimented with takePropagate, which will mostly be used by libraries:

var ch = chan();

go(function*() {
  yield timeout(1000);
  yield put(ch, new Failure("fail!"));
});

var c = go(function*() {
  console.log(yield takePropagate(ch));
});

go(function*() {
  yield take(c);
});

The error passed down ch is forwarded to the c channel from the second go block, and the third go block throws an error.

I think a fundamental issue is what you talk about in the last paragraph in #14 (comment): should errors in go blocks crash the whole system or not. Whether they are forwarded, when they are logged, etc are all just semantic details. In my above system without automatic error propagation we could still catch all errors in log them instead of crashing the system. Note that as far as I know core.async for ClojureScript would crash the system.

[jlongster]

Two other notes:

• In Process.prototype.error, when propagating, we could snag a new Error each time and create a stack of errors in the Failure object and we would get a complete stack trace history.
• It's really important to me to try to keep the act of throwing errors in tact. For example, in a JS debugger you can turn on "pause on uncaught exceptions" and it will pause and take to you to the line where an error is being thrown. We should try to keep this in tact so that the user isn't taken inside js-csp.

[jlongster]

We could also potentially have a global flag which specifies whether or not to catch errors and log them.

[jlongster]

I keep subconsciously brewing on this :) Also worth noting the Go also sides on the error of manual propagation.

[ubolonton]

I haven't had time to check this yet. I'll check it soon and reply.

[jlongster]

No worries, I'd actually like to flesh it out a little more before you give a long thoughtful reply about it. I may change a few things, and I'd like to show some more complex examples. Give me another day or two and I'll add some more stuff here.

[jlongster]

Ok, I just forced pushed my latest research. The commit about the arguments removal is already in another PR I just opened (so that can be merged first and it will go away here).

I settled on the following, and personally I'm really happy with it. Any feedback is good.

• We differentiate errors from channels from general user errors in go processes. For example, we don't run your generator in a try/catch and thus any normal error (like a simple typo) appears like normally, meaning you can use "pause on uncaught exception" and everything will just work like normal. This means that go processes cannot catch these errors, but that's ok. These are real bugs in your own code, and it's the same story in callback-world; you don't normally want to stick a big try/catch around all your code. Calling an undefined function (if you typed foo2 instead of foo) will just error out. None of these kinds of errors will be silenced by default like Promises.
• Users have the option of passing Error objects around and manually checking them (or some other kind of error object or even just close the channel to indicate error). This is akin to checking the return value of a function. It's a valid pattern for some APIs. There's no additional support for this; just pass values around.
• The other pattern is to throw an error. These are errors that are exceptional and in many cases you just want to bail and log the error (i.e. and IO or network error). We want to support this (meaning using generators .throw()), so I introduced a Throw object (similar to your Failure object). A library can put a Throw object on a channel and any attempt to take it off will automatically throw. This puts the decision in the library author, and the user doesn't have to worry about different take operations. I think JS users will expect this, as we are used to promises where things automatically abort. The pattern should be to pass Throw an Error object so you get the right stack:

put(ch, Throw(new Error("this is bad")));

Throw objects also keep track of a "stack of error stacks" or async stacks. Right now you can enable this by calling csp.stackHistory(). Unfortunately we can't quite do this by default; see my note at the end.

• The last problem is propagation. We don't want users to always have to wrap things in try/catch if they just want to propagate the error. I made a go process configurable, so by default the error doesn't propagate and it will throw a real uncaught error. However, if you tell it to propagate, if a take tries to get a Throw object it will automatically put it on the return channel of the process:

go(function*() {
  var x = yield take(ch);
}, { propagate: true });

In the worst case, if you forget to add this flag, you won't be able to catch the error but you will at least see the error, and it would be easy to fix. The running philosophy here is "sane defaults"; you will always see errors but you have options to fine-tune how they are handled. Honestly I think only libraries will use this flag a lot; in application code there may be patterns for using channels to route errors where you don't even need this. Need to use this more though.

• Lastly, in case you are in node land it can be nice to have a way for an uncaught Throw error to not terminate the process. Therefore I introduced setDefaultExceptionHandler which will be called if a take finds a Throw and doesn't handle it (or propagate). Example:

csp.setDefaultExceptionHandler(function(err) {
  console.log('exception', err);
});

This is much safer than node's process.on('uncaughtException', ...) because this will only be for Throw objects intentionally put on channels, which in the node world will most likely be IO or network errors much of the time. Of course, you could re-throw from here if you wanted to.

So that's the design I came up with after several days of thinking really hard about it. What I love about it is that it's actually pretty simple, and aligns well with the CSP philosophy. Obviously I haven't used it very much yet as it's new, so we can hold off on merging it if you want. I'm going to dive into some projects with this and see how it works out.

A few last unresolved things:

• We could still also have a builtin takem or takeMaybe that automatically throws when it hits an Error object if we think that would help some people's workflows
• I thought we could get async stacks even in production by grabbing each stack when a take hits an error. Unfortunately I was wrong. When a take happens, if a put is not available it will queue up for it, and it will "sleep" and it's stack completely goes away. We don't know if it takes an error or not until it's too late. I still think there might be something we could do help track the async flow though, but more research is needed.

[ubolonton]

I think this sounds good! I don't have access to a real computer now. Will
check the code tomorrow.

On Monday, November 10, 2014, James Long notifications@github.com wrote:

Ok, I just forced pushed my latest research. The commit about the
arguments removal is already in another PR I just opened (so that can be
merged first and it will go away here).

I settled on the following, and personally I'm really happy with it. Any
feedback is good.

• We differentiate errors from channels from general user errors in go
  processes. For example, we don't run your generator in a try/catch and thus
  any normal error (like a simple typo) appears like normally, meaning
  you can use "pause on uncaught exception" and everything will just work
  like normal. This means that go processes cannot catch these errors,
  but that's ok. These are real bugs in your own code, and it's the same
  story in callback-world; you don't normally want to stick a big try/catch
  around all your code. Calling an undefined function (if you typed foo2
  instead of foo) will just error out. None of these kinds of errors
  will be silenced by default like Promises.
• Users have the option of passing Error objects around and manually
  checking them (or some other kind of error object). This is akin to
  checking the return value of a function. It's a valid pattern for some
  APIs. There's no additional support for this; just pass values around.
• The other pattern is to throw an error. These are errors that are
  exceptional and in many cases you just want to bail and log the error (i.e.
  and IO or network error). We want to support this (meaning using generators
  .throw()), so I introduced a Throw object (similar to your Failure
  object). A library can put a Throw object on a channel and any attempt
  to take it off will automatically throw. This puts the decision in the
  library author, and the user doesn't have to worry about different take
  operations. I think JS users will expect this, as we are used to promises
  where things automatically abort. The pattern should be to pass Throw
  an Error object so you get the right stack:

put(ch, Throw(new Error("this is bad")));

• The last problem is propagation. We don't want users to always have
  to wrap things in try/catch if they just want to propagate the error. I
  made a go process configurable, so by default the error doesn't
  propagate and it will throw a real uncaught error. However, if you tell it
  to propagate, if a take tries to get a Throw object it will
  automatically put it on the return channel of the process:

go(function*() {
var x = yield take(ch);
}, { propagate: true });

In the worst case, if you forget to add this flag, you won't be able to
catch the error but you will at least see the error, and it would be easy
to fix. The running philosophy here is "sane defaults"; you will always see
errors but you have options to fine-tune how they are handled. Honestly I
think only libraries will use this flag a lot; in application code there
may be patterns for using channels to route errors where you don't even
need this. Need to use this more though.

• Lastly, in case you are in node land it can be nice to have a way
  for an uncaught Throw error to not terminate the process. Therefore I
  introduced setDefaultExceptionHandler which will be called if a take
  finds a Throw and doesn't handle it (or propagate).

This is a simple way to give authors the power of signaling a throw.
Additionally, the Throw class supports "error stacks" which means it can
keep a list of async stacks asthe

—
Reply to this email directly or view it on GitHub
#15 (comment).

Nguyễn Tuấn Anh

[jlongster]

I think this is the right direction, but we could tweak it after trying it out. I'm going to build an app with this soon and see how it feels.

[jlongster]

(been busy with work, but will write more code with this soon and if I feel comfortable with it I will recommend a merge!)

[zeroware]

Hello @jlongster , any news on this one ?

[jlongster]

Sorry about this. I got sucked in at work and lost context for working on all of this, and holidays came. I'm giving another talk in 3 weeks about this & react so you can be sure that I'm going to start working on js-csp again.

[jlongster]

Another quick update, one of the reasons I haven't been as active is because I've been developing a side project using js-csp. I felt it was critical that I get some experience as a user and not just a developer, and it really has helped. Soon I will start reporting my experiences and some improvements I think we could do (and mainly, finish error handling)

[zeroware]

I like the Throw approach.
Is there a branch where I can test it in my current project ?

[ghost]

This PR needs rebasing.

[jlongster]

Well, unfortunately I don't use this project much anymore. I still think channels are the best approach, but I can't use them for work and I haven't been doing many side projects. The next one I'll do will probably be in ClojureScript.

Sorry, but I just don't have time to contribute here anymore. I'll leave these PRs open unless another maintainer wants to close them.