except KeyboardInterrupt: pass in neural_network/multilayer_perceptron.py

[denis-bz]

Description

in 0.18, neural_network/multilayer_perceptron.py _fit_stochastic has

try:
...
except KeyboardInterrupt: 
    pass

Looks like a late-night patch ?

[amueller]

Not really? It stops the training on ctrl-C.

[jnothman]

perhaps it should pass only after a warning

On 8 October 2016 at 03:30, Andreas Mueller notifications@github.com
wrote:

Not really? It stops the training on ctrl-C.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#7596 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEz6zVIFJ7Reggv8cqhiZjYZD2ykVdNks5qxnOXgaJpZM4KQ5Vu
.

[amueller]

It should probably at least say that it's interrupting training.

[denis-bz]

Won't an interrupt in the middle of updating leave data corrupt, unusable ?

For a clean split between an optimization engine and the user's outer loop,
I'd prefer callbacks as in scipy.optimize:

for ... optimization loop:
    ...
    if self.callback is not None:
        if self.callback( locals() ):
            break

so the user can print / plot locals, change parameters, break ...
as he/she pleases.

[amueller]

@denis-bz no that should be fine. It's SGD after all.
Having a better call-back mechanism would be great. Does scipy actually provide all locals? That's interesting... Also see #7574.

This was kind of me punting on writing a callback-framework. It works really well in many cases ;)
How would you interactively break with the callback, though? You need a separate process to control the execution, right? Or how would you do that?

[amueller]

actually callbacks in scipy only get the current iterate and don't allow early termination as far as I can see.

[denis-bz]

Andreas
yes, callbacks in scipy.optimize don't pass locals(), but should --
makes it easy to print / plot / save stuff without changing the source.

On callback frameworks / general optimization frameworks:
different thread, tell me more (on a scipy.optimize forum ?)
Do you have a rough description of what you mean, or a model to copy
< matlab optimization toolbox :)
It's a seductive idea, lot of commonality in optimization outer loops,
but everyone has their own way of doing things.

[amueller]

Sorry, I don't fully understand your comments.
So I'm training a neural network and I want to stop it and look at the results. How would you do that?

[denis-bz]

(Years later) a callback() with no args can easily get its callers' locals with

callers_frame = sys._getframe(1)
callers_locals = callers_frame.f_locals

The main purpose of locals is to get some insight into what the caller is doing
by printing / plotting e.g. f, x, grad at each iteration.
In practice even finding these can be messy, though.
Easier is just print ... if verbose >= 2, e.g.

niter  func       -df      |dx|_max    |dgrad|_max   x                   grad 
  0  1.24988      nan       dx nan      dg nan       x [0.0001 0.0001]   g [-2.25 1]
  1  0.405001     0.8       dx 0.8      dg 4         x [0.773 0.198]     g [1.75 -1]
  2  0.119108     0.3       dx -0.1     dg -4        x [0.623 0.271]     g [-2.25 1]
  3  0.119108     -0        dx 0        dg 0         x [0.623 0.271]     g [-2.25 1]
  ...

verbose can help both novices and experts to see, understand, a program.