WebREPL doesn't return cleanly after a reset

[cefn]

After powerup, my fairly standard webrepl startup routine is successful, but if the ESP8285 resets for any reason, the WebREPL startup fails with an ENOMEM out of memory error in its setup_conn call, which can be seen in the console as follows when triggered by either CTRL+D or CTRL+B.

This happens before really any of my own code runs, so must be due to a fairly low-level issue. I am using esp8266-20171101-v1.9.3.bin as the firmware image...

>>> 
>>> Exception need more than 0 values to unpack

WebREPL connection from: ('192.168.4.2', 35936)
dupterm: EOF received, deactivating

WebREPL connection from: ('192.168.4.2', 36378)
dupterm: EOF received, deactivating
Exception need more than 0 values to unpack

PYB: soft reboot
#5 ets_task(40100130, 3, 3fff837c, 4)
Traceback (most recent call last):
  File "boot.py", line 14, in <module>
  File "webrepl.py", line 72, in start
  File "webrepl.py", line 21, in setup_conn
OSError: [Errno 12] ENOMEM

If I have the right source file, then this is the line at which it fails...
https://github.com/micropython/micropython/blob/master/ports/esp8266/modules/webrepl.py#L21

This may be the same as the less detailed report at #33

Is there something which is not fully reset without a complete power-down. Any alternative workarounds?

P.S. The occasional errors you can see in the log which read "Exception need more than 0 values to unpack" I haven't traced and don't know how to, but they don't seem to prevent the functioning of the system.

[dpgeorge]

Is there something which is not fully reset without a complete power-down.

Yes, the open TCP connections are not closed on a soft reset. In particular for WebREPL there is an open listening socket when it is started, in addition to any open WebREPL connection. To do a clean soft reset you need to:

1. close all WebREPL connections cleanly
2. shutdown webrepl using webrepl.stop()

After that you can safely do a soft reset and restart webrepl.

[cefn]

Given there can be only one listen socket and one client socket then why does webrepl.stop() not itself close the WebREPL connections properly?

https://github.com/micropython/micropython/blob/37282f8fc135a16ff36e2afe0de907cbde531ed0/ports/esp8266/modules/webrepl.py#L51-L57

Do you mean that a browser/webrepl_cli.py retaining the socket means that the ESP8266 causes LwIP to keep zombie resources hanging around even after weprepl.py has released them from the micropython side?

[cefn]

In fact it seems I had one of my own sockets s not shutting down as well, so finally after running both...

webrepl.stop()
s.close()

...it was able to complete a reset cycle without ENOMEM.

Is there a way to wrap a 'finally' clause around the exit of the REPL process. This would permit learners to author their own main.py but automatically get the proper tidyup of any resources which were committed in boot.py (where the webrepl and replserver start up https://github.com/vgkits/vanguard/blob/master/utilities/replserver/boot.py ).

As a workaround in the short term I will author a tidyUp() function in boot.py which should be visible to the REPL namespace and people can be encouraged to add their own finally clause if they care about elegant resets, and there's always 'have you tried turning it off and on again'.

[dpgeorge]

Is there a way to wrap a 'finally' clause around the exit of the REPL process.

Not currently. Two options to do this would be 1) have the C runtime execute "shutdown.py" before doing a soft reset, and the user can put any commands in there to do cleanup; 2) provide a way to register cleanup functions that are executed on soft reset.

But such a mechanism is only really needed if the REPL has been used. For cases where it's automated, ie just a running script, then that script should always be able to run "finally" code before doing a soft reset.

[cefn]

Could I suggest unboot.py for symmetry. Might indicate the design intent to push pre-config into boot.py and that anything you reserve in boot.py should be undone?

Is inserting an unboot.py launcher something which an inexpert under-the-micropython-hood hacker can achieve with their own image build? Is there a natural place for this to be invoked?

My concern is that without being able to force-handle tidyup, it appears to have had a clean reset, but hasn't really, which will create weird bugs for people relying on the proper functioning of anything in boot.py. Those will be exactly the same people who will be hard to reach with the message 'you should write a try:finally wrapper around your code. You cannot report the situation in the form of an error message to the potential REPL user, since the boot routine will have already completed.

[dpgeorge]

Is inserting an unboot.py launcher something which an inexpert under-the-micropython-hood hacker can achieve with their own image build? Is there a natural place for this to be invoked?

Yes it's a 1-liner: just add pyexce_file('unboot.py'); as the first line of the function soft_reset(void) in ports/esp8266/main.c.

If this turns out to be a useful construct we can consider adding it upstream. There's also a need for some way to handle a system failure (eg out-of-memory exception leading to a reset) which could be part of this "unboot" feature.

[larryfast]

Thanks for the discussion. It helped me resolve a similar problem: Closing the WebREPL browser tab without Disconnect gives the ESP8266(Huzzah) a persistent startup failure. The WebREPL port is identified as occupied. My hacky fix is to add webrepl_stop() ... webrepl_start() to all my main.py scripts.