Unexpected interruption: file save error and spawn failed

[benjimin]

While working in a notebook on the sandbox, I had a message popup a couple times:

File Save Error for my.ipynb with a bunch of raw html code, including a comment tag about "padding to disable MSIE and Chrome friendly error page".

This recurred a few times at intervals of a few minutes, and then (when I tried clicking the "Give Feedback" permanent overlay) I got a flurry of messages, indicating the JupyterHub server was no longer running and needed to be restarted. (I think one included an option to restart and not just dismiss the message.)

This led to this screen:

Your server is starting up.
You will be redirected automatically when it's ready for you.
100% Complete
Spawn failed: Server at http://10.95.116.198:8888/user/ben/ didn't respond in 30 seconds
Event log
Server requested
2020-09-04 06:08:48+00:00 [Warning] 0/12 nodes are available: 1 node(s) were unschedulable, 10 node(s) didn't match node selector.
2020-09-04 06:08:48+00:00 [Normal] Successfully assigned sandbox/jupyter-ben to ip-10-95-96-213.ap-southeast-2.compute.internal
2020-09-04 06:08:50+00:00 [Normal] AttachVolume.Attach succeeded for volume "pvc-20e3e749-24fa-400c-909c-959a6010a2d2"
2020-09-04 06:08:58+00:00 [Normal] Container image "geoscienceaustralia/sandbox:0.0.10" already present on machine
2020-09-04 06:08:58+00:00 [Normal] Created container notebook
2020-09-04 06:08:58+00:00 [Normal] Started container notebook
Spawn failed: Server at http://10.95.116.198:8888/user/ben/ didn't respond in 30 seconds

Afterwards, a log out and back in appeared to resolve the issue for now.

[benjimin]

Have found a similar issue is recurring on the dev sandbox. From time to time, a flurry of error messages and jupyter becomes broken.

[benjimin]

[MatthewJA]

Oh, I get this error a bunch (most recently today) - it seems to correspond to using a lot of memory at once. I think that when you exceed your memory allowance the pod stops responding entirely. There's no memory errors etc, it just fails with a 502 bad gateway.