How it works:
- Javascript app sends requests to backend: e.g. load notebook, code commands to execute
- backend processes and executes the
code
, and sends back the result DIRECTLY to javascript viaobservable/websockets
- in Backend stuff is passed over as akka actor messages
- all reactive widgets (e.g.
display(dataFrame)
) work like this: https://github.com/spark-notebook/spark-notebook/wiki/Reactive-outputs----widgets,-renderers,-Rx-and-knockout (this doc might be slightly old)
- then Javascript app stores/send the notebook file which contain cell results
- i.e. it's not the backend which stores cell outputs/results into a SNB
Main interactions (data-flow):
JavaScript [web] <-> Application.scala <-> WebSocketKernelActor (play app) <-> CalcWebSocketService <- different JVM process -> ReplCalculator <-> Repl
Code structure:
conf/routes
- makes URLs to play Application backendapp
: is a Play-app web-backendmodules/
(some sub-projects have a README.md file inside!)core
- basic stuff, e.g. for reading/writing SNB filescommon
- containswidgets
andcharts
likedisplay(df)
,PivotChart()
etc (included by most modules/JVM processes)kernel
- a REPL which is run a separate JVM process, per each notebookReplCalculator
process the commands, and passes them toRepl
for execution (a patched/improved spark-shell/repl for apache-spark)
observable
- websockets based complex stuff used to pass over live messages between backend and web-appspark
- spark/scala version dependent stuff. contains low-levelRepl
implementations, and some utilssubprocess
- a helper that spins up a separate JVM for REPLsbt-dependency-manager
- helpers for resolving and downloading custom dependenciesgit-notebook-provider
- git storage support, used only if enabled
P.S. IntellijIDEA Ultimate helps to orient yourself inside a large Play! framework
project (e.g. conf/routes
are navigatable there).