Instructions for launching spark-notebook on EMR 5.x

[SaiKiranBurle]

Hi,

I see that you have detailed instruction to launch the spark-notebook on EMR, but there are no instructions for EMR 5.x versions. Do you plan to release them? I tried the instructions described for 4.x, but I'm facing the same issues described in

#368

but none of the solutions mentioned work for me. I wonder if that is because the solution is specific to EMR 4.1.

[frbayart]

Hi,

we don't have test platform on EMR for now ... but I can see on AWS documentation the Spark version on last EMR 5.20.0 is using Spark 2.4 and for now we don't support it. Did you try EMR 5.16 ?

Else could you send us more details about your error ?

[vidma]

P.S. didn't test it much, but spark-notebook seems to work OK with spark 2.4.

[yeikel]

How do you download spark-notebook for 2.4? Website does not allow that

[vidma]

at the moment for spark 2.4 you need to build it yourself. see https://github.com/spark-notebook/spark-notebook/blob/master/docs/build_from_source.md

[SaiKiranBurle]

Hi, my question is specific to EMR version and not the spark version. The issues I'm facing are related to setting the class path for the interpreter. Are there any plans to release instructions to set up the spark-notebook on EMR 5.x versions?
Currently, the documentation has instructions up to EMR 4.5.
https://github.com/spark-notebook/spark-notebook/blob/master/docs/clusters_clouds.md