Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Classes publicly available for dependency? #994

Open
kaelte opened this issue Mar 2, 2021 · 2 comments
Open

Classes publicly available for dependency? #994

kaelte opened this issue Mar 2, 2021 · 2 comments

Comments

@kaelte
Copy link

kaelte commented Mar 2, 2021

I would like to develop some common functions in a local scala library so that they can be imported and used by multiple notebooks. For example a function like this:
def getGraph(dfNodes: DataFrame, dfEdges: DataFrame, nodeRadius: Byte): Array[Graph[String]] = ???

However the class Graph needs to imported, but I do not know how. The following does not work:
import org.XYZ...notebook.front.widgets.magic.Graph

How can I add the notebook classes as a dependency to my scala project?

@vidma
Copy link
Contributor

vidma commented Mar 2, 2021

i think if you'd run sbt -Dspark.version="?" -Dscala.version="?" publishLocal (fill in the params) inside spark-notebook source/github dir you'd get it published.

then it'd should be available in common jar, probably io.kensu % common % "some-version". you'll see it when getting published...

spark-notebook/build.sbt

Lines 405 to 432 in 6865818

lazy val common = Project(id = "common", base = file("modules/common"))
.dependsOn(observable, sbtDependencyManager)
.settings(
version := versionShortWithSpark.value
)
.settings(
libraryDependencies ++= Seq(
akka,
log4j
),
libraryDependencies += scalaTest,
unmanagedSourceDirectories in Compile += (sourceDirectory in Compile).value / ("scala-" + scalaBinaryVersion.value),
unmanagedSourceDirectories in Compile += (sourceDirectory in Compile).value / commonProjSparkDir
)
.settings(
gisSettings
)
.settings(sharedSettings: _*)
.settings(sparkSettings: _*)
.settings(buildInfoSettings: _*)
.settings(
sourceGenerators in Compile += buildInfo,
buildInfoKeys := buildInfoValues,
buildInfoPackage := "notebook"
)
.settings(
Extra.commonSettings
)

https://github.com/spark-notebook/spark-notebook/blob/master/modules/common/src/main/scala/notebook/front/widgets/magic/Magic.scala

@kaelte
Copy link
Author

kaelte commented Mar 2, 2021

Thanks @vidma for your answer!
But I was hoping to find a publicly published version, instead of compiling it locally
I will edit my question.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants