Package maintenance and future

[ericsun95]

Hey, I see this package is struggling to catch up latest changes. Will it be in the maintenance mode in future? I personally feel it is a very useful library. Also there are some pull requests that haven't been updated for years but looks very helpful to the community.

What's the plan for this package in future? I am interested in helping in any sense.

[rjurney]

@ericsun95 They just did a release! Having used this library extensively, I would say... many people use it to build knowledge graphs owing to its connected components feature. They mostly don't contribute. Databricks has an improved version internally but if you depend on GraphFrames, expect to contribute to GraphFrames.

[ericsun95]

Gotcha, however it is more like bumping the dependencies without actively improving it. For example, if I want to improve some of the algorithms here, how should I do that? Directly submit a pull request?

Also it seems the current build system is isolated between scala and python. It is super hard for me to figure out how to index the code using intellij for example.

[ericsun95]

I feel GraphX/GraphFrame is super useful for large scale graph data processing. I hope we can also have proposals to merge these two together in the Spark main repo. Not sure if it is in the road map as well.

[ericsun95]

@WeichenXu123 @EnricoMi since you are the most recent active maintainers in this repo.

[rjurney]

@ericsun95 yes, submit a pull request. I also feel this library is very useful. I wrote all about it in a blog post this week and teach it in my class on Full Stack Graph Machine Learning. Some new features went out in the latest release, actually. Databricks also maintains a souped up version of this and has documentation on how to use it as part of their platform. Investment is limited but no it isn't abandoned. Matei the CTO of Databricks has asked me before what features I would like to see added, so if you'd like Databricks to add features then please ask him to do so on LinkedIn 😀

https://blog.graphlet.ai/knowledge-graph-factories-f50466fb7512

https://www.linkedin.com/pulse/knowledge-graph-construction-russell-jurney?utm_source=share&utm_medium=member_ios&utm_campaign=share_via