Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to set Spark storage level to DISK_ONLY #670

Open
jayprajapati857 opened this issue Oct 16, 2017 · 0 comments
Open

How to set Spark storage level to DISK_ONLY #670

jayprajapati857 opened this issue Oct 16, 2017 · 0 comments

Comments

@jayprajapati857
Copy link

jayprajapati857 commented Oct 16, 2017

i want to persist the data into the spark application and set the storageLevel to DISK_ONLY. currently i am setting the storage level to spark-defaults.conf like

spark.storage.StorageLevel DISK_ONLY

while running spark application this storage level is set to DISK_ONLY in environment tab on spark ui but when the data persistance is complete the spark ui shows that memory used is something MBs and Disk Used 0.

in spark-shell i am able to set this storage level but i dont know how to set it using Mobius.

so how can i set that storageLevel?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant