Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changes to Dataset class #106

Open
BrianWilson1 opened this issue Aug 11, 2016 · 4 comments
Open

Changes to Dataset class #106

BrianWilson1 opened this issue Aug 11, 2016 · 4 comments

Comments

@BrianWilson1
Copy link
Collaborator

Rename Dataset class to SciDataset.
Add method so sciDataset object can write itself to a netCDF4 file.

@rahulpalamuttam
Copy link
Member

@BrianWilson1

Writing to netCDF4 is going to be a challenge because the user needs to install the netCDF4 C library to their systems. The java api to write netCDF4 is dependent on this requirement.

See here :
http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/tutorial/NetcdfWriting.html

We can write to netCDF3 just fine, and is what the above pull request does.

@kwhitehall
Copy link
Member

@rahulpalamuttam how about making the installation of the NetCDF-4 C Library a dependency of SciSpark (to be noted in the README.md file to install), and provide the support?

@rahulpalamuttam
Copy link
Member

rahulpalamuttam commented Aug 16, 2016

@kwhitehall @BrianWilson1 @pzimdars
We can do that, but that library needs to be installed on every node in the cluster.
If we ever get to point where we're running on a cluster with a large number of nodes - it's going to get very cumbersome.

This brings up a more important point.
There should be some work towards a SciSpark install script.
Here's https://www.ansible.com/ which we could use.

With respect to the PR - I think users would much rather prefer having to just build SciSpark and run, without installing C libraries manually. What I can do is having a switch to toggle between writing netcdf3 and netcdf4. For the netcdf4 case users must have the C library installed.

This would require updating travis ci to have additional installation procedures so our builds pass.

@rahulpalamuttam
Copy link
Member

@kwhitehall @BrianWilson1

I'm having a tough time getting Java to see the netcdf4 library installed.
I'm following the instructions here : http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/tutorial/NetcdfWriting.html

Are there any members of our team who have done this successfully and used the java api to write netcdf4?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants