Save DataArray to file

[jgrss]

@mmann1123 @rdenham this branch has a new raster save method in the works. The method is called save() and an example of use can be found here. I need to document the additions but the details are here.

Much of the code used in save() already existed in geowombat. However, it was buried in to_raster(). Most of the tests with to_raster() were done on an HPC with large raster data. We implemented to_raster() because the dask task graphs were too slow to generate on rasters we were using. However, I found that the save() method, which uses dask.store() and only calls compute() once is faster on smaller images (e.g., Landsat or Sentinel scenes). And, maybe faster now on large rasters, but I haven't tested it.

If you get a chance and are interested, test it out using the branch jgrss/store. And, if you do, let me know what you think and if you find it faster than to_raster() (on small or large rasters). My idea is that the two methods (save() and to_raster()) can both be used, depending on the use case.

[jgrss]

@mmann1123 Are you able to pull jgrss/store and test it? I've been using the branch for a while and I think it's pretty close to being ready for a PR.

I think it should address Issue #183.

[jgrss]

@mmann1123 I think I've fixed support for using a Dask Client to write data with save(). See branch jgrss/store_time_series.

It would be great if you could test it out on any of your data before I move it over to a PR. See below for an example. If you want to watch the computation in real time, you can open http://localhost:8787/status in a separate tab or browser. Or open the "dask-report.html" file that will be created.

Experiment with threading, e.g.

with LocalCluster(
    processes=False,
    n_workers=4,
    threads_per_worker=1,
    memory_limit="2GB",
) as cluster:

and combinations of processes and threads.

from dask.distributed import Client, LocalCluster, performance_report
import geowombat as gw
from geowombat.data import l8_224078_20200518

with LocalCluster(
    processes=True,
    n_workers=4,
    threads_per_worker=2,
    memory_limit="2GB",
) as cluster:
    with Client(cluster) as client:
        print(client)
        with performance_report(filename="dask-report.html"):
            # This is only here to increase the computation
            with gw.config.update(ref_res=10.0):
                with gw.open(l8_224078_20200518) as src:
                    # Something more complex than this would be good
                    data = src.gw.set_nodata(0, 32768, dtype="uint16")
                    data = (data * 10.0).assign_attrs(**data.attrs)
                    data.gw.save(
                        filename="test.tif",
                        overwrite=True,
                        # I don't think we can compress with a client
                        compress="none",
                        client=client,
                    )