Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

get_elev_raster error - cannot create RasterLayer object from file #62

Closed
NTeich opened this issue Sep 29, 2021 · 21 comments
Closed

get_elev_raster error - cannot create RasterLayer object from file #62

NTeich opened this issue Sep 29, 2021 · 21 comments

Comments

@NTeich
Copy link

NTeich commented Sep 29, 2021

Having issues getting get_elev_raster to run correctly. I am trying to obtain DEMs of different national parks. I believe I am inputting correct file type but I am new R user, so anything is possible.

NPS_shp = st_read("G:/Trails + PP/Shapefiles/nps_boundary/nps_boundary.shp")
GLAC_shp = subset (NPS_shp, UNIT_CODE== "GLAC")
GLAC_boundary <- as_Spatial(GLAC_shp)
GLAC_elevation <- get_elev_raster(location = GLAC_boundary, z = 10, clip = "location") 

From there I receive the following output:

Accessing raster elevation [=========================] 100% Mosaicing & Projecting Error in .rasterObjectFromFile(x, band = band, objecttype = "RasterLayer", : Cannot create a RasterLayer object from this file.

Alternatively, when I play with the z value I receive a different output:

Accessing raster elevation [=========================] 100% Mosaicing & Projecting Error in sf::gdal_utils(util = "warp", source = files, destination = destfile, : gdal_utils warp: an error occured

Any thoughts?

@jhollist
Copy link
Owner

@NTeich I think I know what was happening with the different z value and that is fixed in the current version on GitHub, but I suspect there is an underlying issue that is causing the first problem, but that it is hard to say without a reproducible example.

If you could work up an example that I can run (e.g. doesn't rely on a local dataset) and reproduce the error, then I can dig into it.

@NTeich
Copy link
Author

NTeich commented Oct 8, 2021

Sorry for the delay! I tried using the USAboundaries package and received the same error.

install.packages("USAboundaries")
library(USAboundaries)
library(sp)
library(sf)
library(elevatr)

Colorado <- us_states(states = c("Colorado"))
class(Colorado)
Colorado_sp <- as_Spatial(Colorado)
class(Colorado_sp)

Colorado_elev <- get_elev_raster(locations = Colorado_sp, z = 8)

Still received the same error

Accessing raster elevation [=========================] 100% Mosaicing & Projecting Error in .rasterObjectFromFile(x, band = band, objecttype = "RasterLayer", : Cannot create a RasterLayer object from this file.

I also recieved the same warp error when I attempted to a larger z value

> Colrado_elev <- get_elev_raster(locations = Colorado_sp, z = 10, prj = ) Accessing raster elevation [=========================] 100% Mosaicing & Projecting Error in sf::gdal_utils(util = "warp", source = files, destination = destfile, : gdal_utils warp: an error occured

Any feedback would be appreciated!

@jhollist
Copy link
Owner

jhollist commented Oct 8, 2021

Install the dev version from here and try it again.

install.packages("remotes")
remotes::install_github("jhollist/elevatr")

There was a problem with how I was doing the parallel requests that I think I have fixed.

@NTeich
Copy link
Author

NTeich commented Oct 10, 2021

Still receiving the same error message, even with the new code:

install.packages("USAboundaries")
install.packages("remotes")
remotes::install_github("jhollist/elevatr")

library(USAboundaries)
library(sp)
library(sf)
library(elevatr)

Colorado <- us_states(states = c("Colorado"))
class(Colorado)
Colorado_sp <- as_Spatial(Colorado)
class(Colorado_sp)

Colorado_elev <- get_elev_raster(locations = Colorado_sp, z = 8)

Colorado_elev <- get_elev_raster(locations = Colorado_sp, z = 8)
Accessing raster elevation [=========================] 100%
Mosaicing & Projecting
Error in .rasterObjectFromFile(x, band = band, objecttype = "RasterLayer", :
Cannot create a RasterLayer object from this file.

@Antoine-CSQN
Copy link

Hello,

I had the same problem as Nteich with the current version on CRAN.

Mosaicing & Projecting
Error in sf::gdal_utils(util = "warp", source = files, destination = destfile,  : 
  gdal_utils warp: an error occured
In addition: Warning messages:
1: In showSRID(SRS_string, format = "PROJ", multiline = "NO", prefer_proj = prefer_proj) :
  Discarded datum European Terrestrial Reference System 1989 in Proj4 definition
2: In showSRID(SRS_string, format = "PROJ", multiline = "NO", prefer_proj = prefer_proj) :
  Discarded datum European Terrestrial Reference System 1989 in Proj4 definition

For my case, the dev version solved the the problem.

Raster was in EPSG 3035, z =10, area approx 1500 km * 1000 km.
The call was :
elev_raster <- get_elev_raster(locations = so_sf, z = 10)

The temp tif was approx 2.3 GB,1.1 GB after writing with raster::writeRaster()
Unexpectedly, I did not have to tune the parameter override_size_check

Thanks for writing and maintaining this package, despite the little issue, it is a significant gain of time for my workflow.
I wish I had heard from it before.

@jhollist
Copy link
Owner

jhollist commented Nov 7, 2021

@NTeich can you confirm again that your code is NOT working. When I run it on my machine with the GitHub version it works at z=8 and z=10. If you are still having issues, i'd like to try and find out what the problem is.

Thanks! If I don't hear back in a few weeks, I'll close the issue.

@jhollist
Copy link
Owner

@Antoine-CSQN apologies for not responding earlier. Glad to hear that the problem is solved in the dev version.

And my size estimates are done by hand and obviously are a bit off!! I need to look at how I am doing that to see if I can improve it. I think my # of pixel estimates aren't good.

And you are welcome, nice to hear that the package has helped you out. Keeps me going!

@jordansread
Copy link

@jhollist a heads up that I'm getting this odd error too, and am struggling to come up with a simpler example that I can share that doesn't require me sending an sf points file to reproduce it.

Error in sf::gdal_utils(util = "warp", source = files, destination = destfile,  : 
  gdal_utils warp: an error occured
In addition: Warning messages:
1: In CPL_gdalwarp(source, destination, options, oo, doo, quiet, "-overwrite" %in%  :
  GDAL Error 1: Error: Failed to build program executable!
Build Log:

2: In CPL_gdalwarp(source, destination, options, oo, doo, quiet, "-overwrite" %in%  :
  GDAL Error 1: Error at file gdalwarpkernel_opencl.cpp line 2479: CL_INVALID_BUILD_OPTIONS
3: In CPL_gdalwarp(source, destination, options, oo, doo, quiet, "-overwrite" %in%  :
 
 Error in sf::gdal_utils(util = "warp", source = files, destination = destfile, : 
gdal_utils warp: an error occured

I am snagging elevation using the package for lakes (~10k of them) by making calls for lake centroids within 0.5° lat/lon cells (or 1°, 2°, I've tried a few variations). At the 9 zoom level I run into this error for 42% of the cells. But, when I try to reproduce it for you in the same cell as the failures,

point_elev <- tibble(long = c(-89, -88.5), lat = c(43.5, 44)) %>% 
  sf::st_as_sf(coords = c("long", "lat"), 
           crs = 4326) %>% 
  sf::st_make_grid(square = TRUE, cellsize = 0.01, what = 'centers') %>% 
  sf::st_as_sf() %>% 
  elevatr::get_aws_points(zoom = 9)
   Accessing raster elevation [=========================] 100%
Mosaicing & Projecting
Note: Elevation units are in meters.

There is no error. So I think it may be related to the non-uniform layout of the lake centroids in this or the slight difference in sf classes. Will do some more digging.

@jhollist
Copy link
Owner

jhollist commented Mar 6, 2022 via email

@jhollist
Copy link
Owner

jhollist commented Mar 6, 2022 via email

@jordansread
Copy link

Thanks @jhollist !

Turns out my simpler example can trigger this error, but I wasn't using the right argument name for z (was using zoom instead, which I think gets passed to httr::GET eventually and ignored), so I think I was getting z = 5, the default, which wasn't erroring.

Instead, this simple reprex should trigger it:

library(dplyr)
point_elev <- tibble(long = c(-89, -88.5), lat = c(43.5, 44)) %>% 
  sf::st_as_sf(coords = c("long", "lat"), 
           crs = 4326) %>% 
  sf::st_make_grid(square = TRUE, cellsize = 0.1, what = 'centers') %>% 
  sf::st_as_sf() %>% mutate(site_id = paste0('r_', row_number())) %>% 
  elevatr::get_aws_points(z = 9)

Accessing raster elevation [=========================] 100%
Mosaicing & Projecting
Error in sf::gdal_utils(util = "warp", source = files, destination = destfile,  : 
  gdal_utils warp: an error occured
In addition: Warning messages:
1: In CPL_gdalwarp(source, destination, options, oo, doo, quiet, "-overwrite" %in%  :
  GDAL Error 1: Error: Failed to build program executable!
Build Log:

2: In CPL_gdalwarp(source, destination, options, oo, doo, quiet, "-overwrite" %in%  :
  GDAL Error 1: Error at file gdalwarpkernel_opencl.cpp line 2479: CL_INVALID_BUILD_OPTIONS
3: In CPL_gdalwarp(source, destination, options, oo, doo, quiet, "-overwrite" %in%  :
  GDAL Error 1: OpenCL routines reported failure (-43) on line 4619.
4: In CPL_gdalwarp(source, destination, options, oo, doo, quiet, "-overwrite" %in%  :
  GDAL Error 1: Error: Failed to build program executable!
Build Log:

5: In CPL_gdalwarp(source, destination, options, oo, doo, quiet, "-overwrite" %in%  :
  GDAL Error 1: Error at file gdalwarpkernel_opencl.cpp line 2479: CL_INVALID_BUILD_OPTIONS
6: In CPL_gdalwarp(source, destination, options, oo, doo, quiet, "-overwrite" %in%  :
  GDAL Error 1: OpenCL routines reported failure (-43) on line 4619.

@jhollist
Copy link
Owner

jhollist commented Mar 7, 2022

@jread-usgs I ran you code on windows and ubuntu and was unable to reproduce the error. I can try pushing something via Actions on Mac to see if it reproduces there... Will take me a bit to get that done.

@jhollist jhollist reopened this Mar 7, 2022
@jordansread
Copy link

Thanks Jeff - I am on mac but I'm going to make sure all of my system dependencies are up to date. They should be pretty current (this machine was re-imaged a couple of months ago) but better to check. Seems I am running into something on GDAL that you aren't hitting on win/*nix.
I will also ping another person in the office on mac to give this a spin and see what they hit.

@jhollist
Copy link
Owner

jhollist commented Mar 7, 2022

Do you have any other GDAL installs? Sometimes (e.g. #48 (comment)) other installs will change the GDAL path.

@jordansread
Copy link

It doesn't seem like I have an issue with the gdal path, but I did just do a small upgrade to GDAL (via brew) and sf. I was very close to current on both of these, but am now current. Same issue appears after the updates.

sessionInfo()
R version 4.1.2 (2021-11-01)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Catalina 10.15.7

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] sf_1.0-7        readr_2.1.1     dplyr_1.0.7     elevatr_0.4.2   scipiper_0.0.24

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.8         compiler_4.1.2     pillar_1.6.5       progressr_0.10.0   progress_1.2.2     prettyunits_1.1.1  class_7.3-19       remotes_2.4.2      tools_4.1.2       
[10] digest_0.6.29      pkgbuild_1.2.0     remake_0.3.0       lattice_0.20-45    lifecycle_1.0.1    tibble_3.1.6       pkgconfig_2.0.3    rlang_1.0.2        DBI_1.1.2         
[19] cli_3.1.1          rstudioapi_0.13    rgdal_1.5-23       yaml_2.2.1         curl_4.3.2         e1071_1.7-9        httr_1.4.2         storr_1.2.5        withr_2.4.3       
[28] hms_1.1.1          generics_0.1.1     vctrs_0.3.8        tidyselect_1.1.1   classInt_0.4-3     rprojroot_2.0.2    grid_4.1.2         glue_1.6.2         R6_2.5.1          
[37] processx_3.5.2     fansi_1.0.2        sp_1.4-6           tzdb_0.2.0         tidyr_1.1.4        callr_3.7.0        purrr_0.3.4        magrittr_2.0.2     ps_1.6.0          
[46] ellipsis_0.3.2     units_0.8-0        assertthat_0.2.1   slippymath_0.3.1   KernSmooth_2.23-20 utf8_1.2.2         proxy_0.4-26       crayon_1.4.2   

library(rgdal)
Loading required package: sp
rgdal: version: 1.5-23, (SVN revision 1121)
Geospatial Data Abstraction Library extensions to R successfully loaded
Loaded GDAL runtime: GDAL 3.2.1, released 2020/12/29
Path to GDAL shared files: /Library/Frameworks/R.framework/Versions/4.1/Resources/library/rgdal/gdal
GDAL binary built with GEOS: TRUE 
Loaded PROJ runtime: Rel. 7.2.1, January 1st, 2021, [PJ_VERSION: 721]
Path to PROJ shared files: /Library/Frameworks/R.framework/Versions/4.1/Resources/library/rgdal/proj
PROJ CDN enabled: FALSE
Linking to sp version:1.4-5
To mute warnings of possible GDAL/OSR exportToProj4() degradation,
use options("rgdal_show_exportToProj4_warnings"="none") before loading rgdal.
Overwritten PROJ_LIB was /Library/Frameworks/R.framework/Versions/4.1/Resources/library/rgdal/proj

I'm kind of wondering if this is an issue due to being too current, since I wasn't seeing this prior to my machine being re-imaged and I ran the same code with no issues back in 2021.

@jhollist
Copy link
Owner

jhollist commented Mar 7, 2022

It worked on a MacOS action: https://github.com/jhollist/elevatr/runs/5453029623?check_suite_focus=true

Your rgdal and sp are a little old (I have rgdal 1.5-28 and sp 1.4-6), but I don't think that is it since the error is coming from sf and the GDAL versions are both the same (I think).

@jordansread
Copy link

Bummer, I'm not seeing a smoking gun here. I upgraded sp and rgdal to current and I'm still hitting that error.

One key difference is that my old run of this code was on Mac 10.14 (Mojave) and it didn't trigger this error. Your GH action has Mac 11 (Big Sur) and works. My fails are on 10.5 (Catalina) which is the most recent MacOS that is approved for Department of the Interior at this time.

@jordansread
Copy link

Looking for some other differences - the github mac build is
GEOS 3.9.1, GDAL 3.4.0, PROJ 8.1.1
and I am
GEOS 3.10.2, GDAL 3.4.1, PROJ 9.0.0

A co-worker also on Mac 10.5 had an older set of
GEOS 3.8.1, GDAL 3.2.1, PROJ 7.2.1
but my code snippet worked for that ☝️

update
Thanks to a suggestion by David Watkins (co-worker), I installed sf from the CRAN binary v1.0.6 (not source) which knocked my versions down to
GEOS 3.9.1, GDAL 3.4.0, PROJ 8.1.1 and now the snippet works!! 🎉

So it seems that either these PROJ/GDAL versions or the sf version is the culprit.

@jhollist
Copy link
Owner

jhollist commented Mar 7, 2022 via email

@jhollist
Copy link
Owner

jhollist commented Mar 8, 2022

So, I changed the Action (https://github.com/jhollist/elevatr/runs/5465927334?check_suite_focus=true) and have it building sf 1.0.7 from source. That triggers the error you saw! It also errors on MacOS 11. So we at least have a solution for now and a reproducible example! Maybe the binary for 1.0.7 won't suffer from the same issue?!?! I think I will close this issue and open another one to track the possible impact of 1.0.7 on MacOS...

@alexkrohn
Copy link

I'm still having this problem on Ubuntu 22.04, with sf 1.0.9. Here's my session info and a reproducible example:

> sessionInfo()
R version 4.2.2 Patched (2022-11-10 r83330)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.1 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.10.0
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.10.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8      
 [8] LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] marmap_1.0.8       elevatr_0.4.3.9999 sf_1.0-9           ggtree_3.6.2       forcats_0.5.2      stringr_1.5.0      dplyr_1.0.10       purrr_0.3.5        readr_2.1.3        tidyr_1.2.1       
[11] tibble_3.1.8       ggplot2_3.4.0      tidyverse_1.3.2   

loaded via a namespace (and not attached):
  [1] googledrive_2.0.0   colorspace_2.0-3    ellipsis_0.3.2      class_7.3-20        rgdal_1.6-2         fs_1.5.2            aplot_0.1.9         rstudioapi_0.14     proxy_0.4-27       
 [10] furrr_0.3.1         listenv_0.8.0       farver_2.1.1        ggsn_0.5.0          remotes_2.4.2       bit64_4.0.5         fansi_1.0.3         lubridate_1.9.0     xml2_1.3.3         
 [19] codetools_0.2-18    ncdf4_1.20          cachem_1.0.6        jsonlite_1.8.4      broom_1.0.1         dbplyr_2.2.1        png_0.1-8           compiler_4.2.2      httr_1.4.4         
 [28] backports_1.4.1     assertthat_0.2.1    Matrix_1.5-1        fastmap_1.1.0       lazyeval_0.2.2      gargle_1.2.1        cli_3.4.1           s2_1.1.1            prettyunits_1.1.1  
 [37] tools_4.2.2         ggmap_3.0.1         igraph_1.3.5        gtable_0.3.1        glue_1.6.2          reshape2_1.4.4      USAboundaries_0.4.0 wk_0.7.1            Rcpp_1.0.9         
 [46] cellranger_1.1.0    raster_3.6-11       vctrs_0.5.1         ape_5.6-2           nlme_3.1-160        progressr_0.11.0    globals_0.16.2      rvest_1.0.3         timechange_0.1.1   
 [55] lifecycle_1.0.3     googlesheets4_1.0.1 future_1.29.0       terra_1.6-47        scales_1.2.1        vroom_1.6.0         hms_1.1.2           slippymath_0.3.1    parallel_4.2.2     
 [64] curl_4.3.3          memoise_2.0.1       ggfun_0.0.9         yulab.utils_0.0.5   stringi_1.7.8       RSQLite_2.2.19      maptools_1.1-5      tidytree_0.4.1      e1071_1.7-12       
 [73] shape_1.4.6         RgoogleMaps_1.4.5.3 rlang_1.0.6         pkgconfig_2.0.3     bitops_1.0-7        lattice_0.20-45     treeio_1.22.0       patchwork_1.1.2     labeling_0.4.2     
 [82] bit_4.0.5           tidyselect_1.2.0    parallelly_1.32.1   plyr_1.8.8          magrittr_2.0.3      R6_2.5.1            generics_0.1.3      DBI_1.1.3           pillar_1.8.1       
 [91] haven_2.5.1         foreign_0.8-82      withr_2.5.0         units_0.8-1         sp_1.5-1            modelr_0.1.10       crayon_1.5.2        KernSmooth_2.23-20  utf8_1.2.2         
[100] tzdb_0.3.0          progress_1.2.2      jpeg_0.1-10         grid_4.2.2          readxl_1.4.1        blob_1.2.3          reprex_2.0.2        digest_0.6.31       classInt_0.4-8     
[109] gdistance_1.6       adehabitatMA_0.3.15 gridGraphics_0.5-1  munsell_0.5.0       ggplotify_0.1.0 

Example:

library(sf)
library(elevatr)
library(USAboundaries)

states <- us_states(states = c("AL", "FL", "LA", "MI", "GA", "TN", "AR", "SC", "NC", "VA", "IN", "KY", "OH", "WV", "IL", "MO", "AR"))
 
se.elevations <- get_elev_raster(locations = states, z = 10, clip = "locations") 
Note: Your request will download approximately 984.1Mb.
 Accessing raster elevation [=========================] 100%
Mosaicing & Projecting
Error in sf::gdal_utils(util = "warp", source = files, destination = destfile,  : 
  gdal_utils warp: an error occured
In addition: There were 50 or more warnings (use warnings() to see the first 50)

Downgrading sf to 1.0.6 still produces this error. I don't believe I can build packages from binary on Linux. Any thoughts?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants