Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

proxy_stars vs stars using st_apply #349

Open
alexyshr opened this issue Nov 8, 2020 · 0 comments
Open

proxy_stars vs stars using st_apply #349

alexyshr opened this issue Nov 8, 2020 · 0 comments

Comments

@alexyshr
Copy link

alexyshr commented Nov 8, 2020 •
edited by edzer

Hi!. Here playing again with 5D (five dimensions) stars object! (from #348)

It is possible to use st_apply with the non-proxy stars object and the personalized function theFunction.

  1. How can I improve theFunction to work with the stars_proxy version of the object?
  2. How to remove NA plots when plotting the object st_functionNoProxy?

Purpose of the function theFunction: Look for Monday values (single attribute Temperature) of the FIRST level dimension and FIRST number dimension; for other days returns NA.

Data can be download here!

library(stars)
#> Loading required package: abind
#> Loading required package: sf
#> Linking to GEOS 3.8.0, GDAL 3.0.4, PROJ 6.3.1
library(lubridate)
#> 
#> Attaching package: 'lubridate'
#> The following objects are masked from 'package:base':
#> 
#>     date, intersect, setdiff, union
library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union
Sys.setenv(GDAL_MAX_BAND_COUNT=1200000)
path= "./"
ncname = "download5Dcolombia"
ncfile5d = paste0(path, ncname, ".nc")
(st_col5d = read_stars(ncfile5d, proxy=TRUE))
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: Recode from UTF-8 to CP_ACP failed with
#> the error: "Invalid argument".
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: The dataset has several variables
#> that could be identified as vector fields, but not all share the same primary
#> dimension. Consequently they will be ignored.
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: dimension #1 (number) is not a Time
#> dimension.
#> stars_proxy object with 1 attribute in file:
#> $download5Dcolombia.nc
#> [1] "[...]/download5Dcolombia.nc"
#> 
#> dimension(s):
#>        from  to         offset   delta  refsys point
#> x         1  25         -79.35     0.5      NA    NA
#> y         1  35          12.75    -0.5      NA    NA
#> level     1  37             NA      NA udunits    NA
#> number    1  10              0       1      NA    NA
#> time      1 480 2020-01-01 UTC 3 hours POSIXct    NA
#>                                               values x/y
#> x                                               NULL [x]
#> y                                               NULL [y]
#> level  [1,2) [millibars],...,[1000,1025) [millibars]    
#> number                                          NULL    
#> time                                            NULL

#Read timestamp values
st_time = st_col5d %>% slice(x, 1) %>% slice(y, 1) %>%
  slice(level, 1) %>% slice(number, 1) %>% st_as_stars()
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: Recode from UTF-8 to CP_ACP failed with
#> the error: "Invalid argument".
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: The dataset has several variables
#> that could be identified as vector fields, but not all share the same primary
#> dimension. Consequently they will be ignored.
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: dimension #1 (number) is not a Time
#> dimension.
timePOSIXlt = as.POSIXlt(seq(from = as_datetime(c(0), origin=st_dimensions(st_time)$time$offset),
                   by=st_dimensions(st_time)$time$delta, length.out = st_dimensions(st_time)$time$to))

(st_col5dNoProxy = st_col5d %>% slice(level, 1:2) %>%
                        slice(number, 1:2) %>% st_as_stars())
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: Recode from UTF-8 to CP_ACP failed with
#> the error: "Invalid argument".
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: The dataset has several variables
#> that could be identified as vector fields, but not all share the same primary
#> dimension. Consequently they will be ignored.
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: dimension #1 (number) is not a Time
#> dimension.
#> stars object with 5 dimensions and 1 attribute
#> attribute(s), summary of first 1e+05 cells:
#>  download5Dcolombia.nc [K]
#>  Min.   :237.1            
#>  1st Qu.:248.2            
#>  Median :258.9            
#>  Mean   :258.0            
#>  3rd Qu.:267.9            
#>  Max.   :275.3            
#> dimension(s):
#>        from  to         offset   delta  refsys point
#> x         1  25         -79.35     0.5      NA    NA
#> y         1  35          12.75    -0.5      NA    NA
#> level     1   2             NA      NA udunits    NA
#> number    1   2              0       1      NA    NA
#> time      1 480 2020-01-01 UTC 3 hours POSIXct    NA
#>                                      values x/y
#> x                                      NULL [x]
#> y                                      NULL [y]
#> level  [1,2) [millibars], [2,3) [millibars]    
#> number                                 NULL    
#> time                                   NULL

#theFunction description:
#Filtering temperatures registered on "Mondays"(theDay = 1),
#belonging to the first LEVEL (dimension), and
#belonging to the first NUMBER (dimension).
#The function returns values only for Mondays, for other days it returns NA

theFunction = function(theStars, timePlt, theDay){
  #theDay values:
  #Dom: 0
  #Lun: 1  <<<< Monday
  #Mar: 2
  #Mie: 3
  #Jue: 4
  #Vie: 5
  #Sat: 6
  #Negative or > 6: all days
  
  theValues = rep(NA, length.out=length(timePlt))

  if (theDay == 1){
    mondayIndexes = which(timePlt$wday == 1)
    
    
    
    arraySliced = theStars[1,1,mondayIndexes] #First Index is LEVEL
                                              #Second Index is NUMBER
                                              #Third Index is TIME
    theValues[mondayIndexes]= arraySliced[[1]]#The only attribute is Temperature
  }  
  return(theValues)
}

#With no proxy stars work perfect!
  st_functionNoProxy <- st_col5dNoProxy %>%
    st_apply(MARGIN = c("x", "y"), FUN = theFunction, timePlt = timePOSIXlt, theDay = 1) %>%
    st_set_dimensions("theFunction", timePOSIXlt$wday)
  
  #How to remove NA plots?
  plot(st_functionNoProxy)

#With stars_proxy the function DOES NOT receive extra parameters timePlt and theDay
st_functionProxy <- st_col5d %>%
    st_apply(MARGIN = c("x", "y"), FUN = theFunction, timePlt = timePOSIXlt, theDay = 1) %>%
    st_as_stars()
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: Recode from UTF-8 to CP_ACP failed with
#> the error: "Invalid argument".
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: The dataset has several variables
#> that could be identified as vector fields, but not all share the same primary
#> dimension. Consequently they will be ignored.
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: dimension #1 (number) is not a Time
#> dimension.
#> Error in FUN(array(newX[, i], d.call, dn.call), ...): argument "timePlt" is missing, with no default
Session info 
devtools::session_info()
#> - Session info ---------------------------------------------------------------
#>  setting  value                       
#>  version  R version 4.0.2 (2020-06-22)
#>  os       Windows 10 x64              
#>  system   x86_64, mingw32             
#>  ui       RTerm                       
#>  language (EN)                        
#>  collate  English_United States.1252  
#>  ctype    English_United States.1252  
#>  tz       Europe/Berlin               
#>  date     2020-11-08                  
#> 
#> - Packages -------------------------------------------------------------------
#>  package     * version date       lib source                          
#>  abind       * 1.4-5   2016-07-21 [1] CRAN (R 4.0.0)                  
#>  assertthat    0.2.1   2019-03-21 [1] CRAN (R 4.0.2)                  
#>  backports     1.1.10  2020-09-15 [1] CRAN (R 4.0.3)                  
#>  callr         3.5.1   2020-10-13 [1] CRAN (R 4.0.3)                  
#>  class         7.3-17  2020-04-26 [2] CRAN (R 4.0.2)                  
#>  classInt      0.4-3   2020-04-07 [1] CRAN (R 4.0.2)                  
#>  cli           2.1.0   2020-10-12 [1] CRAN (R 4.0.3)                  
#>  crayon        1.3.4   2017-09-16 [1] CRAN (R 4.0.2)                  
#>  curl          4.3     2019-12-02 [1] CRAN (R 4.0.2)                  
#>  DBI           1.1.0   2019-12-15 [1] CRAN (R 4.0.2)                  
#>  desc          1.2.0   2018-05-01 [1] CRAN (R 4.0.2)                  
#>  devtools      2.3.2   2020-09-18 [1] CRAN (R 4.0.2)                  
#>  digest        0.6.27  2020-10-24 [1] CRAN (R 4.0.3)                  
#>  dplyr       * 1.0.2   2020-08-18 [1] CRAN (R 4.0.2)                  
#>  e1071         1.7-4   2020-10-14 [1] CRAN (R 4.0.3)                  
#>  ellipsis      0.3.1   2020-05-15 [1] CRAN (R 4.0.2)                  
#>  evaluate      0.14    2019-05-28 [1] CRAN (R 4.0.2)                  
#>  fansi         0.4.1   2020-01-08 [1] CRAN (R 4.0.2)                  
#>  fs            1.5.0   2020-07-31 [1] CRAN (R 4.0.2)                  
#>  generics      0.0.2   2018-11-29 [1] CRAN (R 4.0.2)                  
#>  glue          1.4.2   2020-08-27 [1] CRAN (R 4.0.2)                  
#>  highr         0.8     2019-03-20 [1] CRAN (R 4.0.2)                  
#>  htmltools     0.5.0   2020-06-16 [1] CRAN (R 4.0.2)                  
#>  httr          1.4.2   2020-07-20 [1] CRAN (R 4.0.2)                  
#>  KernSmooth    2.23-17 2020-04-26 [2] CRAN (R 4.0.2)                  
#>  knitr         1.29    2020-06-23 [1] CRAN (R 4.0.2)                  
#>  lifecycle     0.2.0   2020-03-06 [1] CRAN (R 4.0.2)                  
#>  lubridate   * 1.7.9   2020-06-08 [1] CRAN (R 4.0.2)                  
#>  lwgeom        0.2-5   2020-06-12 [1] CRAN (R 4.0.2)                  
#>  magrittr      1.5     2014-11-22 [1] CRAN (R 4.0.2)                  
#>  memoise       1.1.0   2017-04-21 [1] CRAN (R 4.0.2)                  
#>  mime          0.9     2020-02-04 [1] CRAN (R 4.0.0)                  
#>  pillar        1.4.6   2020-07-10 [1] CRAN (R 4.0.2)                  
#>  pkgbuild      1.1.0   2020-07-13 [1] CRAN (R 4.0.2)                  
#>  pkgconfig     2.0.3   2019-09-22 [1] CRAN (R 4.0.2)                  
#>  pkgload       1.1.0   2020-05-29 [1] CRAN (R 4.0.2)                  
#>  prettyunits   1.1.1   2020-01-24 [1] CRAN (R 4.0.2)                  
#>  processx      3.4.4   2020-09-03 [1] CRAN (R 4.0.2)                  
#>  ps            1.4.0   2020-10-07 [1] CRAN (R 4.0.3)                  
#>  purrr         0.3.4   2020-04-17 [1] CRAN (R 4.0.2)                  
#>  R6            2.4.1   2019-11-12 [1] CRAN (R 4.0.2)                  
#>  Rcpp          1.0.5   2020-07-06 [1] CRAN (R 4.0.2)                  
#>  remotes       2.2.0   2020-07-21 [1] CRAN (R 4.0.2)                  
#>  rlang         0.4.8   2020-10-08 [1] CRAN (R 4.0.3)                  
#>  rmarkdown     2.3     2020-06-18 [1] CRAN (R 4.0.2)                  
#>  rprojroot     1.3-2   2018-01-03 [1] CRAN (R 4.0.2)                  
#>  sessioninfo   1.1.1   2018-11-05 [1] CRAN (R 4.0.2)                  
#>  sf          * 0.9-7   2020-11-04 [1] Github (r-spatial/sf@92959fd)   
#>  stars       * 0.4-4   2020-11-04 [1] Github (r-spatial/stars@4eaaa76)
#>  stringi       1.5.3   2020-09-09 [1] CRAN (R 4.0.3)                  
#>  stringr       1.4.0   2019-02-10 [1] CRAN (R 4.0.2)                  
#>  testthat      2.3.2   2020-03-02 [1] CRAN (R 4.0.2)                  
#>  tibble        3.0.4   2020-10-12 [1] CRAN (R 4.0.3)                  
#>  tidyselect    1.1.0   2020-05-11 [1] CRAN (R 4.0.2)                  
#>  units         0.6-7   2020-06-13 [1] CRAN (R 4.0.2)                  
#>  usethis       1.6.3   2020-09-17 [1] CRAN (R 4.0.2)                  
#>  vctrs         0.3.4   2020-08-29 [1] CRAN (R 4.0.2)                  
#>  withr         2.3.0   2020-09-22 [1] CRAN (R 4.0.3)                  
#>  xfun          0.18    2020-09-29 [1] CRAN (R 4.0.3)                  
#>  xml2          1.3.2   2020-04-23 [1] CRAN (R 4.0.2)                  
#>  yaml          2.2.1   2020-02-01 [1] CRAN (R 4.0.2)                  
#> 
#> [1] E:/Documents/R/win-library/4.0
#> [2] C:/Program Files/R/R-4.0.2/library
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@alexyshr