Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refusal to scan all scripts either with scan_project_files() & create_checkpoint() #308

Open
ThomasHoussoy opened this issue Aug 3, 2021 · 15 comments

Comments

@ThomasHoussoy
Copy link

Hello,

Trying to implement a checkpoint for my project resulted in the functions scan_project_files() & create_checkpoint() refusing to scan any of my script (either R or RMarkdown).

I put copies of the R & Rstudio session infos, the script setting up the checkpoint with the results on the console below.

I'm a bit confused the package & functions failed this way. After tweaking and reseraching the issues of this Github page (both opened and closed), I'm still unable to pinpoint what exactly failed.

This are relevant (I think) information I collected while trying to debug the issue :

  • changing the namming format of the script files, even to bare minimum (one word with no special characters) doesn't solve the issue
  • copying the script to the main directory instead of putting them in their own folder doesn't solve the issue
  • using a brand new Rproject with no version control (in my case Git/Github) and running the code with a test script containing a copy of the library() calls does solve the issue
  • I branched my project in order to introduce the chekpoint package

I search the Issues page for both the main console result ('Following files could not be scanned') adn 'git" to check if this issue was solved. In the first cas, I see no previous issue published, for the second there is one on git integration but is closed (I supposed it is solved) but do not contain info pointing me to a way to solve the issue in my original R project.

I strongly suspect that the problem originates in the fact I use version control for this project.

Do you confirm there is an incompatibility between usig Git/GitHub and the package ?
I fell the need to post this issue despite the results exposed above because version control is most of the time part of a reproductible research/data analysis R project and find odd the package failed in this circumstances.

If not, is there a piece of documentation (not found yet after seaching) indicating special parmaters or procedures for using the package with Git/GitHub ?

Best regards,

R & R Studio session info

> sessionInfo()
R version 4.0.5 (2021-03-31)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19042)

Matrix products: default

locale:
[1] LC_COLLATE=French_France.1252  LC_CTYPE=French_France.1252    LC_MONETARY=French_France.1252 LC_NUMERIC=C                   LC_TIME=French_France.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] checkpoint_1.0.0

loaded via a namespace (and not attached):
 [1] zip_2.1.1         compiler_4.0.5    pillar_1.6.1      prettyunits_1.1.1 tools_4.0.5       uuid_0.1-4        digest_0.6.27     pkgbuild_1.2.0    jsonlite_1.7.2    tibble_3.1.1      lifecycle_1.0.0  
[12] pkgconfig_2.0.3   rlang_0.4.11      cli_2.5.0         rstudioapi_0.13   filelock_1.0.2    curl_4.3.1        xfun_0.22         withr_2.4.2       stringr_1.4.0     knitr_1.33        rappdirs_0.3.3   
[23] desc_1.3.0        vctrs_0.3.8       rprojroot_2.0.2   glue_1.4.2        R6_2.5.0          processx_3.5.2    fansi_0.4.2       callr_3.7.0       magrittr_2.0.1    rematch2_2.1.2    pkgdepends_0.1.2 
[34] ps_1.6.0          ellipsis_0.3.2    assertthat_0.2.1  lpSolve_5.6.15    pkgcache_1.2.2    tinytex_0.31      utf8_1.2.1        stringi_1.5.3     crayon_1.4.1 
> rstudioapi::versionInfo()
$citation

To cite RStudio in publications use:

  RStudio Team (2021). RStudio: Integrated Development Environment for R. RStudio, PBC, Boston, MA URL http://www.rstudio.com/.

A BibTeX entry for LaTeX users is

  @Manual{,
    title = {RStudio: Integrated Development Environment for R},
    author = {{RStudio Team}},
    organization = {RStudio, PBC},
    address = {Boston, MA},
    year = {2021},
    url = {http://www.rstudio.com/},
  }


$mode
[1] "desktop"

$version
[1] ‘1.4.1106’

$release_name
[1] "Tiger Daylily"

Script and Consol results

> #ETAPE 00 - Checkpoint de reproductivité
> 
> #1 - Chargement des packages ----
> 
> library(checkpoint)

checkpoint: Part of the Reproducible R Toolkit from Microsoft
https://mran.microsoft.com/documents/rro/reproducibility/
> 
> #2 - Options générales ----
> 
> scan_project_files()
$pkgs
[1] "rmarkdown"

$errors
[1] "./editeurs/02-Analyse_de_structure_brute.R"           "./editeurs/02-Analyse_de_structure_brute.Rmd"         "./editeurs/03-Production_des_tableaux_de_travail.R"  
[4] "./editeurs/03-Production_des_tableaux_de_travail.Rmd"

Warning message:
Following files could not be scanned:
./editeurs/02-Analyse_de_structure_brute.R
./editeurs/02-Analyse_de_structure_brute.Rmd
./editeurs/03-Production_des_tableaux_de_travail.R
./editeurs/03-Production_des_tableaux_de_travail.Rmd 
> 
> create_checkpoint(snapshot_date = "2021-08-02",
+                   r_version = "4.0.5",
+                   checkpoint_location = ".")
Creating checkpoint directory D:/[R] PSCE2009_these/PSCE2009_these/.checkpoint/2021-08-02/lib/x86_64-w64-mingw32/4.0.5
v Loading metadata database ... done
i No downloads are needed                                                  
Warning messages:
1: Following files could not be scanned:
./editeurs/02-Analyse_de_structure_brute.R
./editeurs/02-Analyse_de_structure_brute.Rmd
./editeurs/03-Production_des_tableaux_de_travail.R
./editeurs/03-Production_des_tableaux_de_travail.Rmd 
2: The `x` argument of `as_tibble.matrix()` must have unique column names if `.name_repair` is omitted as of tibble 2.0.0.
Using compatibility `.name_repair`.
This warning is displayed once every 8 hours.
Call `lifecycle::last_warnings()` to see where this warning was generated.
@hongooi73
Copy link
Contributor

Can you try running scan_project_files in an Rgui.exe window, rather than Rstudio?

@ThomasHoussoy
Copy link
Author

The results are identical and displayed below.

Disclaimer : first time using the R GUI. I modified the script to add a code-line setting the directory to the Rproject location (seems to me the closer way to reproduce what I did with RStudio while using the R GUI).

R session info

> sessionInfo()
R version 4.0.5 (2021-03-31)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19042)

Matrix products: default

locale:
[1] LC_COLLATE=French_France.1252  LC_CTYPE=French_France.1252   
[3] LC_MONETARY=French_France.1252 LC_NUMERIC=C                  
[5] LC_TIME=French_France.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] checkpoint_1.0.0

loaded via a namespace (and not attached):
 [1] zip_2.1.1         pillar_1.6.1      compiler_4.0.5    prettyunits_1.1.1
 [5] tools_4.0.5       digest_0.6.27     pkgbuild_1.2.0    uuid_0.1-4       
 [9] jsonlite_1.7.2    lifecycle_1.0.0   tibble_3.1.1      pkgconfig_2.0.3  
[13] rlang_0.4.11      cli_2.5.0         rstudioapi_0.13   filelock_1.0.2   
[17] curl_4.3.1        xfun_0.22         withr_2.4.2       stringr_1.4.0    
[21] knitr_1.33        desc_1.3.0        vctrs_0.3.8       rappdirs_0.3.3   
[25] rprojroot_2.0.2   glue_1.4.2        R6_2.5.0          processx_3.5.2   
[29] fansi_0.4.2       callr_3.7.0       magrittr_2.0.1    rematch2_2.1.2   
[33] pkgdepends_0.1.2  ps_1.6.0          ellipsis_0.3.2    assertthat_0.2.1 
[37] lpSolve_5.6.15    pkgcache_1.2.2    utf8_1.2.1        stringi_1.5.3    
[41] crayon_1.4.1

Script and Consol results

> #ETAPE 00 - Checkpoint de reproductivité
> 
> setwd("D:/[R] PSCE2009_these/PSCE2009_these")
> 
> #1 - Chargement des packages ----
> 
> library(checkpoint)

checkpoint: Part of the Reproducible R Toolkit from Microsoft
https://mran.microsoft.com/documents/rro/reproducibility/
> 
> #2 - Options générales ----
> 
> scan_project_files()
$pkgs
[1] "rmarkdown"

$errors
[1] "./editeurs/02-Analyse_de_structure_brute.R"          
[2] "./editeurs/02-Analyse_de_structure_brute.Rmd"        
[3] "./editeurs/03-Production_des_tableaux_de_travail.R"  
[4] "./editeurs/03-Production_des_tableaux_de_travail.Rmd"

Warning message:
Following files could not be scanned:
./editeurs/02-Analyse_de_structure_brute.R
./editeurs/02-Analyse_de_structure_brute.Rmd
./editeurs/03-Production_des_tableaux_de_travail.R
./editeurs/03-Production_des_tableaux_de_travail.Rmd 
> 
> create_checkpoint(snapshot_date = "2021-08-02",
+                   r_version = "4.0.5",
+                   checkpoint_location = ".")
Creating checkpoint directory D:/[R] PSCE2009_these/PSCE2009_these/.checkpoint/2021-08-02/lib/x86_64-w64-mingw32/4.0.5


i Loading metadata database
v Loading metadata database ... done

 Found  1  deps for  0/1  pkgs [-] Resolving standard (CRAN/BioC) packages
 Found  1  deps for  0/1  pkgs [\] Resolving standard (CRAN/BioC) packages
                                                                          
i Getting 13 pkgs (4.89 MB), 5 (20.68 MB) cached


          (             )   0% |  0/13 pkg | ETA ??s  | Connecting...
          (             )   0% |  0/13 pkg | ETA ??s  | Connecting...
                                                                     
v Cached copy of base64enc 0.1-3 (windows) is the latest build
          (             )   0% |  0/13 pkg | ETA ??s  | Connecting...
                                                                     
v Cached copy of digest 0.6.27 (windows) is the latest build
          (             )   0% |  0/13 pkg | ETA ??s  | Connecting...
                                                                     
v Cached copy of evaluate 0.14 (windows) is the latest build
          (             )   0% |  0/13 pkg | ETA ??s  | Connecting...
                                                                     
v Cached copy of glue 1.4.2 (windows) is the latest build
          (             )   0% |  0/13 pkg | ETA ??s  | Connecting...
                                                                     
v Cached copy of highr 0.9 (windows) is the latest build
          (             )   0% |  0/13 pkg | ETA ??s  | Connecting...
                                                                     
v Cached copy of htmltools 0.5.1.1 (windows) is the latest build
          (             )   0% |  0/13 pkg | ETA ??s  | Connecting...
                                                                     
v Cached copy of jsonlite 1.7.2 (windows) is the latest build
          (             )   0% |  0/13 pkg | ETA ??s  | Connecting...
                                                                     
v Cached copy of knitr 1.33 (windows) is the latest build
          (             )   0% |  0/13 pkg | ETA ??s  | Connecting...
                                                                     
v Cached copy of magrittr 2.0.1 (windows) is the latest build
          (             )   0% |  0/13 pkg | ETA ??s  | Connecting...
                                                                     
v Cached copy of markdown 1.1 (windows) is the latest build
          (             )   0% |  0/13 pkg | ETA ??s  | Connecting...
                                                                     
v Cached copy of rlang 0.4.11 (windows) is the latest build
          (             )   0% |  0/13 pkg | ETA ??s  | Connecting...
                                                                     
v Cached copy of stringr 1.4.0 (windows) is the latest build
          (             )   0% |  0/13 pkg | ETA ??s  | Connecting...
                                                                     
v Cached copy of yaml 2.2.1 (windows) is the latest build
          (             )   0% |  0/13 pkg | ETA ??s  | Connecting...
          (#############) 100% | 13/13 pkg | ETA ??s  | Connecting...
                                                                     
v No downloads needed, all packages are cached

Installing...
(-------------) | [BLD]  18/18     | [INS]  0/18     | 
(-------------) | [BLD]  18/18     | [INS]  0/18 / 1 | installing base64enc
(-------------) | [BLD]  18/18     | [INS]  0/18 \ 1 | installing base64enc
(-------------) | [BLD]  18/18     | [INS]  0/18 / 1 | installing base64enc
(-------------) | [BLD]  18/18     | [INS]  0/18 \ 1 | installing base64enc
(-------------) | [BLD]  18/18     | [INS]  0/18 / 1 | installing base64enc
                                                                           
v Installed base64enc 0.1-3  (523ms)
(-------------) | [BLD]  18/18     | [INS]  0/18 / 1 | installing base64enc
(-------------) | [BLD]  18/18     | [INS]  0/18 \ 1 | installing base64enc
(-------------) | [BLD]  18/18     | [INS]  1/18     |                     
(-------------) | [BLD]  18/18     | [INS]  1/18 \ 1 | installing digest
(-------------) | [BLD]  18/18     | [INS]  1/18 / 1 | installing digest
                                                                        
v Installed digest 0.6.27  (255ms)
(-------------) | [BLD]  18/18     | [INS]  1/18 / 1 | installing digest
(-------------) | [BLD]  18/18     | [INS]  1/18 \ 1 | installing digest
(#------------) | [BLD]  18/18     | [INS]  2/18     |                  
(#------------) | [BLD]  18/18     | [INS]  2/18 \ 1 | installing evaluate
(#------------) | [BLD]  18/18     | [INS]  2/18 / 1 | installing evaluate
(#------------) | [BLD]  18/18     | [INS]  2/18 \ 1 | installing evaluate
(#------------) | [BLD]  18/18     | [INS]  2/18 / 1 | installing evaluate
                                                                          
v Installed evaluate 0.14  (311ms)
(#------------) | [BLD]  18/18     | [INS]  2/18 / 1 | installing evaluate
(#------------) | [BLD]  18/18     | [INS]  2/18 \ 1 | installing evaluate
(##-----------) | [BLD]  18/18     | [INS]  3/18     |                    
(##-----------) | [BLD]  18/18     | [INS]  3/18 \ 1 | installing glue
(##-----------) | [BLD]  18/18     | [INS]  3/18 / 1 | installing glue
                                                                      
v Installed glue 1.4.2  (220ms)
(##-----------) | [BLD]  18/18     | [INS]  3/18 / 1 | installing glue
(##-----------) | [BLD]  18/18     | [INS]  3/18 \ 1 | installing glue
(##-----------) | [BLD]  18/18     | [INS]  4/18     |                
(##-----------) | [BLD]  18/18     | [INS]  4/18 \ 1 | installing highr
(##-----------) | [BLD]  18/18     | [INS]  4/18 / 1 | installing highr
                                                                       
v Installed highr 0.9  (250ms)
(##-----------) | [BLD]  18/18     | [INS]  4/18 / 1 | installing highr
(##-----------) | [BLD]  18/18     | [INS]  4/18 \ 1 | installing highr
(###----------) | [BLD]  18/18     | [INS]  5/18     |                 
(###----------) | [BLD]  18/18     | [INS]  5/18 \ 1 | installing htmltools
(###----------) | [BLD]  18/18     | [INS]  5/18 / 1 | installing htmltools
                                                                           
v Installed htmltools 0.5.1.1  (197ms)
(###----------) | [BLD]  18/18     | [INS]  5/18 / 1 | installing htmltools
(###----------) | [BLD]  18/18     | [INS]  5/18 \ 1 | installing htmltools
(####---------) | [BLD]  18/18     | [INS]  6/18     |                     
(####---------) | [BLD]  18/18     | [INS]  6/18 \ 1 | installing jsonlite
(####---------) | [BLD]  18/18     | [INS]  6/18 / 1 | installing jsonlite
(####---------) | [BLD]  18/18     | [INS]  6/18 \ 1 | installing jsonlite
(####---------) | [BLD]  18/18     | [INS]  6/18 / 1 | installing jsonlite
                                                                          
v Installed jsonlite 1.7.2  (336ms)
(####---------) | [BLD]  18/18     | [INS]  6/18 / 1 | installing jsonlite
(####---------) | [BLD]  18/18     | [INS]  6/18 \ 1 | installing jsonlite
(#####--------) | [BLD]  18/18     | [INS]  7/18     |                    
(#####--------) | [BLD]  18/18     | [INS]  7/18 \ 1 | installing knitr
(#####--------) | [BLD]  18/18     | [INS]  7/18 / 1 | installing knitr
(#####--------) | [BLD]  18/18     | [INS]  7/18 \ 1 | installing knitr
(#####--------) | [BLD]  18/18     | [INS]  7/18 / 1 | installing knitr
(#####--------) | [BLD]  18/18     | [INS]  7/18 \ 1 | installing knitr
(#####--------) | [BLD]  18/18     | [INS]  7/18 / 1 | installing knitr
(#####--------) | [BLD]  18/18     | [INS]  7/18 \ 1 | installing knitr
(#####--------) | [BLD]  18/18     | [INS]  7/18 / 1 | installing knitr
(#####--------) | [BLD]  18/18     | [INS]  7/18 \ 1 | installing knitr
                                                                       
v Installed knitr 1.33  (897ms)
(#####--------) | [BLD]  18/18     | [INS]  7/18 \ 1 | installing knitr
(#####--------) | [BLD]  18/18     | [INS]  7/18 / 1 | installing knitr
(#####--------) | [BLD]  18/18     | [INS]  8/18     |                 
(#####--------) | [BLD]  18/18     | [INS]  8/18 / 1 | installing magrittr
(#####--------) | [BLD]  18/18     | [INS]  8/18 \ 1 | installing magrittr
(#####--------) | [BLD]  18/18     | [INS]  8/18 / 1 | installing magrittr
                                                                          
v Installed magrittr 2.0.1  (264ms)
(#####--------) | [BLD]  18/18     | [INS]  8/18 / 1 | installing magrittr
(#####--------) | [BLD]  18/18     | [INS]  8/18 \ 1 | installing magrittr
(######-------) | [BLD]  18/18     | [INS]  9/18     |                    
(######-------) | [BLD]  18/18     | [INS]  9/18 \ 1 | installing markdown
(######-------) | [BLD]  18/18     | [INS]  9/18 / 1 | installing markdown
(######-------) | [BLD]  18/18     | [INS]  9/18 \ 1 | installing markdown
                                                                          
v Installed markdown 1.1  (337ms)
(######-------) | [BLD]  18/18     | [INS]  9/18 \ 1 | installing markdown
(######-------) | [BLD]  18/18     | [INS]  9/18 / 1 | installing markdown
(#######------) | [BLD]  18/18     | [INS]  10/18     |                   
(#######------) | [BLD]  18/18     | [INS]  10/18 / 1 | installing mime
(#######------) | [BLD]  18/18     | [INS]  10/18 \ 1 | installing mime
                                                                       
v Installed mime 0.11  (158ms)
(#######------) | [BLD]  18/18     | [INS]  10/18 \ 1 | installing mime
(#######------) | [BLD]  18/18     | [INS]  10/18 / 1 | installing mime
(#######------) | [BLD]  18/18     | [INS]  11/18     |                
(#######------) | [BLD]  18/18     | [INS]  11/18 / 1 | installing rlang
(#######------) | [BLD]  18/18     | [INS]  11/18 \ 1 | installing rlang
(#######------) | [BLD]  18/18     | [INS]  11/18 / 1 | installing rlang
(#######------) | [BLD]  18/18     | [INS]  11/18 \ 1 | installing rlang
                                                                        
v Installed rlang 0.4.11  (306ms)
(#######------) | [BLD]  18/18     | [INS]  11/18 \ 1 | installing rlang
(#######------) | [BLD]  18/18     | [INS]  11/18 / 1 | installing rlang
(########-----) | [BLD]  18/18     | [INS]  12/18     |                 
(########-----) | [BLD]  18/18     | [INS]  12/18 / 1 | installing rmarkdown
(########-----) | [BLD]  18/18     | [INS]  12/18 \ 1 | installing rmarkdown
(########-----) | [BLD]  18/18     | [INS]  12/18 / 1 | installing rmarkdown
(########-----) | [BLD]  18/18     | [INS]  12/18 \ 1 | installing rmarkdown
(########-----) | [BLD]  18/18     | [INS]  12/18 / 1 | installing rmarkdown
(########-----) | [BLD]  18/18     | [INS]  12/18 \ 1 | installing rmarkdown
(########-----) | [BLD]  18/18     | [INS]  12/18 / 1 | installing rmarkdown
(########-----) | [BLD]  18/18     | [INS]  12/18 \ 1 | installing rmarkdown
(########-----) | [BLD]  18/18     | [INS]  12/18 / 1 | installing rmarkdown
(########-----) | [BLD]  18/18     | [INS]  12/18 \ 1 | installing rmarkdown
(########-----) | [BLD]  18/18     | [INS]  12/18 / 1 | installing rmarkdown
(########-----) | [BLD]  18/18     | [INS]  12/18 \ 1 | installing rmarkdown
(########-----) | [BLD]  18/18     | [INS]  12/18 / 1 | installing rmarkdown
(########-----) | [BLD]  18/18     | [INS]  12/18 \ 1 | installing rmarkdown
                                                                            
v Installed rmarkdown 2.9  (1.4s)
(########-----) | [BLD]  18/18     | [INS]  12/18 \ 1 | installing rmarkdown
(########-----) | [BLD]  18/18     | [INS]  12/18 / 1 | installing rmarkdown
(#########----) | [BLD]  18/18     | [INS]  13/18     |                     
(#########----) | [BLD]  18/18     | [INS]  13/18 / 1 | installing stringi
(#########----) | [BLD]  18/18     | [INS]  13/18 \ 1 | installing stringi
(#########----) | [BLD]  18/18     | [INS]  13/18 / 1 | installing stringi
(#########----) | [BLD]  18/18     | [INS]  13/18 \ 1 | installing stringi
(#########----) | [BLD]  18/18     | [INS]  13/18 / 1 | installing stringi
(#########----) | [BLD]  18/18     | [INS]  13/18 \ 1 | installing stringi
(#########----) | [BLD]  18/18     | [INS]  13/18 / 1 | installing stringi
                                                                          
v Installed stringi 1.7.3  (790ms)
(#########----) | [BLD]  18/18     | [INS]  13/18 / 1 | installing stringi
(#########----) | [BLD]  18/18     | [INS]  13/18 \ 1 | installing stringi
(##########---) | [BLD]  18/18     | [INS]  14/18     |                   
(##########---) | [BLD]  18/18     | [INS]  14/18 \ 1 | installing stringr
(##########---) | [BLD]  18/18     | [INS]  14/18 / 1 | installing stringr
(##########---) | [BLD]  18/18     | [INS]  14/18 \ 1 | installing stringr
                                                                          
v Installed stringr 1.4.0  (271ms)
(##########---) | [BLD]  18/18     | [INS]  14/18 \ 1 | installing stringr
(##########---) | [BLD]  18/18     | [INS]  14/18 / 1 | installing stringr
(##########---) | [BLD]  18/18     | [INS]  15/18     |                   
(##########---) | [BLD]  18/18     | [INS]  15/18 / 1 | installing tinytex
(##########---) | [BLD]  18/18     | [INS]  15/18 \ 1 | installing tinytex
                                                                          
v Installed tinytex 0.32  (157ms)
(##########---) | [BLD]  18/18     | [INS]  15/18 \ 1 | installing tinytex
(##########---) | [BLD]  18/18     | [INS]  15/18 / 1 | installing tinytex
(###########--) | [BLD]  18/18     | [INS]  16/18     |                   
(###########--) | [BLD]  18/18     | [INS]  16/18 / 1 | installing xfun
(###########--) | [BLD]  18/18     | [INS]  16/18 \ 1 | installing xfun
(###########--) | [BLD]  18/18     | [INS]  16/18 / 1 | installing xfun
                                                                       
v Installed xfun 0.24  (252ms)
(###########--) | [BLD]  18/18     | [INS]  16/18 / 1 | installing xfun
(###########--) | [BLD]  18/18     | [INS]  16/18 \ 1 | installing xfun
(############-) | [BLD]  18/18     | [INS]  17/18     |                
(############-) | [BLD]  18/18     | [INS]  17/18 \ 1 | installing yaml
(############-) | [BLD]  18/18     | [INS]  17/18 / 1 | installing yaml
(############-) | [BLD]  18/18     | [INS]  17/18 \ 1 | installing yaml
                                                                       
v Installed yaml 2.2.1  (231ms)
(############-) | [BLD]  18/18     | [INS]  17/18 \ 1 | installing yaml
(############-) | [BLD]  18/18     | [INS]  17/18 / 1 | installing yaml
                                                                       
Warning messages:
1: Following files could not be scanned:
./editeurs/02-Analyse_de_structure_brute.R
./editeurs/02-Analyse_de_structure_brute.Rmd
./editeurs/03-Production_des_tableaux_de_travail.R
./editeurs/03-Production_des_tableaux_de_travail.Rmd 
2: The `x` argument of `as_tibble.matrix()` must have unique column names if `.name_repair` is omitted as of tibble 2.0.0.
Using compatibility `.name_repair`.
This warning is displayed once every 8 hours.
Call `lifecycle::last_warnings()` to see where this warning was generated.

@hongooi73
Copy link
Contributor

I can run scan_project_files in a directory with a git repo, so I don't think that's the problem. For some reason it's refusing to scan those specific files. If they don't contain any confidential info, is it possible for you to send one of them to me?

@ThomasHoussoy
Copy link
Author

The Rmd files are sure to contain text with confidential information.
The code in the Rmd files and the R files may contain elements breachin the security and confidential agreement between the University, the State organization and I.

However :

  • the R files contain the exact copy of the code in the Rmd files ;
  • each files starts with the code sections related to charging the packages, setting the general options, loading the files (their names are not descriptive enough to be an issue) and creating specific functions for each task of data analysis.

I can produce a copy of these files only containing these code sections right away. Would it be ok in terms of reproductibility ?

Otherwise, I will have to review the terms of the agreement and anonymized the files if necessary/accordingly ; but I will take a bit of time to provide them.

@hongooi73
Copy link
Contributor

Sure, you can send the copy.

@ThomasHoussoy
Copy link
Author

Here are the copy of the accessible part of the file "02-Analyse_de_structure_brute.R"

#ETAPE 02 - Analyse de structure brute

#1 - Chargement des packages ----

library(haven)
library(tidyverse)
library(questionr)
library(gtsummary)
library(survey)
library(networkD3)
library(scales)

#2 - Options générales ----

options(OutDec = ",")
options(scipen = 999)
theme_gtsummary_language("fr",
                         decimal.mark = ",",
                         big.mark = " ")

#3 - Chargement des données ----

  #3.01 - Chargement des données brutes ----

A_base_etablissement_mr004 <- read_sas("donnees/original/base_etab_mr004.sas7bdat", NULL)
A_base_contrat_mr004 <- read_sas("donnees/original/base_contrat_mr004.sas7bdat", NULL)
A_base_salarie_mr004 <- read_sas("donnees/original/base_salaries_mr004.sas7bdat", NULL)

  #3.02 - Création des objets "survey" ----

A_survey_etablissement_mr004 <- svydesign(id = ~newid_etab,
                                          weights = ~pond_etab,
                                          data = A_base_etablissement_mr004)

B_survey_contrat_pond_mr004 <- A_base_etablissement_mr004 %>%
  select(newid_etab, pond_etab)
B_survey_contrat_mr004 <- merge(A_base_contrat_mr004, B_survey_contrat_pond_mr004,
                                by = "newid_etab")
rm(B_survey_contrat_pond_mr004)
A_survey_contrat_mr004 <- svydesign(id = ~newid_etab,
                                    weights = ~pond_etab,
                                    data = B_survey_contrat_mr004)
rm(B_survey_contrat_mr004)

A_survey_salarie_mr004 <- svydesign(id = ~newid_sal,
                                    weights = ~pond_sal,
                                    data = A_base_salarie_mr004)

#4 - Fonctions propres ----

  #4.01 - Tableau de fréquence d'une variable catégorielle ----

frequence_categorielle <- function(data_brute, data_ponderee, variable) {
  
  effectif <- data_brute %>%
    tbl_summary(include = c(variable),
                type = list(variable ~ "categorical"),
                statistic = list(all_categorical() ~ "{n}"),
                missing = "always",
                label = variable ~ "",
                digits = list(variable ~ c(0))) %>%
    modify_header(list(label ~ "**Modalites**",
                       stat_0 ~ "*Effectif*"))
  
  frequence_brute <- data_brute %>%
    tbl_summary(include = c(variable),
                type = list(variable ~ "categorical"),
                statistic = list(all_categorical() ~ "{p}"),
                missing = "no",
                label = variable ~ "",
                digits = list(variable ~ c(2))) %>%
    modify_header(list(label ~ "**Modalites**",
                       stat_0 ~ "*Fréquence brute*"))
  
  frequence_ponderee <- data_ponderee %>%
    tbl_svysummary(include = c(variable),
                   type = list(variable ~ "categorical"),
                   statistic = list(all_categorical() ~ "{p}"),
                   missing = "no",
                   label = variable ~ "",
                   digits = list(variable ~ c(2))) %>%
    modify_header(list(label ~ "**Modalites**",
                       stat_0 ~ "*Fréquence pondérée*"))
  
  table_frequence_categorielle <- tbl_merge(list(effectif,frequence_brute,frequence_ponderee))
  
  return(table_frequence_categorielle)
  
}

  #4.02 - Tableau de fréquence d'une variable dichotomique ----

frequence_dichotomique <- function(data_brute, data_ponderee, variable) {
  
  effectif <- data_brute %>%
    tbl_summary(include = c(variable),
                type = all_dichotomous() ~ "categorical",
                statistic = list(all_categorical() ~ "{n}"),
                missing = "always",
                label = variable ~ variable,
                digits = list(variable ~ c(0))) %>%
    modify_header(list(label ~ "**Modalites**",
                       stat_0 ~ "*Effectif*"))
  
  frequence_brute <- data_brute %>%
    tbl_summary(include = c(variable),
                type = all_dichotomous() ~ "categorical",
                statistic = list(all_categorical() ~ "{p}"),
                missing = "no",
                label = variable ~ variable,
                digits = list(variable ~ c(2))) %>%
    modify_header(list(label ~ "**Modalites**",
                       stat_0 ~ "*Fréquence brute*"))
  
  frequence_ponderee <- data_ponderee %>%
    tbl_svysummary(include = c(variable),
                   type = all_dichotomous() ~ "categorical",
                   statistic = list(all_categorical() ~ "{p}"),
                   missing = "no",
                   label = variable ~ variable,
                   digits = list(variable ~ c(2))) %>%
    modify_header(list(label ~ "**Modalites**",
                       stat_0 ~ "*Fréquence pondérée*"))
  
  table_frequence_categorielle <- tbl_merge(list(effectif,frequence_brute,frequence_ponderee))
  
  return(table_frequence_categorielle)
  
}

And of the file "03-Production_des_tableaux_de_travail.R"

#ETAPE 03 - Production des tableaux de travail

#1 - Chargement des packages ----

library(haven)
library(tidyverse)
library(questionr)
library(gtsummary)
library(forcats)
library(labelled)
library(networkD3)

#2 - Options générales ----

options(OutDec=",")
options(scipen=999)
theme_gtsummary_language("fr", decimal.mark = ",", big.mark = " ")

#3 - Chargement des données des données brutes ----

A_base_etablissement_mr004 <- read_sas("donnees/original/base_etab_mr004.sas7bdat", NULL)
A_base_contrat_mr004 <- read_sas("donnees/original/base_contrat_mr004.sas7bdat", NULL)
A_base_salarie_mr004 <- read_sas("donnees/original/base_salaries_mr004.sas7bdat", NULL)

@ThomasHoussoy
Copy link
Author

I haven't put the equivalent Rmd files because they contain the same code.

However, weirdly enough, I notice the warning messages only concerns R and Rmd files containing library() calls ... which would mean the function know somehow which scripts to scan if it worked through without accessing them ?

@hongooi73
Copy link
Contributor

I can scan the files provided.

I suspect there's an encoding issue here, which may be masked by how you've copied the contents to Github. Can you attach the files rather than putting them in a comment?

@ThomasHoussoy
Copy link
Author

I wanted to do just that but file attachment is not supported for .R files in the comment box.
I've got a copy of the first file, with the rest of the code modified to comply as well as I could understand with the agreement I'm bind to.
I only managed to do that to one of the two files for now on, I've got to do the same for the second one.

@hongooi73
Copy link
Contributor

You can zip up the files and attach them that way.

@ThomasHoussoy
Copy link
Author

I should have thought about it.
Here there are both.
R files.zip

@hongooi73
Copy link
Contributor

Ok, I can reproduce the error now.

> z <- parse("~/misc/checkpoint/repro/02-Analyse_de_structure_brute.R")
Error in parse("~/misc/checkpoint/repro/02-Analyse_de_structure_brute.R") : 
  ~/misc/checkpoint/repro/02-Analyse_de_structure_brute.R:271:37: unexpected input      
270: 
271: R_022_etab_absenceCSE_X_combi$Synthè

And line 271 of the file is

R_022_etab_absenceCSE_X_combi$Synthèse <- c("[Non-concerné] Présence d'une CSE",

When I remove the è from Synthèse, it works. It also works if I put the variable name in backquotes ``.

I'm not actually sure if R supports non-ASCII characters in symbols, or if it does, what assumptions it makes. I'd suggest using only ASCII varnames, because I'm sure there are other areas where this is going to bite you.

@hongooi73
Copy link
Contributor

hongooi73 commented Aug 10, 2021

Opening via a connection (so that I can specify the encoding) also works, but as said, this may be fragile:

f <- file("~/misc/checkpoint/repro/02-Analyse_de_structure_brute.R", encoding="UTF-8", open="r")
z <- parse(f)

@hongooi73
Copy link
Contributor

hongooi73 commented Aug 10, 2021

You can also specify the default encoding for files via options(encoding="UTF-8"), so this also works:

options(encoding="UTF-8")
z <- parse("~/misc/checkpoint/repro/02-Analyse_de_structure_brute.R")

I think this is the best solution, because there will be other places where encoding becomes an issue.

PS. your other file doesn't parse correctly because #CENSORED# is not a valid variable name. If I replace it with CENSORED then it works with the encoding set as above.

@ThomasHoussoy
Copy link
Author

I must say I'm embarassed that it came down to such a silly mistake on my part : I thought about avoiding special characters in naming objects but it didn't cross my mind it must also apply to column names of dataframe if I call one of them.

I see no problem in just renaming this element with ASCII characters. Since these characters does not cause issues when in a character string, I can always rename the column once I've finished when I need to print/save/display them.

I just did that and it works as expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants