Bayes Factors now printed out under both fixed and random for variables specified only as fixed #351

alecristia · 2020-12-08T11:10:31Z

Question and context
Below I provide full code and data to show that parameters are printed out under "random" even though they are only fixed. I fit a model back in July and again now; the problem I describe emerged in between these two dates. I had also kept a .rds of the July model fit, and applying BF analyses to that rds also results in a few parameters being printed out under "random effects" -- but not all of them. Similar to the full example below, the actual BFs are identical across fixed & random printouts in the BF analysis of the July model fit.

Example
Full code & data: BFfixed-random.zip

This is the model fitting code, notice that the only random effects are var6 and var5:

mod1 = brm(var_9 | weights(var_17) ~ 
             var_20 + var_19   + 
             var_13* var_24* var_2 + 
             var_12* var_24* var_2 + 
             (1 |var_6) + (1 | var_6:var_5), data = mydat,
           prior = out_priors_B,
           iter=niter, warmup=nwarmup, chains=nchains,cores=2,
           seed=12,
           control = list(adapt_delta = 0.9)
)

And the BF estimation code

bayesfactor_parameters(mod1, null = 0)

produces this output, where all variables appear under both fixed and random, with the exact same BF values:

Bayes Factor (Savage-Dickey density ratio)

*Fixed Effects (Conditional Model)*

Parameter      |       BF
-------------------------
Intercept      |     0.03
var_20M        |     0.01
var_194        |     0.02
var_195        |     0.03
var_13N        |     0.24
var_24         | 4.96e+06
var_2          | 3.43e+19
var_12N        |     0.02
var_13N.var_24 |     0.01
var_13N.var_2  | 4.42e+03
var_24.var_2   | 4.63e+03
var_24.var_12N |     0.01
var_2.var_12N  |     0.02
va13Nvar_2     |     0.04
va24var_12N    |     0.02

*Random Effects (Conditional Model)*

Parameter      |       BF
-------------------------
var_20M        |     0.01
var_194        |     0.02
var_195        |     0.03
var_13N        |     0.24
var_24         | 4.96e+06
var_2          | 3.43e+19
var_12N        |     0.02
var_13N.var_24 |     0.01
var_13N.var_2  | 4.42e+03
var_24.var_2   | 4.63e+03
var_24.var_12N |     0.01
var_2.var_12N  |     0.02
va13Nvar_2     |     0.04
va24var_12N    |     0.02

If you can tell me whether I'm doing something wrong, or whether this is just a bug in the print out that I can safely ignore, I would be grateful.

The text was updated successfully, but these errors were encountered:

mattansb · 2020-12-08T14:39:29Z

Hmm... I am unable to recreate the printing issue you encountered.

mod1 = brm(var_9 | weights(var_17) ~ 
             var_20 + var_19   + 
             var_13* var_24* var_2 + 
             var_12* var_24* var_2 + 
             (1 |var_6) + (1 | var_6:var_5), data = mydat,
           prior = out_priors_B,
           iter=niter, warmup=nwarmup, chains=nchains,cores=2,
           seed=12,
           control = list(adapt_delta = 0.9)
)

bayesfactor_parameters(mod1, null = 0)
#> # Bayes Factor (Savage-Dickey density ratio)
#> 
#> Parameter      |        BF
#> --------------------------
#> Intercept      |     0.030
#> var_20M        |     0.012
#> var_194        |     0.018
#> var_195        |     0.024
#> var_13N        |     0.201
#> var_24         | 3.705e+07
#> var_2          | 5.463e+13
#> var_12N        |     0.020
#> var_13N.var_24 |     0.013
#> var_13N.var_2  |   588.455
#> var_24.var_2   |  3295.248
#> var_24.var_12N |     0.014
#> var_2.var_12N  |     0.024
#> va13Nvar_2     |     0.043
#> va24var_12N    |     0.022
#> 
#> * Evidence Against The Null: [0]

@alecristia can you copy you sessionInfo()? What versions of bayestestR and insight are you running?
@DominiqueMakowski @strengejacke Is one of you getting the same issue?

strengejacke · 2020-12-08T15:24:25Z

No, can't reproduce, either. I got the same output as @mattansb

alecristia · 2020-12-09T11:00:36Z

sessionInfo()

R version 3.6.3 (2020-02-29)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Catalina 10.15.7

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] brms_2.12.0 Rcpp_1.0.4.6 bayestestR_0.5.3 readr_1.3.1

loaded via a namespace (and not attached):
[1] Brobdingnag_1.2-6 gtools_3.8.1 StanHeaders_2.21.0-1 threejs_0.3.3 shiny_1.4.0.2 assertthat_0.2.1
[7] stats4_3.6.3 backports_1.1.8 pillar_1.4.3 lattice_0.20-40 glue_1.4.1 digest_0.6.25
[13] promises_1.1.0 colorspace_1.4-1 htmltools_0.4.0 httpuv_1.5.2 Matrix_1.2-18 plyr_1.8.6
[19] dygraphs_1.1.1.6 pkgconfig_2.0.3 rstan_2.19.3 logspline_2.1.16 purrr_0.3.3 xtable_1.8-4
[25] mvtnorm_1.1-0 scales_1.1.0 processx_3.4.2 later_1.0.0 tibble_3.0.0 bayesplot_1.7.1
[31] ggplot2_3.3.0 ellipsis_0.3.0 DT_0.13 shinyjs_1.1 cli_2.0.2 magrittr_1.5
[37] crayon_1.3.4 mime_0.9 ps_1.3.2 fansi_0.4.1 nlme_3.1-145 xts_0.12-0
[43] pkgbuild_1.0.6 colourpicker_1.0 rsconnect_0.8.16 tools_3.6.3 loo_2.2.0 prettyunits_1.1.1
[49] hms_0.5.3 lifecycle_0.2.0 matrixStats_0.56.0 stringr_1.4.0 munsell_0.5.0 callr_3.4.3
[55] compiler_3.6.3 rlang_0.4.9 grid_3.6.3 ggridges_0.5.2 rstudioapi_0.11 htmlwidgets_1.5.1
[61] crosstalk_1.1.0.1 igraph_1.2.5 miniUI_0.1.1.1 base64enc_0.1-3 codetools_0.2-16 gtable_0.3.0
[67] inline_0.3.15 abind_1.4-5 markdown_1.1 reshape2_1.4.4 R6_2.4.1 gridExtra_2.3
[73] rstantools_2.0.0 zoo_1.8-7 bridgesampling_1.0-0 dplyr_0.8.5 fastmap_1.0.1 shinystan_2.5.0
[79] shinythemes_1.1.2 insight_0.8.2 stringi_1.4.6 parallel_3.6.3 vctrs_0.3.5 tidyselect_1.0.0
[85] coda_0.19-3

I have this issue in two computers - pasting the other one in another comment (soon)

strengejacke · 2020-12-09T11:13:46Z

some of the easystats-core packages are out of date. I suggest updating your packages, and then try again.

alecristia · 2020-12-09T14:12:08Z

issue persisted after all packages except for data.table were updated.
data.table couldn't be installed, with error "clang: error: unsupported option '-fopenmp'"
followed instructions: Rdatatable/data.table#2875
which meant also doing steps 1-3 of https://github.com/Rdatatable/data.table/wiki/Installation#openmp-enabled-compiler-for-mac
But even so, installation for data.table failed with the same error.
In any case, this doesn't seem to be a bug that affects results, just the print-out, so I guess I can trust the results.

strengejacke · 2020-12-14T07:13:09Z

Strange, indeed. After updating packages, can you please provide a reprex, probably also using the reprex package? Including session info?

alecristia · 2020-12-15T09:13:25Z

I'm not sure I can attest to this being reproducible for someone else. I can reproduce it in my system, using the same data and code I linked in my first comment: BFfixed-random.zip

The output just now:

bayesfactor_parameters(mod1, null = 0)

Computation of Bayes factors: sampling priors, please wait...
Loading required namespace: logspline

Bayes Factor (Savage-Dickey density ratio)

Fixed Effects (Conditional Model)

Parameter | BF

Intercept | 0.03
var_20M | 0.01
var_194 | 0.02
var_195 | 0.03
var_13N | 0.27
var_24 | 5.55e+05
var_2 | 9.16e+13
var_12N | 0.02
var_13N.var_24 | 0.01
var_13N.var_2 | 1.92e+03
var_24.var_2 | 2.22e+04
var_24.var_12N | 0.01
var_2.var_12N | 0.03
va13Nvar_2 | 0.04
va24var_12N | 0.02

Random Effects (Conditional Model)

Parameter | BF

var_20M | 0.01
var_194 | 0.02
var_195 | 0.03
var_13N | 0.27
var_24 | 5.55e+05
var_2 | 9.16e+13
var_12N | 0.02
var_13N.var_24 | 0.01
var_13N.var_2 | 1.92e+03
var_24.var_2 | 2.22e+04
var_24.var_12N | 0.01
var_2.var_12N | 0.03
va13Nvar_2 | 0.04
va24var_12N | 0.02

Evidence Against The Null: [0]

sessionInfo()

R version 3.6.3 (2020-02-29)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Catalina 10.15.7

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] brms_2.12.0 Rcpp_1.0.5 bayestestR_0.5.3 readr_1.3.1 reprex_0.3.0

loaded via a namespace (and not attached):
[1] Brobdingnag_1.2-6 gtools_3.8.1 StanHeaders_2.21.0-1 threejs_0.3.3 shiny_1.4.0.2
[6] assertthat_0.2.1 stats4_3.6.3 backports_1.2.0 pillar_1.4.3 lattice_0.20-40
[11] glue_1.4.2 digest_0.6.25 promises_1.1.0 colorspace_1.4-1 htmltools_0.4.0
[16] httpuv_1.5.2 Matrix_1.2-18 plyr_1.8.6 dygraphs_1.1.1.6 pkgconfig_2.0.3
[21] rstan_2.19.3 logspline_2.1.16 purrr_0.3.3 xtable_1.8-4 mvtnorm_1.1-0
[26] scales_1.1.0 processx_3.4.5 later_1.0.0 tibble_3.0.0 bayesplot_1.7.1
[31] generics_0.1.0 ggplot2_3.3.0 ellipsis_0.3.1 DT_0.16 shinyjs_1.1
[36] cli_2.0.2 magrittr_1.5 crayon_1.3.4 mime_0.9 ps_1.3.2
[41] fs_1.5.0 fansi_0.4.1 nlme_3.1-145 xts_0.12-0 pkgbuild_1.1.0
[46] colourpicker_1.0 prettyunits_1.1.1 rsconnect_0.8.16 tools_3.6.3 loo_2.2.0
[51] hms_0.5.3 lifecycle_0.2.0 matrixStats_0.56.0 stringr_1.4.0 munsell_0.5.0
[56] callr_3.5.1 compiler_3.6.3 rlang_0.4.9 grid_3.6.3 ggridges_0.5.2
[61] rstudioapi_0.11 htmlwidgets_1.5.1 crosstalk_1.1.0.1 igraph_1.2.5 miniUI_0.1.1.1
[66] base64enc_0.1-3 codetools_0.2-16 gtable_0.3.0 inline_0.3.15 abind_1.4-5
[71] markdown_1.1 reshape2_1.4.4 R6_2.4.1 gridExtra_2.3 rstantools_2.0.0
[76] zoo_1.8-7 bridgesampling_1.0-0 dplyr_1.0.2 fastmap_1.0.1 shinystan_2.5.0
[81] shinythemes_1.1.2 insight_0.8.2 stringi_1.4.6 parallel_3.6.3 vctrs_0.3.5
[86] tidyselect_1.1.0 coda_0.19-3

strengejacke · 2020-12-15T10:23:22Z

Looking at your sessionInfo, I still see:

insight_0.8.2
bayestestR_0.5.3

Both packages are out of date, the current versions are insight 0.11.1 and bayestestR 0.8.0. Thus, I'd say that your problem is still related to the older package versions.

IndrajeetPatil · 2021-01-02T07:54:17Z

Closing for now. Please reopen if the issue persists even after updating packages.

alecristia · 2021-02-25T09:49:47Z

The error has disappeared from my mac mini, documenting current state of affairs :

sessionInfo()
R version 3.6.3 (2020-02-29)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Catalina 10.15.7
Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] brms_2.12.0 Rcpp_1.0.6 bayestestR_0.8.2 readr_1.3.1
loaded via a namespace (and not attached):
[1] Brobdingnag_1.2-6 gtools_3.8.1 StanHeaders_2.21.0-1 threejs_0.3.3 shiny_1.4.0.2 assertthat_0.2.1
[7] stats4_3.6.3 sessioninfo_1.1.1 backports_1.2.1 pillar_1.4.3 lattice_0.20-40 glue_1.4.2
[13] digest_0.6.25 promises_1.1.0 colorspace_1.4-1 htmltools_0.4.0 httpuv_1.5.2 Matrix_1.2-18
[19] plyr_1.8.6 dygraphs_1.1.1.6 pkgconfig_2.0.3 rstan_2.19.3 logspline_2.1.16 purrr_0.3.3
[25] xtable_1.8-4 mvtnorm_1.1-0 scales_1.1.0 processx_3.4.5 later_1.0.0 tibble_3.0.0
[31] bayesplot_1.7.1 generics_0.1.0 ggplot2_3.3.0 ellipsis_0.3.1 DT_0.17 withr_2.4.1
[37] shinyjs_1.1 cli_2.0.2 magrittr_1.5 crayon_1.3.4 mime_0.9 ps_1.3.2
[43] fansi_0.4.1 nlme_3.1-145 xts_0.12-0 pkgbuild_1.2.0 colourpicker_1.0 prettyunits_1.1.1
[49] rsconnect_0.8.16 tools_3.6.3 loo_2.2.0 hms_0.5.3 lifecycle_0.2.0 matrixStats_0.56.0
[55] stringr_1.4.0 munsell_0.5.0 callr_3.5.1 compiler_3.6.3 rlang_0.4.10 grid_3.6.3
[61] ggridges_0.5.2 rstudioapi_0.11 htmlwidgets_1.5.1 crosstalk_1.1.0.1 igraph_1.2.5 miniUI_0.1.1.1
[67] base64enc_0.1-3 codetools_0.2-16 gtable_0.3.0 inline_0.3.15 abind_1.4-5 DBI_1.1.1
[73] markdown_1.1 reshape2_1.4.4 R6_2.4.1 gridExtra_2.3 rstantools_2.0.0 zoo_1.8-7
[79] bridgesampling_1.0-0 dplyr_1.0.3 fastmap_1.0.1 shinystan_2.5.0 shinythemes_1.1.2 insight_0.12.0
[85] stringi_1.4.6 parallel_3.6.3 vctrs_0.3.6 tidyselect_1.1.0 coda_0.19-3

strengejacke added the bug 🐛 Something isn't working label Dec 8, 2020

mattansb added reprex 💾 We need a reproducible example for further investigation waiting for response 💌 Need more information from people who submitted the issue labels Dec 8, 2020

IndrajeetPatil closed this as completed Jan 2, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bayes Factors now printed out under both fixed and random for variables specified only as fixed #351

Bayes Factors now printed out under both fixed and random for variables specified only as fixed #351

alecristia commented Dec 8, 2020 •

edited by mattansb

mattansb commented Dec 8, 2020 •

edited

strengejacke commented Dec 8, 2020

alecristia commented Dec 9, 2020

strengejacke commented Dec 9, 2020

alecristia commented Dec 9, 2020

strengejacke commented Dec 14, 2020

alecristia commented Dec 15, 2020

Bayes Factor (Savage-Dickey density ratio)

Fixed Effects (Conditional Model)

Parameter | BF

Random Effects (Conditional Model)

Parameter | BF

strengejacke commented Dec 15, 2020

IndrajeetPatil commented Jan 2, 2021

alecristia commented Feb 25, 2021

Bayes Factors now printed out under both fixed and random for variables specified only as fixed #351

Bayes Factors now printed out under both fixed and random for variables specified only as fixed #351

Comments

alecristia commented Dec 8, 2020 • edited by mattansb

mattansb commented Dec 8, 2020 • edited

strengejacke commented Dec 8, 2020

alecristia commented Dec 9, 2020

strengejacke commented Dec 9, 2020

alecristia commented Dec 9, 2020

strengejacke commented Dec 14, 2020

alecristia commented Dec 15, 2020

Bayes Factor (Savage-Dickey density ratio)

Fixed Effects (Conditional Model)

Parameter | BF

Random Effects (Conditional Model)

Parameter | BF

strengejacke commented Dec 15, 2020

IndrajeetPatil commented Jan 2, 2021

alecristia commented Feb 25, 2021

alecristia commented Dec 8, 2020 •

edited by mattansb

mattansb commented Dec 8, 2020 •

edited