Informative priors

[DominiqueMakowski]

Now that I think of it @strengejacke, your tutorial paper about informative priors could be boosted by the implementation of an easy-to-use function (let's call it informative_priors) that would facilitate the set-up of informative priors.

Formatted priors object

I believe that most researchers would be interested in using more informative priors, if only that was simpler. One of the function of such function could be to provide a unified interface to prior specification, that would return a prior object compatible with what they want to use, i.e., brms, rstanarm etc.

Standardized informative priors based on interpreted categorical sizes

Of course, our initial reaction is that informative priors have to be decided on a case by case basis, in regards to prior literature and expectations. While this is, of course, true, one could potentially imagine a second approach to the problem.

For instance, it could be interesting to allow for a more naturalistic prior specification. Similar to what sometimes we think, and what we'd like to say, e.g. "I expect this effect to be small". Now how to transform "small" into something mathematically acceptable? We could refer to an interpretation grid (we could transfer some utilities for that from report). Let's say that a standardized coef for "small" is 0.2. We could unstandardize this value based on the mean/SD of the data, that would give us the location of a prior corresponding to a "small" effect. The scale could be automatically adjusted accordingly.

E.g. informative_priors_rstanarm(y ~ A + B, size =c("small", "medium"), data = df) would return a list with priors set that one could then just pass to the rstanarm model.

Believers vs. Agnostic vs. Sceptic priors

Finally, an argument could control the "type" of prior. Three of them make particular sense to me. First of all, the agnostic prior is similar to the one set by default, for instance in rstanarm (although I find its automatic scaling to be way too generous), i.e., one located at 0. This one is suited for true exploratory analysis. On the contrary, a believer's prior can be useful, for instance in small samples or noisy data, would be located at the expected effect value. If we take the example above, and I expect the effect to be "small", I would place a believer's prior at 0.2 (or its unstandardized equivalent). However, in large datasets or for very robust data, one might want to set sceptic priors, that would be located at the same location than a believer's, but on the opposite direction. A sceptic prior for the example above would be set at -0.2. Such feature could be easily added to our function.

informative_priors_brms(y ~ A, size ="medium", data = df, type = "sceptic")

[strengejacke]

ok, got it. sounds like a good idea! might be some hassle for more complex brms-models, though, probably it's safer to pass a model instead of a formula, or we can check if insight can do the work for us.

[DominiqueMakowski]

This needs:

• implementation of unstandardize() in effectsize, that would take standardized parameters and convert them back to a raw scale (based on the SD/MAD and mean/median of the response variable).
• informative_priors() that would use unstandardize to figure out the priors based on a formula and return it in a stan/compatible format.

We could write an example illustrating a full real-data "study" (the master tutorial?) (showing all facets of bayestestR):

1. We have a question (the relationship between x and y)
2. We make a general hypothesis/prediction (x is positively related to y)
3. We operationalize this hypothesis:
   • Come up with the model information (type of model, formula etc.)
   • Come up with operationalized predictions (I expect a small and positive coefficient)
4. We get the data
5. Prior generation (informative_priors(y ~ x, size="small", direction="positive", data=df)). This returns 3 priors (agnostic, believer and skeptic).
6. We fit the believer's model.
7. Is it a good model? Model evaluation (performance, posterior checks etc.)
8. Yes, -> Interpretation of results (the parameters).
9. We fit the agnostic and skeptics. Does adopting a different perspective significantly change our initial interpretation? "No, the results are consistent"
10. Conclude

[mattansb]

For factors we would need to incorporate contr.bayes, and make sure the scale of prior of continuous is half that of the factor (just like how you can get Beta coefs with 2sd).

[DominiqueMakowski]

nice, we could add that indeed, so this "walk-through" would be fairly complete.

[strengejacke]

reminder to myself: continue working on this paper...

[DominiqueMakowski]

aha! I found your demon, now I will bump this issue just as you bump the news-file one 😈

[DominiqueMakowski]

Looking back at this, this really would be a nice thing to have 🤩

[strengejacke]

Yes yes, it's on its way, and you'll be updated. 🙂

[DominiqueMakowski]

haha nah but I was referring to the bayestestR function to make stan-compatible priors objects

[strengejacke]

Yeah, but that could be part of the work on the paper, so we might introduce that function there...