Mention robust statistics and regularization in guiding principles

[dragice]

In Chapter 1, paragraph "robustness", would be great to have examples of actually robust statistics (currently we only have ANOVA and bootstrapping). I think Matt you also mentioned regularization in a google doc comment, so it could be another example but I'm not entirely sure where it should go (perhaps in "Faithfulness" or "Resilience"?).

[mjskay]

Good idea. I think these are two different things:

• When I hear "robust" I think of things that are robust to outliers or departures from assumptions. Wikipedia seems to agree, as does the principle as currently written in the doc. So things like bootstrapping (as in the doc), robust standard errors, using fat-tailed error distributions in regression, etc.

• Regularization is more about avoiding overfitting by reducing variance (reducing the "dance of the (p values|CIs|Bayes factors|etc)"). So I think it fits in "resilience". Things like regularized regression (e.g. lasso or ridge regression), hierarchical models / random effects, or Bayesian estimation with skeptical priors could be examples there.

[mjskay]

Per discussion on 2018-10-3: the "resilience" section (section 3) seems a good place to put regularization