In an experiment, the averages of the control group and the experimental group are 0.7169 and 0.7552. Is the experimental group better than the control group? Or is the difference just due to the noise?
In this talk, I will introduce how to calculate the p-value in Python by examples, the common misunderstandings of p-values, calculating β and sample size, the relationships among confidence level, α, β, power, and the common tests.
Also, the second part includes the notebooks to explain the theories lively, which covers p-value, α, raw effect size, β, sample size, actual negative rate, inverse α (like false discovery rate), and inverse β (like false omission rate).
The notebooks are available on https://github.com/moskytw/hypothesis-testing-with-python .