effectsize is a comprehensive Python package for computing effect sizes (ESs), also known as standardized differences. The package implements the methodology outlined by Yang and Dalton (2012) and it provides complex functionality, such as the ability to deal with skewed variables, multinomial categories, and weighted statistics.
Before you begin, ensure you have installed Python 3.7 or higher
effectsize has four dependencies: numpy, pandas, scipy, statsmodels
Binary installers for the latest released version are available at the Python Package Index (PyPI):
pip install effectsizeAs well as through the conda-forge channel on Conda:
conda install -c conda-forge effectsizeSource code for effectsize is hosted on its GitHub repository
To use effectsize, it must first be imported as a Python module:
import effectsizeFrom here, all of effectsize's functionality is accessible through a single function named compute, which is called via. effectsize.compute(). This function takes up to 8 arguments, which are outlined below along with their default values:
effectsize.compute(data,
group,
continuous = [],
categorical = [],
skewed = [],
weights = None,
decimals = 2,
intervals = None)Given a Pandas DataFrame and a variable specifying 2 groups, effectsize.compute() will return another Pandas DataFrame containing ESs for all variables that are requested by the user. Detailed description for each argument of effectsize.compute() is presented below:
- data (
Pandas DataFrame): Each row should be an observation and each column should be a variable. In other words, all variables for which the user would like to compute an ES must be a column within the DataFrame. - group (
str): This should be the variable defining the two groups, specified as a string. Ideally, these two groups should be coded as 0 (control) and 1 (treatment), buteffectsizewill work regardless of the coding system used, provided it specifies two groups. Note that if the coding is switched then, the sign of the ES for continuous variables will be reversed, but the magnitude will stay the same. This is typically not an issue as the direction can be inferred from summary statistics. - continuous (
list): This should contain the names of all of the continuous variables for which the user would like an ES computed. This must be specified as a list containing the variable names as strings e.g.,continuous = ["age", "salary", "bmi"]would be syntactically correct butcontinuous = [age, salary, bmi]would not. If there are no continuous variables for which an ES needs to be computed, then continuous should be passed an empty list, which is also the default object passed to the argument. - categorical (
list): This should contain the names of all of the categorical variables for which the user would like an ES computed. In the exact same way as the continuous argument, this must be passed a list containing the variable names as strings. If there are no categorical variables for which an ES needs to be computed, then categorical should be passed an empty list, which is also the default object passed to the argument. - skewed (
list): This should contain the names of all of the continuous variables which have a skewed distribution, for which the user would like an ES computed. Note that the skewed variables must be specified in both the continuous argument and the skewed argument. For example, if age follows a skewed distribution, then this should be specified aseffectsize.compute(..., continuous = ["age", "salary", "bmi"], skewed = ["age"]). If this were to be specified aseffectsize.compute(..., skewed = ["age"]), then the age variable will simply be ignored and no ES returned. In the exact same way as the continuous argument, this must be passed a list containing the variable names as strings. If there are no skewed variables for which an ES needs to be computed, then skewed should be passed an empty list, which is also the default object passed to the argument. - weights (
Noneorstr): This should be the variable defining weights, specified as a string (examples of weights include sampling weights or propensity scores). Note that the sum of all of the weights must be >= 1, else the computed ES will not be correct. If there are no weights, then weights should be passed the valueNone, which is also the default value passed to the argument. - decimals (
int): This should be an integer which specifies the number of decimals to which the ESs should be computed, the default value is 2. - intervals (
Noneorfloat): This should be a value between 0 and 1 specifying the level of confidence interval (CI) which the user would like e.g., to compute a 95% CI, this should be specified asintervals = 0.95. Note if CIs do not need to be computed then intervals should be passed the valueNone, which is also the default value passed to the argument.
effectsize excludes all observations for which data is missing on group (i.e., it is not clear to which of the 2 groups the observation belongs), or if data is missing on the variable for which the user would like ESs computed (i.e., those in continuous and/or categorical). Therefore, it is advised that users deal with missing data in the most appropriate manner for their analyses prior to computing ESs.
The order in which the ESs appear in the output of effectsize.compute() is the same order in which the variables appear in the DataFrame passed to data. This is to ensure consistency between the output of effectsize and the user's original DataFrame. It does not matter in which order users specify the variable names inside of continuous and categorical, the results will always be output so that they correspond to the same order as the original DataFrame.
To demonstrate examples of how to use effectsize, we simulated 2 groups, each containing 100 observations. In each group, we simulated 4 variables of interest: var1 is a Normally distributed continuous variable, var2 is an exponentially ditribusted (i.e., skewed) continuous variable, var3 is a 2-level categorical variable, and var4 is a 3-level categorical variable. We used different parameter values to ensure that the distributions of the variables were different between the 2 groups. Summary statistics for the simulated dataset are presented in Table 1.
Table 1. Summary of simulated data. Continuous variables are presented as mean (standard deviation) and categorical variables are presented as n (%).
| Variable | Group 1 (n = 100) | Group 2 (n = 100) | |
|---|---|---|---|
| var1 | 1.02 (0.7) | 1.15 (0.9) | |
| var2 | 0.37 (0.4) | 0.30 (0.3) | |
| var3 | Level 0 | 81 (81%) | 65 (65%) |
| Level 1 | 19 (19%) | 35 (35%) | |
| var4 | Level 0 | 24 (24%) | 32 (32%) |
| Level 1 | 46 (46%) | 38 (38%) | |
| Level 2 | 30 (30%) | 30 (30%) |
We will assume that the Pandas DataFrame in which these data are stored is named df, and the name of the variable specifying the group to which each observation belongs is named group.
To compute ESs for the continuous variables only:
effectsize.compute(data = df,
group = "group",
continuous = ["var1", "var2"])| ES | |
|---|---|
| var1 | 0.16 |
| var2 | -0.20 |
However we know var2 is skewed, so we may want to account for this:
effectsize.compute(data = df,
group = "group",
continuous = ["var1", "var2"],
skewed = ["var2"])| ES | |
|---|---|
| var1 | 0.16 |
| var2 | -0.11 |
We see a small change in the ES for var2 after accounting for the fact that it is skewed. We can also compute ESs for the categorical variables only:
effectsize.compute(data = df,
group = "group",
categorical = ["var3", "var4"],)| ES | |
|---|---|
| var3 | 0.37 |
| var4 | 0.20 |
Finally, if we wish to compute ESs for all variables at once:
effectsize.compute(data = df,
group = "group",
continuous = ["var1", "var2"],
categorical = ["var3", "var4"],
skewed = ["var2"])| ES | |
|---|---|
| var1 | 0.16 |
| var2 | -0.11 |
| var3 | 0.37 |
| var4 | 0.20 |
Obtraining extra precision is also straightforward:
effectsize.compute(data = df,
group = "group",
continuous = ["var1", "var2"],
categorical = ["var3", "var4"],
skewed = ["var2"],
decimals = 4)| ES | |
|---|---|
| var1 | 0.1632 |
| var2 | -0.1150 |
| var3 | 0.3664 |
| var4 | 0.1961 |
Conifdence intervals are easily computed, in this case 95% CIs:
effectsize.compute(data = df,
group = "group",
continuous = ["var1", "var2"],
categorical = ["var3", "var4"],
skewed = ["var2"],
intervals = 0.95)| ES | 95.0% CI | |
|---|---|---|
| var1 | 0.16 | [-0.12, 0.44] |
| var2 | -0.11 | [-0.39, 0.17] |
| var3 | 0.37 | [0.09, 0.65] |
| var4 | 0.20 | [-0.08, 0.48] |
Similarly, for 99% CIs:
effectsize.compute(data = df,
group = "group",
continuous = ["var1", "var2"],
categorical = ["var3", "var4"],
skewed = ["var2"],
intervals = 0.99)| ES | 99.0% CI | |
|---|---|---|
| var1 | 0.16 | [-0.20, 0.52] |
| var2 | -0.11 | [-0.47, 0.25] |
| var3 | 0.37 | [0.00, 0.74] |
| var4 | 0.20 | [-0.17, 0.57] |
We then create simulated weights for the observations by taking 200 samples from a Normal distribution with mean = 100 and standard deviation = 15. The variable containing the weights is named wgt, and we can compute a weighted ES by specifying this in the function call:
effectsize.compute(data = df,
group = "group",
continuous = ["var1", "var2"],
categorical = ["var3", "var4"],
skewed = ["var2"],
weights = "wgt")| ES | |
|---|---|
| var1 | 0.20 |
| var2 | -0.14 |
| var3 | 0.40 |
| var4 | 0.19 |
To demonstrate use on a real-world example, we use data from the 2017-2018 National Health and Nutrition Examination Survey (NHANES), a cross-sectional study which samples the noninstitutionalized US population. Participants are first interviewed at home and then invited to a mobile examination center for further interviews, tests, and examinations. We will use data from NHANES to evaluate differences between smokers versus non-smokers, across 6 variables:
- Age (in years)
- Sex (male / female)
- Ethnicity (White / Black / Hispanic / Other)
- Education (below high school / high school graduate / college graduate)
- BMI (in kilograms per square metre)
- Blood cholesterol (in milligrams per deciliter)
Age, BMI, and blood cholesterol were measured as continuous variables, whilst sex, ethnicity, and education were measured as categorical variables. Summary statistics for the distribution of the variables amongst smokers and non-smokers are presented in Table 2.
Table 2. Summary of NHANES data. Continuous variables are presented as mean (standard deviation) and categorical variables are presented as n (%).
| Non-smokers (n = 3981) | Smokers (n = 870) | ||
|---|---|---|---|
| Age | 52.2 (17.9) | 47.5 (15.6) | |
| Sex | Male | 1836 (46.1%) | 495 (56.9%) |
| Female | 2145 (53.9%) | 375 (43.1%) | |
| Ethnicity | White | 1334 (33.5%) | 371 (42.6%) |
| Black | 841 (21.1%) | 254 (29.2%) | |
| Hispanic | 999 (25.1%) | 124 (14.3%) | |
| Other | 807 (20.3%) | 121 (13.9%) | |
| Education | Below high school | 751 (18.9%) | 205 (23.6%) |
| High school graduate | 2155 (54.1%) | 580 (66.7%) | |
| College graduate | 1075 (27.0%) | 85 (9.8%) | |
| BMI | 30.0 (7.3) | 29.2 (7.8) | |
| Cholesterol | 188.0 (40.8) | 189.6 (43.5) |
We will assume that the Pandas DataFrame in which these data are stored is named nhanes, and the name of the variable specifying whether indivduals are smokers or non-smokers is named smoking.
To compute ESs for all variables:
effectsize.compute(data = nhanes,
group = "smoking",
continuous = ["age", "BMI", "cholesterol"],
categorical = ["sex", "ethnicity", "education"])| ES | |
|---|---|
| age | -0.28 |
| sex | 0.22 |
| ethnicity | 0.37 |
| education | 0.46 |
| BMI | -0.12 |
| cholesterol | 0.04 |
On further investigation, we may find that BMI is not distributed symmetrically and want to account for this:
effectsize.compute(data = nhanes,
group = "smoking",
continuous = ["age", "BMI", "cholesterol"],
categorical = ["sex", "ethnicity", "education"],
skewed = ["BMI"])| ES | |
|---|---|
| age | -0.28 |
| sex | 0.22 |
| ethnicity | 0.37 |
| education | 0.46 |
| BMI | -0.15 |
| cholesterol | 0.04 |
In NHANES, probability sampling weights are used and stored in a variable named wtmec2yr. If we wish to account for these sampling weights:
effectsize.compute(data = nhanes,
group = "smoking",
continuous = ["age", "BMI", "cholesterol"],
categorical = ["sex", "ethnicity", "education"],
skewed = ["BMI"],
weights = "wtmec2yr")| ES | |
|---|---|
| age | -0.32 |
| sex | 0.11 |
| ethnicity | 0.17 |
| education | 0.56 |
| BMI | -0.18 |
| cholesterol | 0.02 |
The level of precision and CIs can be modified within the decimals and intervals arguments, as demonstrated in earlier simulation examples.
Finally, to demonstrate the effect of how the group coding, we switched how smokers and non-smokers were coded. In the examples above, smokers were coded as 1 and non-smokers as 0. We now re-run the final example from above, but having switched smokers to 0 and non-smokers to 1:
effectsize.compute(data = nhanes,
group = "smoking_switched",
continuous = ["age", "BMI", "cholesterol"],
categorical = ["sex", "ethnicity", "education"],
skewed = ["BMI"],
weights = "wtmec2yr")| ES | |
|---|---|
| age | 0.32 |
| sex | 0.11 |
| ethnicity | 0.17 |
| education | 0.56 |
| BMI | 0.18 |
| cholesterol | -0.02 |
We see that the magnitude of the ESs has remained unchanged, but the direction for the continuous variables has reversed. By referring to Table 2, we can see that the smokers tend to be younger, have a lower BMI, and have a higher blood cholesterol. Therefore, if smokers are our reference group, then we would expect the ESs for age and BMI to be negative, whilst the ES for cholesterol would be positive. However, if we were to take non-smokers as our reference group, then we would expect the ESs for age and BMI to be positive, whilst the ES for cholesterol would be negative, explaining the change in sign of the ESs in the above example. ESs for the categorical variables are always positive as computing them involves taking squares of matrices, which will yield positive values.
Users are actively encouraged to test and implement effectsize in their projects, as well as leave feedback and make contributions to the package. In particular, we welcome contributions relating to improving computational efficiency, adding features which are likely to be widely used, and developing the underlying mathematical theory. Users can fork the software and create pull requests on GitHub, or get in touch regarding any relevant developments in statistical theory.
If you wish to contact me you can reach me at nbashir562@gmail.com