Split out large modules

[jpreszler]

Some of the modules, such as pymc_experiments.py are getting a bit large. I think it would be good to split them into smaller files for each experiment (or groups of experiments).

I propose starting with breaking pymc_experiments.py into

|-- pymc_experiments/
|    |-- __init__.py
|    |-- experimental_design.py
|    |-- pre_post_fit.py (likely including ITS and SC, for now)
|    |-- difference_in_differences.py
|    |-- regression_discontinuity.py
|    |-- pre_post_negd.py
|    |-- instrumental_variable.py

with logic in the __init__.py to ensure that existing import patterns continue to work. This would mean the change has no impact on package usage, but makes the code easier to navigate and allows people to work on different areas without running into merge conflicts (for example someone can be adding multivariate synthetic control features and someone else fuzzy RD and they're changing different files rather than the same one).

I'm planning on starting this, but wanted to give people a heads up in case there were questions, concerns, opposition, or someone has tried this and knows of issues that may come up.

[jpreszler]

PR #254 takes care of this for pymc_experiments.

[drbenvincent]

I think this makes sense. Just to sketch out some vague plans along these lines...

At the moment we have duplicated / parallel class hierarchies for the Bayesian and scikit-learn implementations of the experiments. This is not ideal as it creates a lot of duplicated code and allows for unintentional divergence in the code.

There is already a discussion about what direction we might go in in terms of non-Bayesian backends (see #250), so depending on the outcome of that I could potentially see the following:

We could consolidate pymc_experiments.py and skl_experiments.py into a single backend agnostic experiments.py module. The core experiment logic is nearly independent of the backend at the moment. The main differences are in plotting related code and there are some minor differences relating to dealing with point estimates vs distributions. So there is an opportunity to raise the level of abstraction in experiments.py.

I plan on creating a Discussion topic on that soon, so let's not dive into that particular issue on this thread. But I wanted to mention it now so that it's on the radar. I hope to start a Discussion thread on that soon (but maybe early next month) to lay out a few possible implementation approaches. I don't think there is a massive rush on this, and it depends on what we end up doing with #250.

[jpreszler]

This makes sense and would remove lots of code redundancy. I think splitting up the larger existing modules could be a first step towards a more abstract experiments api since it would make it a bit easier to deal with the two backends of each experiment type. Especially since PR #254 is pretty simple - the only thing left for pymc experiments is to increase some test coverage, which isn't a bad thing. While this is happening, a plan for improving the experiment abstraction can be put together and implemented in smaller parts or work divided more easily.