Use uncertainty in bead population statistic as weights for least squares regression

[castillohair]

Oleg suggested that the standard curve fits could be improved if some measure of estimation uncertainty of the population statistic be used as part of a weighted least squares regression method during the standard curve fitting step. For example, if the statistic of population x is the mean, the uncertainty is given by std(x)/sqrt(n).

Two things that would have to be addressed as part of this issue:

• We need a way to measure how "robust" the fit is with traditional vs weighted least squares.
• If weighted least squares turns out to produce more robust fits, we need a way to incorporate this into the workflow of mef.get_transform_fxn. Right now, statistic calculation and standard curve fitting are separate steps that share only one single number resulting from the statistic calculation step. @JS3xton

[JS3xton]

In general, I agree with this idea, although I feel like it may not change much.

• Agree we need some way to compare different methods. Has this actually been demonstrated as a problem (robustness of standard curve fit)? That might help guide how we should compare methods.
  • If so, I would again highlight some changes I suggested here: MEF module cleanup #165 (initializing the fit, more thorough investigation into which fitting algorithm we use).
  • I did an initial investigation into how the parameters change as a function of both gain and fitting algorithm a while back [https://wiki.rice.edu/confluence/display/~jts6/JS20150713 and JS3xton/fc#5], and you've put similar data in the paper. I also just started tracking bead parameters over time: https://wiki.rice.edu/confluence/display/~jts6/Flow+Cytometer (using whatever the current FlowCal default is at the time). I could envision using this framework as a backdrop for an investigation into this issue, as this issue is basically just an alternative fitting algorithm.
• Would also be nice to resolve the funny behavior of the fit bead autofluorescence term at gains <500.
• May need a more concrete model of bead peaks, which may benefit from switching the population statistic (e.g. mean, as you mention). I don't know enough / haven't seen a compelling argument / concrete model for the most appropriate way to summarize the bead subpopulation with an uncertainty measure and incorporporate that into model fitting.

Practically speaking, I think this would require redoing the interface between fitting functions and mef.get_transform_fxn.

• Again, see get_transform_fxn signature #146.
• Seems like there should be some kind of duck-typing way to have the fitting function ask for whatever statistic/uncertainty measure it needs and throw an error if it can't find it (again, NamedTuples are coming to mind to package a value and its uncertainty). This way, mef.get_transform_fxn can continue to ignorantly just pass the fitting data to the fitting function.