-
Notifications
You must be signed in to change notification settings - Fork 96
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Speed up sleuth_fit, custom shrinkage function, reconfigure transform API, misc bug fixes #213
Open
warrenmcg
wants to merge
25
commits into
pachterlab:devel
Choose a base branch
from
warrenmcg:speedy_fit
base: devel
Could not load branches
Branch not found: {{ refName }}
Could not load tags
Nothing to show
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…t check for obj so that this function can be used with p-value aggregation mode
+ this is done by using the whole matrix rather than modeling by row + an additional small change is to make target_id the first column, which means that the final summary table in the sleuth_fit object will have target_id first, instead of in the middle. This will ease user readability if decide to view the summary table directly for columns not selected in sleuth_results + add documentation for me_model
…uth_fit API: + the models slot for the sleuth_model object is now the one model with the full matrix + these functions have retained the old code to remain backwards compatible with older versions of sleuth
…bjects: + speed up the calculation using matrix multiplication rather than lapply + calculate beta covariances for later use + bonus that this reduces memory footprint of the sleuth_model object further + update 'extract_model' and 'sleuth_wt' to use the new matrix format
+ this will allow the option to supply custom shrinkage procedures + the default is now 'basic_shrink_fun', which contains the old sleuth shrinkage procedure done by 'sliding_window_grouping' and then 'shrink_df' + any custom function must take the list produced by me_model, and then output the mes_df within that list modified with at least one new column: "smooth_sigma_sq", the smooth variances + add updated documentation to 'sleuth_fit' + add documentation for 'basic_shrink_fun' + shrink_fun is now an additional slot on the sleuth_model object, to provide provenance for how the variance estimation was done
+ remove specified extra options needed in sleuth_fit call + use 'do.call' instead of directly calling shrink_fun + this solution was discussed [here](https://stackoverflow.com/a/7028630)
…odels in sleuth_lrt
+ in testing, this was capturing the full sleuth object, essentially doubling the size of the final sleuth object
+ now only happens if there is more than one row, indicating that there is a beta beyond the intercept-only + this prevents an error occurring if the intercept-only model is used
… function + process bootstrap now normalizes TPMs by TPM size factors instead of indirectly by count size factor + add 'sf' argument to the default log_transform function, taking a single size factor when normalizing bootstraps, or a vector of size factors equal in length to the number of samples when normalizing the observed counts + add 'tpm_transform' as an equivalent default function for TPMs. + this will facilitate an alternative approach for sleuth-ALR that decouples normalization and transformation steps
…n the target IDs in the fit + this now guarantees that the fit object, the summary table, and the beta_covars table are all in the same order
+ switch to rlang::eval_tidy + add rlang to NAMESPACE and import list + fixes issue pachterlab#194
This was referenced Mar 16, 2019
Merged
pimentel
reviewed
Dec 17, 2019
@@ -84,5 +87,7 @@ importFrom(data.table,fread) | |||
importFrom(dplyr,"%>%") | |||
importFrom(lazyeval,interp) | |||
importFrom(lazyeval,lazy) | |||
importFrom(limma,squeezeVar) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this used anywhere? I don't see it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just kidding, see it now. hm. I'd like to not depend on yet another package, ideally
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi @pimentel,
There are a few different things that have been changed here:
sleuth_fit
by doing the measurement error and beta covariance calculations with the full matrix rather than by rowsleuth_fit
using theshrink_fun
argument. The default sleuth shrinkage function is a new function calledbasic_shrink_fun
. I also implemented limma's shrinkage function in a new function calledsqueezeVar
. This should address concerns raised by NA in likelihood ratio test with small number of transcripts #173.log_transform
and added one for TPMs (tpm_transform
), so that they can take a single size factor when normalizing bootstraps, or can use a vector of size factors equal in length to the number of samples when transforming the full matrix. This allows for sleuth-ALR to separate the normalization and transformation steps.I have also fixed various small bug fixes. I'll update this description when I've merged in more:
gene_from_gene
. This incorporates the same patch included with pull request Emergency patch to fix #190, part 2 #192.eval
statements in the shiny scatter plot panel withrlang::eval_tidy
to fix scatter plots -> Error in as.data.frame.default #194sleuth_results
to better handle whengene_mode
andpval_aggregate
are bothTRUE
, to fix strange error from sleuth_results in gene mode #202To-dos:
sleuth_prep
documentation mentioned in typo in ?sleuth_prep #212