Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[cudamapper] Add anchor chain output functions to chainer_utils.cuh. #590

Open
wants to merge 29 commits into
base: dev
Choose a base branch
from

Conversation

edawson
Copy link
Contributor

@edawson edawson commented Oct 19, 2020

Adds the necessary functions for outputting the chains of anchors that represent an overlap.

@edawson edawson requested review from tijyojwad and mimaric and removed request for tijyojwad October 19, 2020 21:41
@edawson
Copy link
Contributor Author

edawson commented Oct 19, 2020

This adds the functions needed to generate the anchor chains from a list of overlaps, assuming the overlap list has not been previously filtered (though masking using the select_mask is permissible).

… However, tests fail because the allocator used in testing can't actually allocate memory.
@mimaric mimaric added the cudamapper GPU-based overlapper label Oct 20, 2020
Copy link
Member

@mimaric mimaric left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a few basic comments. Can you please add more documentation?

cudamapper/src/chainer_utils.cu Show resolved Hide resolved
cudamapper/src/chainer_utils.cuh Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cuh Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cu Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cu Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cuh Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cuh Outdated Show resolved Hide resolved
@edawson edawson requested a review from mimaric October 22, 2020 13:53
Copy link
Member

@mimaric mimaric left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So far reviewed allocate_anchor_chains() and left some general comments

cudamapper/tests/Test_CudamapperChainerUtils.cu Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cuh Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cuh Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cuh Outdated Show resolved Hide resolved
/// \param start The first anchor in the chain.
/// \param end The last anchor in the chain.
/// \param num_anchors The total number of anchors in the chain.
__host__ __device__ Overlap create_simple_overlap(const Anchor& start,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is only called by backtrace_anchors_to_overlaps(). I understand that it cannot be in an anonymous namespace in order be able to call it form unittest, but it could be moved to details namespace in order to signal the users that they do not have to think about it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think creating an overlap from two anchors and a number of residues is useful and could be a user-facing API function, but the function could be better named.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. That's why I was suggesting moving it closed to Overlap's definition: #590 (comment)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. I've renamed it to create_overlap in c72e5c9.

cudamapper/src/chainer_utils.cu Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cu Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cuh Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cu Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cu Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cuh Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cuh Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cu Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cu Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cu Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cuh Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cuh Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cu Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cu Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cu Outdated Show resolved Hide resolved
cudaaligner/samples/sample_cudaaligner.cpp Show resolved Hide resolved
cudamapper/samples/sample_cudamapper.cpp Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cu Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cu Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cu Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cu Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cu Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cu Outdated Show resolved Hide resolved
- Use thrust transform_reduce and transform_exclusive_scan (rather than make_transform iterator) whe>
- Constify pointers where possible.
- Remove unnecessary includes.
- Tweak docs for backtrace_anchors_to_overlap
@edawson edawson requested a review from mimaric November 6, 2020 15:35
Copy link
Member

@mimaric mimaric left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please have a look at still unresolved comments from previous reviews as well (you can ignore comment about create_overlap)

cudamapper/src/chainer_utils.cu Outdated Show resolved Hide resolved
cudamapper/src/overlapper_triggered.cu Show resolved Hide resolved
cudamapper/tests/Test_CudamapperChainerUtils.cu Outdated Show resolved Hide resolved
cudamapper/tests/Test_CudamapperChainerUtils.cu Outdated Show resolved Hide resolved
cudamapper/src/chainer_utils.cu Outdated Show resolved Hide resolved
cudamapper/tests/Test_CudamapperChainerUtils.cu Outdated Show resolved Hide resolved
cudamapper/tests/Test_CudamapperChainerUtils.cu Outdated Show resolved Hide resolved
Primarily, this adds a number of tests for output_overlap_chains_by_RLE,
output_overlap_chains_by_backtrace, and backtrace_anchors_to_overlaps.
It also fixes a bug where the last anchor position in a chain was not returned
from backtrace_anchors_to_overlaps, causing faulty output when later extracting anchor
chains.
@edawson edawson requested a review from mimaric November 10, 2020 04:18
@mimaric
Copy link
Member

mimaric commented Nov 16, 2020

rerun tests

/// \param unrolled_anchor_chains An array of int32_t. Will be resided on return.
/// \param anchor_chain_starts An array holding the index in the anchors array of the first
/// anchor in an overlap.
/// \param num_overlaps The number of overlaps in the overlaps array.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

num_overlaps does not exist anymore

ASSERT_EQ(overlaps_d.size(), 3);

ASSERT_EQ(overlaps[0].num_residues_, 1);
ASSERT_EQ(overlaps[0].target_read_id_, UINT32_MAX);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better use std::numeric_limits<decltype(overlaps[0].target_read_id_)>::max()
https://en.cppreference.com/w/cpp/types/numeric_limits

#include <algorithm>

#ifndef NDEBUG
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is algorithm needed in release mode as well?

Comment on lines +116 to +117
/// Launch configuration: any number of blocks/threads, no dynamic shared memory, cuda_stream must be the same in which anchors/overlaps/scores/mask/predecessors/chains/starts were allocated.
/// example: backtrace_anchors_to_overlaps<<<2048, 32, 0, cuda_stream>>>(...)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Further descriptions should be placed between \brief and \param

void allocate_anchor_chains(const device_buffer<Overlap>& overlaps,
device_buffer<int32_t>& unrolled_anchor_chains,
device_buffer<int32_t>& anchor_chain_starts,
int64_t& num_total_anchors,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessary? I see it's used to set the size of unrolled_anchor_chains

Base automatically changed from dev-v0.6.0 to dev February 8, 2021 22:44
@ohadmo ohadmo closed this Feb 8, 2021
@ohadmo ohadmo deleted the branch NVIDIA-Genomics-Research:dev February 8, 2021 22:54
@ohadmo ohadmo reopened this Feb 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cudamapper GPU-based overlapper
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants