fix multiple cell ids match with the same dataset id

[myushen]

[multimeric]

At this point, counts_path$cpm_path seems like it's a vector of filepaths, one for each file_id_db in the metadata. However, get_counts_per_million expects a single file path. Does this work in practice?

[myushen]

I think this API requires users to import a SingleCellExperiment object one at a time, and each object should have the same file_id_db derived from dataset_id.

[multimeric]

Ah yes, I see that file_id_db = .data$dataset_id |> openssl::md5() |> as.character(), so it there will only be one file_id_db. I think to make this clearer, you could just create file_id_db as a separate variable that you add to the metadata data frame, and also use to generate the paths. That way you won't have to filter out the redundant copies at all.

[stemangiola]

Thanks for that, @multimeric; I had a discussion with @myushen and concluded that an additional argument is not needed at this stage, just good messaging and documentation, in case the user input and multiple data sets in the same single experiment.

[multimeric]

I wasn't suggesting adding a new argument. I'm just saying that we have a single file ID, which gets duplicated as a result of putting it into the data frame, and then we filter it back down again to a single value using distinct. This can be avoided by using a separate variable:

file_id_db <- .data$dataset_id |> openssl::md5() |> as.character()

...

metadata_tbl <-
    metadata_tbl |> mutate(file_id_db = file_id_db)

...


original_path <- file.path(original_dir, basename(file_id_db))
cpm_path <- file.path(cache_dir, "cpm", basename(file_id_db))

[stemangiola]

but the redundant file_if_db column will be carried in the metadata, that is what we use to do future queries. So, in the metadata, the redundancy will remain.

Or am I missing something?

[multimeric]

Yes, I realise it will be like that in the metadata. My suggestion is just to make the code simpler and clearer.

[stemangiola]

I am not sure if the additional argument would be in the front end (cost of each additional argument is $1M) or in the back end (cost $1)

sure in the backend @myushen follow @multimeric lead.

I think @myushen intended that as modifying the front end (code interface to the user)

[myushen]

I will add a check point at the front end and rename the function for clarification