[REVIEW]: pivmet: an R package proposing pivotal methods for consensus clustering and mixture modelling

[editorialbot]

Submitting author: @LeoEgidi (LEONARDO EGIDI)
Repository: https://github.com/LeoEgidi/pivmet
Branch with paper.md (empty if default branch): master
Version: v0.6.0
Editor: @skanwal
Reviewers: @adriancorrendo, @larryshamalama
Archive: Pending

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/27410f99af44fd07417b87b8ec193ef5"><img src="https://joss.theoj.org/papers/27410f99af44fd07417b87b8ec193ef5/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/27410f99af44fd07417b87b8ec193ef5/status.svg)](https://joss.theoj.org/papers/27410f99af44fd07417b87b8ec193ef5)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@adriancorrendo & @larryshamalama, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review.
First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @skanwal know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @adriancorrendo

📝 Checklist for @larryshamalama

[editorialbot]

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

[editorialbot]

Software report:

github.com/AlDanial/cloc v 1.90  T=0.02 s (1098.3 files/s, 258564.0 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
R                               14            441            941           2031
HTML                             2             70              5            983
TeX                              2             76              0            518
Markdown                         4            108              0            305
Rmd                              3            187            245            206
YAML                             1              1              1              3
-------------------------------------------------------------------------------
SUM:                            26            883           1192           4046
-------------------------------------------------------------------------------

Commit count by author:

   503	Leonardo Egidi
   168	LeoEgidi
     6	Leonardo

[editorialbot]

Paper file info:

📄 Wordcount for paper.md is 1196

✅ The paper includes a Statement of need section

[editorialbot]

License info:

🔴 Failed to discover a valid open source license

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- None

MISSING DOIs

- No DOI given, and none found for title: A conceptual introduction to Hamiltonian Monte Car...
- No DOI given, and none found for title: K-means seeding via MUS algorithm
- 10.1007/978-3-319-73906-9_7 may be a valid DOI for title: Maxima Units Search (MUS) algorithm: methodology a...
- 10.1007/s11222-017-9774-2 may be a valid DOI for title: Relabelling in Bayesian mixture models by pivotal ...
- No DOI given, and none found for title: Cluster ensembles - A knowledge reuse framework fo...
- 10.1109/tpami.2005.113 may be a valid DOI for title: Combining multiple clusterings using evidence accu...
- No DOI given, and none found for title: k-means++: The advantages of careful seeding
- No DOI given, and none found for title: Bayesian solutions to the label switching problem
- 10.1214/09-ba414 may be a valid DOI for title: Improved criteria for clustering based on the post...
- 10.1111/1467-9868.00265 may be a valid DOI for title: Dealing with label switching in mixture models
- 10.1080/03610926.2010.526741 may be a valid DOI for title: Bayesian mixture labeling and clustering
- 10.1214/088342305000000016 may be a valid DOI for title: Markov chain Monte Carlo methods and the label swi...
- 10.1198/jasa.2009.0237 may be a valid DOI for title: Bayesian mixture labeling by highest posterior den...
- 10.1198/016214501750333063 may be a valid DOI for title: Markov chain Monte Carlo estimation of classical a...
- 10.1198/0003130043286 may be a valid DOI for title: Difficulties in drawing inferences with finite-mix...
- 10.1080/00949655.2012.707201 may be a valid DOI for title: An online Bayesian mixture labelling method by min...
- 10.1198/jcgs.2010.09008 may be a valid DOI for title: An artificial allocations based solution to the la...
- No DOI given, and none found for title: label.switching: An R package for dealing with the...
- No DOI given, and none found for title: label.switching: Relabelling MCMC Outputs of Mixtu...
- No DOI given, and none found for title: bayesmix: Bayesian Mixture Models with JAGS
- 10.2307/2531224 may be a valid DOI for title: Statistical analysis of finite mixture distributio...
- No DOI given, and none found for title: Finite Mixture Models
- No DOI given, and none found for title: Comparing partitions
- 10.1080/10618600.2012.735624 may be a valid DOI for title: Label switching in Bayesian mixture models: Determ...
- 10.1007/978-0-387-38983-7 may be a valid DOI for title: Bayesian core: a practical approach to computation...
- 10.1016/s0169-7161(05)25016-2 may be a valid DOI for title: Bayesian modelling and inference on mixtures of di...
- 10.2307/2669477 may be a valid DOI for title: Computational and inferential difficulties with mi...
- 10.1007/s11222-009-9129-8 may be a valid DOI for title: Probabilistic relabelling strategies for the label...
- No DOI given, and none found for title: Bayesian inference for mixture models via Monte Ca...
- 10.1007/bf00143556 may be a valid DOI for title: Sampling from multimodal distributions using tempe...
- No DOI given, and none found for title: On Bayesian analysis of mixtures with an unknown n...
- 10.2307/2289993 may be a valid DOI for title: Density estimation with confidence sets exemplifie...
- No DOI given, and none found for title: foreign: Read Data Stored by ‘Minitab’, ‘S’, ‘SAS’...
- No DOI given, and none found for title: MASS: Support Functions and Datasets for Venables ...
- No DOI given, and none found for title: mclust: Gaussian Mixture Modelling for Model-Based...
- No DOI given, and none found for title: pivmet: Pivotal Methods for Bayesian Relabelling a...
- No DOI given, and none found for title: RStan: the R interface to Stan
- No DOI given, and none found for title: rjags: Bayesian graphical models using MCMC
- No DOI given, and none found for title: runjags: An R package providing interface utilitie...
- 10.1214/08-aoas191 may be a valid DOI for title: A weakly informative default prior distribution fo...
- No DOI given, and none found for title: Prior distributions for variance parameters in hie...
- 10.1214/009053604000001147 may be a valid DOI for title: Spike and slab variable selection: frequentist and...
- 10.2307/1390653 may be a valid DOI for title: Markov chain sampling methods for Dirichlet proces...
- No DOI given, and none found for title: Bayesian density estimation by mixtures of normal ...
- 10.2307/2291069 may be a valid DOI for title: Bayesian density estimation and inference using mi...
- No DOI given, and none found for title: dirichletprocess: An R Package for Fitting Complex...
- No DOI given, and none found for title: A Bayesian analysis of some nonparametric problems
- 10.1007/s11634-018-0329-y may be a valid DOI for title: From here to infinity: sparse finite versus Dirich...
- 10.1007/s11222-014-9500-2 may be a valid DOI for title: Model-based clustering based on sparse finite Gaus...
- 10.1111/anzs.12350 may be a valid DOI for title: Spying on the prior of the number of data clusters...
- No DOI given, and none found for title: Model-based clustering, discriminant analysis, and...

INVALID DOIs

- None

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[adriancorrendo]

Review checklist for @adriancorrendo

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the https://github.com/LeoEgidi/pivmet?
• License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@LeoEgidi) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
• Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
• Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
• Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[arfon]

@LeoEgidi – this may have been discussed already but what license is this software released under? JOSS requires software published here to have a file named LICENSE at the root of the repository with the whole text of an OSI-approved license.

[LeoEgidi]

@LeoEgidi – this may have been discussed already but what license is this software released under? JOSS requires software published here to have a file named LICENSE at the root of the repository with the whole text of an OSI-approved license.

Hi @arfon , the License, as documented by the description file, is GPL-2

[LeoEgidi]

I investigate a bit more

[LeoEgidi]

Ok, now I put a LICENSE file at the repository root as suggested

@LeoEgidi – this may have been discussed already but what license is this software released under? JOSS requires software published here to have a file named LICENSE at the root of the repository with the whole text of an OSI-approved license.

Hi @arfon , the License, as documented by the description file, is GPL-2

[adriancorrendo]

Hi @LeoEgidi .
How are you? Thanks for developing the pivmet package and submitting it for review here.

I've just created an issue on your repo with my review comments at LeoEgidi/pivmet#1 (comment)

Could you please take a look?

Thanks,
Adrian

[LeoEgidi]

Hi @adriancorrendo , thanks a lot for your comments, I took a first look. I'll try to get a deeper look this week, or at most next week. Should I reply your points here or directly in the repo? What do you prefer? Thanks a lot

[adriancorrendo]

Hi @adriancorrendo , thanks a lot for your comments, I took a first look. I'll try to get a deeper look this week, or at most next week. Should I reply your points here or directly in the repo? What do you prefer? Thanks a lot

Hi Leo, how are you?
You would need to answer on the issue I created in the package repo.
Once you address those comments, I'll start checking the remaining boxes until giving my full ok. That's what I did when I published my paper here.

Hope it helps!
Adrian

[larryshamalama]

Review checklist for @larryshamalama

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the https://github.com/LeoEgidi/pivmet?
• License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@LeoEgidi) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
• Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
• Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
• Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[larryshamalama]

Hi @LeoEgidi, thanks for the submission. Overall, looks good, but I do have some doubts with the paper. Some comments below align with @adriancorrendo's review.

Major comments/questions

• A statement of need and State of the field: You define pivotal units in the first paragraph without diving into the details of the math; this is nice. Was this concept introduced in your 2018 paper or was it previously introduced? The main comment here is that I cannot find any "literature (software) review" of any packages that have any capabilities in finding pivotal units. If the concept was introduced in your 2018 paper, it makes sense that there is no software that specifically provides functionality for identifying pivotal units. It would be nice to either 1) if no other software can be used, identify what mixture modelling packages could potentially be helpful in identifying pivotal units or 2) summarize what this package contributes on top of other pivotal unit software.

Minor comments

• I think that it would be good to include a quickstart akin to what is available in the README in the software paper itself.
• Community guidelines seem to be missing. For instance, you can add a CONTRIBUTING.md file outlining how to ask questions (via GitHub issues or discussions) and how to contribute to the package via pull requests.
• I found some typos in the article. I tried solving the ones I found in my PR. I may have missed some, please have a look again.

[LeoEgidi]

Hi @adriancorrendo , thanks a lot for your comments, I took a first look. I'll try to get a deeper look this week, or at most next week. Should I reply your points here or directly in the repo? What do you prefer? Thanks a lot

Hi Leo, how are you? You would need to answer on the issue I created in the package repo. Once you address those comments, I'll start checking the remaining boxes until giving my full ok. That's what I did when I published my paper here.

Hope it helps! Adrian

@arfon @adriancorrendo

I replied the comments from Adrian in the issue he created. Now time to work on the @larryshamalama comments

[LeoEgidi]

Hi @LeoEgidi, thanks for the submission. Overall, looks good, but I do have some doubts with the paper. Some comments below align with @adriancorrendo's review.

Major comments/questions

• A statement of need and State of the field: You define pivotal units in the first paragraph without diving into the details of the math; this is nice. Was this concept introduced in your 2018 paper or was it previously introduced? The main comment here is that I cannot find any "literature (software) review" of any packages that have any capabilities in finding pivotal units. If the concept was introduced in your 2018 paper, it makes sense that there is no software that specifically provides functionality for identifying pivotal units. It would be nice to either 1) if no other software can be used, identify what mixture modelling packages could potentially be helpful in identifying pivotal units or 2) summarize what this package contributes on top of other pivotal unit software.

Thanks. In line with the comments from @adriancorrendo , we added a new paragraph at the end the "Statement of need" section to make an overview of other R packages related to pivmet. For this reason, we summarized why pivmet is so beneficial in the search of the pivotal units and how this package is naturally related to other existing packages, such as 'rstan', 'rjags' and 'bayesmix'.

Minor comments

• I think that it would be good to include a quickstart akin to what is available in the README in the software paper itself.

Ok, If I well understood your point, in line with one of the comments from @adriancorrendo , we included now in the two examples in the paper, #Example 1 and #Example 2, some relevant lines of R code, basically the same included in the README file.

• Community guidelines seem to be missing. For instance, you can add a CONTRIBUTING.md file outlining how to ask questions (via GitHub issues or discussions) and how to contribute to the package via pull requests.

Thanks, done. We added a new .md file "CODE OF CONDUCT" in the main repo to properly suggest the users how to contribute and report bugs.

• I found some typos in the article. I tried solving the ones I found in my PR. I may have missed some, please have a look again.

Thanks, I read another time, and so far no other typos have been found. I merged the pull request.

[LeoEgidi]

@arfon @larryshamalama @adriancorrendo

Thanks a lot;)
I replied all your comments, for any doubt/clarification I am available

[crvernon]

👋 - @skanwal will you check in on this submission? It looks like the author and reviewers are making good progress. Thanks!

[skanwal]

Thanks so much @adriancorrendo and @larryshamalama.
The checklists look fairly complete to me. Are you happy for this submission to be accpeted? I wanted to confirm there are no outstanding tasks?

Also, thanks @LeoEgidi for staying on top of comments and updates.

[larryshamalama]

Thanks so much @adriancorrendo and @larryshamalama. The checklists look fairly complete to me. Are you happy for this submission to be accpeted? I wanted to confirm there are no outstanding tasks?

Also, thanks @LeoEgidi for staying on top of comments and updates.

Yes looks good to me!

[adriancorrendo]

I just asked authors to add a R-CMD badge in the README that helps users to see upfront if the package is passing automated tests, ideally also a coverage test like codecov. However, this could be more a "suggestion" than a required revision for acceptance. So, I'm happy with the answers and revisions from the authors.

[LeoEgidi]

Thanks a lot!!

I also added the R-CMD suggestion from @adriancorrendo in the readme

[skanwal]

Thanks @LeoEgidi. Your submission is ready to be accepted. Can you please create a new tagged release of the software (if changed), and archive it (on Zenodo, figshare, or other). Once you're done can you please also post the version number and archive DOI in this REVIEW issue.

[skanwal]

Post-Review Checklist for Editor and Authors

Additional Author Tasks After Review is Complete

• Double check authors and affiliations (including ORCIDs)
• Make a release of the software with the latest changes from the review and post the version number here. This is the version that will be used in the JOSS paper.
• Archive the release on Zenodo/figshare/etc and post the DOI here.
• Make sure that the title and author list (including ORCIDs) in the archive match those in the JOSS paper.
• Make sure that the license listed for the archive is the same as the software license.

Editor Tasks Prior to Acceptance

• Read the text of the paper and offer comments/corrections (as either a list or a pull request)
• Check that the archive title, author list, version tag, and the license are correct
• Set archive DOI with @editorialbot set <DOI here> as archive
• Set version with @editorialbot set <version here> as version
• Double check rendering of paper with @editorialbot generate pdf
• Specifically check the references with @editorialbot check references and ask author(s) to update as needed
• Recommend acceptance with @editorialbot recommend-accept

[LeoEgidi]

@editorialbot generate pdf

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[LeoEgidi]

@editorialbot generate pdf

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[LeoEgidi]

@editorialbot generate pdf

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[skanwal]

Hello @LeoEgidi - how are you going with final proof reading and creating the DOI?

[LeoEgidi]

Hello @LeoEgidi - how are you going with final proof reading and creating the DOI?

Hi @skanwal , thanks a lot. Just give us a bunch of days to finalize proofs and DOI, we are working on it!

[LeoEgidi]

@editorialbot generate pdf

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[LeoEgidi]

Post-Review Checklist for Editor and Authors

Additional Author Tasks After Review is Complete

• Double check authors and affiliations (including ORCIDs)

 DONE

• Make a release of the software with the latest changes from the review and post the version number here. This is the version that will be used in the JOSS paper.

 DONE, the current version for pivmet is 0.6.0 (available on Github and submitted to CRAN)

• Archive the release on Zenodo/figshare/etc and post the DOI here.

 DONE. DOI is: 10.5281/zenodo.11243277

• Make sure that the title and author list (including ORCIDs) in the archive match those in the JOSS paper.

 DONE 

• Make sure that the license listed for the archive is the same as the software license.

 DONE (License is GPL-2)

Here we are @skanwal

[LeoEgidi]

Hello @LeoEgidi - how are you going with final proof reading and creating the DOI?

just posted my final replies here above. Hopefully we are there! Best, and thanks for everything

[LeoEgidi]

@editorialbot generate pdf

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[LeoEgidi]

@editorialbot generate pdf

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[LeoEgidi]

(ok, super last edits to the r reproducible code. done)

[skanwal]

@editorialbot set 10.5281/zenodo.11243277 as archive

[editorialbot]

That doesn't look like a valid DOI value

[skanwal]

@LeoEgidi - did you use similar instructions as this to create DOI?

[skanwal]

@editorialbot set v0.6.0 as version

[editorialbot]

Done! version is now v0.6.0

[LeoEgidi]

@LeoEgidi - did you use similar instructions as this to create DOI?

yes!
I copy here the whole DOI url, this works:

https://doi.org/10.5281/zenodo.11243277