Prepare for cellfinder merge (#54)

* Update to new dependencies

* Update README text

* cellfinder_core -> cellfinder.core

With the removal of tox.ini from main branch, also remove the now-redundant check on the development version of cellfinder

* Update cellfinder CLI name to brainmapper

* Prune file that has escaped the rebase

* Move cellfinder CLI tests to tests/brainmapper as per rename

* Remove confusing cellfinder description

* Rename lingering cellfinder references to brainmapper

* Update README.md

Co-authored-by: Adam Tyson <code@adamltyson.com>

---------

Co-authored-by: Adam Tyson <code@adamltyson.com>

.github/workflows/test_and_deploy.yml

@@ -8,11 +8,6 @@ on:
       - "*"
   pull_request:
   workflow_dispatch:
-    inputs:
-      coredev:
-        description: 'Tests with development version of cellfinder-core'
-        required: true
-        default: 'false'
 
 env:
   REGISTRY: ghcr.io
@@ -35,8 +30,6 @@ jobs:
     needs: [linting, manifest]
     name: ${{ matrix.os }} py${{ matrix.python-version }}
     runs-on: ${{ matrix.os }}
-    env:
-      INPUT_COREDEV: ${{ github.event.inputs.coredev }}
     strategy:
       fail-fast: false
       matrix:

README.md

@@ -24,68 +24,67 @@ You can also find the documentation for the backend BrainGlobe tools these workf
 
 At present, the package offers the following workflows to users:
 
-- [cellfinder](#cellfinder): a command-line tool for whole-brain detection, registration, and analysis.
+- [brainmapper](#brainmapper-command-line-interface-cli): A command-line tool for whole-brain detection, registration, and analysis.
 
-Additionally, this repository provides functionalities to support code developers. See [Developers documentation](#developers-documentation) for further details.
+Additionally, this repository provides functionalities to support code developers. See the [developer documentation](#developer-documentation) for further details.
 
 ## User documentation
 
+### Installation
 
-### Installation of the cellfinder CLI tool
-At the moment, users can install the cellfinder CLI tool as a standalone tool, by running `pip install` in your desired environment:
+At the moment, users can install all available workflows by running `pip install` in your desired environment:
 
 ```bash
 pip install brainglobe-workflows
 ```
 
 `brainglobe-workflows` is built using BrainGlobe tools, and it will automatically fetch the tools that it needs and install them into your environment.
-Once BrainGlobe version 1 is available, this package will fetch all BrainGlobe tools and handle their install into your environment, to prevent potential conflicts from partial-installs.
+Once BrainGlobe version 1 is available, this package will fetch all BrainGlobe tools and handle their install into your environment, to prevent potential conflicts from partial installs.
 
+See the sections below for more information about the workflows and command-line tools provided.
 
-### Cellfinder Command Line Interface (CLI)
+#### `brainmapper` Command Line Interface (CLI)
 
 Whole-brain cell detection, registration and analysis.
 
-If you want to just use the cell detection part of `cellfinder`, please see the standalone [cellfinder-core](https://github.com/brainglobe/cellfinder-core) package, or the [cellfinder plugin](https://github.com/brainglobe/cellfinder-napari) for [napari](https://napari.org/).
+If you want to just use the cell detection part of `brainmapper`, please see the standalone [cellfinder](https://github.com/brainglobe/cellfinder) package and its [`napari`](https://napari.org/) plugin.
 
-`cellfinder` is a collection of tools developed by [Adam Tyson](https://github.com/adamltyson), [Charly Rousseau](https://github.com/crousseau) and [Christian Niedworok](https://github.com/cniedwor) in the [Margrie Lab](https://www.sainsburywellcome.org/web/groups/margrie-lab), generously supported by the [Sainsbury Wellcome Centre](https://www.sainsburywellcome.org/web/).
-
-`cellfinder` is designed for the analysis of whole-brain imaging data such as [serial-section imaging](https://sainsburywellcomecentre.github.io/OpenSerialSection/) and lightsheet imaging in cleared tissue.
+`brainmapper` is a workflow designed for the analysis of whole-brain imaging data such as [serial-section imaging](https://sainsburywellcomecentre.github.io/OpenSerialSection/) and lightsheet imaging in cleared tissue.
 The aim is to provide a single solution for:
 
-- Cell detection (initial cell candidate detection and refinement using  deep learning) (using [cellfinder-core](https://github.com/brainglobe/cellfinder-core)),
+- Cell detection (initial cell candidate detection and refinement using  deep learning) (using the [cellfinder](https://github.com/brainglobe/cellfinder) backend package),
 - Atlas registration (using [brainreg](https://github.com/brainglobe/brainreg)),
 - Analysis of cell positions in a common space.
 
 Basic usage:
 
 ```bash
-cellfinder -s signal_images -b background_images -o output_dir --metadata metadata
+brainmapper -s signal_images -b background_images -o output_dir --metadata metadata
 ```
 
-Full documentation can be found [here](https://brainglobe.info/documentation/cellfinder/index.html).
+Full documentation can be found [here](https://brainglobe.info/documentation/brainglobe-workflows/brainmapper/index.html).
 
+NOTE: The `brainmapper` workflow previously used the name "cellfinder", but this has been discontinued following the release of the [unified `cellfinder`](https://github.com/brainglobe/cellfinder) backend package to avoid conflation of terms.
+See our [blog post](https://brainglobe.info/blog/version1/cellfinder-core-and-plugin-merge.html) from the release for more information.
 
 ## Developer documentation
 
 This repository also includes workflow scripts that are benchmarked to support code development.
 These benchmarks are run regularly to ensure performance is stable, as the tools are developed and extended.
-* Developers can install these benchmarks locally via `pip install .[dev]`. By executing `asv run`, the benchmarks will run with default parameters on a small dataset that is downloaded from [GIN](https://gin.g-node.org/G-Node/info/wiki). See [the asv docs](https://asv.readthedocs.io/en/v0.6.1/using.html#running-benchmarks) for further details on how to run benchmarks.
-* Developers can also run these benchmarks on data they have stored locally, by specifying the relevant paths in an input (JSON) file.
-* We also maintain an internal runner that benchmarks the workflows over a large, exemplar dataset, of the scale we expect users to be handling. The result of these benchmarks are made publicly available.
+
+- Developers can install these benchmarks locally via `pip install .[dev]`. By executing `asv run`, the benchmarks will run with default parameters on a small dataset that is downloaded from [GIN](https://gin.g-node.org/G-Node/info/wiki). See [the asv docs](https://asv.readthedocs.io/en/v0.6.1/using.html#running-benchmarks) for further details on how to run benchmarks.
+- Developers can also run these benchmarks on data they have stored locally, by specifying the relevant paths in an input (JSON) file.
+- We also maintain an internal runner that benchmarks the workflows over a large, exemplar dataset, of the scale we expect users to be handling. The result of these benchmarks are made publicly available.
 
 Contributions to BrainGlobe are more than welcome.
 Please see the [developer guide](https://brainglobe.info/developers/index.html).
 
-
 ## Citing `brainglobe-workflows`
 
-**If you use any tools in the [brainglobe suite](https://brainglobe.info/documentation/index.html), please [let us know](mailto:code@adamltyson.com?subject=cellfinder), and we'd be happy to promote your paper/talk etc.**
+**If you use any tools in the [brainglobe suite](https://brainglobe.info/documentation/index.html), please [let us know](mailto:code@adamltyson.com?subject=BrainGlobe), and we'd be happy to promote your paper/talk etc.**
 
-If you find [`cellfinder`](#cellfinder) useful, and use it in your research, please cite the paper outlining the cell detection algorithm:
+If you find `brainmapper` useful, and use it in your research, please cite the paper outlining the cell detection algorithm:
 > Tyson, A. L., Rousseau, C. V., Niedworok, C. J., Keshavarzi, S., Tsitoura, C., Cossell, L., Strom, M. and Margrie, T. W. (2021) “A deep learning algorithm for 3D cell detection in whole mouse brain image datasets’ PLOS Computational Biology, 17(5), e1009074
 [https://doi.org/10.1371/journal.pcbi.1009074](https://doi.org/10.1371/journal.pcbi.1009074)
 >
-If you use any of the image registration functions in `cellfinder`, please also cite [`brainreg`](https://github.com/brainglobe/brainreg#citing-brainreg).
-
----
+If you use any of the image registration functions in `brainmapper`, please also cite [`brainreg`](https://github.com/brainglobe/brainreg#citing-brainreg).

benchmarks/cellfinder_core.py

@@ -4,8 +4,8 @@
 
 import pooch
 from brainglobe_utils.IO.cells import save_cells
-from cellfinder_core.main import main as cellfinder_run
-from cellfinder_core.tools.IO import read_with_dask
+from cellfinder.core.main import main as cellfinder_run
+from cellfinder.core.tools.IO import read_with_dask
 
 from brainglobe_workflows.cellfinder_core.cellfinder import (
     CellfinderConfig,

brainglobe_workflows/cellfinder_brainreg/analyse/analyse.py → brainglobe_workflows/brainmapper/analyse/analyse.py

@@ -18,7 +18,7 @@
 from brainglobe_utils.general.system import ensure_directory_exists
 from brainglobe_utils.pandas.misc import safe_pandas_concat, sanitise_df
 
-from brainglobe_workflows.cellfinder_brainreg.export.export import (
+from brainglobe_workflows.brainmapper.export.export import (
     export_points,
 )
 

brainglobe_workflows/cellfinder_brainreg/export/export.py → brainglobe_workflows/brainmapper/export/export.py

@@ -1,7 +1,7 @@
-from brainglobe_workflows.cellfinder_brainreg.export.abc4d import (
+from brainglobe_workflows.brainmapper.export.abc4d import (
     export_points as abc4d_export,
 )
-from brainglobe_workflows.cellfinder_brainreg.export.brainrender import (
+from brainglobe_workflows.brainmapper.export.brainrender import (
     export_points as brainrender_export,
 )
 

brainglobe_workflows/cellfinder_brainreg/extract/extract_cubes.py → brainglobe_workflows/brainmapper/extract/extract_cubes.py

@@ -22,10 +22,10 @@
 from tifffile import tifffile
 from tqdm import tqdm
 
-from brainglobe_workflows.cellfinder_brainreg.tools import (
+from brainglobe_workflows.brainmapper.tools import (
     image_processing as img_tools,
 )
-from brainglobe_workflows.cellfinder_brainreg.tools import system
+from brainglobe_workflows.brainmapper.tools import system
 
 
 class StackSizeError(Exception):

brainglobe_workflows/cellfinder_brainreg/figures/figures.py → brainglobe_workflows/brainmapper/figures/figures.py

@@ -1,6 +1,6 @@
 import logging
 
-from brainglobe_workflows.cellfinder_brainreg.figures import heatmap
+from brainglobe_workflows.brainmapper.figures import heatmap
 
 
 def run(args, atlas, downsampled_shape):

brainglobe_workflows/cellfinder_brainreg/main.py → brainglobe_workflows/brainmapper/main.py

@@ -17,7 +17,7 @@
 from brainglobe_utils.cells.cells import MissingCellsError
 from brainglobe_utils.general.system import ensure_directory_exists
 from brainglobe_utils.IO.cells import get_cells, save_cells
-from cellfinder_core.main import suppress_tf_logging, tf_suppress_log_messages
+from cellfinder.core.main import suppress_tf_logging, tf_suppress_log_messages
 
 BRAINREG_PRE_PROCESSING_ARGS = None
 
@@ -44,7 +44,7 @@ def main():
     suppress_tf_logging(tf_suppress_log_messages)
     from brainreg.core.main import main as register
 
-    from brainglobe_workflows.cellfinder_brainreg.tools import prep
+    from brainglobe_workflows.brainmapper.tools import prep
 
     start_time = datetime.now()
     args, arg_groups, what_to_run, atlas = prep.prep_cellfinder_general()
@@ -100,14 +100,14 @@ def main():
 
 
 def run_all(args, what_to_run, atlas):
-    from cellfinder_core.classify import classify
-    from cellfinder_core.detect import detect
-    from cellfinder_core.tools import prep
-    from cellfinder_core.tools.IO import read_with_dask
-
-    from brainglobe_workflows.cellfinder_brainreg.analyse import analyse
-    from brainglobe_workflows.cellfinder_brainreg.figures import figures
-    from brainglobe_workflows.cellfinder_brainreg.tools.prep import (
+    from cellfinder.core.classify import classify
+    from cellfinder.core.detect import detect
+    from cellfinder.core.tools import prep
+    from cellfinder.core.tools.IO import read_with_dask
+
+    from brainglobe_workflows.brainmapper.analyse import analyse
+    from brainglobe_workflows.brainmapper.figures import figures
+    from brainglobe_workflows.brainmapper.tools.prep import (
         prep_candidate_detection,
         prep_channel_specific_general,
     )

brainglobe_workflows/cellfinder_brainreg/tools/parser.py → brainglobe_workflows/brainmapper/tools/parser.py

@@ -18,11 +18,11 @@
 )
 from brainreg.core.cli import atlas_parse, geometry_parser, niftyreg_parse
 from brainreg.core.cli import backend_parse as brainreg_backend_parse
-from cellfinder_core.download.cli import (
+from cellfinder.core.download.cli import (
     download_directory_parser,
     model_parser,
 )
-from cellfinder_core.tools.source_files import source_custom_config_cellfinder
+from cellfinder.core.tools.source_files import source_custom_config_cellfinder
 
 from brainglobe_workflows import __version__
 

brainglobe_workflows/cellfinder_brainreg/tools/prep.py → brainglobe_workflows/brainmapper/tools/prep.py

@@ -18,9 +18,9 @@
 from fancylog import fancylog
 
 import brainglobe_workflows as program_for_log
-import brainglobe_workflows.cellfinder_brainreg.tools.parser as parser
-from brainglobe_workflows.cellfinder_brainreg.tools import system, tools
-from brainglobe_workflows.cellfinder_brainreg.tools.parser import (
+import brainglobe_workflows.brainmapper.tools.parser as parser
+from brainglobe_workflows.brainmapper.tools import system, tools
+from brainglobe_workflows.brainmapper.tools.parser import (
     cellfinder_parser,
 )
 

pyproject.toml

@@ -28,7 +28,7 @@ classifiers = [
 # (i.e., only what users will need for the CLI)
 dependencies = [
     "brainreg>=1.0.0",
-    "cellfinder-core>=0.2.4,<1.0.0",
+    "cellfinder>=1.0.0",
     "configobj",
     "fancylog>=0.0.7",
     "imio",
@@ -62,17 +62,9 @@ dev = [
 # (i.e., everything needed to install this package without the CLI tool)
 # Once the cellfinder CLI tool is deprecated, these will move to the
 # default dependencies.
-asv_version = [
-    "asv",
-    "pooch",
-    "cellfinder-core",
-]
+asv_version = ["asv", "pooch", "cellfinder-core"]
 
-napari = [
-    "napari[pyqt5]",
-    "brainglobe-napari-io",
-    "cellfinder-napari<1.0.0",
-]
+napari = ["napari[pyqt5]", "brainglobe-napari-io", "cellfinder[napari]>=1.0.0"]
 
 [project.urls]
 "Bug Tracker" = "https://github.com/brainglobe/brainglobe-workflows/issues"
@@ -82,7 +74,7 @@ napari = [
 
 [project.scripts]
 cellfinder-workflow = "brainglobe_workflows.cellfinder_core.cellfinder:main_app_wrapper"
-cellfinder = "brainglobe_workflows.cellfinder_brainreg.main:main"
+brainmapper = "brainglobe_workflows.brainmapper.main:main"
 
 [build-system]
 requires = ["setuptools>=45", "wheel", "setuptools_scm[toml]>=6.2"]
@@ -94,7 +86,12 @@ zip-safe = false
 
 [tool.setuptools.packages.find]
 include = ["brainglobe_workflows"]
-exclude = ["brainglobe_workflows.cellfinder_core","tests", "resources", "benchmarks"] # it's not excluding "brainglobe_workflows.cellfinder_core"!
+exclude = [
+    "brainglobe_workflows.cellfinder_core",
+    "tests",
+    "resources",
+    "benchmarks",
+] # it's not excluding "brainglobe_workflows.cellfinder_core"!
 
 [tool.black]
 target-version = ["py39", "py310"]

tests/cellfinder_brainreg/conftest.py → tests/brainmapper/conftest.py

@@ -3,7 +3,7 @@
 
 import pytest
 from brainglobe_utils.general.config import get_config_obj
-from cellfinder_core.download.cli import main as cellfinder_download
+from cellfinder.core.download.cli import main as cellfinder_download
 
 test_data_dir = pathlib.Path(__file__) / ".." / ".." / "data"
 data_dir = test_data_dir / "brain"

tests/cellfinder_brainreg/test_integration/test_detection.py → tests/brainmapper/test_integration/test_detection.py

@@ -6,7 +6,7 @@
 import brainglobe_utils.IO.cells as cell_io
 import pytest
 
-from brainglobe_workflows.cellfinder_brainreg.main import (
+from brainglobe_workflows.brainmapper.main import (
     main as cellfinder_run,
 )
 

tests/cellfinder_brainreg/test_integration/test_extract.py → tests/brainmapper/test_integration/test_extract.py

@@ -12,7 +12,7 @@
 from brainglobe_utils.IO.cells import get_cells
 from tifffile import tifffile
 
-from brainglobe_workflows.cellfinder_brainreg.extract import (
+from brainglobe_workflows.brainmapper.extract import (
     extract_cubes as extract_cubes,
 )
 

tests/cellfinder_brainreg/test_integration/test_registration.py → tests/brainmapper/test_integration/test_registration.py

@@ -7,7 +7,7 @@
 import pytest
 from imio.load import load_any
 
-from brainglobe_workflows.cellfinder_brainreg.main import (
+from brainglobe_workflows.brainmapper.main import (
     main as cellfinder_run,
 )
 

tests/cellfinder_brainreg/test_unit/test_analyse.py → tests/brainmapper/test_unit/test_analyse.py

@@ -4,7 +4,7 @@
 
 import pytest
 
-from brainglobe_workflows.cellfinder_brainreg.analyse.analyse import (
+from brainglobe_workflows.brainmapper.analyse.analyse import (
     Point,
     get_region_totals,
 )

tests/cellfinder_brainreg/test_unit/test_tools/test_image_processing.py → tests/brainmapper/test_unit/test_tools/test_image_processing.py

@@ -2,7 +2,7 @@
 
 import numpy as np
 
-from brainglobe_workflows.cellfinder_brainreg.tools import (
+from brainglobe_workflows.brainmapper.tools import (
     image_processing as img_tools,
 )
 

tests/cellfinder_brainreg/test_unit/test_tools/test_prep.py → tests/brainmapper/test_unit/test_tools/test_prep.py

@@ -3,7 +3,7 @@
 import pytest
 from brainglobe_utils.general.exceptions import CommandLineInputError
 
-from brainglobe_workflows.cellfinder_brainreg.tools import prep
+from brainglobe_workflows.brainmapper.tools import prep
 
 # import shutil
 

tests/cellfinder_brainreg/test_unit/test_tools/test_system.py → tests/brainmapper/test_unit/test_tools/test_system.py

@@ -6,7 +6,7 @@
 from brainglobe_utils.general.exceptions import CommandLineInputError
 from brainglobe_utils.general.system import ensure_directory_exists
 
-import brainglobe_workflows.cellfinder_brainreg.tools.system as system
+import brainglobe_workflows.brainmapper.tools.system as system
 
 data_dir = Path(__file__).parents[4] / Path("tests", "data")
 background_im_dir = os.path.join(data_dir, "background")

tests/cellfinder_brainreg/test_unit/test_tools/test_tools_general.py → tests/brainmapper/test_unit/test_tools/test_tools_general.py

@@ -3,7 +3,7 @@
 import numpy as np
 import pytest
 
-import brainglobe_workflows.cellfinder_brainreg.tools.tools as tools
+import brainglobe_workflows.brainmapper.tools.tools as tools
 
 a = [1, "a", 10, 30]
 b = [30, 10, "c", "d"]