0.5.0

This release primarily focuses on the new addition of cell type annotation to DeepCell Label. In addition to implementing the data structures and state machines required to track cell type data, we have introduced a large number of UI and QOL features to maximize the efficiency of cell type labeling, including channel expression calculations / plotting and in-browser training with uncertainty quantification for active learning labeling workflows. Another major addition is an overhaul of canvas rendering by changing the data structure used to store cell-value mapping, allowing for huge increases in performance in actions that involve canvas renders.

🚀 Features

Canvas rendering overhaul, greatly improved speed and overlap support… @ykevu (#502)

… for cell types

What

• Canvas renderings have previously relied on a cellMatrix of size (numValues, numCells), which is both memory inefficient and led to very slow responsiveness due to the reliance on looping through the matrix even in the gpu.js kernel
• We replace cellMatrix with a cell-value mapping { mappings, lengths }. Here is the format of the data structure:
  • mappings: An array of arrays where the i th array contains all cells that map to value i. The array is then padded with zeroes such that each array is the same length
  • lengths: An array where the i th element denotes how many cells value i maps to
• Now, we have a very simple and fast algorithm to color cell label interiors for a given pixel:

value = labelArray[y][x]
numCells = numCellsArray[value]
for (let i = 0; i < numCells; i++) {
    cell = cellMapping[numCells[i]]
    color = colorMap[cell]
    appendColorToOverlap(color)
}

• Anecdotally, on a Keren dataset this led to changing label opacity taking almost 3 seconds to less than 0.3 seconds (more than an order of magnitude speedup)
• The algorithm for anything involving outlines is a bit more complicated, and looks quite ugly, but it should still be significantly faster even with the nested for loops, due to the fact that overlaps should basically always be very sparse.
• Memory-wise, this is also more efficient, even with the padding (which is required for these algorithms to work with gpu.js, since jagged arrays result in incorrect array lookups since the GPU is probably just looking forward by a set amount of memory to a pointer or something)--in the worst, case, even with a certain pixel where every cell in the entire frame overlaps at that point, the mappings array is actually the exact same size as cellMatrix :D

Areas of improvement

• It is likely that the ugly outline algorithms could be cleaned up a bit or made more efficient
• cellSelectionCanvas.js may need some work; it still loops through the total number of selected cells, which can be quite a bit (as much as the number of cells in a canvas), and the max loop iteration is still 5000, so any selection with more than 5000 cells will have invisible selections

Increase visibility of multiselected cells for general use @ykevu (#481)

What

• Addressing #480 and #464
• We change the multiselect canvas rendering to increase its visibility--the width of the white outlines is increased to 2, and the interior is lit up with white 50% opacity pixels. Additionally, an animation can be enabled that alternates the interior on and off to create a light-up flashing effect (there is a warning on the toggle for people sensitive to flashing lights)
• We leverage this new multiselect to add a button + keybinding to multiselect all cells with multiple labels, for reasoning explained in #464
• Some additional UI and keybinding tweaks were made for QOL, such as moving tooltip positions and changing keybindings, most notably hover toggle from Shift to X
• Importantly, this canvas can be extended naturally to account for, say, high uncertainty cells from the in-browser trainer or imported from a model prediction for future features/use

Track opened cell types for UI logic and new features @ykevu (#478)

What

• The cell type panel is open is now tracked in the editCellTypes machine state context
• The cell type that is open determines what celltype you are adding to when you are in add or remove mode now, rather than which button you pressed.
• You can now cycle between the cells within that cell type with buttons / arrow keys if you have the cell info panel open
• Adds keybindings for various cell type functions

Cell info @ykevu (#476)

What

• Adds a "CELL" panel to the right-side UI of cell types that shows, for the current selected cell, the cell ID, the current cell types it is assigned to, the ability to add it to a new cell type and remove labels, and a bar chart showing the normalized channel expressions if the calculation has been made

Feature: drag celltype cards for reordering @ykevu (#469)

What

• Enables the user to drag cell type cards to reorder them--this is purely a UI feature and does not affect the underlying cell types data at all
• Introduces this with the react-beautiful-dnd library

Notes

• The opacity slider is now wrapped in a bit of a gross div as a workaround for the drag behavior not recognizing MUI sliders and thus not allowing you to drag the slider

Export channels on submit/download @ykevu (#468)

What

• Channel names were not exported when a project is exported, meaning that most importantly submitting a project makes the channel names disappear upon loading from the output bucket
• This is now implemented

Add log y scale button for histogram and save layout changes @ykevu (#443)

The UI currently looks pretty ugly and the interfacing between scatter and histogram is still pretty gross (a TODO to potentially also wrap this up in custom plotly buttons could maybe solve this, alternatively hooking into state machine but that might be overkill) but it works. Plotly does indeed have log axes for histograms, I was just looking at the wrong thing.

Also, Plotly related UI stuff is pretty hard to test unfortunately.

Closes #442

Refactor idbWebWorker to separate indexedDB storage into multiple obj… @ykevu (#471)

…ect stores

What

• We noticed that any time some sort of edit is made (including adding a single cell to a cell type), there would be a large amount of disk usage and waiting before the IndexedDB was done updating.
• This was because we were using one object to store all persisted context in the IndexedDB, so even adding one element to the cellTypes json, for example, entailed rewriting the entire raw image, labeled mask, etc. to IndexedDB, which was very undesireable especially with larger images
• By separating into separate object stores instead, we can update cellTypes.json, for example, while leaving the raw arrays untouched

Celltype UI toolbar with new toggles @ykevu (#461)

What

• The cell type controls panel on the left now has a toolbar instead of the giant ADD CELLS button. It includes:

1. Add Cells button (same as before, but as an icon now)
2. Marker Panel open button (same as before, but as part of the toolbar now)
3. Cell type writing mode toggle (Defaults to "overwrite," so that adding Cell 1 which is labeled Type A to Type B removes it from Type A). This addresses #453.
4. Cell type hovering toggle (Defaults to off, but if on, you can hover over cells to view what cell type(s) it is in). This addresses #452.

• There are also tooltips for each button, but a TODO is to add keybindings for them in some form

Add toggle to NavBar to switch between light and dark mode @ykevu (#450)

Adds an icon button to the navbar in the top right to toggle between light and dark mode themes. It turned out that two lines of default dark mode from MUI and adding the cssbaseline component made this essentially good enough out of the box.

Lower priority TODOs:

• The plotly components are still blinding white in dark mode, but they would definitely be a pain to sync up
• Cypress test

Marker panel visualization and editing modal @ykevu (#449)

Adds a popup component that allows the user to visualize the marker panel and make edits. There are still a number of features that would be useful to implement, but I wanted to get this in soon and move on to more important things, since, though it looks pretty, in practicality this feature shouldn't really be used much (hopefully, if our project creation scripts work as intended, not at all) by an annotator such as Martin, other than as a non-interactive reference.

Specifically, the following is implemented by these changes:

• The user can click an icon that pops out the modal with a table visualizing the marker panel, with colors/lit up chips that denote whether the cell type or channel is matched with the current image
• The user can double click cells to make edits to the cell types or channels in the marker panel
• Channels now must be case-sensitive, exact matches to be detected (this can lead to inconveniences for sure, so am still thinking about how best to approach this, but the projects for Martin should have properly mapped channel names and such anyway)

The followin...

0.4.1

🚀 Features

Change raw rescaling to happen on frontend + export original raw unscaled @ykevu (#383)

PR Addressing #373, #379, #381

• Non-png raw inputs are no longer rescaled at all on the backend in loaders.py (.png files are still rescaled to uint8)
• Instead, the frontend performs the rescaling on the raw array, and creates a second array "rawOriginal" which is passed to the arrayMachine.
  • This is a "TypedArray" which can support Uint8, Int8, Uint16, Int16, Uint32, Int32, Float32, and Float64. This will require further testing (by @steveyu323?) for a variety of dtypes since I only had access to uint8 and float64 files
• On export, rawOriginal is zipped instead of the rescaled raw array, so download or submission will give the user the original raw unscaled tiff.

Testing

• So far, I have tested uploading a file from Changhua that is float64, and the rendering on the site was fine. Upon downloading, the X.ome.tiff file inside retained float64 and the unscaled values. After reuploading this downloaded file to DCL, it looked fine as well. Functionality of all the editing and such still worked throughout all of this.

Potential Issues / Enhancements

• The loading times do not seem to have suffered very significantly, but will look into using gpu.js to speed up the reshaping operation (which currently has 4 nested for loops).

Small changes

• REPLACE tool was renamed to COMBINE on its button, tooltip, and instructions. The hotkey and code referencing the tool is unchanged.

🐛 Bug Fixes

BUG: fix bug when no cells on canvas. @ykevu (#397)

What

• Fixes a bug where an error is thrown when trying to load in an image/frame with 0 cells, as a relevant function assumed a non-empty list of cells.

Fix GPU rendering bugs, cell 0 bugs, and bug in saving raw array @ykevu (#384)

PR Addressing #380, #382, and more

• Fixed GPU rendering bugs for the COMBINE and FLOOD tools (#380).
  • These tools also rendered outlines when used and ran into the same issues as those fixed in #376, so similar fixes were applied.
• Fixed bugs related to "Cell 0" edits (#382).
  • If no cell was selected, it was possible to create cells with label 0 on the canvas. Possible from:
    1. Hitting esc while using the BRUSH tool. Fixed by switching to SELECT when esc is hit.
    2. Hitting esc to select nothing and then switching to the THRESHOLD tool. Fixed by selecting a new cell when hitting the THRESHOLD tool if current cell is 0.
    3. Hitting esc to select nothing and then pressing the autofit tool. Fixed by not using the autofit tool if current cell is 0.
• Fixed bug in how raw array was saved in the reshaping.
  • Threshold and watershed tools were broken because these tools pass the raw array to the backend. Originally the unscaled raw array was simply being scaled and the scaling function was called with "as Uint8Array[][][]" but this doesn't work due to how the array buffers work, so a new array is made when the reshaping is done to fix this.

Fix bugs breaking brush tool and unintended undo behavior @ykevu (#378)

Significant Changes (Fix #377)

• Fix bug where brush tool broke when switching to another tab and back.
  • EditMachine.js now sends ENTER_TAB to EditSegmentMachine.js when the SEGMENT tab is selected. EditSegmentMachine will
    accordingly send SET_PAN_ON_DRAG depending on the state it is in so as not to potentially break the state.
• Fix bug where undoing from a pan tool to a non-pan tool (brush) would break the brush.
  • panOnDrag state is now stored in the undo history for CanvasMachine.js, and upon restoring, a CHECK_DRAG event is sent to restore the appropriate state.

Small changes

• Fix bug where using CTRL+Z to undo would always switch to the DISPLAY tab.
  • An event listener was listening for Z presses to switch to DISPLAY tab, so Z was removed from this and a mousetrap binding was instead created to not trigger on combination hotkeys
• Drop uploading now actually uses the user-selected axes
• .zip added to list of acceptable file formats (application/zip was not working on my Windows machine)
• Added hotkey tooltip on select (V)

Fix canvas rendering bugs caused by issues with gpu rendering kernel @ykevu (#376)

What

Fixes canvas rendering bugs, namely #366 and #367, which were both caused by issues in the gpu rendering kernel. Specifically, a check is now called to ensure that a value is contained within the bounds of expected values before trying to render the label associated with that value (out of bounds indexing was resulting not in errors but the cells reappearing upon deletion). Additionally, the max number of cells displayed has been increased to 5000.

These changes were tested by Changhua and me and appear to be working properly.

Why

Users can now render up to 5000 cells at once on a canvas and will not experience bugs when deleting the highest valued cells in a given frame.

Fix bug causing segmentations to be lost upon reload @ykevu (#372)

What

My guess is that after Tom fixed the bug related to adding c to cells, the "feature" field was renamed to "c", so the idbWebWorker was unable to find what segmentation feature was being edited, and segmentations were thus not being tracked.

Why

Users can now reload without losing their segmentation changes (eg. brush adds/removes).

Fix bugs related to cells not properly deleting @ykevu (#368)

What

Addressed the bug from Issue #365, in which after deleting a cell, one cannot write over the deleted cell's area when using exclusion mode.

Additionally, added some snippets of code required for the tests and source code to run on Windows.

Fix

The bug arose because the deleted cell was only removed from the cells matrix but its label persisted in the labels matrix. Added a new function to the Edit class called clean_labels() which deletes any labels that do not correspond to existing cells that is called every time an action is performed that adds masks (eg. brush, flood, etc.). Additionally, a more esoteric bug was found involving overlapped cells reappearing after deletion, which should be addressed by this change as well.

Why

Users can now properly edit areas previously occupied by deleted cells, rendering the delete tool usable in exclusion mode.

🧰 Maintenance

Bump json5 from 1.0.1 to 1.0.2 in /frontend @dependabot (#391)

Bumps json5 from 1.0.1 to 1.0.2.

Release notes

Sourced from json5's releases.

v1.0.2

• Fix: Properties with the name __proto__ are added to objects and arrays. (#199) This also fixes a prototype pollution vulnerability reported by Jonathan Gregson! (#295). This has been backported to v1. (#298)

Changelog

Sourced from json5's changelog.

Unreleased [code, diff]

v2.2.3 [code, diff]

• Fix: json5@2.2.3 is now the 'latest' release according to npm instead of v1.0.2. (#299)

v2.2.2 [code, diff]

• Fix: Properties with the name __proto__ are added to objects and arrays. (#199) This also fixes a prototype pollution vulnerability reported by Jonathan Gregson! (#295).

v2.2.1 [code, diff]

• Fix: Removed dependence on minimist to patch CVE-2021-44906. (#266)

v2.2.0 [code, diff]

• New: Accurate and documented TypeScript declarations are now included. There is no need to install @types/json5. (#236, #244)

v2.1.3 [code, diff]

• Fix: An out of memory bug when parsing numbers has been fixed. (#228, #229)

v2.1.2 [<...

0.4.0

This release adds new types of labels like spots, overlapping cells, and divisions and rearchitects the client-server relationship to manage project data on the client.

The core change is switching to a new file format, where all the files for a project are bundled into a zip. Each file is a modular type of labels, like the segmentation, cells, divisions, and spots. We intend to extend this format, adding further labels that can leverage existing labels or add new types of annotations entirely.

Merge 0.4-dev before releasing 0.4 @tddough98 (#352)

This brings all the work on DeepCell Label so far this year into the main branch for the 0.4 release. As the release drafter hasn't been including updates from PRs into this branch, I added some high level bullet points with the largest features and improvements such as

• overlapping labels (#323)
• spots labels (#317)
• cells labels that can reassign segmented regions without rewriting the segmentation image (#330)
• division labels to replace the lineage (#331)
• replacing the database driven architecture with a client side approach where the frontend stores and updates the labeled data locally (#311)
  • client side undo redo (#325)
  • client side data persistence in IndexedDB (#326)
• end-to-end integration tests with cypress (#314)

Switch master to main @tddough98 (#354)

I changed the default branch to repo to be named main. This updates references to master to main instead.

Add c to cells @tddough98 (#350)

Addresses #347. I decided this was a more critical issue as loading and editing any mesmer results exposes the issue. I did not add "channel mode" editing controls to edit cells across multiple segmentation channels because mesmer does not have consistent cells between channels anyways.

Please user test this (do a handful of actions, undo them, redo them, reload them, combinations of these) on label-dev.deepcell.org!

Fix undoing edits to the segmentation @tddough98 (#349)

This fixes undo after editing the segmentation. It also catches a bug with the cells not persisting on reload immediately after editing the segmentation and adds TODO comments for #347.

Add python-magic-bin @tddough98 (#351)

When setting up the deepcell label repository with Kevin Yu this week, our incoming schmidt software engineer, there was an issue with using python-magic on windows. Installing python-magic-bin as a python dependency solved the issue.

Bump pillow version to address security alerts @tddough98 (#346)

Bumps pillow version to 9.0.1 to deal with dependabot security alerts

Finish merging dev-documentation in 0.4-dev @tddough98 (#345)

This is the same issue as #332 where I forgot to delete a branch and then merged more PRs into the hanging branch.

Reorganize subprojects @tddough98 (#343)

This renames the visualizer folder to frontend and the deepcell_label folder to backend. It also moves resources specific to the Python backend into the backend folder.

Fix mesmer loading @tddough98 (#344)

This fixes an issue with loading results from mesmer from deepcell.org/predict. It came from slight differences in dimension order between the segmentation pipeline and the mesmer pipeline, as documented here: vanvalenlab/kiosk-frontend#202

Add developer documentation @tddough98 (#342)

This adds docstrings to all state machine files and adds ARCHITECUTE.md to cover the high level architecture of the frontend and backend, with closer attention to the tree of state machines.

Fix loading from deepcell.org/predict @tddough98 (#341)

This addresses some quick fixes for loading from deepcell.org/predict (now online!).

The first issue was from an incorrect test case that caused loading zips of tiffs with batches to drop the X dimension.
The second issue was assuming the smallest dimension is the channels when axes are not provided, causing tracking timelapses to use their time axis as channels instead.

Add label zip format documentation @tddough98 (#337)

This adds a markdown file explaining the contents of the project zips created and exported by DeepCell Label. It also includes a section on the supported input for /api/project to create these zips.

There is a new documentation folder to store this and other markdown files.

Handle batches when loading TIFFs and zips of TIFFs @tddough98 (#319)

As we support batch dimensions on the kiosk-frontend, we need a way to handle differently shaped input images and predicted labels from the predict page.

• Adds back a dimension order for the image file
• Checks if tiff filenames contain batch_XX when loading a zip of tiffs and groups and loads files by their batch number

Add forceLoadOutput URL parameter @tddough98 (#334)

To better support the Anolytics job workflow, we add a query string forceLoadOutput. When the query string is true, the page loads from the deepcell-label-output bucket. If the project is already in the local browser DB, we show a modal to confirm overwriting the local data. When the project has not been submitted, we show a missing project page with instructions to submit the project first.

For the backend to receive the query string to load from the output bucket, CloudFront needs to forward query strings, which it does not do by default. Setting the Origin request policy for the Elastic Beanstalk backend to AllViewer will forward all cookies, headers, and query strings. I will document this and all other cloudfront/s3/elastic beanstalk configuration in an upcoming deployment guide.

Bug fixes

• prevent canvas from moving when changing tabs
• bug fix for undoing an action and then reloading before doing another action
• fix for loading PNGs with I or F mode (32 bit single channels)
• fix for setting dynamic range to 0 in grayscale mode

Add deployment guide @tddough98 (#335)

This adds a markdown file to walk through creating and configuring the cloud resources to put DeepCell Label online. As I'll be leaving next month, it's intended to serve as a resource for future work on the project to make sure it can be redeployed and updated. As we discover new required configuration and switch the resources we use, we should update this guide with the up to date steps for deployment.

Update layout, instructions, exporting and remove visualizers @tddough98 (#333)

This contains some of the final changes needed before cutting the 0.4 release of DeepCell Label. It includes

• updating the in app drop down instructions
• move the display controls into a tab
• switching between tabs when using a keybind from another tab
• fixing bugs with too many WebGL contexts by having a shared GPU object with a single context

The changes are online at label-dev.deepcell.org

Tweak .env.example to avoid environment variables set to "" @tddough98 (#336)

When a line like SQLALCHEMY_DATABASE_URI= is in .env, the environment variable is set to "" instead of using the default value. By commenting these fields out in the example environment, this common setup issue is avoided.

This also switches fmd_config.cfg.example to fmd_config.cfg in .env.example to further prevent using fmd_config.cfg.example accidentally.

Replace lineage with divisions and add division editing @tddough98 (#331)

Overview

This overhauls how divisions labels are encoded, changing to a event focused format divisions.json

[{"parent": 1, "daughters": [2, 3], "t": 1}]

from the existing lineage.json format

{
  "1": {
    "label": 1,
    "frames": [0],
    "frame_div": 1,
    "capped": true,
    "daughters": [2, 3],
    "parent": null,
  },
  "2": {
    "label": 2,
    "frames": [1],
    "frame_div": null,
    "capped": false,
    "daughters": [],
    "parent": 1,
  },
  "3": {
    "label": 3,
    "frames": [1],
    "frame_div": null,
    "capped": false,
    "daughters": [],
    "parent": 1,
  },
}

Loading

The loader on the backend now looks for divisions.json in the input zip. For backwards compatibility, it supports loading lineage.json from .trk files and converts the lineage to divisions.json.

Editing

Editing the divisions is handled by divisionsMachine. There is another separate editDivisionMachine to manage the UI state of editing divisions, like whether the user is being prompted to add a daughter to a division, which then sends events to divisionsMachine. Once the divisions change, we send a DIVISIONS event to the divisions event bus.

When the cell labels are edited, we forward these events (REPLACE, SWAP, or DELETE) so the divisions can also replace, swap, or delete the same cells presen...

0.3.0

This release adds GPU acceleration for client side image processing and adds controls to track cells in live cell movies. Add track=true to any URL to display the tracking controls and add division labels to any project. Add download=true to a project URL to display a download button instead of a submit button. Downloading with track=true in the URL will download a .trk file and otherwise will download an .npz file.

There is also a new API route to add channels to an existing project. Posting to /api/raw/{projectID} with an attached .npz will add the array in the .npz as an additional channel for that project. Make sure that the attached array is 4-dimensional with axes order ZYXC and that there is only one channel in the attached array.

The release also improves the development environment in the repository by adding pre-commit hooks, updating the Python and node versions tested, and updating the Elastic Beanstalk support to Amazon Linux 2.

Wrap GPU.js kernel functions in templates @tddough98 (#299)

Our minified production code replaces expression like if (x) { y } to x && y, breaking how GPU.js compiled functions into a kernel. By wrapping the functions in ticks, they won't be minified and all the syntax is preserved for GPU.js when it compiles the function at runtime.

These issues on the GPU.js repo detail this:
https://github.com/gpujs/gpu.js/issues/152
gpujs/gpu.js#584

Fix canvas states for changing cursor @tddough98 (#298)

After changing the canvas in #294 to be easier to mock, the states changes slightly but were not updated in the Canvas component, so the cursor does not change to a grabbing hand as before. This updates the state names in the component.

Fixes for tracking before release @tddough98 (#297)

This adds in a couple of fixes, mostly for tracking jobs, including

• enabling uploading .trk from the homepage
• removing an extra FrameSlider above the image controls
• hiding the Frame label when FrameSlider is hidden
• fixing the colors provided to the Division components to be provided in hex format instead of a list of [r, g, b]

Fix GPU.js handling of WebGL @tddough98 (#296)

With the most recent pull request #294, I noticed that Firefox and Chrome handled the new canvas as expected, but Safari was not behaving correctly. It seems that that Safari only received WebGL2 support in Safari 15 released a few months ago with macOS 15: https://caniuse.com/webgl2.

I fixed the initialization of the GPU-accelerated contexts to check which context is available in the browser and set one or the other up as needed. I also had issues with the WebGL canvases clearing their data after being drawn, causing it to appear once and then to appear blank in later renders. To fix this, I drew the WebGL kernel output onto a separate canvas as soon as it is created, and this other canvas with a standard 2d context does not get cleared when drawn. This strategy works just as well for WebGL2, WebGL and canvas based kernels, so I implemented it as the standard approach regardless of WebGL support

Add tracking and QC available on label.deepcell.org to main branch @tddough98 (#282)

The deployed app on label.deepcell.org is ahead of the main branch as it contains tracking and quality control features.

Code to support tracking includes

• actions in label.py like add_daughter, remove_daughter, and replace_with_parent,
• tests for these actions
• Tracking UI components in visualizer/src/Controls/Tracking
• trackMachine.js to send events to edit the lineage
• toolMachine.js to switch between trackMachine to edit the lineage and segmentMachine to edit the segmentation

Code to support quality control includes

• QC components in visualizer/src/Controls/QualityControl
• qualityControlMachine to make judgments on projects

Other small improvements include

• adding ETags to routes to avoid unnecessarily resending data,
• cleaning label arguments for actions to be valid integers between 0 and a new label

Add prettier pre commit hook @tddough98 (#290)

I added our prettier setup for formatting the javascript side of the project to the pre-commit configuration and action. I also updated the configuration to be closer to prettier's recommended style, only overriding to use single quotes instead of double quotes as we do for the Python formatting as well. Specifically, I removed "semi": true, from .prettierrc because that's already the prettier default and I removed "arrowParens": "avoid", after reading prettier's reasoning here.

I ran the updated prettier config on all files, which is why there are so many changed files. The only changes of note are in .prettierrc and .pre-commit-config.yaml.

Add pre-commit hooks to automatically fix Python code @tddough98 (#289)

Here we enforce autoformatting of Python code with black.

There are three contenders for python autoformatters: autopep8, yapf, and black. Of these three, black is the most popular with over 23k stars on github. A number of high profile libraries are now using black, including Django, PyTest, tox, pandas, sqlalchemy, and virtualenv.

The main selling point of black is that it is very opinionated so that black makes all the decisions about code formatting. The only configuration I'm aware of is max line length (default is 88), and whether to replace single quotes with double quotes (default is to replace with double quotes). I left both of these on their defaults.

I've also added an action to run black on pull requests to enforce its usage. The action checks that the code is consistent with black, and if not, it adds a commit to the PR to autoformat the code.

Add testing for python 3.9 and 3.10 @tddough98 (#288)

In #287, we drop support for Python 3.6, but did not test for the newer Python releases 3.9 and 3.10. Here we add them to our test matrix to make sure our codebase works for these versions for when we'll need to switch to them down the road.

However, pytest-pep8 is incompatible with Python 3.10. Python 3.10 requires at least pytest 6.2.4 per this issue and pytest-pep8 is incompatible with pytest 6, as seen in this error message

Direct construction of Pep8Item has been deprecated, please use Pep8Item.from_parent.
See https://docs.pytest.org/en/stable/deprecations.html#node-construction-changed-to-node-from-parent for more details.

As pytest-pep8 is not an actively mantained project and the last release was in 2014, I switched to flake8 and pytest-flake8. Flake8 checks for pep8 as well as pyflakes and circular complexity, keeping the style checks and adding some error checks.

In summary, this PR contains:

• additions to tests.yaml to test Python 3.9 and 3.10
• switches out pep8 checks for flake8 checks
• fixes issues highlighted by flake8

Migrate Elastic Beanstalk to Python 3.8 running on Amazon Linux 2 @tddough98 (#287)

Our Elastic Beanstalk instances have been running on a deprecated platform Python 3.6 running on 64bit Amazon Linux. Here, we migrate to a supported platform Python 3.8 running on Amazon Linux 2, updating the EB configuration to work with the new platform.

This PR also drops support for Python 3.6, which reaches its end of life and will stop receiving security updates in 2 weeks on December 23 2021. I bumped all our dependencies and removed any references to Python 3.6. I did not add Python 3.9 and 3.10 to our testing workflow as there are only Python 3.7 and 3.8 platforms available on Elastic Beanstalk at the moment.

Add instructions on .env to README @tddough98 (#285)

Issue #284 had issues with setting environment variables. Adding some instructions to the readme on how to use the example .env.example will point users to the resources to configure their envvars correctly.

Add route to add channels to an existing project @tddough98 (#283)

Here we add an API route to add additional channels to a project. This supports displaying predictions from Impartial which creates probability maps from scribbles created in DeepCell Label, so these maps need to be added later in the project lifecycle. By showing the Impartial results in the same project, a user can compare their scribbles to Impartial predictions and update the scribbles to repeat the process.

Create an npz with the new channel with shape (frames, height, width, 1) and post the channel to the /api/raw/{projectId} route with the npz as an attached file.
For example, with curl this looks like

curl --location --request POST 'label.deepcell.org/api/raw/EXAMPLE-ID' --form 'file=@"newChannel.npz"'

Areas for improvement could include loading channels from more filetypes, specifying a flexible dimension order, and addin...

0.2.0

This release marks the complete rebuild of the client with React! It has feature parity with 0.1 release, adds features like more control over image display, and improves the usability and performance of the user interface.

🚀 Features

Rework DeepCell Label with React @tddough98 (#248)

This PR provides a complete rework of user interface for DeepCell Label. We use React to build the UI. This PR also build upon previous work with XState and integrates it with React so that state is managed by actors instantiated by XState and React components use hooks to access and render that state into UI elements.

We also include some changes to the backend to better serve the React-based frontend, namely changed

React components

We bootstrapped this React app with create-react-app, so this PR contains the tooling and dependencies installed by CRA.

The UI is made up of Controls on the left, and a Canvas that fills the available space to the right.

Canvas

The Canvas folder contains everything for the interactive canvas. The top-level Canvas component manages styling and event handling, the RawCanvas folder handles adjusting and showing the raw images, the LabeledCanvas folder handles overlaying the labels on the raw image, and the ToolCanvas folder has components to show tools like brush or threshold.

canvasUtils has functions to manipulate ImageData objects.

Controls

The ImageControls folder has the controls on the far left used that control what and how images are shown. There is a SubmitButton on top, then a FrameSlider, then the LabelControls, then the RawControls.

The LabelControls include a FeatureSelect, a OutlineToggle, and a OpacitySlider.
The RawControls include a ColorModeToggle to switch between viewing one or multiple channels, then either GrayscaleControls when viewing one channel or RGBControls when viewing multiple channels.
The GrayscaleControls have a switch to invert the channel and sliders for range, brightness, and contrast, and the RGBControls have a LayerController for each channel that controls the layer's color, its range, and whether the layer is shown.

The Toolbar folder has the controls just to the left of the canvas that manage how we edit the labels (like picking a tool, selecting labels.
The UndoRedoButtons are on top, then the Toolbar, then the ActionButtons (swap/replace), and finally a palette that shows if there is a foreground/background selected.

Other components

Other simple components include the Navbar, the Footer, and the Instructions.

State management with XState

To provide the state machines to components throught the React, we define a top level ServiceContext component that wraps the whole application. In this component with define a Context that provides the top level state machine to any child of ServiceContext. These children can access the state through hooks defined in this module. At a high level, we can image the React component tree and the XState actor hierarchy as two separate trees that touch each other through these hooks.

Components subscribe to some part of these state machines and rerender when the state updates.

Actor hierarchy

The root actor is still deepCellLabelMachine, which fetches initial project data, and then spawns canvas, image, tool, undo, and api.

canvas tracks where the canvas is within the image, such as zoom, position, and mouse
position. It also manages mouse events and forwards them to tool.

image tracks the current frame, and spawns labeled and raw. feature manages the current segmentation, and spawns a feature machine to fetch the data for each segmentation. raw manages the current raw image, and spawns a channel machine to fetch the data for each channel. raw also spawns two "color modes" grayscale and color and passes them all the channel actors. The grayscale color mode tracks which single channel is displayed in grayscale mode. The color color mode spawns a layer for each displayed channel and each layer manages its own image adjustment settings for that channel.

tool tracks the current tool we're using to interact with the labels. The tool will spawns an actor for the current tool that has custom behavior for mouse events sent from canvas. The tool actor also tracks the selected labels (the foreground and the background).

api handles requests to the Label backend that edit the labels. The events that can edit labels include EDIT, BACKEND_UNDO, and BACKEND_REDO. Once api gets a response from the backend, it sends a EDITED event to the labeled machine to reload the data that changed.

undo can manage any machine that responds to SAVE and RESTORE events. When undo receives an ADD_ACTOR event, it creates a history machine for that actor that will send a SAVE event to the actor when we edit the labeling, and the actor must respond with a RESTORE event that contains the state that the actor should restore on undo/redo. Later, when undo is undoing or redoing an edit, the history machine will send that RESTORE event back to the actor and the actor is responsible for restoring their state based on this event.

Hotkeys

Hotkeys are implemented as invoked callbacks, where the hotkey listeners are setup when entering the states/machines that use the hotkeys and are torn down on exit.

Project management

We've added prettier to the project, an opinionated code formatter than enforces consistent style. Prettier formats javascript, css, html and other browser focused formats, complementing out existing linting for python. We also use husky and lint-staged to add a pre-commit hook so any code committed to the repo will be formatted with prettier.

Typescript has been added as a dependency, but we have no types defined yet. Adding typescript now improves the intellisense for the project in VSCode and will make it easier to add types later.

Changes to Flask backend

The backend changes include:

• a new URL based approach to creating projects, which includes
  • loading raw assets and labeled assets separately
  • support for loading from ZIPs, where separate files in a zip are treated as different channels or features
  • reshaping input arrays based off an 'axes' string like 'ZYXC'
• routes to serve project data separately, rather than in a bundled JSON, including

Here is a table of the new routes, what data they serve, and the data format:

Route	data served	data format
/api/project	project metadata, like image dimensions	JSON
/api/raw	raw image slices	grayscale PNG
/api/labeled	labeled image slices	color PNG
/api/array	array of label at each pixel, with label borders in negative	compressed JSON
/api/instances	label metadata at the moment, only the frames each label is present in used on frontend to know which labels in each feature	JSON

Issues

@willgraf has reviewed outstanding issues and this PR closes these issues

Fixes #11
Fixes #52
Fixes #96
Fixes #97
Fixes #99
Fixes #102
Fixes #110
Fixes #115
Fixes #131
Fixes #132
Fixes #140
Fixes #155
Fixes #158
Fixes #169
Fixes #171
Fixes #203
Fixes #234
Fixes #244
Fixes #220

Add landing page @tddough98 (#268)

Here we add a landing page to DeepCell Label with an introduction to the app, a dropzone to upload TIFFs, NPZs and PNGs, and dropdown menu with two example files. The landing page is a more user-friendly entry point to using Label, giving users another option than the Label API to create projects.

Here's an overview of the changes:

• created a base Loader class and separate URLLoader and FileLoader implements
• the app now has two pages: / for landing page and /project for interacting with a project
  • this is a breaking change for Anolytics jobs as the we'll have to add /project to our job URLs
• the project page can show a download button instead of a submit button by adding a download=true URL parameter

There are also some smaller bug fixes and improvements included:

• loading PNGs is more flexible and robust, now handling palette-based and grayscale PNGs
• the watershed state machine no longer exits its clicked state immediately due to the state selecting the foreground label
• controls for white channels are now shown in black instead so they don't blend into the background

🐛 Bug Fixes

Fix tool keybinds @tddough98 (#265)

When using tool hotkeys to switch out the brush and threshold currently causes UI crashes as the tool changes before the BrushCanvas or ThresholdCanvas is removed, causing these canvases to access a brush machine...

0.1.0

The first release of deepcell-label checkpoints all of the great work by all contributors to the project before migrating from Flask-based Server Side Rendering to a React frontend with a pure JSON API backend.

Special thanks to all the contributors to the project that got us this far, especially @tddough98, @geneva-miller, @jannieyu, and @enricozb.