Code and GUI improvements, NoRMCorre and additional file formats support, etc.

Main features and changes:

• Increment to CIAtah version 4.5.7.
• NoRMCorre is now supported within main CIAtah GUI and as a switch within turboregMovie (to keep support for surrounding motion correction processing code). This gives users another option for motion correction with non-rigid registration is needed.
  • Note this runs a custom forked version of NoRMCorre in which I moved all code within a normcorre package and re-wrote to make compatible to improve namespace and integration: https://github.com/bahanonu/NoRMCorre.
• Improved support for Bio-Formats files (better file handling, ability to grab data via internal series name, etc.). Additional integration into main CIAtah GUI in next release.
• The main module for pre-processing (modelPreprocessMovieFunction) has several new options, speed/code improvements, NoRMCorre integration, etc. Further support for read-from-disk pre-processing in next release.
• Added support for SlideBook (.sld) files.
• Support for MATLAB's new Name=value syntax for passing Name-Value arguments to functions.
• General switch in many functions to fast thresholding.
• Switching away from using nanmean to mean('omitnan'), to be completed in next release.
• Improved loading of CIAtah directory (with ciapkg.loadBatchFxns).

New functions

• ciapkg.bf.getSeriesNoFromName - Returns series number for Bio-Formats series with name given by user.
• ciapkg.bf.dispFileSeries - Returns series number for Bio-Formats series with name given by user.
• writeMatfileToHDF5 - Writes a MAT-file dataset to HDF5 by reading the matrix in parts to avoid large overhead.
• mergeStructs - Copies fields in fromStruct into toStruct, if there is an overlap in field names, fromStruct overwrites toStruct unless specified otherwise.
• writeDataToMatfile - Converts 3D movie matrix stored in a file (HDF5, TIF, AVI, NWB) into a MAT-file for use with larger-than-memory processing.
• getNoRMCorreParams - Automatically returns default NoRMCorre parameters or ask user to input parameters.

Updated @ciatah

• ciatahMainGui - Improve speed of movie display by better passing of movie information to ciapkg.io.readFrame. If processed and raw movie are the same length, sliders move in sync. By default use 'painters' renderer as that can produce smoother rendering than opengl. Added file sizes to the folder files preview. Reset movie dimension on folder select change so movie previews run without outputting initial error (movies would still preview, users just might be confused by the warning). Feature to allow preview of HDF5 datasets. Expanding to Bio-Formats and other types of data with internal structures for data. Added ability to open folder in OS GUI, e.g. Windows Explorer. If manual input for filtering folders set back to no filter to avoid GUI loops. Make sure preview data and open folder buttons are associated with correct previewDataHandle and openFoldersHandle handles.
• getRegistrationSettings - Add additional black background support. Added largeMovieLoad setting.
• modelAddNewFolders - Add additional check for when user adds blank folder paths (NOT empty folders) and remove them. Empty folder check update.
• modelExtractSignalsFromMovie - CELLMax support for loading prior settings. Update to EXTRACT support: added additional options, do not automatically eliminate summary section, and more. Switch order of mergeStructs and supplying cell radius to EXTRACT, else empty vector can be passed depending on user input.
• modelGetSignalsImages - Ensure when loading cell extraction CIAtah-style that only MAT files are loaded.
• modelModifyRegionAnalysis - Misc code standard updates.
• modelPreprocessMovieFunction - Change all nested and local functions to start with "nestedfxn_" or "localfxn_" prefixes for clarity. Added option to fix "broken" frames, e.g. frames in which the camera gives out garbled or improperly formatted data. Fix with mean of the movie to reduce effect on downstream analysis. Disable local function "getMovieFileType" in favor or CIAtah package function. Added option to place prefix on all output files. Detrending now does everything within nested function instead of calling normalizeMovie() to reduce memory overhead. Added support for MAT-file based processing of movies. Add additional black background support. Added largeMovieLoad setting support, improve memory allocation for select movies. Make sure readFrame is compatible with all HDF5 dataset names. NormCorre integrated into the pipeline for end users.
• modelSaveImgToFile - Update PaperPosition to auto to avoid error "Positioning Figure for ResizeFcn. Set PaperPositionMode to 'auto' (match figure screen size) to avoid resizing and this warning." Further change to PaperPositionMode to avoid errors.
• modelVarsFromFiles - Enforce that NWB loading only looks for .nwb files and ensure that manual or automatic classifications are loaded when loading NWB files. Ensure when loading cell extraction CIAtah-style that only MAT files are loaded.
• viewMovieRegistrationTest - Updated so more settings are pulled from CIAtah object settings.
• viewSubjectMovieFrames - Update code standards.

Updated +ciapkg

• signalSorter - Misc updates.
• downloadCnmfGithubRepositories - Update URLs to include a backup URL for cvx since would often timeout.
• appendDataToHdf5 - comments updated.
• downsampleHdf5Movie - Added Bio-Formats support. Switch chunking to automatic by default, generally improved reading from data later than small chunks. Improved Name-Value comments.
• readHDF5Subset - minor updates.
• computeImageFeatures - Make default thresholding fast thresholding. Improved annotation of code and added options.fastThresholding. Also by default plots are not made. Refactored extra features code and finding centroid to avoid unnecessary function calls, speeding up code.
• filterImages - Make default thresholding fast thresholding.
• findCentroid - Make default thresholding fast thresholding. Update to ensure threshold based on max value if user inputs non-normalized (e.g. distribution forced to zero to one) thresholded images.
• groupImagesByColor - Added fast thresholding option and other threshold image options.
• thresholdImages - manageParallelWorkers now passed options.waitbarOn value to reduce command line clutter if user request in thresholdImages.
• getFileInfo - Add trial # to output.
• getFileList - Improved support multiple directory input. Added folder filter.
• getMovieFileType - Added MAT-file support. Add SlideBook Bio-Formats support.
• getOptions - In 2021a MATLAB introduced Name=value syntax for passing name-value arguments. These are passed within varargin as a string array instead of a char array as was the case with comma-separated syntax. getOptions now checks for isstring in addition to ischar to add support for this syntax to CIAtah functions. See https://www.mathworks.com/help/matlab/release-notes.html?rntext=&startrelease=R2021a&endrelease=R2021a&groupby=release&sortby=descending&searchHighlight=#mw_77c0932f-4a31-44e6-b550-db5a736c2de3.
• load_tif_movie - Speed improvements related to imfinfo calls. Added tiffread support and ScanImageTiffReader support (not user accessible currently).
• loadDependencies - Added PatchWarp (https://github.com/ryhattori/PatchWarp). Code clean up. Updated NoRMCorre to use https://github.com/bahanonu/NoRMCorre as that is a package version, cleaner namespace. Added CIAtah utilities repository (https://github.com/bahanonu/ciatah_utils) to separate outside code from main repository. Add SlideBook .jar reader to Bio-Formats download. Ensure NoRMCorre is in default options. Ensure bfmatlab_readers directory exists, else websave errors occur.
• loadMovieList - Speed improvements related to imfinfo calls. Add option to load movie into a MAT-file. Improve Bio-Formats support, including using bfGetPlane to partially load data along with adding ndpi support. Add SlideBook Bio-Formats support. Remove local function getMovieFileType to force use of CIAtah getMovieFileType function. Updated getIndex call to avoid issues with certain Bio-Formats. Ensure getting correct Bio-Formats frames and loadMovieList output. Hide TIF warnings.
• loadSignalExtraction - Added option to load manual sorting as well and output as standard vector.
• manageParallelWorkers - Added displayInfo option.
• readFrame - Ensure subfxn_loadFrame nested function outputs frame data (even if file does not contain the requested frame, e.g. empty matrix).
• saveMatrixToFile - Use Fast_Tiff_Write to write out TIF files instead of saveastiff by default. Give option with options.tifWriter.
• computeManualMotionCorrection - Check if inputs are sparse, convert to single. By default use 'painters' renderer as that can produce smoother rendering than opengl. Update to make sure titles update correctly for test images. Add renderer option to allow users to choose painters or opengl.
• turboregMovie - Refactor code to remove need to transform movie into cell array, performance and memory improvements. Added NoRMCorre support, function will eventually be merged with "registerMovie" or renamed to indicate support for multiple algorithms. Reason to integrate NoRMCorre into this function is takes advantage of existing pre-processing pipeline and integration with other algorithms (e.g. cross-session). Check if inputs are sparse, convert to single. Use custom bahanonu NoRMCorre that is within a package and update to use reference picture instead of template. Updated to which('normcorre.normcorre') from which('normcorre') since normcorre now inside a package. Additional NoRMCorre support and getNoRMCorreParams checking before running NoRMCorre.
• downsampleMovie - Update information for users/comments.
• normalizeMovie - Switch away from using nanmean to mean('omitnan'). Detrend flight code ref...

Improved initialization, movie processing selection, GUI speed-ups, and misc. improvements

Main features and changes:

• Increment to CIAtah version to 4.2.0.
• GUI improvements to ciapkg.motion_correction.computeManualMotionCorrection that significantly speed up GUI responsiveness, e.g. when doing hundreds of sessions.
• computeMatchObjBtwnTrials now supports manual alignment of sessions before automated alignment and matching of cells across sessions.
• Replace calls to suptitle with ciapkg.overloaded.suptitle due to function being dropped from newer Matlab releases.
• Several fixes to improve usability and initializing CIAtah.
• Addition of detrending function to API, alias for existing functionality.
• Misc fixes for edge cases and other improvements.

New functions

• ciapkg.api.getMovieFileType - API access for getMovieFileType.
• ciapkg.api.getMovieInfo - API access for getMovieInfo.
• ciapkg.api.readFrame - API access for readFrame.
• ciapkg.movie_processing.detrendMovie - Detrend a movie to account for photobleaching, etc. Alias for call to normalizeMovie.

Updated @ciatah

• ciatah - Misc code fixes to conform to better Matlab language standards.
• ciatahMainGui - Additional check only for movie previews of supported movie files. "Folder files" list selection additional regexp support. Changed so that 'Start selected method' button selects the correct folders like occurs when pressing enter. The folder list will no longer reset when selecting certain GUI elements that should not impact folder list.
• computeMatchObjBtwnTrials - Added manual alignment option before automated for cases in which there are large shifts or rotations along with changes in bulk cells identified that might be hard for automated to align.
• modelDownsampleRawMovies - Change TIF finding to split apart the information file before using for regular expression. Additional updates to handle CIAtah v4.0 API switch.
• modelModifyRegionAnalysis - Fix modelVarsFromFilesCheck and modelVarsFromFiles recursion edge case.
• modelPreprocessMovieFunction - Updated suptitle to ciapkg.overloaded.suptitle. Fix "No movies" dialog box displaying when movies successfully run. Add modelPreprocessMovieFunction settings saved in CIAtah class to saved output file for later retrieval. For selecting turboreg crop coordinates, switched to ciapkg.io.readFrame when single frame is requested as this is faster and avoids long read times of all frame information as occurs in certain types of TIFF files or hard drives. Misc code fixes to conform to better Matlab language standards.
• modelVerifyDataIntegrity - Updated suptitle to ciapkg.overloaded.suptitle.
• runPipeline - Eliminate the close all figure when initializing the main GUI. Unexpected for some users and undesirable in certain cases.
• viewMovie - Updated suptitle to ciapkg.overloaded.suptitle.
• getRegistrationSettings - Handle "User selects specific value" going into infinite loop. Allow cancel option to prevent loop.
• initializeObj - Wrap calls to diverging_map to avoid potential ciatah initialization errors on macOS.
• modelPreprocessMovieFunction - Add verification that turboreg MEX function is in the path.
• viewMovieRegistrationTest - Additional updates to handle CIAtah v4.0 API switch.

Updated +ciapkg

• signalSorter - Updated suptitle to ciapkg.overloaded.suptitle. Also limit number of GUI re-runs to prevent infinite looping. Fixed issue with gca.XRuler.Axle.LineStyle being an empty object when axes created, leading to failure to assign and error loops. Make all openFigure to ciapkg.api.openFigure.
  downloadGithubRepositories - Update _external_programs to call ciapkg.getDirExternalPrograms() to standardize call across all functions. If multiple repos requested but one already downloaded, previously exited before downloading the other repos.
• readHDF5Subset - Fix display of information on error.
• createPeakTriggeredImages - Ensure only unique signal peaks used.
• fftImage - Misc code fixes to conform to better Matlab language standards.
• getMovieFileType - Update all strcmp to endsWith to ensure finding file extension as there are cases where software will export metadata into files with naming schema like NAME.tif.xml for NAME.tif and this can cause issues when using strcmp without endsWith-like checks. Update docs. Added varargin support.
• getMovieInfo - Directly call loadMovieList and bypass ciapkg.api.
• loadMovieList - AVI now read(...,'native') is faster.
• jsonRead - Fix edge case where the end of the imported string is a space, causing jsondecode to fail.
• loadMovieList - AVI now read(...,'native') is faster.
• manageMiji - Added forceLoad option.
• manageParallelWorkers - Added nCoresFree option for users to set the number of cores to remain free. Added ability to input just the number of workers to open as 1st single input argument that aliases for the "setNumCores" Name-Value input, still support other input arguments as well.
• modelAddOutsideDependencies - No longer use Fiji-based Miji.
• readFrame - AVI now read(...,'native') is faster.
• resetMiji - Update to use manageMiji to setup Miji, avoid certain issues. Also search for imagej instead of Fiji path.
• computeManualMotionCorrection - Change number of pixels to translate per click and UI improvements. Improved pre-allocation, minimize GUI slowdown for large inputs over time. Improve speed of GUI and responsiveness. Force altInputImages to initialize as input class type to avoid automatic conversion to double (save space/ram). Significant speed improvements by removing certain calls to figure, axis, imagesc, etc. that slow down UI over time when many sessions aligned.
• downsampleMovie - Update for Matlab standards.
• viewAddTextToMovie - Added varargin for future additions.
• changeFont - Updated so doesn't change Axes backgroundcolor when changing font color, only Axes text.
• displayMatchingObjs - Updated suptitle to ciapkg.overloaded.suptitle
• openFigure - Option to add a title to the figure, e.g. "Figure 1" becomes "Fig. 1 - Lorem ipsum". Also make user accessible change figure background color.
• playMovie - AVI now read(...,'native') is faster. Also add primaryTrackingPointNback option to allow a trailing number of points from prior frames to appear.
• exampleFxn - updated for better help.
• getMovieFileType - Added support for checking if a cell was accidentally input instead of a string path along with verifying that input was a string. Added support for oir and czi/lsm Olympus and Zeiss file formats that already was in loadMovieList.
• computeManualMotionCorrection - Improved pre-allocation, minimize GUI slowdown for large inputs over time. Improve speed of GUI and responsiveness.
• turboregMovie - Additional display of information. Added verification that turboreg MEX function is in the path.
• changeFont - Updated so can update font size, name, color more independent of one another.
• downloadCnmfGithubRepositories - Update display of information if CVX is not automatically found.

Updated +ciapkg.api

• convertInscopixIsxdToHdf5 - Update +image package to +inscopix.

CIAtah 4.0

Main features and changes

• All CIAtah functions have been moved to +ciapkg folder so they are part of the ciapkg package or sub-packages. The ciapkg folder has been removed.
  • This improves namespace handling (e.g. eliminates naming conflicts) to make it easier to integrate CIAtah with other packages or when using the GUI or API with user scripts.
• Major version incremented due to large change in organization of many CIAtah/ciapkg functions and thus the underlying API.
  • All relevant CIAtah functions can now be called via ciapkg.api.FUNCTION_NAME, which allows for a central location to call all functions as well as allowing importing of relevant CIAtah functions into a function if want to reduce visual clutter. Full function can be found in respective +ciapkg sub-packages.
• Updated to handle CIAtah v4.0 switch to all functions inside ciapkg package and ciatah class.
• Current support is MATLAB 2019b or later due to function of nested packages.
  • See https://www.mathworks.com/help/matlab/matlab_prog/upgrade-code-for-r2019b-changes-to-function-precedence-order.html#mw_8a31cdce-6dc1-4ae0-ae4a-ca0fbae7d1ff.
• Updates to certain functions for speed, UI, and other improvements.
• Move all NWB functions to ciapkg.nwb sub-package.
• Re-organized several File Exchange functions and removed unused ones.

Example of movie slider and cell-extraction in the main GUI.

Updated @ciatah functions

• ciatahMainGui - Added support for file list when selecting a specific folder. Font scaling support.
• modelPreprocessMovieFunction - Fix to handle folders with no files.
• modelExtractSignalsFromMovie - Added nwbpkg support. Updated EXTRACT support.

Updated +ciapkg functions

• Move Turboreg to _external_programs
• loadBatchFxns - Improved handling of external programs both in adding and removing from path. Additional support for removing specific packages that are not always needed.
• cropMatrix - Updated to allow specifying the size of the rectangle.
• getFileList - Added support to exclude adding the input directory to each file path.
• getMovieFileType - Added HDF5 capitalized file extension.
• getOptions - Ensure no warnings are shown. Added fix to handle users calling API version of getOptions with getOptions variable input arguments.
• loadDependencies - Added Turboreg (moved from within ciapkg) to make explicit that this is an external program. Update Schnitzer NWB package to new URL (https://github.com/schnitzer-lab/nwbpkg). Default to download GRAMmar.
• loadMovieList - Added HDF5 capitalized file extension. Ensure that loadMovieList has all output arguments set no matter return conditions.
• computeManualMotionCorrection - Added acceleration based on rapid user clicks or holding down direction keys. Add support for 90 degree quick rotation, fix passing of flip dims. Change number of pixels to translate per click and UI improvements. Improved pre-allocation.
• turboregMovie - Additional matlab disk normalizeType options. Additional display of information.
• normalizeVector - Added percentile name-value options for soft zero to one.
• playMovie - Update to avoid caxis with rgb movies, making them hard to view.
• setupNwb - Update to include check for each of the NWB dependencies, moved from saveNeurodataWithoutBorders. Added nwbpkg support.
• cmdLinePipeline - Updated to handle CIAtah v4.0 switch to all functions inside ciapkg package.
• saveNeurodataWithoutBorders - Added nwbpkg support.

Updated to license, move functions into ciapkg package, gramm graphics, detrend support, speed improvements, movie controls to CIAtah main GUI, etc.

Main features and changes:

• Add sliders to allow users to quickly scroll through both movies in main ciatah GUI.
• Added detrend support for ciatah class modelPreprocessMovieFunction.
• Moved ciapkg overloaded functions into sub-package ciapkg.overloaded to clean up namespace and not interfere with users namespace, updated dependent functions accordingly.
• Moved startup into ciapkg to prevent changing user settings at startup.
• Moved loadBatchFxns into ciapkg package and updated all functions accordingly.
• Speed improvements when setting up ciatah class.
• Added downloading of Gramm (https://github.com/piermorel/gramm) for graphics as external program.
• Updated to MIT license from GPL-3.0 License.

Example of movie slider and cell-extraction in the main GUI.

New functions

• None.

Updated @ciatah

• ciatahMainGui - Update to improve handling of output variables when figure closes to avoid calling invalid handles.
• computeCellDistances - Added gramm plots that show the distribution of cell-cell distances.
• getRegistrationSettings - Added support for certain options (e.g. motion correction) to give a vector and also added movie detrend options.
• modelPreprocessMovieFunction - Add support for multiple reference frames, the mean is taken before input to motion correction. Added support for detrending movies.
• ciatahMainGui - Add sliders to allow users to quickly scroll through both movies. Updated to ensure movie callback playback loop operations only occur if handles to movies are still valid, e.g. if main GUI figure is still open.

Updated ciapkg

• signalSorter - Changed call to overloaded functions to ciapkg.overloaded to avoid altering users Matlab workspace native functions. Change selection of cell on the maps to be instantaneous for where the mouse location is, much faster, don't have to wait for ginputCustom cross-hairs to show. Users can still show crosshairs with 'V'.
• thresholdImages - Fast border calculation using convn for options.fastThresholding==1, less precise than bwboundaries or bwareafilt but works for fast display purposes.
• manageMiji - Check that ImageJ already in java path to prevent duplicate loading and wasting time.

Updated +ciapkg

• `io.loadDependencies' - Added backup URL for downloading mij.jar.
• loadBatchFxns - Remove redundant loading and unloading of external programs via additional checks. Moved loadBatchFxns into ciapkg package. Use ciapkg.getDir to get directory as standard IO.
• startup - moved into ciapkg package.
• loadDependencies - Added gramm (https://github.com/piermorel/gramm) support/downloading for graphics plotting.

GUI updates, fast thresholding, and misc updates.

Main features and changes:

• CIAtah main GUI now displays cell extraction outputs.
• Threshold images has a fast thresholding option.
• Improved callback support in various methods.
• viewCellExtractionOnMovie now supports read from disk.

Updated @ciatah

• ciatahMainGui - Update to ensure NWB switch is made after user selects it. Updated cell extraction loading and allow a cache for faster loading if user switches between methods in the same GUI load. Added support for folder loading button and some other additional improvements. Cell extraction now thresholded for cleaner visuals.
• getRegistrationSettings - Switch to callbacks to close instead of using pause, allows for more flexibility going forward and use with MATLAB Online.
• modelAddNewFolders - Add 'next' button to initial menu, in part to add MATLAB Online support.
• modelExtractSignalsFromMovie - Update to add fix for CELLMax with ROI.
• modelGetSignalsImages - Updated to allow loading raw files without calling modelVarsFromFiles.
• viewCellExtractionOnMovie - Add support for view from disk and cell overlays.

Updated ciapkg

• thresholdImages - Added support for fast thresholding using vectorized form, faster than parfor loop.
• loadMovieList - dims.three fix for reading tifs, esp. ImageJ >4GB.
• playMovie - Added feature to input line overlays on input movie (e.g. to overlay cell extraction outputs).

Updated +ciapkg

• readFrame - Updated to have backup read method for different tiff styles.
• updatePkg - Updated comparison to ensure ignoring of lower level versions after higher level versions are found to differ.
• version - Updated to trim version string to make sure new line is removed.

Improved pre-processing UI, Bio-Formats support, easier ImageJ/Miji loading, and more.

Main features and changes:

• modelPreprocessMovieFunction - CIAtah now remembers the user's analysis order and there are options to reset it. Further, the pre-processing settings are now all displayed on a single page.
• Bio-Formats (https://www.openmicroscopy.org/bio-formats/) support added. Currently added support for Zeiss (CZI, LSM) and Olympus (OIR) formats with more to follow.
• Switch support from Miji/Fiji combination to ImageJ/Miji combination using .jar files that is easier to install and should have easier support across Windows/Linux/OSX.
• Update README to include webinars and workshop recordings on calcium imaging analysis and CIAtah.

New functions

• ciapkg.io.getOptions - clone of getOptions in anticipation of moving all functions to ciapkg sub-package.
• @ciatah\modelVarsFromFilesCheck - Checks whether a specific folder has cell extraction variables loaded, if not loads them.

Updated @ciatah

• computeManualSortSignals - Added modelVarsFromFilesCheck() to check and load signals if user hasn't already. Fix check to make sure variables are loaded.
• computeMatchObjBtwnTrials - Added modelVarsFromFilesCheck() to check and load signals if user hasn't already.
• getRegistrationSettings - Switch to single page support. Multi-column layout since most computers are widescreen now.
• modelExtractSignalsFromMovie - Use filesep in getAlgorithmRootPath to avoid issues in Unix-based systems. Fix issue of passing multiple movies to PCA-ICA.
• modelModifyMovies - manageMiji('startStop','closeAllWindows'); added to improve support for ImageJ over Fiji implementation of Miji.
• modelModifyRegionAnalysis - Added modelVarsFromFilesCheck() to check and load signals if user hasn't already.
• modelPreprocessMovieFunction
  • Fixed thisFrame issue when displaying area to use for motion correction if treatMoviesAsContinuous=0 and processMoviesSeparately=0, e.g. would load reference frame from each and if there were 3 movies, would assume RGB, causing display to be white.
  • Also update so the display frame takes into account custom frame list range. Updated checking of options. Also save ordering of options selected.
  • Added manageMiji('startStop','closeAllWindows'); support.
• modelVarsFromFilesCheck - Add check for objLocations.
• viewCellExtractionOnMovie - Added modelVarsFromFilesCheck() to check and load signals if user hasn't already.
• viewCreateObjmaps - Added modelVarsFromFilesCheck() to check and load signals if user hasn't already.
• viewMovie - Added support for simple and advanced settings. Added modelVarsFromFilesCheck() to check and load signals if user hasn't already.
• viewMovieRegistrationTest - Added modelVarsFromFilesCheck() to check and load signals if user hasn't already.
• viewObjmaps - Added modelVarsFromFilesCheck() to check and load signals if user hasn't already.
• viewSubjectMovieFrames - Added modelVarsFromFilesCheck() to check and load signals if user hasn't already.

Updated ciapkg

• signalSorter - Since ciapkg.io.loadSignalExtraction already supports loading NWB or CIAtah MAT-file cell extraction files, remove remove redundancy and only call ciapkg.io.loadSignalExtraction rather than loadNeurodataWithoutBorders. Added support for progress bar when movie not input.
• cropMatrix - Support switched from [nSignals x y] to [x y nSignals]. Updated to make imrect the default method of selecting the coordinates.
• loadMovieList - Added support for Bio-Formats compatible files, specifically Olympus (OIR) and Zeiss (CZI, LSM).
• manageMiji - Add support for setting up ImageJ along with closing all windows to future proof any changes to those calls.
• modelAddOutsideDependencies - Updated to add support for ImageJ call instead of Fiji.
• saveMatrixToFile - updated TIFF saving to add support for export of multi-channel color timeseries TIFF stack if in format [x y C t] where x,y = width/height, C = RGB channels, t = frames.
• computeManualMotionCorrection - Add default gamma option. Update to fix issue of prior figure (even after clf) maintaining previous key press and exiting, causing uiwait to fail.
• normalizeMovie - Detrend movie now uses nanmean to get around issues in motion corrected videos.
• createMovieFromVector - Users can now manually change value assigned to center line or signal.
• playMovie - Add support for playing RGB movie of dimension [x y C t] if input directly as matrix.
• loadBatchFxns - Added manageMiji('startStop','closeAllWindows'); support.

Updated +ciapkg

• cmdLinePipeline - Added CNMF/CNMF-e and EXTRACT to cell extraction examples.
• loadDependencies - Switched to support for original MIJ calling of ImageJ using just jar files, easier compatibility across MATLAB versions and OSes.
• updatePkg - Added additional URLs for compatibility. Updated to ciatah from calciumImagingAnalysis URLs.

Misc

• ciapkg.io.updatePkg - option on whether to update package (only 0 for now). Also alert user if behind a version.
• ciapkgRoot moved into +ciapkg folder.
• ciapkg.version - Update VERSION path.
• ciapkg.versionOnline - Update VERSION path.
• plotSignalsGraph - Fix minIncrementAmount issue.
• • ciapkg.versionOnline - Now checks multiple location of VERSION file to future proof in case it is moved.
• ciatah - checks online for version at startup.
• ciapkg.nwb.setupNwb - Fix for options.defaultObjDir leading to incorrect NWB folder and cores not being generated.

New load/save cell-extraction outputs, EXTRACT support, and misc. updates

New

• Added support for the new cell-extraction method EXTRACT (https://github.com/schnitzer-lab/EXTRACT-public) to CIAtah.
• ciapkg.io.saveSignalExtraction - universal function to save NWB or CIAtah-style cell-extraction outputs
• ciapkg.io.loadSignalExtraction - function to load either NWB or CIAtah-style cell-extraction outputs.
• ciapkg.pkgName - Outputs the package name, e.g. CIAtah.

Updated:

• calciumImagingAnalysisMainGui changed to ciatahMainGui.
• computeManualSortSignals - If user hasn't called modelVarsFromFiles, computeManualSortSignals called the function. However, this lead to a mismatch between computeManualSortSignals fileNum and obj.fileNum, leading to mismatch between xcoords, etc. and input signals/images.
• loadDependencies - Added EXTRACT support to list of functions to download.
• ciapkg.pkgName - Option on whether to update package (only 0 for now). Also alert user if behind a version.
• modelExtractSignalsFromMovie - Update saveRunTimes to handle case in which user selects multiple movies for cell extraction. Convert ndSparse outputs to single from cell-extraction algorithms (e.g. for CELLMax/EXTRACT) when saving as NWB. Updated EXTRACT support to include additional options.
• signalSorter - User can now input just NWB path without a blank variable for inputSignals. Also added support for CIAtah mat files.
• saveNeurodataWithoutBorders - Update to checking if only a single signal input.
• computeCnmfeSignalExtraction_batch - extractedSignalsType, extractedSignalsEstType struct update.
• changeFont - Add font name support.
• Updated docs.

Adding ciapkg API sub-package, improved directory management, etc.

New

ciapkg.io.loadDependencies - Download and load CIAtah dependencies. Branch from class function.

ciapkg.getDirExternalPrograms - Returns the directory where external programs are stored. All functions should call this to find external program directory.

ciapkg.getDirPkg - Standardized location to obtain relevant CIAtah directories, e.g. location of default data folder.

ciapkg.api - This package will eventually contain pass-through functions for all CIAtah functions, e.g. this will allow CIAtah to keep organization clear by putting functions in related subpackages but also allow users to quickly access all functions using ciapkg.api.

Updated

ciapkg.nwb.setupNwb - Update _external_programs to call ciapkg.getDirExternalPrograms() to standardize call across all functions.

ciapkg.versionOnline - Updated to handle new VERSION file that includes datestamp on 2nd line.

example_downloadTestData - Function now calls data directory via standardized ciapkg.getDirPkg('data') to avoid placing data in incorrect folder.

createHdf5File - Close space_id, dset_id, and fid with low-level HDF5 functions before appending data with hdf5write to avoid read/write issues.

readHDF5Subset - Updated support for files with datasets that contain 2D matrices.

loadMovieList - Fixed loading HDF5 datasetname that has only a single frame, loadMovieList would ask for 3rd dimension information that did not exist.

loadNeurodataWithoutBorders - Parse the algorithm associated with the NWB signal extraction data.

saveNeurodataWithoutBorders - Function checks that yaml, matnwb, and nwb_schnitzer_lab loaded, else tries to load to make sure all dependencies are present and active. Update _external_programs to call ciapkg.getDirExternalPrograms() to standardize call across all functions. Added a check for inputs with a single signal and function returns as it is not supported.

computeCnmfeSignalExtraction_batch - Added trace origin type to output structure.

playMovie - Added feature to sub-sample movie to make display run faster for larger movies.

plotSignalsGraph - Function outputs the modified traces for parent functions to use for additional plotting behavior.

CIAtah class

A number of functions change calling to external programs from hardcoded value to calling ciapkg.getDirExternalPrograms().

loadDependencies - Calls non-class function for use in more functions without needing to load CIAtah class.

modelAddNewFolders - Added support for direct input of method type, useful for command-line or unit testing. 'Add CIAtah example folders.' now adds the absolute path to avoid path errors if user changes Matlab current directory.

modelExtractSignalsFromMovie - adding CELLMax and EXTRACT to public repo in anticipation of their repos being made public. Update _external_programs to call ciapkg.getDirExternalPrograms() to standardize call across all functions.

modelPreprocessMovieFunction - _inputMovieF0 now saved to processing subfolder.

README update, image/signal loading streamline, and misc. other updates

• ciapkg.io.matlabToolboxCheck - Checks whether a MATLAB toolbox/feature has a license available and is installed.
• ciapkg.io.dependencyCheck now re-directs to ciapkg.io.matlabToolboxCheck.
• ciapkg.io.updatePkg - notify users if behind on a version.
• ciapkg.versionOnline - check the online version of CIAtah.
• README update.
• modelGetSignalsImages - Reorganized returnType to be outside raw signals flag, so common filtering mechanism regardless of variables loaded into RAM or not. This fixes if a user loads variables into RAM then uses cross-session alignment, the viewMatchObjBtwnSessions method may not display registered images (cross-session alignment is still fine and valid).
• viewMatchObjBtwnSessions - Take out parfor in one loop that calls obj functions to eliminate serialization issues. Also change color mapping of cross session maps so that it covers the full range using the actual global IDs matched across sessions rather than diluting the range by including global IDs that don't meet the criteria. Makes the maps more colorful.
• applyImagesToMovie - Remove parallelization across inputImages to reduce memory overhead and serialization memory issues (e.g. transferring a duplicate of the movie to all workers).
• Misc other changes and update to README.

Command-line processing, GUI/features updates playMovie, signalSorter, increased NWB support, and more.

Changelog
Version: v3.20.20201019131033.

This release contains large usability updates to signalSorter and playMovie to make GUIs more responsive and user friendly. Also updated command-line examples and processing for users along with adding additional helper functions and other updates.

New

• ciapkg.demo.runPreprocessing - Addition to allow users to run pre-processing on a movie from a single function. To be expanded later.
• ciapkg.io.getMovieInfo - Wrapper to quickly get movie information. Options are the same as loadMovieList.
• ciapkg.io.readFrame - Fast reading of frame from files on disk. This is an alternative to loadMovieList that is much faster when only a single frame needs to be read.
• ciapkg.signal_extraction.runPcaIca - Wrapper to run PCA-ICA (Mukamel, 2009) cell extraction using two existing versions on CIAPKG.

Upated

• playMovie - major update to improve GUI. Updated slider and pause movie callbacks to reduce probability that slider starts to move erratically, move focus away from slider after use (so keyboard callbacks work), and improve pause handling. Made angle optional for options.primaryTrackingPoint. Refactored code to make easier to manage. Middle and right-click of mouse now lead to pausing. Check for NWB input file and change dataset name to accomodate. Create a shortcut menu for easier navigation. Add support for displaying in RGB when reading from AVI movies. Switch read from disk support to using ciapkg.io.readFrame so that playMovie uses the new standard interface for fast reading from disk.
• signalSorter - Display context menu for keyboard shortcuts, easier than separate figure. User can select with right-click or via a menu in the GUI. Users can now scroll through cells using mouse scroll wheel.
• ciapkg.demo.cmdLinePipeline - Updated to add support for cross-session analysis and use ciapkg.demo.runPreprocessing() to process the other imaging sessions. Update to use ciapkg.signal_extraction.runPcaIca for PCA-ICA to make easier for users to run in the future.
• getOptions - Added passArgs option, this mimics the ... construct in R, so users can pass along arguments without having to define them in the calling function (e.g. in the case of wrapper functions).
• applyImagesToMovie - Supports inputMovie as a character path to the movie.
• loadMovieList - Improved comments and options descriptions.
• ciapkg.io.getMovieFileType - addition of a second movie type, e.g. to differentiate NWB even though it is HDF5.
• suptitle - Added support for latex intepreter to allow more detailed information display.
• computeSaleaeOutput - Updated to make more easily usable with other miniscope sync signals. Remove eval call.
• runICA - added IcaOutputInfo output.
• runPCA - Compatibility fixes for matrix inputs.
• changeFont - Added support for changing font type.

Additional

• playMovie - Add support for read from disk for major filetypes, e.g. hdf5, tiff, avi, and isxd. User just needs to input filename. Added frameList to support only looping over specific sequences in the movie.
• example_downloadTestData - added two-photon example data.
• ciapkgRoot - added for future functions to quickly locate the root as needed.
• ciapkg.getDir - allow users to get the directory of ciapkg quickly.
• ciapkg.io.loadIDPS - added to allow central place to load Inscopix IDP ISX package.
• ciapkg.demo.cmdLinePipeline - Use ciapkg.getDir() to make sure demo always calls correct path regardless of where user is pointing. Also make playMovie calls have titles to make clearer to new users and allow a GUI-less option.
• loadMovieList - Change warning message output for HDF5 file of certain type. Add option to suppress warnings.
• turboregMovie - Remove references to parfor_progress.
• computeCnmfSignalExtractionClass - allow option to skip CNN, problematic for certain datasets and not necessary if doing manual classification.
• Added updated NWB support for saving out movies during processing step.
• Misc other changes.