Skip to content

Handling of large datasets

Pre-release
Pre-release
Compare
Choose a tag to compare
@epnev epnev released this 11 Apr 19:29
· 412 commits to master since this release

New features

The main new feature with this release is the ability to handle large datasets through a memory mapping process and by processing the different spatially overlapping patches in parallel. This process is described is more detail in the wiki page. Several new functions are introduced for this:

  • memmap_file.m: Save a tiff stack file as a matlab file that can be memory mapped. If the dataset is too large to load in memory, it can be read/saved in pieces.
  • memmap_file_sequence.m: Save a sequence of tiff files as a matlab file that can be memory mapped.
  • construct_patches.m: A simple function that determines the different spatial patches to be processed in parallel.

Another new feature is an unsupervised classifier for classifying the components into true and false. The classifier works by first classifying the observed voxels into active/inactive based on their power spectrum properties and then classifies the identified components as active when they significantly overlap with the set of active voxels.

Other changes

  • update_temporal_components.m and update_temporal_components.m can also accept as inputs memory mapped data files.
  • The output P of preprocess_data.m also contains information about the power spectrum of each pixel and the classification of each pixel to active/inactive.
  • plot_contours.m can accept precomputed coordinates for faster plotting.
  • Several bug fixes, and changes for performance improvement have been implemented.