FOCUS with single particle 2D crystal software included

1.1.0

.

Focus v1.0.0

The first stable release for Focus.

FOCUS is a C++ / Qt front end, running own and third-party softwares in the background. It features a GUI that allows you to define and edit parameters and C-shell or Python scripts, which are then executed in queues. Users can add or edit their own scripts, to adapt FOCUS to specific workflows or tasks. FOCUS comes with a set of default scripts, ready to run.

Installation of FOCUS requires also the installation of any third-party programs that you want FOCUS to utilise, such as IMOD, EMAN, FREALIGN, or MotionCor2, UNBLUR, Zorro, CTFFIND4, gCTF, gAutomatch, or others.

If you install FOCUS on a strong Linux machine on a computer adjacent to the automated SerialEM run, then FOCUS can for example be used to:

• monitor the file system of the SerialEM run
• fetch newly recorded movies from the SerialEM computer, together with the pixel-defect-list and gain reference files,
• unpack the compressed movies (TIFF ZLW), gain correct,
• Fourier crop 8k to 4k, or to any pixel size you want (optionally)
• drift-correct (with ZORRO, MotionCor2, or Unblur)
• computer 2D averages of the movie, and FFTs of all files
• measure defocus (gCTF or CTFFIND4)
• pick particles (gAutomatch)
• present the last recorded image and its FFT (before and after drift correction), and the drift trajectory plot and CTF Thon ring plot on a website that you would have to setup (ours is at https://status.c-cina.unibas.ch. Ours is open and shows blurred images, but you can protect it with a password and then display the crisp images)
• present the statistics of each recorded image on that web site (as plots over time)
• organize all recorded movies in a large table (similar to an Excel sheet), which can be sorted by various parameters such as
  -- grey value (which images are too dark (contaminated, beam lost) or too white (empty hole?)
  -- defocus (which images are in focus or totally out of focus?)
  -- CTF resolution (for which images is CTF determination difficult?)
  -- iciness (which images have a too high ratio of crystalline ice, or show devitrified sample?)
  -- drift (which images have too long or too short or too jittery drift trajectories? which images have too high remaining interframe drift?)
  -- and various other parameters.
  This table can be sorted by any of such parameters, and images can be moved into a TRASH folder with one click.
• remaining images can be exported in folder structures and STAR files ready for subsequent processing in, e.g., Relion.

FOCUS can also be used for tomography sessions, to:

• recognize parameters from the file name (e.g., specimen number and tilt angle) and organize files accordingly
• drift-correct each recorded movie at a certain tilt angle, while taking the current electron dose and the prior electron dose from earlier tilt angle recordings on the same specimen into account for electron-dose-dependent B-factor resolution filtering
• compute one 2D image per tilt angle movie
• re-organize the tilt angle images by tilt angle (-60,-59,…,0,…,59,60) when recorded in the Hagen scheme (0,1,-1,-2,2,3,-3,-4,4,5,…60,-60)
• as above: monitor data collection progress on a web site, sort data, assist in manually pruning of data,
• export data in MRC stacks with one frame per tilt angle.
• write out metadata for the stacks into STAR files

FOCUS can also be used for 2D crystal sessions, where it can do the entire 2D crystal processing automatically (the full functionality of the 2dx software package is included in FOCUS). This allows direct 3D reconstructions during the 2D crystal data collection session.

FOCUS maintains a batch queue, with which is runs all processes above in parallel.

We have installed this on an Ubuntu PC with
2x12 cores (= 48 threads)
256 GB RAM (of which 96GB are used as a RAM disk. More would make life easier.)
70 TB HD (RAID5)
2 x GTX1080 GPU
We also use a 40” 4k monitor, which is great to display the images full-screen in full quality.
We have one such system per automated cryo-EM microscope, which is able to keep up with single particle data collection of 40-frame 8k movies at a rate of 90 movies per hour (Fourier cropping, drift correction wth MotionCor2 on 5x5 patches, CTFFIND4, gautomatch, etc.), and is thereby on par with data collection.

Focus Pre release

Pre-release for the Focus package

2dx Version 3.5.0

• A new tool to automatically switch between maps and references now available
• Parallel processing of images in 2dx_merge is now possible
• Weighted Backprojection introduced instead of Latline fitting
• 2D merge and 3D merge separated
• Movie mode processing for direct electron detectors
• GUI Improvements in 2dx_merge and 2dx_image
• Moved to Qt5.x
• Improved build and packaging, no dependency on Boost
• It is now possible to sharpen maps using bead models
• B-factor sharpening scripts included
• New Volume refinement methods included which helps to encounter problems of Missing cone
• New kernel tools introduced, capable of doing all the major operations on EM-Volume
• Radiation damage measurement for movie-mode processing
• [bug] 2D-ML now works with the newly integrated general processing parameters
• [bug] Classical diffmap-script working for maps with where RES < 6A
• Many other bugfixes

2dx Version 3.4.2

• Full OSX-10.8 and OSX-10.9 support
• Auto-saving added
• Move image to another project now supported on right-click, useful in case of two crystal conformations
• New bug tracker and developer wiki
• TTF correction script added for images with a higher tilt then a given value
• Internal version control system switched to git
• ttfcorfitst removed from Unbend2, as we do not recommend this option at all
• 2dx_automator added
• Automated drift-correction GUI added
• Single Particle Processing scripts added, but not ready to use so far
• Difference-map functions added
• Command line merger
• [bug] Beam-tilt refinement now working
• [bug] Maximal index in CCP4-script changed in order to achieve high resolution (better than 6A)
• Latest version of CTFFIND3 integrated (incl. OMP)
• Refine switch now also refines the tilt-geometry
• [bug] Resolution plot fixed
• [bug] Error on PLTILTK fixed
• [bug] Big in astigmatic handling of CTF fixed
• File initialization improved
• [bug] Synchronization for .raw.mrc files fixed
• [bug] Reference origin resetting works now
• Parameters are reset for a new image now
• Optimal image import for K2 Summit data
• Selection of multiple images with right-click now possible
• Possibility to generate non-CTF-corrected map
• Removed re-unbending from the standard script list
• Added options for tilt-angle range statistics
• Included fomstat in 2dx_merge_merge again
• Masking both lattices at the same time now possible
• [bug] Issue with symmetry p4 in hklsym fixed
• FDF translator added
• Better sync with backup script
• NANs removed from hklsym4-output
• mtzdunp now displays all reflections
• Non-scroll QComboBox added
• refine Defocus now uses only spots up to IQ-4
• FOM calculation for symmetry-constraint lattice lines implemented into 2dx_latlinek.for
• Lattice line in color now
• Tiff translator added
• Better spot scan masking functionally
• Support for TVIPS-headers
• Origtilt output extended
• 2dx_hklsym3 removed
• [bug] Contour plot in p2 now working
• Amplitude scaling added
• Quality evaluation added (2D and 3D)
• Longer image names now supported
• MergeIQMAX added
• [bug] Unsigned/singed integer issue finally fixed
• [bug] TLTPLOT now used correctly
• Second lattice determination
• Shortcuts to image/merge directory added in 2dx_image/2dx_merge
• Shortcut to logbrowser added
• Updated change_dylib.sh
• More stable Linux support
• SBGrid integration
• Full build-process done in cmake now
• Compilation optimized
• SCALIMAMP3d added
• Final map exported also as .tif
• Sub-volume generation of final map
• Use of cpack for Linux package generation
• Improved trash functionality