DIALS 3.14.1

2023-04-20:

• dials.export_bitmaps: Correctly scale resolution rings with image binning= set. (#2392)
• Revert GTK-message-supression change that was causing reciprocal_lattice_viewer errors. (#2395)

For a more complete list of changes and fixed bugs check the dials changelog.

There are no changes to cctbx, dxtbx or xia2 in this release.

DIALS 3.14.0

2023-04-12:

Features

• dials.damage_analysis: Add dose_group_size= option to allow generation of plots for subsets of the data based on dose. (#1903)
• dials.import/dials.export: The image template is now reported in the log file, for rotation data. (#2278)
• dials.merge: Now suggests a resolution limit, based on a fit to CC½. (#2351)
• dials.merge: Save merged half datasets to the MTZ file when additional_stats=True. (#2353)
• dials.export_bitmaps: Option to render resolution rings. (#2360)
• dials.rs_mapper: Now works with multi-panel detectors. (#2362)
• dials.ssx_integrate: Add support for multi-panel detectors, for the ellipsoid model. (#2368)
• dials.find_spots: Add the exclude_images= option. Previously this was only on dials.import. (#2374)
• dials.merge: Generate "FreeR_flag" column in output mtz, optionally copying from a reference mtz containing a FreeR_flag column. (#2378)
• dxtbx: flumpy: Add support for conversion of flex.miller_index arrays to/from numpy. (cctbx/dxtbx#618)
• xia2.ssx: Enable arbitrary grouping of data for merging by specifying a grouping yml, add dose_series_repeat=$n option to indicate dose series for merging. (xia2/xia2#713)
• xia2: Add handling for small-molecule chemical formula data, and extra help output when using xia2.small_molecule. (xia2/xia2#723)
• xia2.ssx: Improve unit cell assessment and reporting when unit cell is not known. (xia2/xia2#731)
• xia2.multiplex: Generate FreeR_flag column in merged mtz output, ensuring flags are consistent across all clusters/filtered reflections. (xia2/xia2#732)

Bugfixes

• dials.refine: Does not call sys.exit when running as a library function. (#2325)
• dials.generate_mask: Fix a failure when the direct beam intersected the exact centre of a pixel. (#2329)
• Providing invalid PHIL parameters no longer causes a traceback. (#2331)
• dials.index: Handle possibility of Krivy-Gruber iteration limit exceeded in Niggli cell reduction. (#2332)
• CPU allocation limits will now be correctly inherited from the Slurm cluster scheduler. (#2335)
• dials.ssx_integrate: Fix cases where the background and centroid algorithms choices were not being used with ellipsoid-model integration. (#2337)
• dials.assign_experiment_identifiers now works with experiment lists without identifiers. (#2342)
• Test data files are no longer patched to correct their trusted range values, and usages of the deprecated DataBlock have been removed. (#2346)
• dials.ssx_integrate: catch rare ValueError in ellipsoid profile model refinement (#2349)
• dials.symmetry: Handle cases when an experiment has no scan object. (#2350)
• dials.ssx_index: Ensure file paths are split correctly on Windows. (#2358)
• Replace the deprecated imp library with importlib. (#2363)
• dials.rs_mapper: Fix bug causing the image data to be accessed with out-of-bounds indices. (#2364)
• dials.image_viewer: Fix error when using newer versions of wxPython. (#2365)
• DIALS viewers: Suppress terminal output saying "Gtk-Message". (#2366)
• Prevent occasional failures testing constrained geometry refinement. (#2367)
• Slicing a reflection_table does not lose the experiment identifiers map. (#2371)
• dials.integrate: Update exclude_images= syntax to match dials.import and dials.find_spots. The older form is now deprecated. (#2374)
• reflection_table.match_by_hkle: Fix failure on Windows with pandas 2.0.0. (#2383)
• dxtbx: Flumpy: Prefer returning flex.int instead of flex.long when they are the same size. This solves ambiguous behaviour when reading images on Windows platforms. (cctbx/dxtbx#607)
• dxtbx.plot_detector_models: Fix display of multiple single-panel detector models. (cctbx/dxtbx#610)
• xia2.ssx: Don't use the beam model from a reference geometry. (xia2/xia2#724)
• xia2.ssx: If rerunning in same directory, make sure correct batch folders are generated. (xia2/xia2#725)
• xia2.ssx: When re-importing with the self-determined reference geometry, don't overwrite the detector model with manually specified phil options. (xia2/xia2#726)

For a more complete list of changes and fixed bugs check the dials, dxtbx and xia2 changelogs.

DIALS 3.13.0

2023-01-26:

Features

• dev.dials.napari_rlv: A reciprocal lattice viewer based on napari. This currently requires the napari module to be manually added into your DIALS installation. (#2229)
• dials.merge: Allow exclude_images= parameter, as in dials.scale. (#2280)
• dials.refine: Use an overall interval_width_degrees= parameter to set the default smoothness in scan-varying refinement for all models. This does not affect which models will be parameterised as scan-varying, which is controlled by their individual force_static= parameters. (#2268)
• dials.scale: Add ability to use a precalculated analytical correction as part of scaling models. (#2313)
• dials.stills_process: Added reflection subsampling. If reflection_subsampling.enable=True, and an image fails to index, the reflections will be randomly subsampled, and indexing tried again. Reflections will be randomly subsampled in smaller amounts until a threshold is reached or the image succesfully indexes. (#2275)
• dials.stills_process: Adds suppressed_logging= option, and minimial progress feedback. (#2263)
• xia2.ssx: Handle data from PAL-XFEL and SACLA. (xia2/xia2#719)
• xia2.ssx: Report r-split metric in merging stats (xia2/xia2#721)
• Add additional_stats option to generate R-split statistic for stills data in dials.merge and dials.scale. (#2314)
• dxtbx: FormatNXmxED: Format support for electron diffraction images converted to be compatible with NXmx by nexgen. (cctbx/dxtbx#583)
• dxtbx: FormatNXmxEDeBIC: Including a mask specific for the SINGLA that is temporarily installed at eBIC, through to early 2023. (cctbx/dxtbx#589)

Bugfixes

• dials.cluster_unit_cell: Correctly report the number of lattices in each cluster, instead of the number of clusters. (#2300)
• dials.combine_experiments: Prevent default clustering.max_clusters=None from raising error when clustering. (#2311)
• dials.import: Fix convert_stills_to_sequences= option for h5 data formats. (#2273)
• dials.integrate: Clearer error message when kapton absorption_correction.apply=True, but algorithm=None. (#2317)
• dials.integrate: Fail for negative profile-fitting intensity variance, not zero variance. (#2271)
• dials.scale: Catch rare crash when making summary stats table. (#2284)
• dials.scale: Fix crash when scaling against reference, and some datasets were removed during scaling. (#2312)
• dxtbx.image_average: Fix a crash from using more processors than images when using MPI. (cctbx/dxtbx#571)
• dxtbx.plot_detector_models now works with newer versions of matplotlib. (cctbx/dxtbx#574)
• Allow reflection_table.remove_on_experiment_identifiers() for an empty flex.reflection_table. (#2298)
• dxtbx: FormatNXmxDLS: Don't process electron diffraction images collected at eBIC that have been converted by nexgen. (cctbx/dxtbx#579)
• dxtbx: Read underload from CBF files. (cctbx/dxtbx#592)
• dxtbx: NXmx: Ensure integer data types get converted to flex.int on all platforms. (cctbx/dxtbx#594)
• dxtbx: Generally better defined trusted_range values all around (cctbx/dxtbx#590, cctbx/dxtbx#595, cctbx/dxtbx#601)

For a more complete list of changes and fixed bugs check the dials, dxtbx and xia2 changelogs.

DIALS 3.12.1

2022-12-05:

Bugfixes

• dxtbx: Fix support for datasets generated by the DECTRIS EIGER filewriter (cctbx/dxtbx#582, cctbx/dxtbx#586, cctbx/dxtbx#588)
• dxtbx: FormatCBFFull: trusted range bug fix - use the minimum valid pixel value rather than the undefined value (cctbx/dxtbx#587)
• dxtbx: NXmx: eliminate potential divide-by-zero warning (cctbx/dxtbx#572)

For a more complete list of changes and fixed bugs check the dxtbx changelog.

There are no changes to cctbx, dials or xia2 in this release.

DIALS 3.12.0

2022-10-31:

Features

• The xia2.ssx, xia2.ssx_reduce, dials.ssx_index and dials.ssx_integrate programs are now considered stable, so have dropped the dev. prefix. (#2265 and xia2/xia2#710)
• dials.background: Add n_checkpoints= parameter, to run the analysis on evenly-spaced images. (#2240)
• dials.image_viewer: Change default brightness to 10. (#2254)
• dials.rs_mapper: Perform the calculation in parallel to improve speed. (#2238)
• dials.stills_process: Add known_orientations= feature, useful for specifying previous processing results to use when reprocessing data, or for multiple detectors in an experiment. (#2110)
• dxtbx: Improve XTC handling from LCLS. (cctbx/dxtbx#517)
• dxtbx: NXmx: Add support for @equipment_component. Note, .expt files will not be backwards compatible for users of the JF16M detector at SwissFEL or the AGIPD detector at EuXFEL. (cctbx/dxtbx#561)

Bugfixes

• dials.find_rotation_axis: Recover a plot that was broken by previous changes. (#2225)
• dials.image_viewer: Fix intensity readout mouseover, and beamcenter half-pixel errors. (#2194)
• dials.reciprocal_lattice_viewer: Set the maximum number of experiments to show in the selector toggles (default 15). Datasets with a number larger than this will not be individually selectable. (#2248)
• dials.scale: Restore consistent dataset id output numbering when a dataset is excluded. (#2246)
• dials.ssx_integrate: Gracefully handle assertion error for particular bad data. (#2264)
• dials.stills_process: Fix crashes for raster scans of stills. (#2128)
• trusted_range is now defined throughout DIALS consistently as the inclusive range between the minimum and maximum trusted values - valid pixels are those less than or equal to the maximum trusted value and greater than or equal to the minimum trusted value. (#2195)
• dxtbx: Improved speed of reading many-panel Nexus images (cctbx/dxtbx#565)
• xia2: Explicitly set the log file encoding to be UTF-8. This could break on systems set to non-native character encodings. (xia2/xia2#709)
• xia2.ssx: Fix error in deciding whether to assess indexing ambiguities. (xia2/xia2#705)
• xia2.ssx: Fix crash in geometry refinement when a block has no indexable images. (xia2/xia2#707)
• xia2.ssx: Avoid crash if no images successfully integrated. (xia2/xia2#711)

For a more complete list of changes and fixed bugs check the dials, dxtbx and xia2 changelogs.

DIALS 3.11.2

2022-09-27:

• dials.scale: Fix bug in intensity combination scoring for multi-sweep datasets, affecting midpoint test values. (#2199)
• dxtbx: NXmx: Cope more gracefully with scalar NXtransformations values. (cctbx/dxtbx#546)
• dxtbx.dlsnxs2cbf: Fix distance and pixel size bugs. (cctbx/dxtbx#548)
• dxtbx: NXmx reading: Handle cases where the detector is read as between the sample and source. This is to compensate for an incorrect definition in the Dectris Eiger file writer. (cctbx/dxtbx#550)
• cctbx: Try to be smarter about which python to use when installing DIALS.

For a more complete list of changes and fixed bugs check the dials and dxtbx changelogs.

There are no changes to xia2 in this release.

DIALS 3.11.1

2022-09-02:

• Revert default installation to Python 3.9, to avoid WXPython incompatibilities. You can still manually request Python 3.10 when building the release, but the binaries we distribute for DIALS 3.11 will be Python 3.9 for the rest of the series. (#2216)
• dxtbx.dlsnxs2cbf: Fix bug introduced by #572. (cctbx/dxtbx#545)

This release includes the changes from the now-retracted DIALS 3.11.0:
2022-08-24:

Features

• DIALS now use Python 3.10 by default in releases and when running bootstrap. Python 3.8 is still supported. (#2206)
• dials.scale: Added the reference= option, with support for using a cif data file as a scaling reference. This replaces the old usages of target_mtz= and target_model=, which are now deprecated. (#2148)
• dials.cosym: Allow use of a reference dataset, to consistently index against. (#2154)
• dials.find_rotation_axis: Dramatically improve execution time by performing the search in parallel. (#2160)
• dials.image_viewer: Score tool now accepts keyboard entry. (#2162)
• dials.symmetry: Add option to score systematic absences using fourier analysis. Thanks to Kevin Dalton for contributing this feature. (#2184)
• Unit cell clustering reports: Add plots of unit cell angle distribution. (#2197)
• dials.merge: Add merging statistics to output html report, and the PHIL option output.json= to write this this to json. (#2178)
• dials.merge: include merging statistics in json output (#2207)
• dxtbx: Replace use of legacy FormatNexusEiger with new FormatNXmx format class. (cctbx/dxtbx#455)
• xia2: dev.xia2.ssx: Add data reduction to experimental ssx pipeline, and expose data reduction as the new standalone dev.xia2.ssx_reduce program. (xia2/xia2#683)

Bugfixes

• dials.find_rotation_axis: Consistently report rotation axis angle at start and end of the search. (#2145)
• dials.generate_mask: Make the log filename consistent with the program name. (#2147)
• dials.image_viewer: Automatically complete an active polygon mask when saving mask parameters. (#2150)
• dials.refine: Correct an error mapping constraint definitions to models. (#2155)
• dials.background: Check for empty experiments. (#2163)
• dials.estimate_gain: Now works on multiple experiments from a single image sequence. (#2164)
• dials.powder_calibrate: More general determination of background mask removal. (#2165)
• dials.symmetry: Ensure data for systematic absences check is in the correct setting for non-conventional minimum cells. (#2183)
• dials.scale: Fix bug in intensity combination scoring for multi-sweep datasets, affecting midpoint test values. (#2199)
• dev.dials.ssx_index: Fix crash when generating plotting data where rmsd values are almost all equivalent. (#2203)
• dev.dials.ssx_integrate: Handle potential assertion errors from FastMCD. (#2179)
• dev.dials.ssx_integrate: Fix divergent refinement bug when n_macro_cycles > 1. (#2204)
• dxtbx: DXTBX now uses the median oscillation width from across the entire scan. This resolved issues where the goniometer scan positions were read-back values instead of set-point values, and a slow rotation start across the first two images would cause the oscillation width for the whole scan to be calculated incorrectly. (cctbx/dxtbx#526)
• dxtbx: ExperimentList.append(): No longer O(N²) with experiment identifiers. (cctbx/dxtbx#528)
• dxtbx: FormatNXmx: Ignore empty pixel masks, instead of printing a confusing error. (cctbx/dxtbx#529)
• dxtbx: Correct assumptions about interpreting multi-axis goniometer axes from full-CBF files. Previously, it was assumed the axis and diffrn_scan_axis categories listed axes in the same order, and that this matched a standard diffractometer axis order. The goniometer model is now build correctly, regardless of the order specified in the file. (cctbx/dxtbx#539)
• xia2.to_shelx: Fix crash when using unmerged mtz with the --cell option. (xia2/xia2#698)

For a more complete list of changes and fixed bugs check the dials, dxtbx and xia2 changelogs.

DIALS 3.10.3

2022-08-02:

• dials.image_viewer: Fix error after loading images with the "Load" button. (#2168)
• dials.merge: Fix crash for P-1 datasets. (#2175)
• dials.export format=mtz: Handle shared experiment models, when converting to cambridge frame. (#2182)
• dxtbx: Fix mask_untrusted_circle() crash when untrusted circle extends outside detector. This affected dials.generate_mask. (cctbx/dxtbx#525)
• dxtbx: FormatNXmx: Allow empty saturation_value field when importing data. (cctbx/dxtbx#534)

For a more complete list of changes and fixed bugs check the dials and dxtbx changelogs.

There are no changes to cctbx or xia2 in this release.

DIALS 3.10.2

This is a minor patch release to fix version numbering in the installer for releases.

This includes changes originally released as DIALS 3.10.1: 2022-07-12:

• dials.cosym: Fix bug whereby the change of basis op was applied twice to the crystal model. (#2151)
• dials.image_viewer: Allow viewing still collections where some images have no reflections. (#2157)
• dials.scale: Avoid crashe when all reflections filtered out during reflection selection, for a dataset in multi-dataset scaling, (#2146)
• dials.ssx_index: Fix potential crash from one datapoint. (#2156)
• dev.dials.ssx_index: Fix reporting when only one image indexed, and fix crash. (#2159)
• dxtbx.install_format: Handle case on MacOS .pkg installations where URL-formats could not be installed. (cctbx/dxtbx#524)
• dev.xia2.ssx: Fix crash when no images indexed in a batch. (xia2/xia2#686)
• Refinement internals: Terminate refinement if the objective is not expected to decrease. (#2135)

For a more complete list of changes and fixed bugs check the dials, dxtbx and xia2 changelogs.

There are no changes to cctbx in this release.

DIALS 3.10.1

2022-07-12:

• dials.cosym: Fix bug whereby the change of basis op was applied twice to the crystal model. (#2151)
• dials.image_viewer: Allow viewing still collections where some images have no reflections. (#2157)
• dials.scale: Avoid crashe when all reflections filtered out during reflection selection, for a dataset in multi-dataset scaling, (#2146)
• dials.ssx_index: Fix potential crash from one datapoint. (#2156)
• dev.dials.ssx_index: Fix reporting when only one image indexed, and fix crash. (#2159)
• dxtbx.install_format: Handle case on MacOS .pkg installations where URL-formats could not be installed. (cctbx/dxtbx#524)
• dev.xia2.ssx: Fix crash when no images indexed in a batch. (xia2/xia2#686)
• Refinement internals: Terminate refinement if the objective is not expected to decrease. (#2135)

For a more complete list of changes and fixed bugs check the dials, dxtbx and xia2 changelogs.

There are no changes to cctbx in this release.