NEWS

=========
Changelog
=========

Here you can see the full list of changes between each ufo-core release.


Version 0.16
============

Enhancements:

- OutputTask signals end via "inputs-stopped" signal
- InputTask: Improve performance
- Add ufo_buffer_set/get_layout and pass through

Fixes:

- Better error propagation
- Fix #116: allow _get_requisition to report errors
- Fix #130: add ufo_buffer_swap_data interface
- Fix #143: invalid object dereference
- Fix #145: remove sphinx-tabs dependency
- Fix #146: multi GPU expansion broke
- Fix #147: do correct comparison
- Fix #148: re-write longest path algorithm
- Fix #154: add UFO_RESOURCES_CHECK_SET_AND_RETURN

Breaks:

- Remove _with_opts functions
- Retire `ufo_graph_{{shallow_}copy|subgraph,flatten}`
- Raise GLib version to 2.38
- Remove address options from CLI tools
- Remove daemon and messenger API


Version 0.15.2
==============

This is another bugfix release to help Debian adoption.


Version 0.15.1
==============

This is bugfix release that fixes #143 which causes segfaults with certain task
graph conditions.


Version 0.15
============

Released on January 4th 2018.

Enhancements:

- Manual and API documentation has been much improved
- Passing -Dintrospection=false to meson disables introspection data
- An error is generated if duplicate node names exist in JSON
- Added ufo_profiler_call_blocking to wait for execution
- Added ufo_resources_get_kernel_from_source_with_opts
- Added an NVIDIA Dockerfile example
- The deadlock warning has been turned into a debug message
- The scheduler returns early if no OpenCL platform is found
- Fix #135: added UFO_GPU_NODE_INFO_NAME

Fixes:

- UFO is compiled with gnu99 now to avoid GLOB_TILDE complaints
- Fixed wrong references and highlighting in the manual
- if UFO_DEVICE_TYPE=cpu is set, no GPUs are used
- Fix #142: fixed multi input and GPU processing
- Fix #142: prune graph correctly for multiple GPU
- Fix #137: support JSON array properties
- Fix #133: output build flag string for each device
- Fix #132: do not segfault on unknown properties
- Fix #126: abort execution without resources
- Fix #129: allocate enough memory for index array
- Fix #125: build and install manpages with meson

Breaks:

- Property sets have been removed from the JSON specification


Version 0.14
============

Released on September 5th 2017.

Enhancements:

- meson build system support has been added
- Improve documentation
- Improve expansion for multi-input node graphs
- Output warning in case expansion is not possible
- Use g_printerr to print errors on stderr instead of stdout
- ufo-launch returns exit code 1 on execution error
- Make device usage debug output unambiguous
- #123: allow vector properties via ufo-launch
- #120: generate buffer timestamps with the --timestamps option

Fixes:

- Fix a minor memory leak in ufo-launch
- Fix problem with CMake install dir
- Fix #119: compile OpenCL programs for each device with device-specific flags
  to allow kernel developers optimizations for each device
- Fix #121: use the same command-line interface for both runjson and launch

Breaks:

- We dropped support for Autotools, CMake and meson are the only options to
  build at the moment.


Version 0.13
============

Released on January 25th 2017.

Enhancements:

- ufo-runjson: Add the -s/--scheduler flag to choose a scheduler different from
  the regular one.
- Restructure docs and add section about broadcasts
- Added ufo_resources_get_kernel_source function to have an API to access
  directly a source file from CL/kernel path.

Fixes:

- Fixup for compilation and installation on MacOS


Version 0.12.1
==============

Bugfix release released on November 28th 2016.

- Do not install the Docbook XML build dir
- Document UFO_BUFFER_DEPTH_INVALID
- Enable Large File Support
- Check ftell and return NULL on error
- Add forgotten manpage to the build list
- Initialize uninitialized variable
- Enable _FORTIFY_SOURCE feature flag


Version 0.12
============

Released on November 24th 2016.

Enhancements:

- ufo-launch: convert string to enum values
- ufo-runjson: add -t/--trace flag doing the same thing as ufo-launch
- Documentation updates

Fixes:

- Fix manual heading
- ufo-mkfilter: fix template and type handling
- Add manpage for ufo-prof

Breaks:

- Add UFO_BUFFER_DEPTH_INVALID with value zero which means adding to the API and
  breaking ABI
- Remove package target from the build system


Version 0.11.1
==============

Bugfix release released on November 12th 2016.

- Install systemd unit file through pkg-config
- Fix #110: install templates correctly
- Do not run xmllint on manpage generation output
- Remove unused CMake modules
- Remove executable bit from source files
- Remove PACKAGE_* variables


Version 0.11
============

Released on November 8th 2016

Enhancements:

- Build manpages for the tools
- Update TomoPy integration
- Add UFO_DEVICE_TYPE environment variable
- Properly build on MacOS
- Unify debug message output format
- ufo-launch: rewrite specification parser allowing more flexible descriptions
- ufo-launch: add --quieter
- ufo-mkfilter: add --type and --use-gpu

Fixes:

- Check if we have with multiple roots to exit early
- Show version information consistently
- ufo-prof: support Python < 3.0

Breaks:

- GNUInstallDirs instead is now used instead of our own ConfigurePaths CMake
  module which might affect installation paths on your system. Please note that
  for example -DPREFIX thus becomes -DCMAKE_INSTALL_PREFIX.
- Remove CPack
- ufod: unused --paths option removed
- ufo-runjson: unused --path option removed
- ufo-launch: do not execute graph when --dump'ing


Version 0.10
============

Released on May 24th 2016

Enhancements:

- Add UFO_GPU_NODE_INFO_MAX_MEM_ALLOC_SIZE
- Fix #103: allow ufo-launch to use arbitrary graphs with new workflow
  specification and more robust parsing
- Look for AMD APP SDK 3.0
- Cache programs to avoid rebuilding them
- ufo-runjson now also outputs the number of processed items
- Fix #104: output type and blurb with -v

Fixes:

- Fix #106: match word characters to find plugins
- Fix #101: really unref non-intermediate nodes
- ufo-launch: query only valid graph
- Fixed misleading documentation
- Fixed some leaks and unreferenced resources
- Stop num-processed from being serialized into JSON field
- ufod: fix segfault if no address is specified

Breaks:

- ufo-query: print errors on stderr instead of stdout
- ufo-launch: removed --progress and --time in favor of --quiet
- Removed public ufo_signal_emit symbol
- Use date and time to differentiate trace profiles
- Replace clprof with ufo-prof


Version 0.9.1
=============

Release on January 12th 2016.

Enhancements:

- Pass a property map to scheduler in the Python wrapper
- Added convenience wrapper to copy into a buffer

Fixes:

- Fix version numbering from 0.8.x to 0.9.x
- Fix documentation issues
- Fix build problems if libzmq is not present
- ufo-launch unrefs non-intermediate nodes


Version 0.9
===========

Release on November 3rd 2015.

Enhancements:

- Generally improved debug output
- Add support for plugin packages as demonstrated by the ART plugins
- Add Docker installation method
- Add simple fabfile to start ufod instances
- Allow reductors to pause processing
- Add ufo-query binary to retrieve info about tasks
- Add ufo_buffer_set_device_array API call
- Python: add ufo.numpy.empty_like
- ufo-launch: add --dump flag to serialize to JSON
- ufo-launch: add Bash completion script
- ufo-launch: pass address list

Fixes:

- Fix segfault with long-running tasks
- Prevent daemon from leaking OpenCL resources
- Fix broken continuous daemon operation
- ufo-launch: fix parsing uint64
- Use same reductor policy for both schedulers
- Fix ufo_buffer_new_with_data
- Fix nano second to second conversion

Breaks:

- Specify device subset with UFO_DEVICES
- Do not prepend . to trace and profile output
- Change ufo_buffer_set_host_array signature
- Use common timestamp unit for both trace types


Version 0.8
===========

Release on May 19th 2015.

This release breaks with the distinction of ArchGraph and Resources. The former
is removed with its functionality moved to the latter. Besides that numerous
improvements have been incorporated:

- ufo-launch: add --time and --trace options
- Added a systemd unit for ufod
- Fixed cluster communication.
- Fixed #86: use CL_INTENSITY instead of CL_R
- Handle kernel path with environment variable UFO_KERNEL_PATH
- ufo-launch now parses boolean properties
- Removed hard dependency on ZeroMQ
- Removed WITH_DEBUG and WITH_PROFILING CMake variables
- Add JSON-based TANGO server for remote computation
- Removed ufo_base_scheduler_get_gpu_nodes()
- Fixed #80: add ufo_gpu_node_get_info()
- Fixed re-running the same task graph
- Removed tifffile dependency
- Fixed #78: reset num-processed before execution
- Fixed #77: initialize threads when accessing the GIL


Version 0.7
===========

Released on February 20th 2015.

This is a major update from the previous version released a year ago. Besides
numerous bug fixes and compatibility enhancements we

- added ufo_signal_emit for threadsafe signal emission
- export plugin and kernel directories via pkg-config so that thirdparty plugins
  easily know where to install themselves
- added the ufo-launch tool to run basic pipelines directly from the command
  line
- merged the ufo-python-tools and ufo-tests repos with the core repository
- added additional buffer depths
- parse the UFO_PLUGIN_PATH environment variable to specify additional plugin
  locations
- added a copy task which copies data from input 0
- added a fixed scheduler for manual assignment of hardware resources to task
  nodes
- added "processed" and "generated" signals to tasks which emit whenever either
  action completed
- added UFO_DEVICE_TYPE_ACC for accelerator devices
- added buffer views for larger-than-GPU data
- output OpenCL profiler information as Chrome JSON
- added ufo_resources_get_kernel_with_opts()
- added ufo_buffer_set_host_array()
- added ufo_buffer_get/set_metadata(), ufo_buffer_get_metadata_keys() and
  ufo_buffer_copy_meta_data()
- added ufo_buffer_get_device_array_with_offset()
- added ufo_buffer_get_location()

and broke compatibility by

- retiring the UfoConfig infrastructure and
- replacing g_message() with g_debug()


Version 0.6
===========

Released on January 24th 2014.

Due to the inclusion of Autotools builds, we restructured and cleaned up
installation paths, along the lines of GLib. That means headers are installed
into major API version dependent directories, e.g. now /usr/include/ufo-0/ufo.
We also split hardware-dependent and -independent files, thus kernels go into
/usr/share/ufo now. In the same vein, we also lowered the SO version down to 0,
so please if you have installed from source, make sure to remove any traces of
an old UFO installation before installing the latest version.

Minor changes include:

- Added "time" property to scheduler.
- Scheduler can now use and existing arch graph and return associated resources.
- By default only GPU devices are used.
- Documentation is now hosted at ufo-core.readthedocs.org.
- Added "num-processed" property.

Developers
----------

- Replace GList for loops with g_list_for macro
- Include compat.h
- Renamed and install mkfilter as ufo-mkfilter.
- ufo-core is now monitored by Travis CI
- Removed the python/ subdir which is replaced by ufo-python-tools
- Removed unnecessary clprof tool.

Bugfixes
--------

- Fix and simplify deploy script
- Require json-glib in pkgconfig
- Link to documentation on rtd.org
- Fix .gir generation
- Fix installation path for header files
- Do not error on deprecated declarations


Version 0.5
===========

Released on October 28th 2013.

- Added MPI support as an alternative to ZMQ.
- Added basic math operations for use with filters.
- UFO can now be used reliably in a multithreaded Python context. That means,
  calling ``ufo_scheduler_run`` in a Python thread will "just work". This change
  allows run-time injection of NumPy buffers into the task graph.

Developers
----------
- Add ``-DDEBUG`` when debug is enabled so we can #ifdef it
- Add GLib version guards
- ufo-core compiles with Clang
- CMake 2.6 is used solely throughout the sources
- Add convenience function ufo_buffer_new_with_size
- Add a shim macros to support both zmq 2 and 3
- Add ``UFO_USE_GPU`` env var to restrict to single GPU
- Added ufo_resource_manager_get_cached_kernel that always returns the same
  kernel object when given the same file and kernel name. Note, that you have to
  guard it properly and do not call ``clSetKernelArg`` from multiple threads at
  the same time.
- Add profile tracing to produce a JSON trace event file, that can be read and
  visualized with Google's Chrome browser. It can be enabled with
  UfoScheduler::enable-tracing set to TRUE.

Bug fixes
---------
- Fix #6: Don't use enum values as bit flags
- Fix bug: no plugin name is sent to remote nodes
- Fix copy segfault in when source has not alloc'd
- Removed dependency on a C++ compiler
- Fix reduction problem


Version 0.4
===========

Released on July 18th 2013.

Major changes
-------------

- Rewrote internal architecture for better scheduling.
- Remove profiler levels and add more output
- Implement input data partitioning: On clusters where distributed data access
  is possible, we can achieve perfect linear scalability by partitioning the
  input data set.
- Install SIGTERM handler for cleanup of node server


Features
--------

- Add ufo_task_graph_get_json_data
- Streamline and simplify scheduling
- Provide function to flatten graph
- Provide graph copy functionality
- Add node indices for copies
- Add all paths as OpenCL include paths
- Write out JSON version
- Search in UFO_PLUGIN_PATH env var


Bug fixes
---------

- Fix problems with AMD platforms
- Fix timestamp readout
- Fix potential single integer overflow
- Exit when JSON tasks could not be found
- Fix remote tasks getting stuck
- Unref expanded nodes explicitly
- Fix #189: don't copy nodes with more than one input
- Fix #219: Warn instead of segfault
- Fix annotation for older GI compiler
- Fix problem with first remote data item
- Fix platform selection
- Fix problems with objects that are not unreffed
- Refactor buffer and add support for #184
- Refactor resources and fix #183
- Fix buffer for broadcast operations


Version 0.3
===========

Released on February 8th 2013.

Major breakage
--------------

- A graph is now a simple data structure and not a specific graph of task nodes.
  This is implemented by a TaskGraph.

- Filters are now called TaskNodes and connected with
  ``ufo_task_graph_connect_nodes`` and ``ufo_task_graph_connect_nodes_full``
  respectively.


Graph expansion
---------------

With 0.2, Using multiple GPUs was possible by manually splitting paths in the
graph and assigning GPUs. Now, task graphs are automatically expanded depending
on the number of available GPUs and remote processing slaves that are started
with the newly added ``ufod`` server software.


Minor improvements
------------------

- A ``deploy.sh`` script has been added for easier deployment of the software
  stack. This is especially useful to install everything in the home directory
  of the user, who only needs to setup ``LD_LIBRARY_PATH`` and
  ``GI_TYPELIB_PATH`` correctly to run the software.


Version 0.2
===========

Released on November 8th 2012.

Major breakage
--------------

- Filters are now prefixed again with ``libfilter`` to allow introspected
  documentation. Thus, any filter built for 0.1 cannot be used because they are
  simply not found.

- :c:func:`ufo_plugin_manager_get_filter` received a new third parameter
  ``error`` that reports errors when opening and loading a UfoFilter from a
  shared object.

- :c:func:`ufo_resource_manager_add_program` is removed.

- The kernel file name must be passed to :c:func:`ufo_resource_manager_get_kernel`.

- The ``CHECK_ERROR`` macro defined in ``ufo-resource-manager.h`` was renamed to
  ``CHECK_OPENCL_ERROR`` to better reflect its real purpose.

- The old JSON specification has been changed to reflect the possibilities of
  the current API. Thus, JSON files that worked under Ufo 0.1 cannot be read
  with Ufo 0.2.

- Removed the otherwise unused :c:func:`ufo_buffer_get_transfer_time` and
  replaced this with the more flexible :c:func:`ufo_buffer_get_transfer_timer`.

- Rename :c:func:`ufo_filter_initialize` to
  :c:func:`ufo_filter_set_plugin_name` that reflects its true meaning.


Scheduling
~~~~~~~~~~

A more scheduable way to run filters has been implemented with the virtual
:c:func:`process_cpu` and :c:func:`process_gpu` methods. Contrary to the old
way, they act on *one* working set at a time that is passed as an array of
pointers to :c:type:`UfoBuffer`. Sometimes, a filter needs to setup data
depending on the input size. For this reason, the virtual method
:c:func:`initialize` takes a second parameter that is again a list of pointers
to buffer objects.

Moreover, the :c:type:`UfoScheduler` class has been added that is combining
the work previously accomplished by :c:func:`ufo_filter_process` and
:c:func:`ufo_graph_run`. The scheduler orchestrates the filters and
assigns resources in a meaningful way.

If written in the new kernel style, producer filters must return a boolean flag
denoting if data was produced or not.


General improvements
--------------------

- The manual was restructured considerably.

- Saving graphs as JSON files has been added via
  :c:func:`ufo_graph_save_to_json()`.

- Filters can now wait until their properties satisfy a condition using
  :c:func:`ufo_filter_wait_until`, see also :ref:`filters-block`.

- A new method :c:func:`ufo_resource_manager_get_kernel_from_source` so that
  filters can load kernels directly from source.

- Streamlined error handling: Filters should not issue ``g_warnings`` or
  ``g_errors`` on their own anymore but create an error with ``g_error_new`` and
  return that.


Version 0.1.1
=============

- Ticket #55: tests/test-channel blocks indefinitely