Release Notes 0.23.6

Release Notes 0.23.6

Overview

The 0.23.6 release is a strongly recommended upgrade for ALL Learning Registry node deployments, as some internal changes that make distribution more compliant towards the LR Service specification cause distribute functionality to fail when interacting with nodes running versions prior to 0.23.6.

Below find the key features addressed in this release.

To upgrade an existing installation, please review the Upgrade Guide for 0.23.6.

Key Features

• Data Services NEW
  • The extract API is included and enabled in this distribution. The Extract API brings a "batteries included" interface for tailoring data extraction and query to suit more narrow and varied use case needs without high resource requirements.
  • Data Service views are NOT installed by default. Node administrators can install any or all of the 3 standards alignment focused data services using python CouchApp. This is done to prevent unneeded consumption of disk space for data services not utilized.
  • Interactive documentation explaining Data Services and how to roll your own for your use case.
• Resource Data Distribution refactoring CHANGED
  • Distribute has been significantly refactored to reduce the need for 2x storage for documents. New distribute uses 2 DB's, incoming and resource_data. New tainted documents can be distributed to incoming, and node may impose an internal policy for untainting documents before moving into resource_data. All harvest services still operate against resource_data.
  • More compatible with Learning Registry 0.23 specification. Document Types of resource_data are distributed, not resource_data_distributable.
  • Upgrade will require some minor configuration changes to NGINX to change or expose incoming endpoint.
  • IMPORTANT Legacy nodes can distribute to a 0.23.6 by adjusting service documents to use the incoming endpoint. Legacy cannot be the destination for a 0.23.6 server.
• Subscription for Distribute NEW
  • You may now visit http://[node address]/register, enter the URL of your node (use https if SSL required to access).
• Support for CouchDB 1.2.0 NEW
  • This is a highly recommended upgrade. Significantly improves the storage and resource utilization, as well paves path for features planned for future releases.
• Many Bug fixes
  • See details on the 0.23.6 Milestone Closed Issues

Minimum Requirements

• Linux/Windows OS
• NGINX w/ uwsgi support
• Python 2.7.3 (+setuptools; +virtualenv)
• CouchDB 1.1.1 (1.2.0 Recommended)

Commits

• 3368700 add a feature to quiet logging during certain operations
• a756613 force incoming to rescan all docs on startup
• 5f13ad1 updated to delete for spec validation errors or conflicts
• d832b01 fixed type of doc -> newDoc
• a5acb7e fixed type of doc -> newDoc
• 01c408a updated to handle -distributable docs and delete properly
• d10bf1f Adding Compaction Handler since @wegrata probably missed it in a checkin
• 0fe98fc updated to run compation in seperate handler, with seperate threshold
• ba2fecf Removed filter for -distributable
• af5d599 catch an error on open a url upon startup.
• 861dea9 assumed default couch.url in config
• 76743b6 wrong node specified in config.
• d15f4a0 added except clause for conflicts to keep from getting logged twice
• a4b60b2 Added reference to SpecValidationError and fixed the expected doc_type
• 829c636 fix for reading possibly the wrong config file
• 2acca36 Validates document and cleans up from incomming
• 0720bc2 exclude the couchapp command extension from being considered a test by nose
• 7d317bc updated to install data_service view for testing
• 8b20a4f make the test wait longer for updater to catch up.
• 0cf759c make the test wait longer for updater to catch up.
• ace426f couch not defined early enough
• b97e660 foobar'd syntax error.
• 9fcf328 foobar'd syntax error.
• ca7e54b try making test wait until indexer is idle before trying to start
• 409416d updated to use deepcopy to store previous config for service
• 5f363b1 updated test_obtain_resource_locator so leftover values in DB don't cause an error, as it's pulling all the content from the DB anyway
• ddadea7 updated to work with current config rather than hard coded url
• 5323e1d updated tests and decorators so they consistently work together
• b72559e Fixed tests expecting flow control to enable it, also added a decorator to make this easier for future tests
• 41dfbd6 attempt to fix slice test
• f4be1bf change destination url to something 'almost' guaranteed.
• 8956db7 fixed bug #203
• e89d98f Fixed bug #203
• 4a4191a added constant in place of magic number
• 1e0fb77 fixed array element type checking
• 286434d add forgotten template
• bc9fd8a added test and code to handle no key being passed in
• d2784a4 fixed missing keys bug
• 1e46363 Fixed bug and added tests
• 859c06f add incoming to node config files automatically
• 8edae39 added incoming values
• 8a9fb84 added incoming values
• bb70b46 reverted back to creating monitors separately
• c1d0b8e forgot to include distribute controller with previous commit
• 6fddc33 copies instead of replicate
• fedfafe changes to fix partial discriminators
• 97777af fix the ddoc name to match folder name
• 67e3e2b add new data service that combines the metadata and paradata
• 8912dd8 switched handlers around
• ab346e9 Found a way to get complex discriminators to to work with "starts with"
• e920c0a Fix issue where when no term is being specified, one was being generated plus adding a generated tstamp, causing over filtering to occur
• 681ea60 fix an issue where we have views with timestamps, but no way to display. Also fix grouping which was grouping more than it should
• 04b538e okay made the parameters work with empty result and with starts-with and complex keys for discriminator
• e36c8c1 fixed unicode encoding
• ef1916f creates incoming db now
• 15f76ce creates DB in run if not already created
• ecfdb8f refactored couchMonitor process
• a7285a7 added try catch for dev.ini value
• 5bca546 added try statement for obtaining config file value
• 558902d added try statement for obtaining value out of config file
• aa444de incoming db handler added
• 05d08b1 changes incorporate incoming DB
• 839afbd added view compaction
• fd43b84 updated tests and error responses
• cb52a4d added basic unit tests
• c5a7b27 added form to register node for replication
• 31cee58 Added tests
• ad9168b added check to verify the design doc is part of data services
• 908e844 fixed some bugs and added some tests
• 4597e1e fixed typo in function call
• 12feb03 fixed so documents don't get emitted more than once
• f340678 updated to use views, assumes the default view is to-json
• 984783d updating the dct_conformsTo and adding a list function
• 3a533b4 mixup dct-conformsTo tests
• e8bf276 move functions around to make more modular
• c9bc092 deprecations of some old functions no longer needed.
• 7966e30 Add support for getting Filter function to evaluate
• 7f6b800 Make loading list function more streamlined
• ef27be8 Cleanup and test framework bug fixes
• cd89f9c Got unit testing working again
• 06627d8 remove log statement
• 30a0543 removing reduce functions and making list function do something useful
• 36a1497 adding basic to-json list function
• 3c49633 fixed spelling
• c4f7354 simple test list function
• daa4290 fix typo
• 216ca6c Cleaned up logging and added noise level to logging
• f9425c7 Updated core test suite, and added tests for paradata
• e51b5a8 updated python layer
• b76a27e Updated to determine param order
• a2720f9 add tests to .couchappignore
• 67a1dad added some unit testing and working on extracting paradata
• 27a6f44 fixed self.couchapps
• 8808861 Updated data services prototype
• e8ae06e Update network diagram
• 52a4ae5 Allow distribution gateway to common node on the same network
• 9dcefee renamed to standards-alignment-dct-conformsTo
• d0fa988 fixed routing
• f1ebc4c Adding skeleton for python layer of data services
• f6800a8 ont the fly
• 119c20f output mockup
• f1564f2 Revert Array Element Type Restriction
• 8bc1b6b Adding some more documentation to README.
• c7687dd Cleanup and verify examples
• e59619e fixed bad readme
• 9c645c0 Checkpoint
• 6a54788 modified to use a long for the datetime, the long represents seconds since midning jan 1 1970
• 975295a changed the work Key to Discriminator
• a8842e8 Initial working revision
• 0c70f3f removed unnecessary constant in model_parser
• d668cc3 added more descriptive error codes/conditions on publish
• d0940dd cleaned up comment
• 4b0d2ea check for string values inside arrays when parsing model
• 6005cda handled missing/invalid doc version with better error msg
• 749b9a9 Fix typo
• 5e11773 Push couchapps for active services only
• 09f1756 Fixed bug #189
• d611e9d Remove unsed code
• ff795a8 Minor clean up
• cafec37 Refactor Node Start/Stop
• 733b722 Update to run on Windows
• e629957 Use BaseMonintorChange Use BaseChangeMonitor class the is inherits from a thread when run on windows,but from process otherwise.
• d94ad54 Runs on Windows
• 07003a6 Update for PT story 23705115, add param to slice that says if the view is updated
• 205a4e6 Fixed to work with newest description doc and with the old incorrect format