We've switched to doing released in order to make sure folks are getting stable code. This release includes many changes (mostly bugfixes) from the past several months.
Here is an abbreviated commit log. If you are updating your local sync engine, note there are several migrations you may need to run.
4df11df [sync] TravisCI testing changes and version bump for OSS release
5619ef9 [developer] More logs display bugfixes
bcef868 [none] Update outdated comment
1b42c6c [admin] Add instructions for perf profiler and make easier to use
3e94a3f [balancer] Don't move Accounts that belong to debug sync instances
d08e69b [api, sync] Handle more IMAP auth errors. Provide better error messages.
aa9b82a [sync] Truncate long calendar names
de16267 [api] Fixes AttributeError where some messages categories are None
5cc00df [sync] Truncate traceback to prevent DataError cont.
dd7bad3 [sync] Truncate traceback to prevent DataError
bcfa1a5 [sync] Make sure the thread exists
4f4bc6f [sync] Don't process event during message reconciliation
0cbd15f [sync] Fixes T7297
c386528 [*] Fix event queue bug
825d9bc [api] Don't schedule draft deletion until message is sent
29a5ed4 [all] Don't include debug instances by default
6123e2a [sync] Handle JSON parsing errors from event queue data
5a38ac6 [sync] Use KillerGreenletTracer for SyncHTTPFrontend
124b57d [sync] Use PPAs instead of CPU history for process limits
9de67a4 [sync] Unpack the right number of values for event queue lpop
2f17ab1 [migrator] Make migrator send heartbeats
3005676 [sync, api, admin, migrator] Guard event listener against multiple commits
622a37f [sync] Use correct end time for all day events
b250782 [sync, api] Mark Accounts that already have post commit listeners
10243b5 [sync] Split shared and private event queue handlers
3b08cf4 [sync] Fix bugs in updated unschedule script
4b0b653 [balancer] Handle when account ID is missing
76f5b13 [sync] Update script to manually unload Accounts
4ed36b8 [balancer] Add stats logging for migration minimization
d58d651 [sync] Flush private queue when receiving an event
2a80cd4 [balancer] Use Hungarian algo to minimize migrations
4b0040f [balancer] Parameterize the timespan for load balancing
2191519 [none] Delete legacy scheduler code
0a9626d [sync] Release owned Accounts that no longer should be
2daba2d [sync] Turn noisy event queue timeout statement down to DEBUG
e850a9f [all] Make database transaction timing metrics optional
2a0719a [sync] Don't discard Account until it's marked as released in the DB
065eb63 [webhooks,sync,api,admin] Consolidate StatsD code, revise metrics names
b950d0a [sync] Notify when Account.sync_should_run changes
8dc90fa [api] [direct_fetching] Handle case where g_msgid is None.
498a9ff [sync] Comment out new statsd counters for now
2c69eb5 [sync] Don't send migrate_from if desired_sync_host is None
be41c73 [sync] Handle more folder roles edge cases
fe8f3db [sync, api] Ignore irrelevent Account updates when sending events
9f65c7b Fix another draft deletion edge-case
1dcf4f9 [api] Add custom provider rule for adaptiveinsights.com
0f323fb [sync] Use gauge for sync_hosts_count metric
82b20fb [sync] Fixup sync restart script
b8b1c27 [sync] Don't manually clear sync host
ed70ec8 [sync] Add function to get vcpu count when balancing fleet
205fc95 [sync] Fix some problems with finding Account loads
1bfe66b [sync] Reduce some code duplication
e67a9a5 [sync] Add script to dump per-Account loads in JSON
88d0184 [sync] Add scripts to dump and restore the assigned sync redis hash
531df0e [api, sync] Handle more errors during device provisioning.
ad6ac16 [sync] Add legacy-balance-fleet to setup.py
1e2aa19 [sync] Add legacy load balancer
8b1ae86 [sync][api] Don't double-save to our legacy S3 bucket anymore
aeae53a Log affected account id.
81e8407 [sync] Add script to restart sync for forgotten Accounts
5ed6047 [api] Direct fetching: handle non-existent UID replies.
d8bb3e4 [syncback] Delete messages in a safe way
36812e6 [migrator] Moar load balancer fixes
aea75f9 [migrator] More sync load balancer fixes
763fa83 [sync] Add script to balance sync fleet
226d322 [api] Try converting g_msgid to a number if we get a string.
3bd4764 [syncback] Merge adjacent SyncbackTasks for change_labels
2103eec [sync] Add service to schedule Account migrations across sync hosts
e7eba03 [sync] Deduplicate migration events
8e015e2 [sync] Add per-Account sync counter for alerts
b81bde9 [sync] Fixup string formatting in a couple scripts
ca24daa Display exception breakdown in direct fetching validation script.
d388df0 [sync] Add /build-metadata endpoint to SyncHTTPFrontend
df7d351 More fixes for account migration
c6c8a40 Fix Account migration stuff
7251aed [schema][sync, api] Add ability to migrate Accounts between sync hosts
b76ad14 Fix SyncHTTPFrontend after refactoring
ed580d6 Commit after undeleting a Thread
c7d1da6 [api] Bump nylas-production-python version
0926449 [sync] Only collect first message metric for inbox
c87a433 [sync] Include application_id in all requests
69df7aa [api] Add provider to api request logs
a4cc16f [api] Log more details on direct fetch failures
8d56dc6 [api] Raise the right exception when trying to fetch a deleted message.
89c78bd Skip null MIME parts.
7aa87f4 Add more graphite logs.
49576e0 [migration, sync] Update imapfolderinfo uidnext to be a bigint instead of int
15b4d17 [direct-fetching] [sync] [api] Phase 2: Stop reading from our permanent bucket
480c75e [sync] Fix truncation for TEXT columns
95cc875 Add the ability to interrupt blocked greenlets
3b0cdce Fix alembic migration script revisions
4ed8cfc More exception catching
0da4345 Log attempts on individual syncback actions
66152e2 Standardize logging actionlog.id as action_log_id
0b77782 Log account_id and num_connections as structured data in crispin pool creation
4d468ee Make the direct fetching code a bit more robust
e74efd2 Add profiler for SyncbackService
e220f21 Add test for correct Contact.uid collation
881548b Revert "Remove Block/Part ForeignKeys"
d7e2af3 Bump nylas-production-python version
ab7bb85 Bump nylas-production-python version
9e4100b Skip Messages that don't have a valid Thread
b8f8f81 [api, sync, schema] Make deleting Threads asynchronous
53fe179 [sync, api] Don't update remote events if the remote event creation failed
f5b5d82 [api] Update fastmail provider settings
25e7a6f [api] Prevent TypeError for recurring calendar events
723d84c Fix incorrect rebase of alembic migration script
9781733 [SCHEMA] Drop foreign keys to and from Contact table
e154aff Remove Block/Part ForeignKeys
fe86721 [Ask @khamidou before shipping] Fetch attachments and raw messages directly from the email provider
3a81f0f [api] Don't leak Exception messages when we don't know what they are!
7ff64a4 Don't send an event invite to the owner.
5b99c86 Set Part.content_id for inline attachments
1f55c83 [api] fix(logging): Add all auth scope logging to WSGI context immediately
727f9dd [api] fix(logging): Log namespace IDs before throwing 404s
e47e6e7 Add support for sending inline images
9a37aab [none] fix(logging): mysql “too many conn” is transient
ed63006 [none] Upgrade coverage dep for more features
9028025 [sync] fix(logging): Get default logger if none provided in callback
f308473 [none] fix(tests): Use a random port for SMTP server
e4aa649 fix(logging): Clip error messages saved to Account objects
2d78db4 fix(logging): Don’t log transient mysql errors
3461444 Standardize test database names
f388883 [sync] Bump IMAPClient to include folder_status fix
20f201f Provide detailed message to users when handling `OperationalError`'s
572b70b Roll back to older pytest
32e2b51 fix(tests): Unbreak local testing, oops
2f8b12e fix(tests): Skip test_block_on_depleted_pool()
7693683 Fix broken JSON in config-test.json
482203a Split test sync-engine shard config out into separate config files
c88471a Remove redwood DB config from sync engine test config
e3eb09b Disable webhooks tests during jenkins runs until they are fixed
af0881f Add iontrading as an exchange provider
3512e7d Prevent ICS organizer CN field from being set to blank
20e121d Bump nylas-production-python version
ab6d56a Don't allow DetachedInstanceError crashes in model repr()
c873997 Fix passing in of echo to engine()
b27868f Quiet noisy 'syncing events' log statement
c48cb33 Fix from address.
f5bf5a8 Fix typo which made code super slow.
5e13deb Turn down the production log volume more
d7828d5 Remove Transaction ForeignKeys
ea8d940 Give more insight about what deletion is doing 😅
485250a Fix dev secrets filename
113b95c Add ZONE in prod test config
4e9603e For prod too
236e968 Restore old SYNCBACK_ASSIGNMENTS values
c860120 Syncback assignments really do have id '0' even though it's not valid JSON
435f06a Add back some keys to the sync-engine test config
7efb16b globs aren't strings
19be6ef Deduplicate test configs
e9f80b7 Make 'syncing contacts' message DEBUG also
2f51947 Further reduce log volume
363f98d Remove extraneous conftest.py
87d8359 Install pytest.ini into package
49c4371 Pass 127.0.0.1 as the hostname to mysql when dropping/creating test tables
f9d9b62 sync-engine system tests depend on unspecified nylas API client library
318e52e s/localhost/127.0.0.1/ on DATABASE_USERS also
7661f81 Use 127.0.0.1 instead of localhost in test mysql configs
1c02fee Give all test_util.py a different name
e24c1d8 Fix SMTP SSL test
cb1d356 Fix more tests
ce226d9 Fix more tests in install
da74be1 Ship JSON files too
511fb55 Fix a bunch of tests
c6901b1 Fix up MANIFEST.in to ship test data
3abcc69 Fix Exchange tests to work in installed form
c22eb3a Combine inbox.test dirs in installation
127e617 Make tests work when package is installed too
34276c6 Install test configs for sync-engine too
5928e13 Add tox to sync-engine deps
01fa0ac Distribute tests with source code