Bulk operations in UI & API

[andrewmyersyapily]

Hi,

I can see in the public roadmap (Q2-2022) "🔥 Discovery Items" has Bulk operations in UI & API listed.

My team has a use case that would hopefully benefit from new features discussed here.

As of speaking we're running 4 Unleash instances and would like the ability to automatically roll out updates in bulk to keep consistency between environments whilst reducing chance for manual errors.

Exploring the documentation there is the option to export toggles, strategies and variants. When re-importing the output file only new/missing toggles are added, updates to toggles are not propagated unless calling PUT via the API on individual toggles. This introduces its own problems with state and rollback if an update fails.

Ideally, we need to version/migrate the database with toggle changes and integrate this into a review process.
e.g. keep toggles, strategies, variants in a configuration file that can be peer reviewed

Would love to hear if anything like this is in plan.

Thanks,
Andrew

[andrewmyersyapily]

Link to card here https://github.com/orgs/Unleash/projects/5#card-80376609

[thomasheartman]

Hey, @andrewmyersyapily!

You're right that the import/export API could use some love and some more features. At the moment, though, if you just want to keep everything in sync, you might be able to get away with passing the drop=true query parameter (mentioned in the API import section) That said, that can be risky (and is not the right thing to do if you've got other toggles that you wanna keep), hence why we generally don't recommend it for production use.

So if that doesn't cover your use case, you might be better served by adding/updating features one by one or by doing database backups and restores. Would that be an option for you?

The feature was not prioritized for the current quarter, but we may look at it in Q4 or Q1 next year. Getting your input is super helpful, by the way! It lets us discover more use cases and user needs and it shows that there is a need for this 😄

[andrewmyersyapily]

Hi @thomasheartman,

Thanks for the reply!

Yes, we are aware of the drop before import option but as you said it's risky on a production environment and event history is lost. I imagine database backups and restores face similar risk as the production database could become unavailable.

I've been exploring a method to apply updates using a custom node start script, we can apply updates one by one on startup. Similar to the stateService example provided in the documentation we can retrieve the featureToggle service like so

const { featureToggleServiceV2 } = services;

And then call

await featureToggleServiceV2.createFeatureToggle('default', dto, 'userName')

However, I'm struggling to make this transactional. For example if an operation fails we'd need to rollback to guarantee consistency between unleash instances - either everything applies or nothing. So far I've tried a database transaction using knex, e.g.

await db.transaction(async trx => { 
              await featureToggleServiceV2.createFeatureToggle('default', dto, 'userName') 
              // apply further updates in here
})

Unfortunately this doesn't work without refactoring the underlying Unleash services (featureToggleStore etc).

I will continue to explore potential solutions, please let me know if I can be any help with providing more feedback!

[thomasheartman]

Got it! Thanks for the input. That does indeed sound like a possible way to go, but as you say, it probably does not have the transactional guarantees you're looking for. At this point I'm a bit out of my depth, but I imagine @chriswk might have some input that could be useful here!

[dsusa72]

Another community member (Shea Styer) provided good insights on This slack thread.