Mutually exclusive flippers

[kykyi]

Would a feature for mutually exclusive flippers be useful?

I have a situation where I have a group has_no_subscription and I want to give 50% of them show_x_product and the other 50% I want to give show_y_product.

As I understand doing this in the Flipper UI would mean assigning each flipper to 50% of actors, which would mean any has_no_subscription user would have a 50% chance of getting show_x_product, 50% chance of show_x_product, 25% chance of both, and 25% chance of neither.

[thetimbanks]

Could you set that up as a single flipper with 50% actor setting and when it is active you can show_y_product and if not active show_x_product? That would give a 50/50 split on each product and keep it under one flipper.

[kykyi]

Thank for the response @thetimbanks! That will def work for the 2 product case, but we will likely add a show_z_product and beyond, and afaik this approach couldn't handle this.

[jnunemaker]

It sounds like you are trying to do multi-variate (a/b) testing or something similar. Is that the case?

For now, flipper is only focused on true/false, not a/b/c so fitting stuff like this in is probably tough.

I will say its hard for me to picture the use case like this so any more detail on why you need to do it this way or the end goal or real names instead of product_x and product_y would be helpful. The more real and concrete the problem the easier it is for me to see the shape of a solution.

[kykyi]

@jnunemaker we are trying to experiment with price anchoring between our subscription products, and to do this are giving flippers to three groups.

The first group is the control, the second only sees the most expensive, and the third sees the cheapest and most expensive. These are all users who do not have a subscription, so are part of the has_no_subscription group.

This was previously how we were managing the flippers through the UI, but as groups on the UI are not mutually exclusive, it was possible for a user to have all 3 flippers which is not ideal for the experiment!

To work around this we are manually assigning the flippers in the backend when the user is created based on a 1/3 probability which is obviously a bit of a hack. If 0-0.33 then we Flipper.enable_actor :control_group, @user and so on for the other two groups.

As you said flipper is more about true/false so we might be stretching the use case for flipper, but wanted to start a discussion to clarify 🙏

[jhubert]

To work around this we are manually assigning the flippers in the backend when the user is created based on a 1/3 probability which is obviously a bit of a hack. If 0-0.33 then we Flipper.enable_actor :control_group, @user and so on for the other two groups.

Just chiming in here two years later to say that I would strongly recommend doing this part outside of flipper entirely. Just add two new models in your Rails app experiments and experiment_participants, and then randomly assign users to experiment buckets based on whatever proportions you want.

Then use the user's participation in an experiment bucket as the condition for the feature flag. You can do all of this in a very simple ruby code block. No visual interface required, which would save a lot of dev time. Then you have a nice database record about what experiments a user has participated in, which will give you even more control down the road if you want to exclude people from certain experiments if they've already taken part in others.

Background: I've built and managed large A/B bucketing systems at Yahoo! and other companies.

[kykyi]

To work around this we are manually assigning the flippers in the backend when the user is created based on a 1/3 probability which is obviously a bit of a hack. If 0-0.33 then we Flipper.enable_actor :control_group, @user and so on for the other two groups.

Just chiming in here two years later to say that I would strongly recommend doing this part outside of flipper entirely. Just add two new models in your Rails app experiments and experiment_participants, and then randomly assign users to experiment buckets based on whatever proportions you want.

Then use the user's participation in an experiment bucket as the condition for the feature flag. You can do all of this in a very simple ruby code block. No visual interface required, which would save a lot of dev time. Then you have a nice database record about what experiments a user has participated in, which will give you even more control down the road if you want to exclude people from certain experiments if they've already taken part in others.

Background: I've built and managed large A/B bucketing systems at Yahoo! and other companies.

Great call 100% agree