Proposal: events for tracking conversion goals

[fahad19]

Background

Featurevisor allows setting up A/B and multivariate tests:

• https://featurevisor.com/docs/use-cases/experiments/
• https://featurevisor.com/docs/features/#variations
• https://featurevisor.com/docs/sdks/javascript/#getting-variations

Experiment activations

When feature (experiment) variations are evaluated, they can also be "activated" for tracking purposes:

• https://featurevisor.com/docs/sdks/javascript/#activation

Analytics

While Featurevisor itself is not an analytics tool, the SDKs provide API to track the experiment activations with your preferred analytics service.

Challenge: tracking conversion goals

The idea of experimenting is to ultimately figure out which variation is converting better. Meaning, which variation is resulting into our users doing what we intend them to do (conversion goal).

Examples include:

• Anonymous user visiting our landing page, and signing up successfully
• User purchasing a product
• User returning to our application daily
• ...and many more

Without conversion goals, we don't have anything to aim towards and see which variation of a feature is winning/losing.

Like having two variations of a button in a landing page, resulting into more/less successful signups.

Proposal: events

(Not to be confused with SDK events like ready, refresh, and update: https://featurevisor.com/docs/sdks/javascript/#events)

Next to:

• attribute: https://featurevisor.com/docs/attributes/
• segment: https://featurevisor.com/docs/segments/
• feature: https://featurevisor.com/docs/features/
• group: https://featurevisor.com/docs/groups/

We could introduce a new event type for defining what are the possible conversion goals that we wish to track.

Defining events

They could reside in the events directory:

# events/user_signup.yml
description: User has signed up successfully (for documentation purpose only)

# optional schema for collecting additional information when tracking
schema:
  - key: userId
    type: string
    description: User's ID in UUID format
  - key: country
    type: string
    description: Country where the user signed up in
  - key: campaignId
    type: string
    description: Campaign ID that brought the user into our app

Or, the schema could be composed based on already defined attributes:

# events/user_signup.yml
description: User has signed up successfully (for documentation purpose only)

# optional schema for collecting additional information when tracking
attributes:
  - userId
  - country
  - campaignId

Tracking with SDK

To track events with SDKs:

// your-application/index.js
import { createInstance } from "@featurevisor/sdk";

const f = createInstance({
  datafileUrl: "https://yoursite.com/datafile.json",
  onReady: () => console.log("SDK is ready"),

  // handle tracked events
  onTrack: (eventName, payload) {
    // ...send logs to your analytics service
  },
});

// track events
f.trackEvent("user_signup", {
  userId: "...",
  country: "nl",
  campaignId: "...",
});

Referencing events in defined features

If we are already defining features with variations, and we know what conversion goals we should measure, we can reference them from our feature definition directly:

# features/landing_page_redesign.yml

# ...

events:
  - user_signup

Code generation

Featurevisor currently supports generating code for your features as documented here: https://featurevisor.com/docs/code-generation/

This is completely optional for you to adopt.

For convenience, we could expand the code generation to also support new event type, where produced code could be used like this:

import { UserSignupEvent } from "@yourorg/features";

UserSignUpEvent.track({
  userId: "...",
  country: "nl",
  campaignId: "...",
});

Since we have schemas defined already for our events, TypeScript will help developers know what to pass when tracking specific events.

[meirroth]

@fahad19 Interesting. What would you benefit by tracking events through Featurevisor SDK? For example, you can track Sign up event directly to your analytics platform, what would you gain by running it through Featurevisor SDK?

[fahad19]

very good question, @meirroth. breaking down my thought process which can lead to a few different directions below:

1: Designing experiments keeping conversion goals in mind

At the moment, Featurevisor allows expressing experiments in the form of variations inside features, but without any ties to what conversion goals (events) they should impact.

From purely tooling perspective, events have no place in Featurevisor.

But from the workflow perspective around experimentation, it does make sense that we:

• encourage/force those who design experiments to also bring in their own events in the same Pull Request
• and associate those events to the experiments
• making everyone aware what they want to achieve with their experiment

It's more of a gap in experimentation workflow that I am looking at, rather than thinking what a particular tool should or should not do.

2: Separate events management solution

Building Featurevisor also gave me ideas about managing schemas in a similar fashion, with a much broader use case covering:

• user activity event schemas (like conversion goals that this proposal is discussing about)
• REST API contracts (for aligning on frontend to backend to backend communication)
• cross-platform interfaces for data structures
• schemas for payloads in an event driven microservices architecture
• generating code out of expressed schemas
• ...many more

If such a tool is built, then we can also guide users to define their conversion goal events there, and then show them how that tool can be integrated with Featurevisor SDKs in their application runtime. Thus, closing the loop of designing experiments and also creating event schemas while tying them together.

---

Just some thoughts in my head, and I haven't really settled on any direction yet.

But tbh, I am leaning a bit more towards the second option. I am yet to study the use cases more thoroughly before I attempt to build something new, because the surface area with schema management is really large.

[fahad19]

Closing this. I will likely create a separate solution for this idea. Featurevisor stays simple focusing on what it does best.