Feature request: Add more telemetry metrics and insight for usage analytics

[akphi]

Similar requests

• I have searched and found no existing similar requests

How are you using Legend?

Legend Studio, Legend Query, Legend Taxonomy, Legend Pure IDE

What problems are you trying to solve?

We have laid out the foundation for telemetry across all the apps, we now should systematically find metrics that worths capturing, they divide roughly into 2 camps:

• Usage metrics: what actions do users take, what do they use (potentially, for deeper study, how do they use that feature, where do they use it from, etc.)
• Performance metrics: For long-running actions, what is the timing, what is the load, etc.

I want the team to work on this thread together to exhaustively find all the interesting metrics to capture, also potentially, we can discuss popular capturing techniques to generalize that into some method inside of ApplicationStore.TelemetryService

And most importantly, after we capture those metrics, what is the best way to present those metrics in dashboard to provide value and insights into users' behavior

Describe the solution you would like to see implemented

• Think about another kind of endpoint that does the absolute, call a bunch of APIs that pulls stats like: total number of queries, dataspaces, projects, etc. and create cron job to push these data to analytics service
• Include generic object to indicate users' workspace: SDLC coordinates, GAV, Query ID, etc.
• Metrics How can we track the number of times people flip between text/form mode - check for windows visibility
  • Leaving/entering text-mode
  • Service test runner
  • Mapping test runner
  • Service registration
  • For compilation / graph load / execution
  • Check entitlement
  • Mapping algorithm
  • Loading model
  • Pushing change
  • Updating workspace
  • Updating project configuration
  • Database builder
  • Sample data generator: metrics for launch, with format info
  • Query Builder: Preview data
  • Query Builder: Typeahead
• THINK Consider the following aspects of metrics:
  • For performance metrics, what kind of timings should we capture? For multi-staged timings, what do we do?
  • How can we justify that a feature is popular, should we based it off the percentage of unique user using it?
  • Think about how we can determine feature to deprecate
  • Think about how we can make use of application session telemetry to study users' behavior per session, such as how frequently they do action a, b, c, etc.
  • Think about the popularity of a feature, percentage of users that use the feature - just to see how popular the actions are

Describe alternatives you have considered

No response

Documentation, Design, Adoption, Migration Strategy

No response

Contribution

• I would like to work on this feature