New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

api: Create experiment abstraction for closed beta features #1615

Merged

victorges merged 25 commits into master from vg/x/experiment

Jan 23, 2023

Member

victorges commented Jan 17, 2023 •

edited

What does this pull request do? Explain your changes. (required)

This is to create a new abstraction called "experiment" to allow us to iterate on
private experiments with specific customers without risking having a feature widely
available to all the users.

This was originally designed here: https://www.notion.so/livepeer/Feature-Experimentation-Playbook-f80717040b254c1ca3591169bcf6611f

Specific updates (required)

Create new experiment schema/table
Create support for GIN indexes to use in that table
Create new experiment API CRUD
Create specific endpoints for the more useful experiment actions (checking, adding/removing users etc)

How did you test each of these updates (required)
yarn test, also putting on staging to test the internal api

Does this pull request close any open issues?
Implements #1617

Checklist

I have read the CONTRIBUTING document.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have added tests to cover my changes.

vercel bot commented Jan 17, 2023 •

edited

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated
livepeer-studio	✅ Ready (Inspect)	Visit Preview	💬 Add your feedback	Jan 23, 2023 at 10:48PM (UTC)

vercel bot had a problem deploying to Preview

January 17, 2023 00:59

Failure

vercel bot deployed to Preview

January 17, 2023 01:02

View deployment

github-advanced-security bot found potential problems

View reviewed changes

packages/api/src/controllers/experiment.ts

+              app.post(
+                "/:experiment/audience",
+                authorizer({ anyAdmin: true }),

Check failure

Code scanning / CodeQL

Missing rate limiting

This route handler performs [authorization](1), but is not rate-limited.

packages/api/src/controllers/experiment.ts Fixed Show fixed Hide fixed

packages/api/src/controllers/experiment.ts

+                }
+              );
+              app.patch("/:experiment", authorizer({ anyAdmin: true }), async (req, res) => {

Check failure

Code scanning / CodeQL

Missing rate limiting

This route handler performs [authorization](1), but is not rate-limited.

packages/api/src/controllers/experiment.ts

+                res.status(204).end();
+              });
+              app.get("/:experiment", authorizer({ anyAdmin: true }), async (req, res) => {

Check failure

Code scanning / CodeQL

Missing rate limiting

This route handler performs [authorization](1), but is not rate-limited.

packages/api/src/controllers/experiment.ts Fixed Show fixed Hide fixed

vercel bot deployed to Preview

January 17, 2023 01:07

View deployment

github-advanced-security bot found potential problems

View reviewed changes

packages/api/src/controllers/experiment.ts

+              app.post(
+                "/",
+                authorizer({ anyAdmin: true }),

Check failure

Code scanning / CodeQL

Missing rate limiting

This route handler performs [authorization](1), but is not rate-limited.

vercel bot deployed to Preview

January 17, 2023 18:14

View deployment

github-advanced-security bot found potential problems

View reviewed changes

packages/api/src/controllers/experiment.ts

+                userId: `experiment.data->>'userId'`,
+              } as const;
+              app.get("/", authorizer({ anyAdmin: true }), async (req, res) => {

Check failure

Code scanning / CodeQL

Missing rate limiting

This route handler performs [authorization](1), but is not rate-limited.

vercel bot deployed to Preview

January 17, 2023 18:21

View deployment

vercel bot deployed to Preview

January 17, 2023 20:34

View deployment

vercel bot deployed to Preview

January 17, 2023 22:32

View deployment

github-advanced-security bot found potential problems

View reviewed changes

packages/api/src/controllers/experiment.ts

+                app.use(`/api/${experiment}`, experimentSubjectsOnly(experiment), api);
+              }
+              app.get("/check/:experiment", authorizer({}), async (req, res) => {

Check failure

Code scanning / CodeQL

Missing rate limiting

This route handler performs [authorization](1), but is not rate-limited.

victorges changed the title ~~Vg/x/experiment~~ api: Create experiment abstraction for closed beta tests

victorges changed the title ~~api: Create experiment abstraction for closed beta tests~~ api: Create experiment abstraction for closed beta features

vercel bot deployed to Preview

January 18, 2023 17:17

View deployment

victorges marked this pull request as ready for review

January 18, 2023 17:22

victorges requested a review from a team as a code owner

January 18, 2023 17:22

victorges force-pushed the vg/x/experiment branch from e08254b to a2e374f Compare

January 18, 2023 17:24

vercel bot deployed to Preview

January 18, 2023 17:26

View deployment

vercel bot deployed to Preview

January 18, 2023 17:42

View deployment

vercel bot deployed to Preview

January 18, 2023 18:04

View deployment

victorges force-pushed the vg/x/experiment branch from 978044b to 525f072 Compare

January 18, 2023 19:13

vercel bot deployed to Preview

January 18, 2023 19:15

View deployment

vercel bot deployed to Preview

January 18, 2023 19:45

View deployment

vercel bot deployed to Preview

January 18, 2023 20:46

View deployment

victorges force-pushed the vg/x/experiment branch from 78ad148 to f2a2f59 Compare

January 18, 2023 20:55

vercel bot deployed to Preview

January 18, 2023 20:57

View deployment

victorges commented

View reviewed changes

packages/api/src/controllers/experiment.ts Outdated Show resolved Hide resolved

gioelecerati reviewed

View reviewed changes

packages/api/src/controllers/access-control.ts Show resolved Hide resolved

packages/api/src/schema/schema.yaml Outdated Show resolved Hide resolved

packages/api/src/schema/schema.yaml Outdated Show resolved Hide resolved

packages/api/src/controllers/experiment.ts Show resolved Hide resolved


          api: Create experiment API to check a user

62a0510

victorges added 15 commits

January 20, 2023 10:10


          api: Create simple experiment CRUD API

d893167


          api: Fix the schema

51c95f1


          api: Add user ID to experiment object

e9da26f


          api: Create createdAt and updatedAt fields

f4491d1


          api: Create generic listing API to replace by-user endpoint

c245025


          api: Create sub-set of experimental API endpoints

1c56eef


          api: Make experiments API experiment-gated

23045c7


          api: Make isExperimentSubjet a static helper

f3719f5

To be used in experimenting code


          api: Make the experiment check API authorized

273aa3e

Always safer to start closed.


          api: Create some tests for experiment table index

bc151b6


          api: nit return next

0838b7f


          amend to base

2bd7664


          api: Fix experiment schema

8d44648


          api/package.json: Upgrade uuid package

e47a8e4


          api: Fix uuid lib imports

0dd33f0

victorges force-pushed the vg/x/experiment branch from f2a2f59 to 0dd33f0 Compare

January 20, 2023 13:10

vercel bot deployed to Preview

January 20, 2023 13:12

View deployment


          api: Add some unrelated logs to parseFilters 🙈

2eec107

I wanna make the filters actually fail the request if they
are malformed. It's very confusing to get a response that
is completely ignoring some input you provided.

vercel bot deployed to Preview

January 20, 2023 19:45

View deployment

codecov bot commented Jan 20, 2023 •

edited

Codecov Report

Merging #1615 (b7d7199) into master (37589b0) will decrease coverage by 0.87027%.
The diff coverage is 19.29825%.

❗ Current head b7d7199 differs from pull request most recent head e918484. Consider uploading reports for the commit e918484 to get more accurate results

@@                 Coverage Diff                 @@
##              master       #1615         +/-   ##
===================================================
- Coverage   53.78187%   52.91160%   -0.87027%     
===================================================
  Files             70          73          +3     
  Lines           4667        4774        +107     
  Branches         886         914         +28     
===================================================
+ Hits            2510        2526         +16     
- Misses          1837        1927         +90     
- Partials         320         321          +1

Impacted Files	Coverage Δ
packages/api/src/controllers/api-token.js	`65.06024% <ø> (ø)`
packages/api/src/controllers/helpers.ts	`54.94505% <0.00000%> (-0.61051%)`	⬇️
packages/api/src/controllers/webhook.ts	`56.55172% <ø> (ø)`
packages/api/src/store/cdn-usage-table.ts	`21.73913% <ø> (ø)`
packages/api/src/store/experiment-table.ts	`0.00000% <0.00000%> (ø)`
packages/api/src/controllers/experiment.ts	`14.45783% <14.45783%> (ø)`
packages/api/src/store/table.ts	`72.12121% <60.00000%> (-0.54960%)`	⬇️
...rc/controllers/experiment/lit-signing-condition.ts	`66.66667% <66.66667%> (ø)`
packages/api/src/controllers/access-control.ts	`83.67347% <100.00000%> (-0.64026%)`	⬇️
packages/api/src/store/db.ts	`82.41758% <100.00000%> (+0.19536%)`	⬆️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 37589b0...e918484. Read the comment docs.

Impacted Files	Coverage Δ
packages/api/src/controllers/api-token.js	`65.06024% <ø> (ø)`
packages/api/src/controllers/helpers.ts	`54.94505% <0.00000%> (-0.61051%)`	⬇️
packages/api/src/controllers/webhook.ts	`56.55172% <ø> (ø)`
packages/api/src/store/cdn-usage-table.ts	`21.73913% <ø> (ø)`
packages/api/src/store/experiment-table.ts	`0.00000% <0.00000%> (ø)`
packages/api/src/controllers/experiment.ts	`14.45783% <14.45783%> (ø)`
packages/api/src/store/table.ts	`72.12121% <60.00000%> (-0.54960%)`	⬇️
...rc/controllers/experiment/lit-signing-condition.ts	`66.66667% <66.66667%> (ø)`
packages/api/src/controllers/access-control.ts	`83.67347% <100.00000%> (-0.64026%)`	⬇️
packages/api/src/store/db.ts	`82.41758% <100.00000%> (+0.19536%)`	⬆️
... and 1 more


          api: Fix schema based on PR comments

b7d7199

victorges requested a review from a team as a code owner

January 23, 2023 21:07

vercel bot deployed to Preview

January 23, 2023 21:13

View deployment

victorges added 2 commits

January 23, 2023 19:45


          api: Use - as the experimental api separator

4edc8ff


          api: Add tests to experiment API

e918484

victorges enabled auto-merge (squash)

January 23, 2023 22:47

victorges disabled auto-merge

January 23, 2023 22:48

vercel bot deployed to Preview

January 23, 2023 22:48

View deployment

victorges merged commit 813a4d9 into master

victorges deleted the vg/x/experiment branch

January 23, 2023 22:48

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment