New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Migration for all the model changes for Project Templates #69816
base: master
Are you sure you want to change the base?
Conversation
|
||
class Meta: | ||
app_label = "sentry" | ||
db_table = "sentry_projecttemplate" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this need any unique together constraints like only 1 per org?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In this case, nope, we want to support multiple templates in an org. The feature use case is an organization wants to have a default template for their python backend services and their js FE services.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should the names be unique per org?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oh snap, i did not read that very well. yeah, they should likely be unique names per org. thanks for the shout Colleen and the catch Dan!
src/sentry/models/project.py
Outdated
@@ -245,6 +245,7 @@ class Project(Model, PendingDeletionMixin, OptionMixin, SnowflakeIdMixin): | |||
# projects that were created before this field was present | |||
# will have their first_event field set to date_added | |||
first_event = models.DateTimeField(null=True) | |||
project_template = FlexibleForeignKey("sentry.ProjectTemplate", null=True) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Someone like Dan would know better than me but idk if you can add this and create the new table in one go
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah, i wasn't sure if i should do all 3 model changes in 1 pr or in 3 PRs. I can ping dan and see if this makes sense.
fwiw the migration did work locally.. unclear if that means it'll work in prod or not though 😅
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah ha! it seems that django uses the operations
key to decide the order it applies the migrations: https://github.com/getsentry/sentry/pull/69816/files#diff-705160f3302cfba81a9ad4b3182878d45a36c8a5f3f55d5be85d5ebde083e309R32 - since django automates all this they figured out the application order for me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is fine, you can add an FK and the table in the same migration
5727491
to
ff8d117
Compare
This PR has a migration; here is the generated SQL for --
-- Create model ProjectTemplate
--
CREATE TABLE "sentry_projecttemplate" ("id" bigint NOT NULL PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY, "date_updated" timestamp with time zone NOT NULL, "date_added" timestamp with time zone NULL, "name" varchar(200) NOT NULL, "organization_id" bigint NOT NULL);
ALTER TABLE "sentry_projecttemplate" ADD CONSTRAINT "sentry_projecttempla_organization_id_b072f5c6_fk_sentry_or" FOREIGN KEY ("organization_id") REFERENCES "sentry_organization" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_projecttemplate" VALIDATE CONSTRAINT "sentry_projecttempla_organization_id_b072f5c6_fk_sentry_or";
CREATE INDEX CONCURRENTLY "sentry_projecttemplate_organization_id_b072f5c6" ON "sentry_projecttemplate" ("organization_id");
--
-- Add field template to project
--
ALTER TABLE "sentry_project" ADD COLUMN "template_id" bigint NULL;
--
-- Create model ProjectTemplateOption
--
CREATE TABLE "sentry_projecttemplateoption" ("id" bigint NOT NULL PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY, "key" varchar(64) NOT NULL, "value" text NOT NULL, "project_template_id" bigint NOT NULL);
ALTER TABLE "sentry_project" ADD CONSTRAINT "sentry_project_template_id_67f0bec7_fk_sentry_pr" FOREIGN KEY ("template_id") REFERENCES "sentry_projecttemplate" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_project" VALIDATE CONSTRAINT "sentry_project_template_id_67f0bec7_fk_sentry_pr";
CREATE INDEX CONCURRENTLY "sentry_project_template_id_67f0bec7" ON "sentry_project" ("template_id");
ALTER TABLE "sentry_projecttemplateoption" ADD CONSTRAINT "sentry_projecttempla_project_template_id_23fa7e71_fk_sentry_pr" FOREIGN KEY ("project_template_id") REFERENCES "sentry_projecttemplate" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_projecttemplateoption" VALIDATE CONSTRAINT "sentry_projecttempla_project_template_id_23fa7e71_fk_sentry_pr";
CREATE INDEX CONCURRENTLY "sentry_projecttemplateoption_project_template_id_23fa7e71" ON "sentry_projecttemplateoption" ("project_template_id");
--
-- Create constraint unique_projecttemplate_name_per_org on model projecttemplate
--
CREATE UNIQUE INDEX CONCURRENTLY "unique_projecttemplate_name_per_org" ON "sentry_projecttemplate" ("name", "organization_id");
ALTER TABLE "sentry_projecttemplate" ADD CONSTRAINT "unique_projecttemplate_name_per_org" UNIQUE USING INDEX "unique_projecttemplate_name_per_org";
--
-- Alter unique_together for projecttemplateoption (1 constraint(s))
--
CREATE UNIQUE INDEX CONCURRENTLY "sentry_projecttemplateop_project_template_id_key_87ae3e18_uniq" ON "sentry_projecttemplateoption" ("project_template_id", "key");
ALTER TABLE "sentry_projecttemplateoption" ADD CONSTRAINT "sentry_projecttemplateop_project_template_id_key_87ae3e18_uniq" UNIQUE USING INDEX "sentry_projecttemplateop_project_template_id_key_87ae3e18_uniq"; |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #69816 +/- ##
===========================================
+ Coverage 51.67% 80.00% +28.32%
===========================================
Files 6460 6494 +34
Lines 288653 290026 +1373
Branches 49758 49963 +205
===========================================
+ Hits 149164 232028 +82864
+ Misses 139074 57587 -81487
+ Partials 415 411 -4
|
ff8d117
to
d4f4f00
Compare
|
||
class Meta: | ||
app_label = "sentry" | ||
db_table = "sentry_projecttemplate" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should the names be unique per org?
src/sentry/models/project.py
Outdated
@@ -245,6 +245,7 @@ class Project(Model, PendingDeletionMixin, OptionMixin, SnowflakeIdMixin): | |||
# projects that were created before this field was present | |||
# will have their first_event field set to date_added | |||
first_event = models.DateTimeField(null=True) | |||
project_template = FlexibleForeignKey("sentry.ProjectTemplate", null=True) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is fine, you can add an FK and the table in the same migration
src/sentry/models/project.py
Outdated
@@ -245,6 +245,7 @@ class Project(Model, PendingDeletionMixin, OptionMixin, SnowflakeIdMixin): | |||
# projects that were created before this field was present | |||
# will have their first_event field set to date_added | |||
first_event = models.DateTimeField(null=True) | |||
project_template = FlexibleForeignKey("sentry.ProjectTemplate", null=True) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might make sense as just template
, otherwise we'll be calling project.project_template
a lot
# is a schema change, it's completely safe to run the operation after the code has deployed. | ||
# Once deployed, run these manually via: https://develop.sentry.dev/database-migrations/#migration-deployment | ||
|
||
is_post_deployment = True |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Schema changes can't be post deployment, this will take down sentry, as per the comment (this is the only blocking comment)
from sentry.testutils.cases import TestCase | ||
|
||
|
||
class ProjectTemplateOptionTest(TestCase): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's up to you, but you don't really need to test that the ORM works
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, agree. I just wanted to create a placeholder test for upcoming PRs :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
migration lgtm
primary_key=True, serialize=False | ||
), | ||
), | ||
("key", models.CharField(max_length=64)), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When querying this table will you always have project_template_id
available? If not you'll need an index on key
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, we'll always need the project template id to select the settings. I don't see us needing queries like "what are all the values an organization has set in all of their templates" or "Get me all the values for email prefixes in my templates" -- the design right now is more that we'll apply these settings to a project when getting those options, or when directly manipulating the template itself.
from sentry.db.models.fields import PickledObjectField | ||
|
||
|
||
@region_silo_only_model |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@region_silo_only_model | |
@region_silo_model |
This is in the process of being renamed.
|
||
project_template = FlexibleForeignKey("sentry.ProjectTemplate", related_name="options") | ||
key = models.CharField(max_length=64) | ||
value = PickledObjectField() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure we should be adding more pickled data to our database. Using a text field with JSON encoded payloads would be an improvement over the current option schema.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The pickled field actually just uses json now, there was a project to convert it a while ago. Not sure why we never renamed it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a fixed set of values, or some sort of schema, we expect for this field, or is it completely freeform? If it's the former, we should at least at add a doc comment with some details (see UserOption
for inspiration).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently, it's freeform. We'll be aligning to a subset of project options and new template level options. Since it's not clear what options are going to be in the initial version (waiting on design for those areas), this is purposefully left open until we have a bit more clarity on those details.
I was pushing off that work until the model code has been merged, i kinda figured there'd be a lot of test updates / random things and wanted to keep the complexity of this pr as close to sql changes as possible.
a0f9de3
to
f48f4e3
Compare
This PR has a migration; here is the generated SQL for --
-- Create model ProjectTemplate
--
CREATE TABLE "sentry_projecttemplate" ("id" bigint NOT NULL PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY, "date_updated" timestamp with time zone NOT NULL, "date_added" timestamp with time zone NULL, "name" varchar(200) NOT NULL, "organization_id" bigint NOT NULL);
ALTER TABLE "sentry_projecttemplate" ADD CONSTRAINT "sentry_projecttempla_organization_id_b072f5c6_fk_sentry_or" FOREIGN KEY ("organization_id") REFERENCES "sentry_organization" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_projecttemplate" VALIDATE CONSTRAINT "sentry_projecttempla_organization_id_b072f5c6_fk_sentry_or";
CREATE INDEX CONCURRENTLY "sentry_projecttemplate_organization_id_b072f5c6" ON "sentry_projecttemplate" ("organization_id");
--
-- Add field template to project
--
ALTER TABLE "sentry_project" ADD COLUMN "template_id" bigint NULL;
--
-- Create model ProjectTemplateOption
--
CREATE TABLE "sentry_projecttemplateoption" ("id" bigint NOT NULL PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY, "key" varchar(64) NOT NULL, "value" text NOT NULL, "project_template_id" bigint NOT NULL);
ALTER TABLE "sentry_project" ADD CONSTRAINT "sentry_project_template_id_67f0bec7_fk_sentry_pr" FOREIGN KEY ("template_id") REFERENCES "sentry_projecttemplate" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_project" VALIDATE CONSTRAINT "sentry_project_template_id_67f0bec7_fk_sentry_pr";
CREATE INDEX CONCURRENTLY "sentry_project_template_id_67f0bec7" ON "sentry_project" ("template_id");
ALTER TABLE "sentry_projecttemplateoption" ADD CONSTRAINT "sentry_projecttempla_project_template_id_23fa7e71_fk_sentry_pr" FOREIGN KEY ("project_template_id") REFERENCES "sentry_projecttemplate" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_projecttemplateoption" VALIDATE CONSTRAINT "sentry_projecttempla_project_template_id_23fa7e71_fk_sentry_pr";
CREATE INDEX CONCURRENTLY "sentry_projecttemplateoption_project_template_id_23fa7e71" ON "sentry_projecttemplateoption" ("project_template_id");
--
-- Create constraint unique_projecttemplate_name_per_org on model projecttemplate
--
CREATE UNIQUE INDEX CONCURRENTLY "unique_projecttemplate_name_per_org" ON "sentry_projecttemplate" ("name", "organization_id");
ALTER TABLE "sentry_projecttemplate" ADD CONSTRAINT "unique_projecttemplate_name_per_org" UNIQUE USING INDEX "unique_projecttemplate_name_per_org";
--
-- Alter unique_together for projecttemplateoption (1 constraint(s))
--
CREATE UNIQUE INDEX CONCURRENTLY "sentry_projecttemplateop_project_template_id_key_87ae3e18_uniq" ON "sentry_projecttemplateoption" ("project_template_id", "key");
ALTER TABLE "sentry_projecttemplateoption" ADD CONSTRAINT "sentry_projecttemplateop_project_template_id_key_87ae3e18_uniq" UNIQUE USING INDEX "sentry_projecttemplateop_project_template_id_key_87ae3e18_uniq"; |
f48f4e3
to
9fbef86
Compare
9fbef86
to
a8593c1
Compare
This PR has a migration; here is the generated SQL for --
-- Create model ProjectTemplate
--
CREATE TABLE "sentry_projecttemplate" ("id" bigint NOT NULL PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY, "date_updated" timestamp with time zone NOT NULL, "date_added" timestamp with time zone NULL, "name" varchar(200) NOT NULL, "organization_id" bigint NOT NULL);
ALTER TABLE "sentry_projecttemplate" ADD CONSTRAINT "sentry_projecttempla_organization_id_b072f5c6_fk_sentry_or" FOREIGN KEY ("organization_id") REFERENCES "sentry_organization" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_projecttemplate" VALIDATE CONSTRAINT "sentry_projecttempla_organization_id_b072f5c6_fk_sentry_or";
CREATE INDEX CONCURRENTLY "sentry_projecttemplate_organization_id_b072f5c6" ON "sentry_projecttemplate" ("organization_id");
--
-- Add field template to project
--
ALTER TABLE "sentry_project" ADD COLUMN "template_id" bigint NULL;
--
-- Create model ProjectTemplateOption
--
CREATE TABLE "sentry_projecttemplateoption" ("id" bigint NOT NULL PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY, "key" varchar(64) NOT NULL, "value" text NOT NULL, "project_template_id" bigint NOT NULL);
ALTER TABLE "sentry_project" ADD CONSTRAINT "sentry_project_template_id_67f0bec7_fk_sentry_pr" FOREIGN KEY ("template_id") REFERENCES "sentry_projecttemplate" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_project" VALIDATE CONSTRAINT "sentry_project_template_id_67f0bec7_fk_sentry_pr";
CREATE INDEX CONCURRENTLY "sentry_project_template_id_67f0bec7" ON "sentry_project" ("template_id");
ALTER TABLE "sentry_projecttemplateoption" ADD CONSTRAINT "sentry_projecttempla_project_template_id_23fa7e71_fk_sentry_pr" FOREIGN KEY ("project_template_id") REFERENCES "sentry_projecttemplate" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_projecttemplateoption" VALIDATE CONSTRAINT "sentry_projecttempla_project_template_id_23fa7e71_fk_sentry_pr";
CREATE INDEX CONCURRENTLY "sentry_projecttemplateoption_project_template_id_23fa7e71" ON "sentry_projecttemplateoption" ("project_template_id");
--
-- Create constraint unique_projecttemplate_name_per_org on model projecttemplate
--
CREATE UNIQUE INDEX CONCURRENTLY "unique_projecttemplate_name_per_org" ON "sentry_projecttemplate" ("name", "organization_id");
ALTER TABLE "sentry_projecttemplate" ADD CONSTRAINT "unique_projecttemplate_name_per_org" UNIQUE USING INDEX "unique_projecttemplate_name_per_org";
--
-- Alter unique_together for projecttemplateoption (1 constraint(s))
--
CREATE UNIQUE INDEX CONCURRENTLY "sentry_projecttemplateop_project_template_id_key_87ae3e18_uniq" ON "sentry_projecttemplateoption" ("project_template_id", "key");
ALTER TABLE "sentry_projecttemplateoption" ADD CONSTRAINT "sentry_projecttemplateop_project_template_id_key_87ae3e18_uniq" UNIQUE USING INDEX "sentry_projecttemplateop_project_template_id_key_87ae3e18_uniq"; |
…t_template' to 'template' on the project model
…rectly for the future
43dcaad
to
8fdf00e
Compare
This PR has a migration; here is the generated SQL for --
-- Create model ProjectTemplate
--
CREATE TABLE "sentry_projecttemplate" ("id" bigint NOT NULL PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY, "date_updated" timestamp with time zone NOT NULL, "date_added" timestamp with time zone NULL, "name" varchar(200) NOT NULL, "organization_id" bigint NOT NULL);
ALTER TABLE "sentry_projecttemplate" ADD CONSTRAINT "sentry_projecttempla_organization_id_b072f5c6_fk_sentry_or" FOREIGN KEY ("organization_id") REFERENCES "sentry_organization" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_projecttemplate" VALIDATE CONSTRAINT "sentry_projecttempla_organization_id_b072f5c6_fk_sentry_or";
CREATE INDEX CONCURRENTLY "sentry_projecttemplate_organization_id_b072f5c6" ON "sentry_projecttemplate" ("organization_id");
--
-- Add field template to project
--
ALTER TABLE "sentry_project" ADD COLUMN "template_id" bigint NULL;
--
-- Create model ProjectTemplateOption
--
CREATE TABLE "sentry_projecttemplateoption" ("id" bigint NOT NULL PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY, "key" varchar(64) NOT NULL, "value" text NOT NULL, "project_template_id" bigint NOT NULL);
ALTER TABLE "sentry_project" ADD CONSTRAINT "sentry_project_template_id_67f0bec7_fk_sentry_pr" FOREIGN KEY ("template_id") REFERENCES "sentry_projecttemplate" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_project" VALIDATE CONSTRAINT "sentry_project_template_id_67f0bec7_fk_sentry_pr";
CREATE INDEX CONCURRENTLY "sentry_project_template_id_67f0bec7" ON "sentry_project" ("template_id");
ALTER TABLE "sentry_projecttemplateoption" ADD CONSTRAINT "sentry_projecttempla_project_template_id_23fa7e71_fk_sentry_pr" FOREIGN KEY ("project_template_id") REFERENCES "sentry_projecttemplate" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_projecttemplateoption" VALIDATE CONSTRAINT "sentry_projecttempla_project_template_id_23fa7e71_fk_sentry_pr";
CREATE INDEX CONCURRENTLY "sentry_projecttemplateoption_project_template_id_23fa7e71" ON "sentry_projecttemplateoption" ("project_template_id");
--
-- Create constraint unique_projecttemplate_name_per_org on model projecttemplate
--
CREATE UNIQUE INDEX CONCURRENTLY "unique_projecttemplate_name_per_org" ON "sentry_projecttemplate" ("name", "organization_id");
ALTER TABLE "sentry_projecttemplate" ADD CONSTRAINT "unique_projecttemplate_name_per_org" UNIQUE USING INDEX "unique_projecttemplate_name_per_org";
--
-- Alter unique_together for projecttemplateoption (1 constraint(s))
--
CREATE UNIQUE INDEX CONCURRENTLY "sentry_projecttemplateop_project_template_id_key_87ae3e18_uniq" ON "sentry_projecttemplateoption" ("project_template_id", "key");
ALTER TABLE "sentry_projecttemplateoption" ADD CONSTRAINT "sentry_projecttemplateop_project_template_id_key_87ae3e18_uniq" UNIQUE USING INDEX "sentry_projecttemplateop_project_template_id_key_87ae3e18_uniq"; |
Description
This will create 2 new models,
ProjectTemplate
andProjectTemplateOptions
. These models will be used to store settings across multiple projects. This PR also creates a 1 to 1 relationship forProjectTemplates
andProjects
.Design Document: https://www.notion.so/sentry/Service-Settings-Templates-ca6fdb38b15f4a6db61d319bc342b86c?pvs=4#60623e7fbaba4e0985f8d4581af18e64