Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(replay): add kafka config for new ingest-feedbacks consumer #66169

Closed
wants to merge 148 commits into from
Closed
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
148 commits
Select commit Hold shift + click to select a range
62dd685
feat(replay): add kafka config for new ingest-feedbacks consumer
aliu3ntry Mar 1, 2024
78cedd6
Merge branch 'master' into aliu/ingest-feedbacks-config
aliu3ntry Mar 1, 2024
2552f1a
Merge branch 'master' into aliu/ingest-feedbacks-config
lynnagara Mar 4, 2024
7292129
Merge branch 'master' into aliu/ingest-feedbacks-config
aliu3ntry Mar 4, 2024
f0f614a
add ingest-feedbacks to KAFKA_TOPIC_TO_CLUSTER and kafka_definition
aliu3ntry Mar 5, 2024
4dc60ad
feat(replay): remove organizations:session-replay-event-linking featu…
aliu3ntry Mar 4, 2024
69e2a34
fix(metrics-extraction): Fix meta lookup in factory (#66256)
k-fish Mar 4, 2024
9d26087
fix(metrics-extraction): Allow overriding split decision (#66259)
k-fish Mar 4, 2024
35a45b3
feat(autofix): timeout with autofix, query state (#66241)
corps Mar 4, 2024
9ae6f76
feat(new-trace): Added logic for red errored rows. (#66194)
Abdkhan14 Mar 4, 2024
1b3e8c4
feat(replay): adds current user to replay in header (#66253)
scefali Mar 4, 2024
8763274
fix(crons): Remove db constraint from historical migrations for monit…
wedamija Mar 4, 2024
0f346d9
feat(backup): Enable shimming deleted fields (#66248)
azaslavsky Mar 4, 2024
8823af4
ref: Move topics to new-style of Kafka topic definition (#66271)
lynnagara Mar 4, 2024
99e584e
fix(replay): delete flaky test test_get_replays_dead_rage_click_cutof…
aliu3ntry Mar 5, 2024
81f6adc
fix(performance): Turn off INP on the browser integration sdk (#66279)
edwardgou-sentry Mar 5, 2024
83e2e1c
ref(crons): Remove unused code branch in mark_failed (#66284)
evanpurkhiser Mar 5, 2024
b1e0f4e
chore(ddm): Remove formula feature flag in BE (#66199)
ArthurKnaus Mar 5, 2024
05b4d67
ref(dynamic-sampling): Remove capture message for timing (#66245)
iambriccardo Mar 5, 2024
cb4ca9d
feat(ddm): Add support for units normalization in formulas (#66213)
iambriccardo Mar 5, 2024
ddbc653
fix(ddm): Sample uniformly the list of requested code locations (#66296)
iambriccardo Mar 5, 2024
9ab3434
Clean up `ArtifactBundle(FlatFile)Index`, step 2 (#66207)
Swatinem Mar 5, 2024
dc7d11c
ref(ddm): Remove normalization logic (#66132)
ArthurKnaus Mar 5, 2024
23a731e
feat(ddm): Allow multiple use case ids to be queried at once and para…
iambriccardo Mar 5, 2024
edd1f49
perf: use pipeline for redis rate limiter (#64428)
anonrig Mar 5, 2024
4a5aa18
perf: do not include projects and teams (#66232)
anonrig Mar 5, 2024
ed56fba
fix(ddm): Duplicate chart sample ids (#66301)
ArthurKnaus Mar 5, 2024
9750bb4
Remove `FileBlob` locking entirely (#66297)
Swatinem Mar 5, 2024
d78df18
ref(perf): Introduce `MetricReadout` component (#66254)
gggritso Mar 5, 2024
06b99fb
Bump Sentry Python SDK to version 2.0.0a2 (#66052)
antonpirker Mar 5, 2024
64bbcdf
fix(metrics-extraction): Fix attribute vs dict lookup (#66306)
k-fish Mar 5, 2024
7be67a1
Revert "Bump Sentry Python SDK to version 2.0.0a2 (#66052)"
getsentry-bot Mar 5, 2024
b611054
fix(ddm): Fix units normalization when * and / are used (#66304)
iambriccardo Mar 5, 2024
d367e7b
fix(performance): Fix a typo in the mri of measurements.inp (#66273)
edwardgou-sentry Mar 5, 2024
7f1dd10
fix(ddm): Handle division by zero (#66311)
iambriccardo Mar 5, 2024
5de51dc
Revert "ref(sdk): Bump js sdk version to 7.105.0 (#66224)"
getsentry-bot Mar 5, 2024
32b35e7
fix(profiling): Add missing transaction condition for profile digest …
Zylphrex Mar 5, 2024
ce126ce
feat(proguard): Add Symbolicator.process_jvm method (#66064)
loewenheim Mar 5, 2024
302e39b
fix(hybridcloud) Add visiblity to region configuration (#66295)
markstory Mar 5, 2024
f79cdb0
fix(hybrid-cloud): Prevent API gateway from resolving redirects (#66291)
dashed Mar 5, 2024
c94ba94
fix padding (#66318)
DominikB2014 Mar 5, 2024
608780f
feat(sdk): bumps js sdk to 7.103.0 and updates browsertracingintegrat…
edwardgou-sentry Mar 5, 2024
8ed8f67
feat(indexed-spans): Support profile.id on indexed spans (#66178)
Zylphrex Mar 5, 2024
1cebf7f
fix(perf): Fix HTTP widget documentation link (#66011)
gggritso Mar 5, 2024
b01c76a
feat(processing) Move redis operations in reprocessing to servicewrap…
markstory Mar 5, 2024
a39c154
ref(metrics): Change aggregation options to be a dictionary (#65666)
evanh Mar 5, 2024
72cf7e3
ref(crons): Remove legacy fingerprint generation (#66250)
evanpurkhiser Mar 5, 2024
2906d16
fix(daily summary): Fix subquery, exit early if no users (#66261)
ceorourke Mar 5, 2024
64ba793
feat(issue-priority): Rename issue stream tab inbox -> priority and r…
malwilley Mar 5, 2024
65c709f
ref: Move more code to the new way of defining kafka topics and overr…
lynnagara Mar 5, 2024
625ae35
feat(issues): Remove quicktrace component, move tracelink (#66333)
scttcper Mar 5, 2024
29cd673
feat(performance): Updates useIndexedSpans to accept an optional fiel…
edwardgou-sentry Mar 5, 2024
fcadc17
ref(feedback): toggles are false by default (#66331)
michellewzhang Mar 5, 2024
3ca6676
feat(issue-priority): Use 'Med' label in priority pills (#66272)
malwilley Mar 5, 2024
1794c57
ref: Move query subscriptions over to new-style kafka config (#66287)
lynnagara Mar 5, 2024
a0359e9
styles(metrics): Use disabled button for no profile (#66341)
Zylphrex Mar 5, 2024
e18b1be
ref(issues): Add typing to issue status and issue platform files (#66…
snigdhas Mar 5, 2024
89bf6ae
ref(feedback): update links and system default for onboarding (#66330)
michellewzhang Mar 5, 2024
c6fa668
chore(commit-context): Remove unused feature flag (#66345)
leeandher Mar 5, 2024
274acff
ref(crons): Remove usage of last_state_change (#66328)
evanpurkhiser Mar 5, 2024
1b4fba0
fix(staff): Set access by calling parent method before checking staff…
schew2381 Mar 5, 2024
468f10f
fix(metrics): Update indexer telemetry to handle no messages (#66350)
john-z-yang Mar 5, 2024
242e6a6
feat(discover): Remove replay_id column from default discover table v…
ryan953 Mar 5, 2024
0029dd8
fix(ui): Remove custom prism text selection styles (#66240)
malwilley Mar 5, 2024
bf65b9e
ref(crons): Use `id__in` query for failed check-in occurrence creatio…
evanpurkhiser Mar 5, 2024
ff74225
fix(hc): Provide region specific urls for chunk-upload config (#66320)
GabeVillalobos Mar 5, 2024
e574575
ref(feedback): add rust to platforms (#66356)
michellewzhang Mar 5, 2024
db28b55
Revert "ref: Move more code to the new way of defining kafka topics a…
getsentry-bot Mar 5, 2024
5a5b331
Revert "ref: Move query subscriptions over to new-style kafka config …
getsentry-bot Mar 5, 2024
9e6f13b
ref(crons): Drop last_state_change column (#66353)
evanpurkhiser Mar 5, 2024
f8649f6
bump more_slow_alerts to 300 alerts (#66367)
nhsiehgit Mar 5, 2024
442f7e4
Activated alert rule apis feature flag (#66361)
nhsiehgit Mar 5, 2024
8631dad
ref(daily summary): Add logger when sending summary (#66369)
ceorourke Mar 5, 2024
b9d94b8
feat(issue-priority): Add last edited by info and feedback button to …
malwilley Mar 5, 2024
1a3dbf6
ref(crons): Clean up mark_failed logic (#66372)
evanpurkhiser Mar 5, 2024
7445433
docs: Wrap stories with OrganizationContainer (#66323)
ryan953 Mar 5, 2024
9e85a14
logging(group-owner): adds logging for create group owners (#66371)
scefali Mar 5, 2024
7d16cb4
fix(crons): Fix backfill script to handle the case where a monitor wi…
wedamija Mar 6, 2024
5005304
logging(codeowners): adds logging when we delete code group owners (#…
scefali Mar 6, 2024
ef8ec57
ref(feedback): remove duplicate native platform (#66384)
michellewzhang Mar 6, 2024
57a37c3
feat(issue-details): Improved UI for viewing object/array values (#66…
malwilley Mar 6, 2024
75f021c
ref: Move code to new-style kafka topics - take 2 (#66381)
lynnagara Mar 6, 2024
8fd0983
test(ui): Add mock date test helpers, remove fake timers (#66342)
scttcper Mar 6, 2024
dde199c
chore(crons): Return a better message from deprecated crons ingest ap…
wedamija Mar 6, 2024
1525c6b
ref(ddm): Remove multiple metrics meta queries (#66305)
ArthurKnaus Mar 6, 2024
7ff66f2
fix(loader-script-setting): Cache update (#66396)
ArthurKnaus Mar 6, 2024
f143d17
Bump Sentry Python SDK to version 2.0.0a2 (#66397)
antonpirker Mar 6, 2024
83faf43
Revert "Bump Sentry Python SDK to version 2.0.0a2 (#66397)"
getsentry-bot Mar 6, 2024
770de43
feat(ddm): Filter empty series from summary table (#66400)
ArthurKnaus Mar 6, 2024
e8cedc2
Clean up `ArtifactBundle(FlatFile)Index`, step 3 (#66208)
Swatinem Mar 6, 2024
e51094f
feat(ddm): Hiding queries (#66398)
ArthurKnaus Mar 6, 2024
910ed95
fix(ddm): Show samples only on selected widget (#66404)
ArthurKnaus Mar 6, 2024
58b07d2
ref(deps): Bump sentry-relay to v0.8.48 (#66401)
iker-barriocanal Mar 6, 2024
0c285c0
feat(discover): Remove commented out replay ID column (#66409)
narsaynorath Mar 6, 2024
3afdb48
chore(hybridcloud) Remove unused parameter from sent_incident_alert_n…
markstory Mar 6, 2024
0ca6661
fix(hybridcloud) Make bitbucket uninstall silo safe (#66354)
markstory Mar 6, 2024
2a749cd
fix(hybridcloud) Add logging for bad RPC requests (#66357)
markstory Mar 6, 2024
01103b7
feat(replay): Add feature flag for mobile replay player (#66276)
billyvg Mar 6, 2024
ce143c1
feat(spans): Tag perf problems created from standalone spans (#66325)
shruthilayaj Mar 6, 2024
1445beb
Remove rollout options for Rust Enhancers (#65966)
Swatinem Mar 6, 2024
de9eedd
feat(profiling): pass internal (profile-scoped) dsn for function metr…
viglia Mar 6, 2024
2b9958a
fix(mql): Handle strings and formula with scalars bug (#66324)
evanh Mar 6, 2024
cfd8949
feat(hybridcloud) Add region tag to errors/transactions (#66370)
markstory Mar 6, 2024
b0a2c9e
feat(perf): Add HTTP module domain overview page scaffold (#66359)
gggritso Mar 6, 2024
345f8d9
revert(sdk): Move us back to 7.102 js sdk (#66412)
k-fish Mar 6, 2024
e3330ff
ref(onboarding): refactor useCurrentProjectState to be generalized (#…
michellewzhang Mar 6, 2024
6e2a7b3
Revert "ref(onboarding): refactor useCurrentProjectState to be genera…
getsentry-bot Mar 6, 2024
739564a
feat(code-mappings): LA automatic code mapping support for PHP projec…
MichaelSun48 Mar 6, 2024
5007096
chore(superuser): Add logs to check u2f (#66393)
schew2381 Mar 6, 2024
703029c
ref(crons): Avoid extra hashing on fingerprint UUIDs (#66373)
evanpurkhiser Mar 6, 2024
5c9084c
ref(onboarding): refactor useCurrentProjectState to be generalized (#…
michellewzhang Mar 6, 2024
8485db2
fix(metrics): Adjust bounding boxes based on operation (#66339)
Zylphrex Mar 6, 2024
90b2ace
chore(superuser): Add loggers to _needs_validation (#66364)
schew2381 Mar 6, 2024
ff5e9d2
feat(discover): allows filtering metric queries on span.op (#66423)
edwardgou-sentry Mar 6, 2024
8bf591b
ref(replay): see full replay in rage click issue goes to breadcrumbs …
michellewzhang Mar 6, 2024
a5e235c
ref(perf): Remove usage of generic-ish `SpanMetricsRibbon` (#66415)
gggritso Mar 6, 2024
300a604
ref: Remove old style topic definition from ingest consumer code (#66…
lynnagara Mar 6, 2024
c7e9a43
feat(feedback/issues): add sidebar to issue details UF empty state (#…
michellewzhang Mar 6, 2024
fb5a7a7
fix(feedback): correct usage of option name (#66430)
JoshFerge Mar 6, 2024
64a74e2
ref(replays): remove rage click creation celery task (#66360)
JoshFerge Mar 6, 2024
3d24962
fix(queries): Add more errors to queryhandler (#66432)
wmak Mar 6, 2024
8b74892
fix(profiling): Add handle_query_errors to profiling (#66433)
wmak Mar 6, 2024
d286f50
feat(crons): Add new broken detection model (#66375)
davidenwang Mar 6, 2024
fd11248
test(ui): Switch children to react nodes (#66390)
scttcper Mar 6, 2024
06ed897
ref(issues): Add object type to KeyValue data item (#66391)
scttcper Mar 6, 2024
c7f8945
ref(ui): Remove render function from dropdown label (#66389)
scttcper Mar 6, 2024
efb1f92
feat(crons): Show project information in details area for crons (#65944)
davidenwang Mar 6, 2024
99de5a5
ref(js): useOrganization in ProjectBadge (#66442)
evanpurkhiser Mar 6, 2024
b1d5691
ref(crons): Drop last_state_change from db (#66429)
evanpurkhiser Mar 6, 2024
2bbcdc7
feat(webvitals): Updates web vital module query hooks to actually que…
edwardgou-sentry Mar 6, 2024
5e372c0
feat(webvitals): update calculatePerformanceScoreFromStoredTableDataR…
edwardgou-sentry Mar 6, 2024
fd1151d
feat(metrics): Support paginating samples via random seed (#66440)
Zylphrex Mar 6, 2024
a23749e
feat(webvitals): Greys out FID in the performance score breakdown cha…
edwardgou-sentry Mar 6, 2024
bd775f9
feat(webvitals): updates webvitals landing page slideout panel to sup…
edwardgou-sentry Mar 6, 2024
3bea14c
ref(sdk): Bump js sdk to 7.105 again (#66436)
k-fish Mar 6, 2024
1537c1f
ref: Move the rest of the consumer definitions to the new style (#66343)
lynnagara Mar 6, 2024
cdaf91c
fix(event_manager): catch kafka exceptions in perf issue production (…
JoshFerge Mar 6, 2024
5c7f1f2
ref: split incident and alert_rule models into separate files (#66386)
nhsiehgit Mar 6, 2024
dfd60b1
fix(ondemand): Wrong column usage here was breaking (#66437)
wmak Mar 6, 2024
a79af6e
fix(event-stats): Remove error from top events (#66455)
wmak Mar 6, 2024
36afa8a
feat(api): Allowlist for API Pagination Check (#66463)
iamrajjoshi Mar 6, 2024
de7fb8c
ref(crons): Move grouphash generation to a field default (#66456)
evanpurkhiser Mar 6, 2024
a63025a
ref(crons): Clarify why incident creation skip happens (#66462)
evanpurkhiser Mar 6, 2024
55b3275
feat(perf): Record occurrence payload size if it's over the limit (#6…
gggritso Mar 6, 2024
c0ede95
chore(inbound-filters): Remove feature flag for default inbound filte…
roggenkemper Mar 6, 2024
b49b81a
ref: Remove references to deprecated settings (#66355)
lynnagara Mar 6, 2024
47f0b8f
test(ui): Vendor `react-select-event`, use RTL user events (#66441)
scttcper Mar 6, 2024
55fede8
rename to ingest-feedback-events
aliu3ntry Mar 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions src/sentry/conf/server.py
Expand Up @@ -3452,6 +3452,7 @@ def build_cdc_postgres_init_db_volume(settings: Any) -> dict[str, dict[str, str]
KAFKA_SESSIONS_SUBSCRIPTIONS_RESULTS = "sessions-subscription-results"
KAFKA_METRICS_SUBSCRIPTIONS_RESULTS = "metrics-subscription-results"
KAFKA_INGEST_EVENTS = "ingest-events"
KAFKA_INGEST_FEEDBACKS = "ingest-feedbacks"
aliu3ntry marked this conversation as resolved.
Show resolved Hide resolved
KAFKA_INGEST_EVENTS_DLQ = "ingest-events-dlq"
KAFKA_INGEST_ATTACHMENTS = "ingest-attachments"
KAFKA_INGEST_TRANSACTIONS = "ingest-transactions"
Expand Down
8 changes: 8 additions & 0 deletions src/sentry/consumers/__init__.py
Expand Up @@ -269,6 +269,14 @@ def ingest_events_options() -> list[click.Option]:
"consumer_type": "events",
},
},
"ingest-feedbacks": {
"topic": settings.KAFKA_INGEST_FEEDBACKS,
"strategy_factory": "sentry.ingest.consumer.factory.IngestStrategyFactory",
Copy link
Member

@lynnagara lynnagara Mar 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each consumer should have it's own strategy. The ingest strategy is already doing far more than it should and we should not add additional logic to it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any tips/references for how to write a new strategy? In ingest/consumer/factory.py, right

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The directory structure in sentry is a bit misleading, but the ingest module really just refers to the errors/transactions ingest consumer today and not any of the rest of the piplines. I would suggest keeping it either entirely separate, or inside a new module ingest.user_feedback or something like that. See https://getsentry.github.io/arroyo/ for docs how to write a strategy.

Copy link
Member

@cmanallen cmanallen Mar 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lynnagara We're not adding additional logic to it. This behavior already exists in that consumer. We're just moving it to a new topic and new consumer infrastructure. I agree we should split out this product into its own consumer. Can that be handled in a follow-up PR?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah makes sense, I thought it was totally new functionality.

"click_options": ingest_events_options(),
"static_args": {
"consumer_type": "events",
Copy link
Member

@lynnagara lynnagara Mar 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you might want to make this a new type. We don't really want to run all the events logic and configuration (even if they share the bulk of code in the initial implementation), and i'm guessing this will diverge in the future?

},
},
"ingest-attachments": {
"topic": settings.KAFKA_INGEST_ATTACHMENTS,
"strategy_factory": "sentry.ingest.consumer.factory.IngestStrategyFactory",
Expand Down