Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(generic-metrics): Don't produce headers unnecessarily #67396

Closed
wants to merge 3 commits into from
Closed

feat(generic-metrics): Don't produce headers unnecessarily #67396

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
94461cd
feat(generic-metrics): Don't produce headers unnecessarily
lynnagara Mar 20, 2024
1c0963a
headers
lynnagara Mar 21, 2024
30f3943
release health too
lynnagara Mar 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
11 changes: 1 addition & 10 deletions src/sentry/sentry_metrics/consumers/indexer/batch.py
View file
Open in desktop
Expand Up @@ -410,10 +410,6 @@ def reconstruct_messages(
fetch_types_encountered.add(metadata.fetch_type)
output_message_meta[metadata.fetch_type.value][str(metadata.id)] = tag

mapping_header_content = bytes(
"".join(sorted(t.value for t in fetch_types_encountered)), "utf-8"
)

numeric_metric_id = mapping[use_case_id][org_id][metric_name]
if numeric_metric_id is None:
metadata = bulk_record_meta[use_case_id][org_id].get(metric_name)
Expand Down Expand Up @@ -501,12 +497,7 @@ def reconstruct_messages(
kafka_payload = KafkaPayload(
key=message.payload.key,
value=rapidjson.dumps(new_payload_value).encode(),
headers=[
*message.payload.headers,
("mapping_sources", mapping_header_content),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just realized, this is technically still used by the last seen updater, so I'm not sure we can safely delete these headers while that consumer is still around.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should be fine, as I think we'll just hit this condition in the last seen updater https://github.com/getsentry/sentry/blob/master/src/sentry/sentry_metrics/consumers/last_seen_updater.py#L52-L65 and the message will be ignored/dropped.

Still, maybe let's make a note of this -- that by removing the headers the last seen updater will no longer be updating the last_seen column in PG.

# XXX: type mismatch, but seems to work fine in prod
("metric_type", new_payload_value["type"]), # type: ignore
],
headers=[],
)
if self.is_output_sliced:
routing_payload = RoutingPayload(
Expand Down
5 changes: 0 additions & 5 deletions tests/sentry/sentry_metrics/test_gen_metrics_multiprocess_steps.py
View file
Open in desktop
Expand Up @@ -63,11 +63,6 @@ def compare_messages_ignoring_mapping_metadata(actual: Message, expected: Messag

assert actual_payload.key == expected_payload.key

actual_headers_without_mapping_sources = [
(k, v.encode()) for k, v in actual_payload.headers if k != "mapping_sources"
]
assert actual_headers_without_mapping_sources == expected_payload.headers

actual_deserialized = json.loads(actual_payload.value)
expected_deserialized = json.loads(expected_payload.value)
del actual_deserialized["mapping_meta"]
Expand Down
5 changes: 0 additions & 5 deletions tests/sentry/sentry_metrics/test_rh_metrics_multiprocess_steps.py
View file
Open in desktop
Expand Up @@ -62,11 +62,6 @@ def compare_messages_ignoring_mapping_metadata(actual: Message, expected: Messag

assert actual_payload.key == expected_payload.key

actual_headers_without_mapping_sources = [
(k, v.encode()) for k, v in actual_payload.headers if k != "mapping_sources"
]
assert actual_headers_without_mapping_sources == expected_payload.headers

actual_deserialized = json.loads(actual_payload.value)
expected_deserialized = json.loads(expected_payload.value)
del actual_deserialized["mapping_meta"]
Expand Down