Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mismatch in count of OTel attributes to index and default label limit #12913

Open
lindeskar opened this issue May 8, 2024 · 2 comments · May be fixed by #12969
Open

Mismatch in count of OTel attributes to index and default label limit #12913

lindeskar opened this issue May 8, 2024 · 2 comments · May be fixed by #12969

Comments

@lindeskar
Copy link
Contributor

Describe the bug
The count of OTel attributes to index (17) and the default label limit (15) do not match:

otlp_config:
# List of default otlp resource attributes to be picked as index labels
# CLI flag: -distributor.otlp.default_resource_attributes_as_index_labels
[default_resource_attributes_as_index_labels: <list of strings> | default = [service.name service.namespace service.instance.id deployment.environment cloud.region cloud.availability_zone k8s.cluster.name k8s.namespace.name k8s.pod.name k8s.container.name container.name k8s.replicaset.name k8s.deployment.name k8s.statefulset.name k8s.daemonset.name k8s.cronjob.name k8s.job.name]]

# Maximum number of label names per series.
# CLI flag: -validation.max-label-names-per-series
[max_label_names_per_series: <int> | default = 15]

The 17 OTel resource attributes are also listed in the docs, which mentions the old default limit (max_label_names_per_series: 30):

- Index labels: Resource attributes map well to index labels in Loki, since both usually identify the source of the logs. Because Loki has a limit of 30 index labels, we have selected the following resource attributes to be stored as index labels, while the remaining attributes are stored as [Structured Metadata]({{< relref "../../get-started/labels/structured-metadata" >}}) with each log entry:
- cloud.availability_zone
- cloud.region
- container.name
- deployment.environment
- k8s.cluster.name
- k8s.container.name
- k8s.cronjob.name
- k8s.daemonset.name
- k8s.deployment.name
- k8s.job.name
- k8s.namespace.name
- k8s.pod.name
- k8s.replicaset.name
- k8s.statefulset.name
- service.instance.id
- service.name
- service.namespace

To Reproduce
Steps to reproduce the behavior:

  1. Read the docs

Expected behavior
The count of attributes and labels match.

Environment:

Screenshots, Promtail config, or terminal output

Comment
I would have created a PR to fix the docs, but I suspect this might lead to a default config change?

@lindeskar
Copy link
Contributor Author

lindeskar commented May 8, 2024

Maybe the idea was that all attributes will never be used at once? (Deployment / StatefulSet / DaemonSet / ...)

If so, this issue is just reporting the old default limit (30) in:

- Index labels: Resource attributes map well to index labels in Loki, since both usually identify the source of the logs. Because Loki has a limit of 30 index labels, we have selected the following resource attributes to be stored as index labels, while the remaining attributes are stored as [Structured Metadata]({{< relref "../../get-started/labels/structured-metadata" >}}) with each log entry:

@sandeepsukhani
Copy link
Contributor

Maybe the idea was that all attributes will never be used at once? (Deployment / StatefulSet / DaemonSet / ...)

Yeah, not all attributes will be set at once. Some of them are mutually exclusive.

If so, this issue is just reporting the old default limit (30) in:

Yeah, it was a mistake. I think I referenced the old limit, which was 30 earlier. Thanks for pointing it out! I will fix it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants