Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

kubernetes_logs source metrics not available #20491

Closed
ShahroZafar opened this issue May 14, 2024 · 2 comments 路 Fixed by #20530
Closed

kubernetes_logs source metrics not available #20491

ShahroZafar opened this issue May 14, 2024 · 2 comments 路 Fixed by #20530
Labels
type: bug A code related bug.

Comments

@ShahroZafar
Copy link

ShahroZafar commented May 14, 2024

A note for the community

  • Please vote on this issue by adding a 馃憤 reaction to the original issue to help the community and maintainers prioritize this request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

We dont see any metrics related to kubernetes_logs source. Placing some for example below:

k8s_format_picker_edge_cases_total
k8s_stream_processed_bytes_total

Configuration

sources:
    kubernetes_logs:
      type: kubernetes_logs
      glob_minimum_cooldown_ms: 5000
      ingestion_timestamp_field: "ingest_timestamp"
      include_paths_glob_patterns:
        - /var/log/pods/istio-ingress_*/**
      namespace_annotation_fields: 
        namespace_labels: ""
      node_annotation_fields:
        node_labels: ""
      pod_annotation_fields:
        container_id: ""
        container_image_id: ""
    internal_metrics:
      type: internal_metrics
  sinks:
    prom_exporter:
      type: prometheus_exporter
      flush_period_secs: 30
      inputs: 
        - internal_metrics
      address: 0.0.0.0:9090
      buffer:
        type: disk
        when_full: block
        max_size: 4000000000
    kafka:
      type: kafka
      inputs:
        - kubernetes_logs
      bootstrap_servers: brokers:9092
      topic: vector-ingress-logs
      encoding:
        codec: json
      compression: zstd
      healthcheck:
        enabled: true
      batch:
        timeout_secs: 0.2
      librdkafka_options:
        client.id: "vector"
        request.required.acks: "1"
        message.timeout.ms: "0"
        linger.ms: "100"
        batch.size: "1000000"
      message_timeout_ms: 0
      buffer:
        type: disk
        when_full: block
        max_size: 4000000000

Version

3.7.1

Debug Output

No response

Example Data

No response

Additional Context

No response

References

No response

@ShahroZafar ShahroZafar added the type: bug A code related bug. label May 14, 2024
@ShahroZafar ShahroZafar changed the title Kubernetes_logs sink metrics not available Kubernetes_logs source metrics not available May 14, 2024
@ShahroZafar ShahroZafar changed the title Kubernetes_logs source metrics not available kubernetes_logs source metrics not available May 14, 2024
@jszwedko
Copy link
Member

Thanks for reporting this @ShahroZafar . Some metrics were removed, but it appears we neglected to remove them from the docs. I opened #20530 to correct that.

k8s_stream_processed_bytes_total was replaced by component_received_bytes_total
k8s_format_picker_edge_cases_total is still in there but will only be reported if actually hit (that is it won't report with the initial value of 0; it'll only report when an edge case was hit)

Hope this helps!

@ShahroZafar
Copy link
Author

@jszwedko Thanks for the clarification

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug A code related bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants