Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logzio plugin: Not rendering output_include_tags nor output_include_time #1723

Open
kloudwrangler opened this issue Apr 18, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@kloudwrangler
Copy link

kloudwrangler commented Apr 18, 2024

Describe the bug:
When creating a clusteroutput of type logz, I explicitly set the following fields

output_include_tags: false
output_include_time: false

However, the configuration produced by this CRD does not set these fields. This causes a problem as these fields default to true when not set.

https://github.com/logzio/fluent-plugin-logzio/blob/master/lib/fluent/plugin/out_logzio_buffered.rb#L15-L16

Here is the CRD

apiVersion: logging.banzaicloud.io/v1beta1
kind: ClusterOutput
metadata:
  name: logzio
  namespace: cattle-logging-system
spec:
  logz:
    buffer:
      chunk_limit_size: 16m
      flush_interval: 5s
      flush_mode: interval
      flush_thread_count: 4
      queue_limit_length: 4096
      type: file
    endpoint:
      port: 8071
      token:
        valueFrom:
          secretKeyRef:
            key: token
            name: logz-token
      url: https://listener-eu.logz.io
    output_include_tags: false
    output_include_time: false

Here is the relevant part of output

<match **>
    @type logzio_buffered
    @id clusterflow:cattle-logging-system:webapp:clusteroutput:cattle-logging-system:logzio
    endpoint_url https://listener-eu.logz.io:8071
    <buffer tag,time>
      @type file
      chunk_limit_size 16m
      flush_interval 5s
      flush_mode interval
      flush_thread_count 4
      path /buffers/clusterflow:cattle-logging-system:webapp:clusteroutput:cattle-logging-system:logzio.*.buffer
      queue_limit_length 4096
      retry_forever true
      timekey 10m
      timekey_wait 1m
    </buffer>
  </match>

Expected behaviour:
When explicitly setting these fields in the ClusterOutput, it should produce a fluentd configuration with those fields explicitly set e.g.

<match **>
    @type logzio_buffered
    @id clusterflow:cattle-logging-system:webapp:clusteroutput:cattle-logging-system:logzio
    endpoint_url https://listener-eu.logz.io:8071
    output_include_tags false
    output_include_time false

Also, if not set, then it shall leave it blank as these fields will default to true if not set

Steps to reproduce the bug:

# Create a fake token
kubectl apply -f - <<EOF
apiVersion: v1
data:
  token: ZmFrZXRva2Vu
kind: Secret
metadata:
  name: logz-token
  namespace: cattle-logging-system
type: Generic
EOF

# Create a clusterflow of type logz.io 
kubectl apply -f - <<EOF
apiVersion: logging.banzaicloud.io/v1beta1
kind: ClusterOutput
metadata:
  name: logzio
  namespace: cattle-logging-system
spec:
  logz:
    buffer:
      chunk_limit_size: 16m
      flush_interval: 5s
      flush_mode: interval
      flush_thread_count: 4
      queue_limit_length: 4096
      type: file
    endpoint:
      port: 8071
      token:
        valueFrom:
          secretKeyRef:
            key: token
            name: logz-token
      url: https://listener-eu.logz.io
    output_include_tags: false
    output_include_time: false

EOF

# After some time, look at the produced configuration by looking at the secret
kubectl get secret logging-operator-fluentd-app -o jsonpath="{.data['fluentd\.conf']}" | base64 --decode | grep "output_include_tags"

Additional context:
None

Environment details:

  • Kubernetes version (e.g. v1.15.2): v1.27.7
  • Cloud-provider/provisioner (e.g. vSphere/Rancher RKE2): vSphere/Rancher RKE2
  • logging-operator version (e.g. 4.5.6): 4.5.6
  • Install method (e.g. helm or static manifests):
  • Logs from the misbehaving component (and any other relevant logs): None
  • Resource definition (possibly in YAML format) that caused the issue, without sensitive data: Provided above

/kind bug

@kloudwrangler kloudwrangler added the bug Something isn't working label Apr 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant