Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[receiver/k8sobject] request giving a option to use raw json object to represent k8s object #25857

Closed
Frapschen opened this issue Aug 17, 2023 · 13 comments
Closed

[receiver/k8sobject] request giving a option to use raw json object to represent k8s object #25857

Frapschen opened this issue Aug 17, 2023 · 13 comments
Assignees
@Frapschen
Labels
enhancement New feature or request receiver/k8sobjects

Comments

@Frapschen
Copy link
Contributor

Component(s)

receiver/k8sobjects

Describe the issue you're reporting

As an example of the event watcher, the receiver uses the map to represent the event object:

opentelemetry-collector-contrib/receiver/k8sobjectsreceiver/unstructured_to_logdata.go

Lines 75 to 78 in 919e970

dest := record.Body()
destMap := dest.SetEmptyMap()
//nolint:errcheck
destMap.FromRaw(e.Object)

this result taht the event JSON format will be like(use file exporter):

{"resourceLogs":[{"resource":{},"scopeLogs":[{"scope":{},"logRecords":[{"observedTimeUnixNano":"1692261066062813364","body":{"kvlistValue":{"values":[{"key":"type","value":{"stringValue":"ADDED"}},{"key":"object","value":{"kvlistValue":{"values":[{"key":"lastTimestamp","value":{"stringValue":"2023-08-17T08:31:06Z"}},{"key":"type","value":{"stringValue":"Normal"}},{"key":"reportingComponent","value":{"stringValue":""}},{"key":"apiVersion","value":{"stringValue":"v1"}},{"key":"message","value":{"stringValue":"Started container shell"}},{"key":"firstTimestamp","value":{"stringValue":"2023-08-17T08:31:06Z"}},{"key":"count","value":{"intValue":"1"}},{"key":"eventTime","value":{}},{"key":"kind","value":{"stringValue":"Event"}},{"key":"metadata","value":{"kvlistValue":{"values":[{"key":"resourceVersion","value":{"stringValue":"8018197"}},{"key":"creationTimestamp","value":{"stringValue":"2023-08-17T08:31:06Z"}},{"key":"managedFields","value":{"arrayValue":{"values":[{"kvlistValue":{"values":[{"key":"operation","value":{"stringValue":"Update"}},{"key":"apiVersion","value":{"stringValue":"v1"}},{"key":"time","value":{"stringValue":"2023-08-17T08:31:06Z"}},{"key":"fieldsType","value":{"stringValue":"FieldsV1"}},{"key":"fieldsV1","value":{"kvlistValue":{"values":[{"key":"f:message","value":{"kvlistValue":{}}},{"key":"f:reason","value":{"kvlistValue":{}}},{"key":"f:source","value":{"kvlistValue":{"values":[{"key":"f:component","value":{"kvlistValue":{}}},{"key":"f:host","value":{"kvlistValue":{}}}]}}},{"key":"f:type","value":{"kvlistValue":{}}},{"key":"f:count","value":{"kvlistValue":{}}},{"key":"f:firstTimestamp","value":{"kvlistValue":{}}},{"key":"f:involvedObject","value":{"kvlistValue":{}}},{"key":"f:lastTimestamp","value":{"kvlistValue":{}}}]}}},{"key":"manager","value":{"stringValue":"kubelet"}}]}}]}}},{"key":"name","value":{"stringValue":"node-shell-4113b2c5-d7e8-4b5d-903f-c8c59f0e537b.177c1e793d86209c"}},{"key":"namespace","value":{"stringValue":"kube-system"}},{"key":"uid","value":{"stringValue":"cffdab10-3631-4af5-bdf6-63ea3b6c02bc"}}]}}},{"key":"source","value":{"kvlistValue":{"values":[{"key":"component","value":{"stringValue":"kubelet"}},{"key":"host","value":{"stringValue":"workload-04"}}]}}},{"key":"involvedObject","value":{"kvlistValue":{"values":[{"key":"kind","value":{"stringValue":"Pod"}},{"key":"namespace","value":{"stringValue":"kube-system"}},{"key":"name","value":{"stringValue":"node-shell-4113b2c5-d7e8-4b5d-903f-c8c59f0e537b"}},{"key":"uid","value":{"stringValue":"1299adf6-a2ae-47a2-a19a-a50eadb32e3a"}},{"key":"apiVersion","value":{"stringValue":"v1"}},{"key":"resourceVersion","value":{"stringValue":"8018190"}},{"key":"fieldPath","value":{"stringValue":"spec.containers{shell}"}}]}}},{"key":"reason","value":{"stringValue":"Started"}},{"key":"reportingInstance","value":{"stringValue":""}}]}}}]}},"attributes":[{"key":"k8s.resource.name","value":{"stringValue":"events"}},{"key":"event.domain","value":{"stringValue":"k8s"}},{"key":"event.name","value":{"stringValue":"node-shell-4113b2c5-d7e8-4b5d-903f-c8c59f0e537b.177c1e793d86209c"}}],"traceId":"","spanId":""}]}]}]}

after structure, its body looks like this:

{
  "body": {
    "kvlistValue": {
      "values": [
        {
          "key": "type",
          "value": {
            "stringValue": "ADDED"
          }
        },
        {
          "key": "object",
          "value": {
            "kvlistValue": {
              "values": [
                {
                  "key": "lastTimestamp",
                  "value": {
                    "stringValue": "2023-08-17T08:31:06Z"
                  }
                },
                {
                  "key": "type",
                  "value": {
                    "stringValue": "Normal"
                  }
                },
                {
                  "key": "reportingComponent",
                  "value": {
                    "stringValue": ""
                  }
                },
                {
                  "key": "apiVersion",
                  "value": {
                    "stringValue": "v1"
                  }
                },
                {
                  "key": "message",
                  "value": {
                    "stringValue": "Started container shell"
                  }
                },
                {
                  "key": "firstTimestamp",
                  "value": {
                    "stringValue": "2023-08-17T08:31:06Z"
                  }
                },
                {
                  "key": "count",
                  "value": {
                    "intValue": "1"
                  }
                },
                {
                  "key": "eventTime",
                  "value": {}
                },
                {
                  "key": "kind",
                  "value": {
                    "stringValue": "Event"
                  }
                },
                {
                  "key": "metadata",
                  "value": {
                    "kvlistValue": {
                      "values": [
                        {
                          "key": "resourceVersion",
                          "value": {
                            "stringValue": "8018197"
                          }
                        },
                        {
                          "key": "creationTimestamp",
                          "value": {
                            "stringValue": "2023-08-17T08:31:06Z"
                          }
                        },
                        {
                          "key": "managedFields",
                          "value": {
                            "arrayValue": {
                              "values": [
                                {
                                  "kvlistValue": {
                                    "values": [
                                      {
                                        "key": "operation",
                                        "value": {
                                          "stringValue": "Update"
                                        }
                                      },
                                      {
                                        "key": "apiVersion",
                                        "value": {
                                          "stringValue": "v1"
                                        }
                                      },
                                      {
                                        "key": "time",
                                        "value": {
                                          "stringValue": "2023-08-17T08:31:06Z"
                                        }
                                      },
                                      {
                                        "key": "fieldsType",
                                        "value": {
                                          "stringValue": "FieldsV1"
                                        }
                                      },
                                      {
                                        "key": "fieldsV1",
                                        "value": {
                                          "kvlistValue": {
                                            "values": [
                                              {
                                                "key": "f:message",
                                                "value": {
                                                  "kvlistValue": {}
                                                }
                                              },
                                              {
                                                "key": "f:reason",
                                                "value": {
                                                  "kvlistValue": {}
                                                }
                                              },
                                              {
                                                "key": "f:source",
                                                "value": {
                                                  "kvlistValue": {
                                                    "values": [
                                                      {
                                                        "key": "f:component",
                                                        "value": {
                                                          "kvlistValue": {}
                                                        }
                                                      },
                                                      {
                                                        "key": "f:host",
                                                        "value": {
                                                          "kvlistValue": {}
                                                        }
                                                      }
                                                    ]
                                                  }
                                                }
                                              },
                                              {
                                                "key": "f:type",
                                                "value": {
                                                  "kvlistValue": {}
                                                }
                                              },
                                              {
                                                "key": "f:count",
                                                "value": {
                                                  "kvlistValue": {}
                                                }
                                              },
                                              {
                                                "key": "f:firstTimestamp",
                                                "value": {
                                                  "kvlistValue": {}
                                                }
                                              },
                                              {
                                                "key": "f:involvedObject",
                                                "value": {
                                                  "kvlistValue": {}
                                                }
                                              },
                                              {
                                                "key": "f:lastTimestamp",
                                                "value": {
                                                  "kvlistValue": {}
                                                }
                                              }
                                            ]
                                          }
                                        }
                                      },
                                      {
                                        "key": "manager",
                                        "value": {
                                          "stringValue": "kubelet"
                                        }
                                      }
                                    ]
                                  }
                                }
                              ]
                            }
                          }
                        },
                        {
                          "key": "name",
                          "value": {
                            "stringValue": "node-shell-4113b2c5-d7e8-4b5d-903f-c8c59f0e537b.177c1e793d86209c"
                          }
                        },
                        {
                          "key": "namespace",
                          "value": {
                            "stringValue": "kube-system"
                          }
                        },
                        {
                          "key": "uid",
                          "value": {
                            "stringValue": "cffdab10-3631-4af5-bdf6-63ea3b6c02bc"
                          }
                        }
                      ]
                    }
                  }
                },
                {
                  "key": "source",
                  "value": {
                    "kvlistValue": {
                      "values": [
                        {
                          "key": "component",
                          "value": {
                            "stringValue": "kubelet"
                          }
                        },
                        {
                          "key": "host",
                          "value": {
                            "stringValue": "workload-04"
                          }
                        }
                      ]
                    }
                  }
                },
                {
                  "key": "involvedObject",
                  "value": {
                    "kvlistValue": {
                      "values": [
                        {
                          "key": "kind",
                          "value": {
                            "stringValue": "Pod"
                          }
                        },
                        {
                          "key": "namespace",
                          "value": {
                            "stringValue": "kube-system"
                          }
                        },
                        {
                          "key": "name",
                          "value": {
                            "stringValue": "node-shell-4113b2c5-d7e8-4b5d-903f-c8c59f0e537b"
                          }
                        },
                        {
                          "key": "uid",
                          "value": {
                            "stringValue": "1299adf6-a2ae-47a2-a19a-a50eadb32e3a"
                          }
                        },
                        {
                          "key": "apiVersion",
                          "value": {
                            "stringValue": "v1"
                          }
                        },
                        {
                          "key": "resourceVersion",
                          "value": {
                            "stringValue": "8018190"
                          }
                        },
                        {
                          "key": "fieldPath",
                          "value": {
                            "stringValue": "spec.containers{shell}"
                          }
                        }
                      ]
                    }
                  }
                },
                {
                  "key": "reason",
                  "value": {
                    "stringValue": "Started"
                  }
                },
                {
                  "key": "reportingInstance",
                  "value": {
                    "stringValue": ""
                  }
                }
              ]
            }
          }
        }
      ]
    }
  }
}

I expect the body just contain the raw event json object which can easy parse by downstream componment like fluenbit and friendly for storage .etc.

I have not context to know why use map but I suggest that we can have option to control it wheather using map or raw json.

I can sent a PR to imporve it.
@Frapschen Frapschen added the needs triage New item requiring triage label Aug 17, 2023
@github-actions
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@Frapschen Frapschen mentioned this issue Aug 17, 2023
Closed
@Frapschen Frapschen self-assigned this Aug 17, 2023
@Frapschen Frapschen added receiver/k8sobjects and removed needs triage New item requiring triage labels Aug 17, 2023
@github-actions
Copy link
Contributor

Pinging code owners for receiver/k8sobjects: @dmitryax @hvaghani221. See Adding Labels via Comments if you do not have permissions to add labels yourself.

@dmitryax
Copy link
Member

dmitryax commented Aug 17, 2023 •
edited

@Frapschen, what's the problem you are trying to solve here? It doesn't seem like a receiver concern but rather a serialization of the file exporter.

@dmitryax
Copy link
Member

I expect the body just contain the raw event json object which can easy parse by downstream componment like fluenbit and friendly for storage .etc.

Why it currently cannot be parsed by fluentbit?

@Frapschen
Copy link
Contributor Author

@Frapschen, what's the problem you are trying to solve here? It doesn't seem like a receiver concern but rather a serialization of the file exporter.

I try to get the raw event object JSON data instead of OTLP JSON data in the log body. it's more like the file exporter concern indeed.

@Frapschen
Copy link
Contributor Author

I expect the body just contain the raw event json object which can easy parse by downstream componment like fluenbit and friendly for storage .etc.

Why it currently cannot be parsed by fluentbit?

I use this data flow to transport event data:

otel-col-contrib -> fluentbit(v2.1.4)-opentelemetry-input

the fluentbit side will output this error log:

[2023/08/17 17:01:04] [error] [otel]: invalid value type in pack_any_value
[2023/08/17 17:01:04] [error] [otel] Failed to convert log record body
[2023/08/17 17:01:04] [error] [otel] marshalling error

After I use Vector to replace the fluenbit in this data flow, I found that the Vector is working fine and I can confirm this is a fluenbit bug fluent/fluent-bit#7840.

@Frapschen
Copy link
Contributor Author

if we consider saving the event data into a file and using other tools like fluentbit tail to collect them, we more like hope the log body contains raw k8sobject JSON data instead of OTLP JSON data.

@TylerHelmuth
Copy link
Member

I agree with @dmitryax that this is not a receiver concern. According to the OTel Logs specification, body should maintain any structured data received from a third-party applications..

I also agree that it is wrong that the fileexporter is "leaking" our internal data model. It doesn't feel right that the fileexporter is printing the pdata structures, I think it should be printing a generic map[string]any.

@github-actions
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Oct 30, 2023
@JaredTan95 JaredTan95 added enhancement New feature or request and removed Stale labels Dec 19, 2023
@JaredTan95
Copy link
Member

I agree with @dmitryax that this is not a receiver concern. According to the OTel Logs specification, body should maintain any structured data received from a third-party applications..

I also agree that it is wrong that the fileexporter is "leaking" our internal data model. It doesn't feel right that the fileexporter is printing the pdata structures, I think it should be printing a generic map[string]any.

After some research, I think we can reuse the current jsonlogencodingextension, but it will need to wait for this(#28683) PR get to merge

@github-actions GitHub Actions
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Feb 20, 2024
@JaredTan95
Copy link
Member

with #31774 merged, we can use json_log_encoding in fileexporter later

@github-actions github-actions bot removed the Stale label Mar 21, 2024
@Frapschen
Copy link
Contributor Author

I use fileexporter with json_log_encoding, and its work well:

{
  "object": {
    "metadata": {
      "uid": "49fc31ec-69a8-4c3c-a872-d6bd2f97491c",
      "resourceVersion": "2000269",
      "creationTimestamp": "2024-03-17T02:14:48Z",
      "managedFields": [
        {
          "time": "2024-03-17T02:14:48Z",
          "fieldsType": "FieldsV1",
          "fieldsV1": {
            "f:lastTimestamp": {},
            "f:message": {},
            "f:reason": {},
            "f:source": {
              "f:component": {}
            },
            "f:type": {},
            "f:count": {},
            "f:firstTimestamp": {},
            "f:involvedObject": {}
          },
          "manager": "kube-controller-manager",
          "operation": "Update",
          "apiVersion": "v1"
        }
      ],
      "name": "bindplane-gateway-agent-lb-1.17bd6b8d713cb9f5",
      "namespace": "bindplane-agent"
    },
    "reportingComponent": "",
    "eventTime": null,
    "reportingInstance": "",
    "source": {
      "component": "horizontal-pod-autoscaler"
    },
    "apiVersion": "v1",
    "involvedObject": {
      "kind": "HorizontalPodAutoscaler",
      "namespace": "bindplane-agent",
      "name": "bindplane-gateway-agent-lb-1",
      "uid": "7890317b-fa6e-4aa9-bdc8-19771b2bfc84",
      "apiVersion": "autoscaling/v2",
      "resourceVersion": "117936"
    },
    "message": "failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server could not find the requested resource (get pods.metrics.k8s.io)",
    "lastTimestamp": "2024-03-20T14:29:49Z",
    "count": 20196,
    "firstTimestamp": "2024-03-17T02:14:48Z",
    "kind": "Event",
    "reason": "FailedGetResourceMetric",
    "type": "Warning"
  },
  "type": "MODIFIED"
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Frapschen Frapschen

Labels
enhancement New feature or request receiver/k8sobjects
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[receiver/k8sobject] support log body format setting Frapschen/opentelemetry-collector-contrib
4 participants
@dmitryax @TylerHelmuth @JaredTan95 @Frapschen