Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

parse_grok not work? #20486

Closed
rightly opened this issue May 13, 2024 · 1 comment
Closed

parse_grok not work? #20486

rightly opened this issue May 13, 2024 · 1 comment
Labels
type: bug A code related bug.

Comments

@rightly
Copy link

rightly commented May 13, 2024

A note for the community

  • Please vote on this issue by adding a 馃憤 reaction to the original issue to help the community and maintainers prioritize this request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

The parse_groke function does not seem to work.

  • input
{"@timestamp":"2024-05-13T02:55:56.849Z","@version":"1","agent":{"ephemeral_id":"ab9177e9-1c6c-4389-a711-84efdca3704a","id":"0d8feb97-be58-445c-924a-e89fd378e558","name":"bind1","type":"filebeat","version":"8.3.1"},"ecs":{"version":"8.0.0"},"headers":{},"host":{"name":"bind1"},"input":{"type":"log"},"log":{"file":{"path":"/bind/logs/dns.log"},"offset":1142632},"log_type":"default","message":"May 13 11:55:51 bind1 named[56126]: lame-servers: info: REFUSED unexpected RCODE resolving 'example.ru/A/IN': 10.10.10.10#53","timestamp":"2024-05-13T02:55:57.953Z"}
  • output
{"log_type":"default","message":"May 13 11:55:51 bind1 named[56126]: lame-servers: info: REFUSED unexpected RCODE resolving 'example.ru/A/IN': 10.10.10.10#53'","timestamp":"2024-05-13T02:55:57.953Z"}
  • expected output
{"log_type":"default","logdate":"~~", "hostname": "bind1", "proc_name": "56126", "category": "lame-servers", "level": "info", "msg": "REFUSED unexpected RCODE resolving 'example.ru/A/IN': 10.10.10.10#53'","timestamp":"2024-05-13T02:55:57.953Z"}

Configuration

sources:
  producer_filebeat:
    type: logstash
    address: 0.0.0.0:10044
    connection_limit: 1024
    receive_buffer_bytes: 65536
transforms:
  producer_filebeat_remap:
    type: remap
    inputs: ["producer_filebeat"]
    source: |
        del(.host.name)
        del(.ecs)
        del(.host)
        del(.input)
        del(.log)
        del(.agent)
        del(.dns_metadata)
        del(.source_type)
        del(.@metadata)
        del(.@timestamp)

        if .log_type == "default" {
          parse_grok!(.message, s'%{SYSLOGTIMESTAMP:logdate} %{NOTSPACE:hostname} %{WORD:proc_name}\[%{INT:pid}\]: %{NOTSPACE:category}: %{NOTSPACE:level}: %{GREEDYDATA:msg}')
        }
        if .log_type == "startup" {
          parse_grok!(.message, s'%{SYSLOGTIMESTAMP:logdate} %{NOTSPACE:hostname} %{WORD:proc_name}\[%{INT:pid}\]: %{GREEDYDATA:msg}')
        }

        # del(.message)
sinks:
  print:
    type: console
    inputs: ["producer_filebeat_remap"]
    encoding:
      codec: json

Version

0.38.0

Debug Output

No response

Example Data

No response

Additional Context

No response

References

No response

@rightly rightly added the type: bug A code related bug. label May 13, 2024
@rightly
Copy link
Author

rightly commented May 20, 2024

my mistake..

@rightly rightly closed this as completed May 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug A code related bug.
Projects
None yet
Development

No branches or pull requests

1 participant