Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ELK 7.x support #9

Closed
nightah opened this issue Aug 12, 2019 · 6 comments · Fixed by #10 · May be fixed by #14
Closed

ELK 7.x support #9

nightah opened this issue Aug 12, 2019 · 6 comments · Fixed by #10 · May be fixed by #14

Comments

@nightah
Copy link
Contributor

nightah commented Aug 12, 2019

I tried upgrading my ELK stack to 7.x over the weekend without success.

I'm wondering if anybody has managed to get the configuration working with ELK 7.x?
It looked to me like that my_index field for the indexes was not being created properly.

If anybody has had success deploying this on 7.x could you please share the appropriate logstash and filebeat config and an object export from Kibana?
@nightah
Copy link
Contributor Author

nightah commented Oct 17, 2019

Alright so I figured this out, didn't quite have to do with the my_index field, because of the changes between 6.x for logstash and filebeat the logstash.conf needed to change to accommodate the new and deprecated fields.

Let me know if you want me to submit a PR to make this 7.x compatible.

@nightah
Copy link
Contributor Author

nightah commented Oct 20, 2019

The following changes need to be made to make a 7.x compatible config:

logstash/logstash.conf

Line 9 in aea8a7c

if [beat][name] =~ "^squid" {

Becomes: if [host][name] =~ "^squid" {

logstash/logstash.conf

Line 34 in aea8a7c

if [beat][name] == "lancache" {

Becomes: if [host][name] == "lancache" {

logstash/logstash.conf

Lines 36 to 43 in aea8a7c

mutate {
remove_field => [ "[host]" ]
}
mutate {
add_field => {
"host" => "%{[beat][hostname]}"
}
}

Needs to be removed.

logstash/logstash.conf

Line 51 in aea8a7c

if [source] =~ "access" {

Becomes: if [log][file][path] =~ "access" {

logstash/logstash.conf

Line 73 in aea8a7c

} else if [source] =~ "error" {

Becomes: } else if [log][file][path] =~ "error" {

@nightah nightah closed this as completed Oct 20, 2019
@VibroAxe
Copy link
Member

@nightah can you submit a PR, but create a new logstash7.conf and rename logstash to logstash6.conf. The changes you suggest don't feel like they are backwards compatible?

1 similar comment
@VibroAxe
Copy link
Member

@nightah can you submit a PR, but create a new logstash7.conf and rename logstash to logstash6.conf. The changes you suggest don't feel like they are backwards compatible?

@VibroAxe VibroAxe reopened this Oct 20, 2019
@nightah
Copy link
Contributor Author

nightah commented Oct 20, 2019 •
edited

They’re definitely not backwards compatible.
There are breaking changes from 6.x to 7.x in filebeat and logstash.

I’ll submit a PR shortly.

@nightah nightah mentioned this issue Oct 20, 2019
Merged
@nightah
Copy link
Contributor Author

nightah commented Nov 14, 2019

@VibroAxe: I've got a question pending in that PR can you have a look please?
My suggestion is that we rename it.

Secondly could someone export all objects related to lancache (including any squid objects)?
I want to import what I'm missing and create a new export which is clean and clearly identifies all objects as relating to LanCache as part of that existing PR.

@VibroAxe VibroAxe linked a pull request Mar 6, 2022 that will close this issue
Add an example ELK stack for collecting information #14
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@VibroAxe @nightah