Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Entity Analytics] Move scripted metric painless scripts to static file & remove category based weighting #182038

Merged
merged 27 commits into from May 17, 2024

Conversation

oatkiller
Copy link
Contributor

@oatkiller oatkiller commented Apr 29, 2024

Summary

The scripted metric aggregation is being deprecated but an exception is being made for our team, moving the painless to static files allows for this exception to be made. We have had to remove category weighting to make the script less dynamic, weights weren't used anyway so not a breaking change.

The scripts are loaded once when they are first used and then cached. A unit test verifies the content of the script hasnt changed.

Tested locally with hosts and users with 100 alerts, risk score docs the same before and after.

Here is a diff of the scripted metric before and after https://www.diffchecker.com/gefuBoYK/

oatkiller and others added 3 commits April 29, 2024 17:00
…based weighting

the category based weighting, like all weighting, isn't used and we probably won't re-implement it in the next version of risk scoring

this commit probably doesn't work. i can't get integration tests running locally

I put the painless scripts in a directory and I load them from the file system at runtime.

We should figure out how to bundle the scripts possibly. I'm not sure that it matters a lot, but I would like any static analysis processes to be able to work with this code as usual. e.g. i'd like to get a static-time error if a file is missing vs needing to run the risk engine task to find out a file is missing
@hop-dev hop-dev changed the title Try making risk engine painless scripts static. also remove category … [Entity Analytics] Move scripted metric painless scripts to static file & remove category based weighting May 10, 2024
@hop-dev hop-dev added release_note:skip Skip the PR/issue when compiling release notes Team:Entity Analytics Security Entity Analytics Team labels May 10, 2024
@hop-dev hop-dev self-assigned this May 10, 2024
@hop-dev hop-dev marked this pull request as ready for review May 10, 2024 09:09
@hop-dev hop-dev requested a review from a team as a code owner May 10, 2024 09:09
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-entity-analytics (Team:Entity Analytics)

@hop-dev
Copy link
Contributor

hop-dev commented May 10, 2024

@elasticmachine merge upstream

Copy link
Member

@machadoum machadoum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks great! 👏

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
securitySolution 15.1MB 15.1MB -188.0B
Unknown metric groups

ESLint disabled line counts

id before after diff
securitySolution 522 523 +1

Total ESLint disabled count

id before after diff
securitySolution 604 605 +1

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @hop-dev

@hop-dev hop-dev merged commit 01cb168 into main May 17, 2024
35 checks passed
@hop-dev hop-dev deleted the risk-score-painless-refactor branch May 17, 2024 12:45
@kibanamachine kibanamachine added v8.15.0 backport:skip This commit does not require backporting labels May 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting release_note:skip Skip the PR/issue when compiling release notes Team:Entity Analytics Security Entity Analytics Team v8.15.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants