Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.14] [Security Solution] [Attack discovery] Overrides default Attack discovery timeouts (#183575) #183581

Merged
merged 1 commit into from
May 16, 2024

Conversation

kibanamachine
Copy link
Contributor

Backport

This will backport the following commits from main to 8.14:

Questions ?

Please refer to the Backport tool documentation

…very timeouts (elastic#183575)

## [Security Solution] [Attack discovery] Overrides default Attack discovery timeouts

### Summary

This PR fixes an issue where Attack discovery requests may be retried when responses from the LLM take longer than two minutes.

In LangSmith, the retry looks like the following _before_ screenshot:

#### Before

![langsmith_before](https://github.com/elastic/kibana/assets/4459398/b02f016c-c260-43f3-a6cc-1260ca8d99c2)

_Above: Before the fix, a retry, shown in LangSmith, for an LLM call > 2 minutes_

After the fix, a single pair for runs > 2 minutes are observed in LangSmith:

#### After

![langsmith_after](https://github.com/elastic/kibana/assets/4459398/864ef2d4-f845-4d62-ab30-686211aadf30)

_Above: After the fix, a single pair in LangSmith, for an LLM call > 2 minutes_

### Details

This PR overrides the following default timeouts:

1) The attack discovery route's `idleSocket` socket timeout in `x-pack/plugins/elastic_assistant/server/routes/attack_discovery/post_attack_discovery.ts`

2) The connector timeout (also in `x-pack/plugins/elastic_assistant/server/routes/attack_discovery/post_attack_discovery.ts`)

3) The chain timeout in `x-pack/plugins/security_solution/server/assistant/tools/attack_discovery/attack_discovery_tool.ts`

with the following defaults:

```typescript
const ROUTE_HANDLER_TIMEOUT = 10 * 60 * 1000; // 10 * 60 seconds = 10 minutes
const LANG_CHAIN_TIMEOUT = ROUTE_HANDLER_TIMEOUT - 10_000; // 9 minutes 50 seconds
const CONNECTOR_TIMEOUT = LANG_CHAIN_TIMEOUT - 10_000; // 9 minutes 40 seconds
```

### Desk testing

1) Verify there are ~ 100 open alerts in the last 24 hours in your testing environment

2) Navigate to Security > Attack discovery

3) Select an Azure / OpenAI connector

4) Click Generate

**Expected results**

- LangSmith displays a single pair of `LLMChain` and `AttackDiscovery` runs when the LLM responds (with the final answer) in less than 2 minutes
- LangSmith displays a single pair of `LLMChain` and `AttackDiscovery` runs when the LLM takes longer than two minutes to respond (with the final answer), as illustrated by the `before` / `after` screenshots in the description above

(cherry picked from commit 1c96c31)
@kibanamachine kibanamachine merged commit 9188192 into elastic:8.14 May 16, 2024
29 checks passed
@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
elasticAssistant 31 32 +1
Unknown metric groups

API count

id before after diff
elasticAssistant 45 46 +1

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @andrew-goldstein

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants