Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Discover][Obs ai assistant] Set the index pattern on the context api to generate correct ES|QL queries etc #181001

Merged
merged 7 commits into from
Apr 30, 2024
Merged

[Discover][Obs ai assistant] Set the index pattern on the context api to generate correct ES|QL queries etc #181001

merged 7 commits into from
Apr 30, 2024

Conversation

stratoula
Copy link
Contributor

@stratoula stratoula commented Apr 17, 2024 •
edited

Summary

Sets the index pattern correctly in the obs ai assistant.
The Obs ai assistant is part of Discover in serverless Oblt project and can also be set up everywhere from the Observability AI Assistant scope advanced setting. It is a very useful tool for the Discover users and for moving from KQL to ES|QL world.

This is important for prompts such as:

  • Transform the KQL and filters to ES|QL

meow

  • More
image

Other interesting prompts:

  • Equivalent of KQL of where filters
image
  • Explain the ES|QL query
image
  • Run questions on your dataset and unified search bar selection
image

@stratoula
Copy link
Contributor Author

/ci

@stratoula stratoula changed the title [Discover][Obs ai assistant] Set the index pattern on the context api to generate correct ES|QL queries [Discover][Obs ai assistant] Set the index pattern on the context api to generate correct ES|QL queries etc Apr 29, 2024
@stratoula stratoula added release_note:skip Skip the PR/issue when compiling release notes backport:skip This commit does not require backporting v8.15.0 Feature:ES|QL Feature:Discover Discover Application labels Apr 29, 2024
@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
discover 635.6KB 635.9KB +287.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
discover 34.8KB 34.9KB +52.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@stratoula stratoula marked this pull request as ready for review April 29, 2024 07:18
@stratoula stratoula requested review from a team as code owners April 29, 2024 07:18
@stratoula stratoula added the Team:DataDiscovery Discover App Team (Document Explorer, Saved Search, Surrounding documents, Graph) label Apr 29, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

@jughosta
Copy link
Contributor

jughosta commented Apr 29, 2024 •
edited

Should be probably aligned with

@stratoula
Copy link
Contributor Author

stratoula commented Apr 29, 2024 •
edited

Maybe it is not clear from my description but this is a bug fix, the index pattern is not correctly passed to the LLM creating wrong ES|QL queries. This functionality is already out in the wild, I am not adding something new here (from the functionality perspective), I am fixing a bug.

dgieselaar
dgieselaar approved these changes Apr 29, 2024
View reviewed changes
Copy link
Member

@dgieselaar dgieselaar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (on the AI Assistant usage)

@stratoula
Copy link
Contributor Author

Gathering here the outcome of an offline discussion I had with some folks:

  • the Obs ai assistant is built as a platform tool and has very few dependencies so we can use it in Discover plugin and we dont need a middleware
"actions",
"features",
"licensing",
"security",
"taskManager",
"dataViews"
  • there is no advantage of adding a middleware (also we are adding xpack plugins dependencies all the time)
  • this PR is fixing an existing but which will also benefit the Logs Discover
  • in the future we might need a middleware (for example when the security ai assistant also provides the same capabilities). At this point this is not possible, we can always adapt later.

jughosta
jughosta approved these changes Apr 30, 2024
View reviewed changes
Copy link
Contributor

@jughosta jughosta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code changes LGTM

@stratoula stratoula merged commit 9edb9d5 into elastic:main Apr 30, 2024
29 checks passed
delanni added a commit to delanni/kibana that referenced this pull request Apr 30, 2024
@delanni
Revert "[Discover][Obs ai assistant] Set the index pattern on the con…
a5bfb2c 
…text api to generate correct ES|QL queries etc (elastic#181001)"

This reverts commit 9edb9d5.
delanni added a commit that referenced this pull request Apr 30, 2024
@delanni
Revert "[Discover][Obs ai assistant] Set the index pattern on the con…
c84e716 
…text api to generate correct ES|QL queries etc (#181001)" (#182088)

This reverts commit 9edb9d5.
@delanni
Copy link
Contributor

delanni commented Apr 30, 2024

Reverted in #182088

@stratoula stratoula mentioned this pull request Apr 30, 2024
Merged
stratoula added a commit that referenced this pull request Apr 30, 2024
@stratoula @kibanamachine
[Discover][Obs ai assistant] Set the index pattern on the context api…
74fdd1b 
… to generate correct ES|QL queries etc (#182090)

## Summary

Reverting the revert of
#181001 (comment)

the only change is the removal of the ml plugin dependency from the
obs-ai-assistant

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
yuliacech pushed a commit to yuliacech/kibana that referenced this pull request May 3, 2024
@stratoula @kibanamachine @yuliacech
[Discover][Obs ai assistant] Set the index pattern on the context api…
0608dc6 
… to generate correct ES|QL queries etc (elastic#181001)

## Summary

Sets the index pattern correctly in the obs ai assistant. 
The Obs ai assistant is part of Discover in serverless Oblt project and
can also be set up everywhere from the Observability AI Assistant scope
advanced setting. It is a very useful tool for the Discover users and
for moving from KQL to ES|QL world.

This is important for prompts such as:

- Transform the KQL and filters to ES|QL


![meow](https://github.com/elastic/kibana/assets/17003240/e9b1be13-667a-4fdb-addb-fb77e485b99e)

- More

<img width="579" alt="image"
src="https://github.com/elastic/kibana/assets/17003240/91b6b0ac-754f-47f3-98a2-ed09e7be646b">


Other interesting prompts:

- Equivalent of KQL of where filters

<img width="640" alt="image"
src="https://github.com/elastic/kibana/assets/17003240/f203bc5e-a037-49c9-9e6c-5e68c9fcb864">

- Explain the ES|QL query
<img width="658" alt="image"
src="https://github.com/elastic/kibana/assets/17003240/b2f9abfb-577d-4c06-b715-553ebd626e28">


- Run questions on your dataset and unified search bar selection
<img width="687" alt="image"
src="https://github.com/elastic/kibana/assets/17003240/e095ced8-5bf9-4846-ac54-644c4fd8de04">

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
yuliacech pushed a commit to yuliacech/kibana that referenced this pull request May 3, 2024
@delanni @yuliacech
Revert "[Discover][Obs ai assistant] Set the index pattern on the con…
f6a3e6e 
…text api to generate correct ES|QL queries etc (elastic#181001)" (elastic#182088)

This reverts commit 9edb9d5.
yuliacech pushed a commit to yuliacech/kibana that referenced this pull request May 3, 2024
@stratoula @kibanamachine @yuliacech
[Discover][Obs ai assistant] Set the index pattern on the context api…
dfe521c 
… to generate correct ES|QL queries etc (elastic#182090)

## Summary

Reverting the revert of
elastic#181001 (comment)

the only change is the removal of the ml plugin dependency from the
obs-ai-assistant

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dgieselaar dgieselaar dgieselaar approved these changes

@jughosta jughosta jughosta approved these changes

@pgayvallet pgayvallet pgayvallet approved these changes

Assignees
No one assigned
Labels
backport:skip This commit does not require backporting Feature:Discover Discover Application Feature:ES|QL release_note:skip Skip the PR/issue when compiling release notes reverted Team:DataDiscovery Discover App Team (Document Explorer, Saved Search, Surrounding documents, Graph) v8.15.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
8 participants
@stratoula @kibana-ci @elasticmachine @jughosta @delanni @dgieselaar @pgayvallet @kibanamachine