Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Grounding with Gemini doesn't work, no metadata is returned #688

Open
1 task done
sanjanalreddy opened this issue May 15, 2024 · 3 comments
Open
1 task done

Comments

@sanjanalreddy
Copy link

File Name

https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/grounding/intro-grounding-gemini.ipynb

What happened?

Grounding with Gemini doesn't work. Gemini model doesn't seem to be using the grounding source to answer the prompt. The same datasource with Palm however returns grounded responses and grounding metadata.

Screenshot 2024-05-09 at 4 27 36 PM

Relevant log output

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct
@holtskinner
Copy link
Collaborator

Can you provide details about the specific queries and data source you're using for grounding?

@sanjanalreddy
Copy link
Author

I'm using the same data source that is mentioned in the notebook that points to cloud.google.com/* and the prompt is When should I use an object table in BigQuery? And how does it store data?

@maralm
Copy link

maralm commented Jun 4, 2024

Similar experience here. The grounding_metadata contains the retrieval_queries property only, but nothing that looks like citations or attributions. For example, the method print_grounding_response in the grounding example indicates that there should be attributions available, but I'm consistently not getting any.

  • SDK: google-cloud-aiplatform-1.53.0
  • Data sources: both custom data set as well as an indexed web site in Vertex AI Search and Conversation
  • Queries: Simple one-shot prompts

I'm a bit perplexed in general because I can set up a prompt and grounding source in the Vertex AI UI, try it out and it works okay, then click "Get Code" and execute that code and get no results. In general, results seem to be much worse using latest SDK and Gemini models, compared to the previous way of doing it and with text-bison.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants