I am a paid Gemini Enterprise customer. I STILL get Frequent google.api_core.exceptions.InternalServerError: 500 An internal error has occurred

[gunawebs]

Description of the bug:

I am a paid Gemini Enterprise customer. I STILL get Frequent google.api_core.exceptions.InternalServerError: 500 An internal error has occurred

THIS IS NOT deterministic.
Sometimes things work and sometime sthings do NOT

File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/google/ai/generativelanguage_v1beta/services/generative_service/client.py", line 566, in generate_content
response = rpc(
^^^^
File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/google/api_core/gapic_v1/method.py", line 131, in call
return wrapped_func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/google/api_core/retry/retry_unary.py", line 293, in retry_wrapped_func
return retry_target(
^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/google/api_core/retry/retry_unary.py", line 153, in retry_target
_retry_error_helper(
File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/google/api_core/retry/retry_base.py", line 212, in _retry_error_helper
raise final_exc from source_exc
File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/google/api_core/retry/retry_unary.py", line 144, in retry_target
result = target()
^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/google/api_core/timeout.py", line 120, in func_with_timeout
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/google/api_core/grpc_helpers.py", line 78, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.InternalServerError: 500 An internal error has occurred. Please retry or report in https://developers.generativeai.google/guide/troubleshooting

Actual vs expected behavior:

No response

Any other information you'd like to share?

No response

[LindaLawton]

This is a known issue and they are working on it 500 An internal error is the model overloaded. There are a lot of people using this system due to the fact that its free. this and 503 are the number one errors reported in the Discord channel right now.

[pkenjora]

Hi, this error occurs when the chat history becomes too long or large. Probably a token issue.

To work around this please prune your chat history periodically as described in the medium post Avoiding InternalServerError When Using Google GenerativeAI Chat.

It took me a while to track this down but it seems to solve the problem until the Gemini team can improve the error codes coming back from the API.

[gunawebs]

Thanks

[MarkDaoust]

his error occurs when the chat history becomes too long or large. Probably a token issue.

Oh, yes. I've seen that error too, I've raised this internally. You can tell if this is the problem by checking the output of genai.count_tokens and comparing that to the model's maximum input context. 1.5-pro has much longer context than 1.0-pro.

ref: https://ai.google.dev/gemini-api/docs/models/gemini

[MarkDaoust]

500 can be a lot of things, overloaded servers, uncaught exceptions, in this case it sounds like @pkenjora pinpointed the actual problem, it was throwing a 500 instead of "invalid argument" when the context gets too long (I've reported this internally). Closing.