Is upload_file thread safe?

[jiupinjiandingshi]

Description of the bug:

when I want use genai , I need to use this method in an interface request, but when I put "genai.upload_file(path=path,display_name=file)" into the interface method, it will report an error SLL_ERROR
@app.route("/xxx",method=["GET"]) def upload_video(): output_frame_folder = requests.output_frame_folder time.sleep(1.0) lists = [] try: for file in os.listdir(f"{output_frame_folder}"): path = os.path.join(f"{output_frame_folder}", file) sample_file = genai.upload_file(path=path,display_name=file)

Actual vs expected behavior:

No response

Any other information you'd like to share?

No response

[jiupinjiandingshi]

Does this mean that this method does not support concurrency?

[jiupinjiandingshi]

the error desc:[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:2621)

[singhniraj08]

@jiupinjiandingshi, I tried running Prompting with media files tutorial and genai.upload_file works without any issues. Can you please share a colab gist or code which we can try to reproduce on our end. Thank you!

[jiupinjiandingshi]

If I run it once, there is no problem, as follows：
for file in os.listdir(f"{output_frame_folder}"): path = os.path.join(f"{output_frame_folder}", file) sample_file = genai.upload_file(path=path,display_name=file)

But if my interface receives two requests at the same time, this code may be called twice in the same time period. In this case, an error will be reported as follows:
httplib2.Http() error:[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:2621)
pit:’output_frame_folder ‘ It is a folder that contains some pictures and an MP3

[sMx7d]

try to run pip install google-generativeai --upgrade that solved it for me!
i was using google-generativeai-0.3.2 but now its google-ai-generativelanguage-0.6.4

[MarkDaoust]

Thanks, please reopen this if the problem comes back in a reproducible way.

[ronfromhp]

i tried upgrading the library as suggested by @sMx7d . But still getting errors

 File "/code/vision_models.py", line 228, in media2concept
video-backend  |     messages = [
video-backend  |                ^
video-backend  |   File "/code/vision_models.py", line 231, in <listcomp>
video-backend  |     "parts": await asyncio.create_task(make_video_prompt(media))
video-backend  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/code/vision_models.py", line 154, in make_video_prompt
video-backend  |     framefiles = await upload_frames_concurrently(framefiles)
video-backend  |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/code/vision_models.py", line 136, in upload_frames_concurrently
video-backend  |     uploaded_frames = await asyncio.gather(*tasks)
video-backend  |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/code/vision_models.py", line 127, in aupload_frame
video-backend  |     res = await asyncio.to_thread(genai.upload_file, ff.file_path, name=ff.name)
video-backend  |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/asyncio/threads.py", line 25, in to_thread
video-backend  |     return await loop.run_in_executor(None, func_call)
video-backend  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
video-backend  |     result = self.fn(*self.args, **self.kwargs)
video-backend  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/google/generativeai/files.py", line 69, in upload_file
video-backend  |     response = client.create_file(
video-backend  |                ^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/google/generativeai/client.py", line 79, in create_file
video-backend  |     return self.get_file({"name": result["file"]["name"]})
video-backend  |                                   ~~~~~~^^^^^^^^
video-backend  | TypeError: string indices must be integers, not 'str'

anyone else finding the same issue?

[sMx7d]

@ronfromhp , uninstall and reinstall it.

[ronfromhp]

@sMx7d somehow its gone back to giving the ssl error

these are the installed libraries:
google-ai-generativelanguage==0.6.4
google-api-core==2.19.0
google-api-python-client==2.130.0
google-auth==2.29.0
google-auth-httplib2==0.2.0
google-generativeai==0.5.4
googleapis-common-protos==1.63.0
grpcio==1.64.0
grpcio-status==1.62.2
h11==0.14.0
httpcore==1.0.5
httplib2==0.22.0
httpx==0.27.0

This is the same old error trace im getting:

video-backend  |     uploaded_frames = await asyncio.gather(*tasks)
video-backend  |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/code/vision_models.py", line 127, in aupload_frame
video-backend  |     res = await asyncio.to_thread(genai.upload_file, ff.file_path, name=ff.name)
video-backend  |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/asyncio/threads.py", line 25, in to_thread
video-backend  |     return await loop.run_in_executor(None, func_call)
video-backend  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
video-backend  |     result = self.fn(*self.args, **self.kwargs)
video-backend  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/google/generativeai/files.py", line 69, in upload_file
video-backend  |     response = client.create_file(
video-backend  |                ^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/google/generativeai/client.py", line 77, in create_file
video-backend  |     result = request.execute()
video-backend  |              ^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
video-backend  |     return wrapped(*args, **kwargs)
video-backend  |            ^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/googleapiclient/http.py", line 902, in execute
video-backend  |     _, body = self.next_chunk(http=http, num_retries=num_retries)
video-backend  |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
video-backend  |     return wrapped(*args, **kwargs)
video-backend  |            ^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/googleapiclient/http.py", line 1007, in next_chunk
video-backend  |     resp, content = _retry_request(
video-backend  |                     ^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/googleapiclient/http.py", line 222, in _retry_request
video-backend  |     raise exception
video-backend  |   File "/usr/local/lib/python3.11/site-packages/googleapiclient/http.py", line 191, in _retry_request
video-backend  |     resp, content = http.request(uri, method, *args, **kwargs)
video-backend  |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/httplib2/__init__.py", line 1724, in request
video-backend  |     (response, content) = self._request(
video-backend  |                           ^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/httplib2/__init__.py", line 1444, in _request
video-backend  |     (response, content) = self._conn_request(conn, request_uri, method, body, headers)
video-backend  |                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/httplib2/__init__.py", line 1396, in _conn_request
video-backend  |     response = conn.getresponse()
video-backend  |                ^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/http/client.py", line 1395, in getresponse
video-backend  |     response.begin()
video-backend  |   File "/usr/local/lib/python3.11/http/client.py", line 325, in begin
video-backend  |     version, status, reason = self._read_status()
video-backend  |                               ^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/http/client.py", line 286, in _read_status
video-backend  |     line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
video-backend  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/socket.py", line 706, in readinto
video-backend  |     return self._sock.recv_into(b)
video-backend  |            ^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/ssl.py", line 1314, in recv_into
video-backend  |     return self.read(nbytes, buffer)
video-backend  |            ^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/ssl.py", line 1166, in read
video-backend  |     return self._sslobj.read(len, buffer)
video-backend  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  | ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:2580)

[sMx7d]

Its stil a ssl error , sorry but i don't know anything in ssl or what it means .
But it seems that genertiveai version is 0.5.4 , IDK if it has upload file function or not , but either way, i don't think upgrading the library will solve as its a ssl error and not related to library functions.
I hope my response was clear, if you needed anything just mention me.
@ronfromhp

[ronfromhp]

@jiupinjiandingshi can you reopen the issue? As currently I havent found a resolution which works

[sMx7d]

video-backend  | TypeError: string indices must be integers, not 'str'

sorry , i just reviewed the error and i saw video-backend | TypeError: string indices must be integers, not 'str'
that error is likely not from the generative-ai library , its from your code but IDK what the solve for it.

[ronfromhp]

as of my previous comment i was still getting the same ssl error even after updating genai @sMx7d