SMTP.quit produces warning "Future exception was never retrieved"

[Danstiv]

I ported this code from my old implementation on standard smtplib module, and perhaps it is not correct for aiosmtplib.
But i think this shouldn't matter.

import asyncio
import aiosmtplib
from email.mime.text import MIMEText
from email.header import Header
async def send_mail(host, login, password, from_addr, to_addr, subject, body_text):
	msg=MIMEText(*body_text)
	msg['Subject']=Header(subject, 'utf-8')
	msg['From']=from_addr
	msg['To']=to_addr
	server=aiosmtplib.SMTP(host)
	await server.connect()
	await server.starttls()
	await server.login(login, password)
	try:
		await server.sendmail(from_addr, [to_addr], msg.as_string())
	except (aiosmtplib.SMTPRecipientsRefused, aiosmtplib.SMTPSenderRefused):
		return 1
	finally:
		await server.quit()
#code for testing
async def start():
	await send_mail('smtp.yandex.ru', 'a@b.c', '1234', 'a@b.c', 'x@y.z', 'Testing', ['Hello', 'plain', 'utf-8'])
	print('success')
#emulating work of other asyncio tasks...
	await asyncio.sleep(3)
asyncio.run(start())```

```Python 3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 23:09:28) [MSC v.1916 64 bit (AMD64)] on win32
>>> import mail_test
success
>>> exit()
Future exception was never retrieved
future: <Future finished exception=SMTPServerDisconnected('Unexpected EOF received')>
aiosmtplib.errors.SMTPServerDisconnected: Unexpected EOF received```

If i comment two last lines of my function, it works good.

```>>> import mail_test
success
>>> exit()```

[cole]

Last two lines meaning?

    print('success')
    #emulating work of other asyncio tasks...
    await asyncio.sleep(3)

[Danstiv]

Last two lines meaning?

    print('success')
    #emulating work of other asyncio tasks...
    await asyncio.sleep(3)

Because "Future exception was never retrieved" happens only if after calling quit passed at least a few seconds.

[potens1]

I get the same error with pytest test (using pytest-asyncio), for my tests it use gmail smtp with tls (if that matters)

setting PYTHONASYNCIODEBUG=1 this is the end of the output (everything before is not relevant)

...
File "/home/dev/development/newgateway/notifications_email_transport/src/notifications/transports/plugins/email/__init__.py", line 45, in send
    response = await aiosmtplib.send(
  File "/home/dev/.virtualenvs/notifications_email_transport-ELxUtyWv/lib/python3.8/site-packages/aiosmtplib/api.py", line 402, in send
    result = await client.sendmail(sender, recipients, message)
  File "/home/dev/.virtualenvs/notifications_email_transport-ELxUtyWv/lib/python3.8/site-packages/aiosmtplib/connection.py", line 151, in __aexit__
    await self.quit()
  File "/home/dev/.virtualenvs/notifications_email_transport-ELxUtyWv/lib/python3.8/site-packages/aiosmtplib/esmtp.py", line 236, in quit
    response = await self.execute_command(b"QUIT", timeout=timeout)
  File "/home/dev/.virtualenvs/notifications_email_transport-ELxUtyWv/lib/python3.8/site-packages/aiosmtplib/connection.py", line 420, in execute_command
    response = await self.protocol.execute_command(*args, timeout=timeout)
  File "/home/dev/.virtualenvs/notifications_email_transport-ELxUtyWv/lib/python3.8/site-packages/aiosmtplib/protocol.py", line 291, in execute_command
    response = await self.read_response(timeout=timeout)
  File "/home/dev/.virtualenvs/notifications_email_transport-ELxUtyWv/lib/python3.8/site-packages/aiosmtplib/protocol.py", line 268, in read_response
    self._response_waiter = self._loop.create_future()
  File "/usr/lib64/python3.8/asyncio/base_events.py", line 422, in create_future
    return futures.Future(loop=self)
aiosmtplib.errors.SMTPServerDisconnected: Unexpected EOF received

By the way, the mail is sent, even with the exception.

[potens1]

OK, trying to have a look at the problem, I just discovered this does not happen every time, I just ran my test bench twice in a row, (full stop start, not twice the same test in code), first time was without problem, the second one has the error... so it something not so easy to reproduce to test...

[potens1]

I'm not 100% sure but, it seems that it depends when the garbage collectors kick because, if I put some print statement in SMTPProtocol.__del__ it print after the warning. So it means the last response waiter created while doing the QUIT command is left until the end of the run for a very fast run because the __del__ is not called before the end (garbage collection not done yet). It means also that, while the warning is boring and not super clean, it should be harmless for normal operations (when the __del__ is called before the very end of the software).
Deleting the references and manually calling gc.collect at the end of tests, I'm not able anymore to get the warning (but... it does not prove anything since it was not 100% reproducible)

[cole]

I'm kind of stuck on a fix for this, since this isn't very reproducible (depends on when garbage collection happens relative to loop shutdown).

Note though that the warning should be harmless.

[SamyCookie]

Hi,

I reproduced this issue:

Future exception was never retrieved
future: <Future finished exception=SMTPServerDisconnected('Connection lost')>
aiosmtplib.errors.SMTPServerDisconnected: Connection lost

I think the issue is not in my code and indeed is random and must be related with garbage collection.

edit: I use version 1.1.6

[suspiciousRaccoon]

I can reliably reproduce this issue with the following code based on the docs.

Python 3.11.2
Version 3.0.0

import asyncio
from email.message import EmailMessage

import aiosmtplib


async def send_hello_world():
    message = EmailMessage()
    message["From"] = "gmailemail@gmail.com"
    message["To"] = "gmailemail@gmail.com"
    message["Subject"] = "Hello World!"
    message.set_content("Sent via aiosmtplib")

    await aiosmtplib.send(
        message,
        hostname="smtp.gmail.com",
        port=587,
        username="gmailemail@gmail.com",
        password="used app password here"
    )
    print("Error happens after this print!")

asyncio.run(send_hello_world())

print("End of error!")

[dokime7]

Same problem for me...

[cole]

Oh nice, at least it's consistent on 3.11.2. Can you try the current main branch? I've made a change that fixes it for me.

[suspiciousRaccoon]

I tried the new branch and the problem is still present with 3.11.2 on windows. Also tried with 3.10.12 on a linux VM and ran into the inconsistency problem, although it still happened.

However, I also found a way to avoid the problem:

import asyncio
from email.message import EmailMessage

import aiosmtplib


async def send_hello_world():
    message = EmailMessage()
    message["From"] = "gmailemail@gmail.com"
    message["To"] = "gmailemail@gmail.com"
    message["Subject"] = "Hello World!"
    message.set_content("Sent via aiosmtplib")

    await aiosmtplib.send(
        message,
        hostname="smtp.gmail.com",
        port=587,
        username="gmailemail@gmail.com",
        password="used app password here"
    )
    print("Error doesn't happens after this print!")

loop = asyncio.get_event_loop()
loop.run_until_complete(send_hello_world())

print("No error!")

I'm still very new to concurrency and the asyncio lib, so I'm not really sure why this works...?

[Danstiv]

Reproduced on 3.10, 3.11 and 3.12 with aiosmtplib 3.0.0.
In 3.12 some changes were made to the garbage collector, so it is strange that the problem appears in 3.11 and 3.10, perhaps changes were made to aiosmtplib itself.

[apepenkov]

I can confirm that I have that issue too. platform win32, python 3.11.6, aiosmtplib is 3.0.0

[polina-koval]

Reproduced on 3.10.9, with aiosmtplib 3.0.0. 😞

[cole]

I'm still not sure how exactly that race condition is happening, but I did remove some __del__ logic in 3.0 on the protocol that seems to have caused it to happen. Please try current main branch if you're able to. If it works, I'll put out a patch release.

[apepenkov]

I'm still not sure how exactly that race condition is happening, but I did remove some __del__ logic in 3.0 on the protocol that seems to have caused it to happen. Please try current main branch if you're able to. If it works, I'll put out a patch release.

can confirm, latest master version does not produce the error. win32, 3.11.6

[polina-koval]

The main branch works without errors 👍🏻

[polina-koval]

I'm eagerly awaiting the patch. Thank you for your prompt responses and work :)

[cole]

3.0.1 is out 🙂

[dokime7]

Unfortunately, sometimes the issue occurs again...

Future exception was never retrieved
future: <Future finished exception=SMTPServerDisconnected('Unexpected EOF received')>

python 3.11.6
aiosmtplib 3.0.1

[cole]

@dokime7 any steps to reproduce?

[dokime7]

No sorry, it's random...
I add that it's on Docker with image tiangolo/uvicorn-gunicorn:python3.11