get(url, stream=True) does not stream

[Kamik423]

When using browser.get(url, stream=True) it does not in fact stream, but wait for the entire content to download and add it to the soup. One has to do browser.session.get(...).

This should be documented somewhere or there should be a get_without_adding_to_soup. I spent 3 hours debugging this yesterday, maybe this can save someone some time in the future.

MechanicalSoup/mechanicalsoup/browser.py

Lines 116 to 128 in c767683

	def get(self, *args, **kwargs):
	"""Straightforward wrapper around `requests.Session.get
	<http://docs.python-requests.org/en/master/api/#requests.Session.get>`__.
	:return: `requests.Response
	<http://docs.python-requests.org/en/master/api/#requests.Response>`__
	object with a *soup*-attribute added by :func:`add_soup`.
	"""
	response = self.session.get(*args, **kwargs)
	if self.raise_on_404 and response.status_code == 404:
	raise LinkNotFoundError()
	Browser.add_soup(response, self.soup_config)
	return response

[hemberger]

Hi, and thanks for reporting this! I agree that there should be some fix for stream=True (or at least documentation to describe how it does/doesn't work). At the moment I'm not exactly sure how we should proceed, but I will note that in #288 another user identified a possible workaround for this issue.