add auto detect charset from http body when http headers not seted

[likexian]

requests detect page's charset/encoding from responsed http headers, but many servers won't set it in headers.

In this case, requests will set the charset/encoding to the default 'ISO-8859-1', and result in decoded error, many Chinese developers get lose and complain.

For example:

print requests.get('http://www.126.com/').text

This commit will detect charset/encoding from http headers, if no, from http body, for most situations, everything goes well now.

[Lukasa]

Thanks for this!

This is a change that has been discussed many, many times. A sample of issues that touch it are: #1604, #1774, #1604, #1589, #1588, #1546, and #2042. We also mention it in the documentation here. Though it's an easy mistake to make, in future you really should search both open and closed issues before making a change like this because you'll save yourself time and effort.

We aren't going to merge this. Previously we wouldn't have merged this because of the explanation given in the issues mentioned above (namely, that the behaviour is deliberate, and that we aren't a HTML library so we won't introspect content). Currently, this behaviour is tracked under the open issue #2086, which will change this behaviour in a different way. If you'd found that issue you'd have saved yourself some time.

I'm sorry that we can't take this change, please don't let it dissuade you from continuing to contribute!