Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

iso-8859-1/windows-1252 support? #194

Open
papandreou opened this issue Aug 23, 2016 · 14 comments · May be fixed by #326
Open

iso-8859-1/windows-1252 support? #194

papandreou opened this issue Aug 23, 2016 · 14 comments · May be fixed by #326
Labels

Comments

@papandreou
Copy link

Hi,

A while ago I developed a web app to replace an old cgi-bin script from the nineties, which means that it had to assume that POST requests with a Content-Type of application/x-www-urlencoded are in iso-8859-1 (and also support "smart quotes" etc., aka. windows-1252). At the same time it had to be possible to switch to utf-8 mode.

I accomplished that by adding support for a defaultCharset option, which can either be utf-8 or iso-8859-1, and for explicitly switching to utf-8 mode by adding utf8=%E2%9C%93 (utf8=✓ as percent-encoded utf-8 octets) to the query string, a trick that I've seen others use as well.

Here are the diffs between my forks of body-parser and the qs module:

master...papandreou:iso-8859-1
ljharb/qs@master...papandreou:interpretNumericEntities

Would anyone be interested in this work? If so, I can clean it up and open PRs.

@dougwilson dougwilson self-assigned this Dec 21, 2016
@Xtrem65
Copy link

Xtrem65 commented Jul 16, 2018

@dougwilson Any update on this ?

I'm interested in this feature but as @papandreou 's branch is way behind the official master. Seems like it's going to be hard to pull...

Is there any proper other way to handle ISO-8859-1 encoded data ?

@dougwilson
Copy link
Contributor

Hi @Xtrem65 I haven't even commented on here, let alone done any work on it :) I don't have any updates.

@dougwilson dougwilson removed their assignment Jul 16, 2018
@papandreou
Copy link
Author

@Xtrem65, I guess the only update is that you're expressing interest, which is also a good thing :)

If @dougwilson and @ljharb think the work looks good and they want to adopt this feature, I'd be willing to bring the branches up to date.

@papandreou
Copy link
Author

@Xtrem65, have you tried switching to my body-parser-papandreou fork to see if it solves your problem? It should still work with current versions of express.

@ljharb
Copy link
Contributor

ljharb commented Jul 16, 2018

I’d be happy to review a PR.

@papandreou
Copy link
Author

papandreou commented Jul 16, 2018

@ljharb, that sounds great. Before I try to rebase it on qs master, does this approach look good? papandreou/qs@9250c4c...interpretNumericEntities

@ljharb
Copy link
Contributor

ljharb commented Jul 16, 2018

I'd need to review it when rebased :-) it looks like a good direction tho.

@papandreou
Copy link
Author

Rebased branch: https://github.com/papandreou/body-parser/tree/feature/iso-8859-1/take2

Requires ljharb/qs#268 to be npm linked in.

@Xtrem65
Copy link

Xtrem65 commented Jul 17, 2018

Wow guys ! so much reactivity :D

I'd be happy to help if needed

@papandreou
Copy link
Author

@Xtrem65, you can help by trying it out :)

  1. Make a clone of https://github.com/papandreou/qs
  2. Check out the feature/iso8859-1 branch
  3. npm link
  4. Make a clone of https://github.com/papandreou/body-parser
  5. Check out the feature/iso-8859-1/take2 branch
  6. npm link qs
  7. npm link
  8. cd to your project
  9. npm link body-parser

... Then see if you can solve your problem. Depending on the requests that you need to support, you might have to specify the defaultCharset option.

@jonchurch
Copy link
Member

Closing as niche interest. Looks like some code was written to get folks who want this started. If someone wants to revive this and open a PR, we can get the defibrillator out on this issue.

@jonchurch jonchurch closed this as not planned Won't fix, can't repro, duplicate, stale Apr 27, 2024
@ljharb
Copy link
Contributor

ljharb commented Apr 27, 2024

indeed, the qs code for this is in v6.6.0+, so only body-parser would need changes.

@papandreou
Copy link
Author

papandreou commented Apr 27, 2024

@jonchurch, it's implemented by this PR, which has been sitting there for almost 5 years, waiting for a new major release that ditches support for node.js 0.10 and below :)

@jonchurch
Copy link
Member

apologies @papandreou I missed that PR link while triaging on mobile. I thought a PR never came through, will open this back up as there's a PR attached.

Getting up to speed on this request, as I have never had to deal with this problem. But Im all for better supporting common bodies on the internet via config options.

Reopening

@jonchurch jonchurch reopened this Apr 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants