-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can't pipe from request that has a POST body #1664
Comments
Try with this one: var express = require('express')
var request = require('request')
var app = express()
// put it before any other middleware
app.use(function(req, res) {
req.pipe(
request[req.method.toLowerCase()]('http://example.com'))
.pipe(res)
})
app.listen(3000, function () {
console.log('Express server listening on port 3000')
}) In case you want to modify the body, just construct a new request and pipe to the response. |
@simov I don't want to modify the body, just the URL. So I want to keep the headers and other properties of the original request but pipe into a new request with a new URL. The problem is that the body is not copied over when the original request is piped into the new one (not sure if this is a bug), and if I try to copy the body over myself by specify the When you say "construct a new request" are you suggesting that I simply create a new request from scratch with the body I want and copy over the headers from the original request myself, rather than piping them in? I can do that but it would be less repetitive and fragile if I could use the built-in pipe support to copy the original request in the new one. I still think one or both of these two issues is probably a bug (i.e. 1) the request body is not copied when piping and 2) the exception mentioned in my originak post). |
I don't know too much about express or request (just using these for a single-page app proxy without really understanding how they work), but this fixed the request.post({ url: 'http://www.example.com', form: { foo: 'bar' }}).pipe(res); Instead of: req.pipe(request.post({ url: 'http://www.example.com', form: { foo: 'bar' }})).pipe(res); |
@matthewgertner just try out my code example, it works. |
@simov It definitely works, but your example doesn't cover the case where |
@matthewgertner can you give me your exact code example that doesn't work? |
@simov It's the sample in my original post. If I use this one then the body is not sent:
If I use this then I get that exception:
|
Can you replace Or you can use my example code from above as well, because that's what I'm using for server. Then I'm executing this script: var request = require('request')
request.post('http://localhost:3000', {form:{some:'data'}}) As you can see in my link I have the |
Let me explain my use case. I implemented a simple web API proxy (see https://github.com/salsita/web-api-proxy). All it does is receive HTTP requests and redirect them to another host, replacing URL parameters if necessary with environment variables on the proxy server. The point is to avoid having to embed secret keys into client code. It works great for GET requests, but I recently had a requirement to use it for POST as well. So I'm not initializing the body of my request as in your previous comment since the body is coming through the Express request. Your most recent example definitely works but that doesn't really help me. What I need to do is:
It seems to me that if
That causes an exception to be thrown due to what I suspect is a bug in Anyway, I can build a completely new |
It doesn't work because you are using a
Besides that if you need to modify the body, again from my first comment:
|
But I'm using
It's a bummer that I basically need to rewrite the code that's already in IMO it should be possible to pipe a request into another request that has a form body without an exception being thrown, or do you think there is some reason for this behavior? |
This is making The constructing of new object applies only when you want to modify the body. |
Sure, the current codebase does not support POST. I'm trying to fix that, which is when I ran into the problems outlined here. I only referenced the existing codebase to explain the use case better. I'm pretty sure I need to create a new object always, even if I don't want to modify the body, since the body is not copied over by the pipe operation. Did you try the example I provided? You need an Express request with a body that you pipe into a |
This one works (your example): 'use strict';
let express = require('express');
let request = require('request');
let app = express();
app.use('/', (req, res) => {
req.pipe(request.post('http://www.example.com'));
});
app.listen(process.env.PORT || 3000); Using this to make the request: var request = require('request')
request.post('http://localhost:3000', {form:{some:'data'}})
Where is that, I'm not following you. |
Hmmm, ok. When I use requestb.in (nice tip!) I can see that the body actually is sent. I actually had (and still have) two problems that led me to believe it was not being sent:
I guess that 2) is irrelevant and the body is simply stored somewhere else. I'd be interested to know if you can reproduce 1) though. If you send the request to |
|
I actually used
It looks to me like when I use |
Yep, if you take a look at my first comment again:
|
Phew! Okay, thanks a ton for your help and patience. Took a while to get to the bottom of that. I still think it's kind of a bummer that I can't pipe in all the headers and stuff and then modify the body, but I guess there's no way around that. |
app.use(function(req, res) {
req.pipe(
request[req.method.toLowerCase()]('http://example.com'))
.pipe(res)
}) will fail for DELETE. as request.del is the method and not req.delete |
I know this is old, but I had a hard time finding a suitable solution to this, and thought it might be useful to others. 'use strict';
let express = require('express');
let request = require('request');
let app = express();
app.use('/', (req, res) => {
req.pipe(request.post({ url: 'http://www.example.com', form: { foo: 'bar' }}), {end: false}).pipe(res);
});
app.listen(process.env.PORT || 3000); As shown in the https://nodejs.org/api/stream.html#stream_readable_pipe_destination_options, adding this makes it so the new stream isn't automatically terminated when req is terminated. While this caused this example to work, I've noticed some issues making a proxy for other methods using this. The body somehow causes it to close everything properly, but without a body it still hangs. If using a json body parser, this shouldn't cause a problem since it causes the body to be read as |
@JeffreyAngell just try your resolution, it works. thx |
This works for me:
}); function createOptions(method, targetUrl, json) {
}` |
I experienced the same issue. Could you explain why it has to be before certain middleware in my case only the EDIT:
https://stackoverflow.com/questions/27117337/exclude-route-from-express-middleware |
I'm trying to create a new request object by piping from an Express request. This works fine for GET requests, but for POST requests that have a body, the body does not seem to be copied into the new request. I tried copying over the body manually like so:
let pipedReq = req.pipe(request({ url: 'http://www.example.com', form: req.body }));
This copies over the body but then I get a "write after end" error when
pipedReq
is cleaned up. Might be related to #1659.It's easy to reproduce the problem with the following simple Express app:
Backtrace:
The text was updated successfully, but these errors were encountered: