Cookies lost during multiple redirects

[nijikokun]

1. Put a cookie on a 302
2. Do another 302
3. View headers, cookie from the first 302 does not exist.

[lalitkapoor]

Hey @nijikokun can you give me more details. What were the domains and protocols of the redirects? Thanks.

[nijikokun]

Hey @lalitkapoor

Can't give the domains as they are internal, should work with any, protocol was https.

Cookie type was: http-only; secure

[lalitkapoor]

thanks! Will look into it. If it's not too much trouble could you create a failing test?

[anklos]

I tried with node https module, it does not preserve cookie from first redirect either.

Is it possible to get the first redirect response rather than the final response?

[lalitkapoor]

@anklos you could set request to not follow redirects and do the redirect manually, in which case you could get the first response.

[anklos]

@lalitkapoor thanks. I just did a test with followRedirect false in the request option. It seems that the code can stop at the first response, as the statusCode is 302, however, the header is the same as after redirects response's header, it does not contain set-cookie content from the first redirect.

[nijikokun]

Cookies are not passed with redirects
On Mar 23, 2015 8:21 PM, "Shane Niu" notifications@github.com wrote:

@lalitkapoor https://github.com/lalitkapoor thanks. I set followRedirect
to false in the option. It seems that I could reach the first response, as
the statusCode is 302, however, the header is the same as after redirects
response's header, it does not contain set-cookie content from the first
redirect.

[image: image]
https://cloud.githubusercontent.com/assets/530145/6777142/bab3d244-d19a-11e4-9c15-4cbc9af23316.png

—
Reply to this email directly or view it on GitHub
#1502 (comment).

[lalitkapoor]

@anklos can you tell me if the domains are different?

[lalitkapoor]

I don't need the exact domains, just need to know if they're the same...if one is a subdomain, etc.

[anklos]

they are on the same subdomain.

[lalitkapoor]

@anklos are you able to modify the cookie on the servers? or do you not have access to do that. I'd like to try something: setting the domain value on the cookie to the hostname (w/subdomain).

@nijikokun do your cookies have the domain value set on them?

[anklos]

@lalitkapoor no I dont. but I can try to set up a https server myself to test the cookie behavior.

[anklos]

although still not make sense to me why browser could get the set-cookie content from the first redirect but not my node.js program?

[anklos]

Just find that I can get set-cookie by setting redirect option false. Originally I did not get it because I send request to the first redirect url, not the destination url.

Also tried other language http libraries to do the same test, the results are the same as nodejs

[lalitkapoor]

@nijikokun does the above satisfy the concern?

[nijikokun]

For me yes

On Fri, Apr 3, 2015 at 9:06 PM, Lalit Kapoor notifications@github.com
wrote:

@nijikokun https://github.com/Nijikokun does the above satisfy the
concern?

—
Reply to this email directly or view it on GitHub
#1502 (comment).

[jazarja]

Having similiar problem with redirection cookie, I use the following code to extract cookie on first redirection:

var httpreq = {
                    ...
                    jar: cookieJar
                };



request(httpreq, function (error, response, body) {                   
                    if (typeof response.headers["set-cookie"]!='undefined') {
                        response.headers["set-cookie"].forEach(function(aCookie)
                        {
                            cookieJar.getCookieString(aCookie);
                        });
                    }
});

[Freyert]

@lalitkapoor I seem to be having this issue on 2.61.

1. I pick up a cookie from my original request.
2. Head to the endpoint in the same domain which starts a chain of redirects.
3. During the automatically followed redirects no cookies are ever used.

When I turn on NODE_DEBUG=request, I see this pattern on all redirects:

REQUEST redirect <url>
REQUEST redirect to <url>
REQUEST {}

I imagine that the third log output should have my cookies in them. The line where this is logged can be found here.

Also, it's good to note here that I am using request.defaults to set a few defaults.

So in request that are not redirects that third line looks a little bit like this:

REQUEST {
  followAllRedirects: true,
  jar:  {
     //All the cookies in the cookie jar.
   },
   method: 'GET',
   url: http://theurl.com
}

[lalitkapoor]

@Freyert Thanks for the details. I'm traveling this week, but will be able to look at it this weekend, please feel free to remind me.

[Freyert]

@lalitkapoor Thanks! I'll keep taking notes as I explore.

[Freyert]

As an extra note:

It seems odd that the third debug statement should be empty on redirects. At a minimum I would expect it to contain followAllRedirects.

Placing a debugger on this line.
I can see that the request object passed to the Redirect constructor has followAllRedirects set to true. This gets nested inside the Redirect object which has it's own followAllRedirects which is set to false.

---

Discovery A: Every Request object receives an instance of Redirect

So when I do request.defaults, I'm creating a new request object with some defaults, but it also has a Redirect object that manages all redirects from any request made with that request object.

For example

req(siteA) -> siteB -> siteC -> siteD

These Redirect Chains are kept in an array on the Redirect object.

Not a ground breaking discovery.

---

Problem B: Why is it not using my request.default settings to make other requests?

This is probably the big tuna. Since request.defaults does not modify the global Request object there are abound to be methods that don't receive these default options. As a hypothesis it makes sense that Redirect is not using my default request object, but instead the global object to make any redirect request.

Therefore, we would see {} in the third debug statement above. Ideally, there would be a way to store the cookies. Perhaps, I need to resort to the global cookie jar?

[leonardean]

Hi @lalitkapoor, I have a very similar issue when dealing with requests that contains redirection in response. I have set followRedirect to false btw. below is the difference of calling the API using Postman when testing and using Request:

when testing in Postman

when using Request

so I use this code to do the request:

var request = require("request");
require('request-debug')(request);

var options = { method: 'POST',
  url: 'something hidden',
  qs: 
   { 
      //something hidden
    },
  followRedirect: false,
  jar: true,
  headers: 
   { 'postman-token': '48a42acc-3fa4-3c3e-e826-34e6deb26b09',
     'cache-control': 'no-cache',
     host: 'hidden again',
     referer: 'hidden, sorry' } };

request(options, function (error, response, body) {
  if (error) throw new Error(error);

  console.log(body);
  console.log(response.headers)
});

and the console log of response.header only shows this:

{ server: 'Cowboy',
  connection: 'close',
  date: 'Tue, 29 Dec 2015 08:59:07 GMT',
  status: '302 Found',
  'x-frame-options': 'SAMEORIGIN',
  'x-xss-protection': '1; mode=block',
  'x-content-type-options': 'nosniff',
  location: 'https://developer.kii.com/',
  'content-type': 'text/html; charset=utf-8',
  'cache-control': 'no-cache',
  'x-request-id': '255fb857-95fa-4aea-8f9e-17036f166083',
  'x-runtime': '0.009318',
  'strict-transport-security': 'max-age=31536000',
  via: '1.1 vegur' }

so you see, the set-cookie header is missing somewhere. May I know your suggestion. If you need any more info, please let me know

[jeffreyleeon]

+1

[patricknn]

+1, once the website that I need to access blocks the connection when you don't have cookies from the redirect, I can't access the page.

[lalitkapoor]

@leonardean @jeffreyleeon @patricknn Can you look at tests/test-redirect.js and create a failing scenario?

[rahulbir]

+1

[rodw]

+1 - this seems to be a problem with other headers (e.g., authorization) as well.

Is the only current workaround to set followRedirects to false and handle the redirects "manually" ?

[melassa]

AFAIK yes

[ezzygemini]

+1

[atMari]

+1

[avadhpatel]

I found a workaround by using .on('redirect', function() {}) event handler on your request object. Use it to copy the cookie jar and save it for later. Its much easier than handling all the redirects manually.

[jmmclean]

@avadhpatel can you add some psuedocode on how you would handle the callback?

[avadhpatel]

Sure:

request.get('http://google.com/')
   .on('redirect', function() {
       // collect the cookies from 'this' object.
    });

[jmmclean]

@avadhpatel yea, that doesn't work too well because to follow redirects, you should be stripping out the 'set-cookie' header in the response headers. As you can see in your example, there is no object passed to the callback. I don't think the 'this' object is referring to the response....seems weird to me.

[avadhpatel]

@jmmclean 'this' object in the function is actually the request object.

The this.response refer to the response that has 'set-cookie' header set. I copy my cookies from that.

[delijati]

The comment from @jazarja worked for me.

const that = this;
if (r.headers.hasOwnProperty('set-cookie')) {
    r.headers['set-cookie'].forEach(function(aCookie) {
        that._request.options.jar.setCookie(aCookie);
    });
}

[saknarak]

+1

[ccgagnon]

+1

[drcrack]

not fixed

[tylinux]

not fixed yet

[eliothxchan]

+1

[rocifier]

+1 I can't use postman for our cross-domain authentication because of this, it's a showstopper for me.

[parveenoodles]

I have the same issue , i didn't get all of the cookies from request module.
But as i set the jar attribute to true, then it returned me all of the Cookies.
I used "request-promise" module for http request.
Below is my code:-
var reqOpt = { url: url, method: 'GET', timeout: 20000, qs: { 'token': token }, form: null, jar: true, headers: { 'User-Agent': 'Request-Promise' }, resolveWithFullResponse: true }

As you can see i have set

jar : true

and it worked for me.
Hope this will help.

[Kamikadze4GAME]

+1

[Revadike]

Can I make a suggestion? Why not return an array of responses when using followRedirect: true or followAllRedirects: true. The array contains the responses of each redirect. This way, we can access all intermediate headers (including cookies).

[AshokChava]

I have multiple redirects in my case (not just one redirect, there are 3 redirects happen when I access a URL). I have tried all the suggestions above, but without success. Can anyone share manual handling of redirects code and retrieving the cookies? Thanks.

[jonmifsud]

@lalitkapoor was this ever tackled? And is it being worked on?

I think this looks similar to what's being done with the authorization header in line 72 for 401 redirects. Haven't done too much nodejs patching; but would be happy to send up a PR if I manage to fix it.

Update: I thought I had this issue but then I've added jar: j, instead of jar: true and cookies seem to be passing through. where var j = request.jar(); - hope this helps the next person.

[esco]

@jonmifsud 👍 using request.jar() result instead of true works for me, thanks!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.