Problems with gzipped sites?

[meub]

I'm having issues making this work with sites on Amazon S3 with gzipped content. I get a "Finished! 0 links found." message back. Other sites without gzip work perfectly though.

Is this a known issue and are there plans to support sites that use gzip?

[stevenvachon]

It should be supported. I'll have to add tests to find out.

[meub]

Thanks for the response. I just published a test gzipped version of my personal site: http://test.alexmeub.com

Running blc http://test.alexmeub.com results in:

Getting links from: http://test.alexmeub.com/
Finished! 0 links found.

Running blc http://alexmeub.com works just fine (my live non-gzipped site). So it seems gzipped sites don't work (at least if hosted on Amazon S3).

[stevenvachon]

bhttp doesn't support gzip. This is another reason to switch away from it.

[MrDys]

FWIW, I locally forked broken-link-checker and replaced bhttp with request-promise. I needed to be able to have cookies and gzip support. From what I can tell, it also addressed the redirects-reported-as-broken issue in #47.

[zeke]

forked broken-link-checker and replaced bhttp with request-promise.

@MrDys I tried this but didn't have much luck. Can you share a link to your working fork?

[stevenvachon]

request/request#3142

[stevenvachon]

Fixed in v0.8.0 branch.