🐛 Bug Report: Error on home page

[0x09AF]

Describe the bug

I'm pretty sure it is a change on Reddit itself and not a bug, but every subreddit, post and even a home page gives me a Failed to parse page JSON data: expected value at line 1 column 1.
I'm running libreddit in a docker container, the container itself is healthy and the logs didn't specify anything related:

libreddit  | Starting Libreddit...
libreddit  | Running Libreddit v0.30.1 on 0.0.0.0:8080!

Steps to reproduce the bug

Steps to reproduce the behavior:

0. Before I ran the latest :arm image from Docker Hub
1. Git cloned latest repo to run latest code
2. Same result

What's the expected behavior?

Give me back my subreddits and posts :)

Additional context / screenshot

N/A

[phinze]

I just had this happen to me and I checked curl to reddit.com from the host that was running libreddit and sure enough it was returning 403 Blocked responses.

Moving to a different egress IP seems to have fixed everything for the moment but it’s unclear how long that will last. Will report back if I learn anything more.

If you can test from a different IP I recommend giving that a shot.

[0x09AF]

I have a reserved static IP with a bunch of other services running on the same host. I would have to re-write allowlists, firewall rules etc. This is not a solution for me, unfortunately

---

EDIT: Funny, of the 4 microservers that I spun up - only one could connect to Reddit via curl. My home IP also was able to connect. Pretty frustrating

[phinze]

That contributes to the theory that Reddit just updated their IP blocklists to be much stricter. My guess is they’re working towards blocking everything but end users - so like only allowing traffic from residential IP blocks. Even if that’s the case, it will always be a cat and mouse game. Still it’s a disappointing development. 😞

[sigaloid]

If anyone has access to an IP that is still being blocked and has the ability to compile and run directly (not just docker), please shoot me an email/matrix message (email is in the bottom of https://matthew.science, matrix is sigaloid:matrix.org).

[maxexcloo]

If anyone has access to an IP that is still being blocked and has the ability to compile and run directly (not just docker), please shoot me an email/matrix message (email is in the bottom of https://matthew.science, matrix is sigaloid:matrix.org).

I've sent an email 👍🏻

[freedit-dev]

How to know that your ip is blocked

1. Failed to parse page JSON data: expected value at line 1 column 1
2. Visit reddit.com directly and get

whoa there, pardner!
reddit's awesome and all, but you may have a bit of a problem.

Make sure your User-Agent is not empty, is something unique and descriptive and try again. if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

If you're attemping to access Reddit via a hosting provider or private VPN, please either login at [reddit.com/login/](https://www.reddit.com/login/) or register/sign in with your developer credentials [here](https://www.reddit.com/wiki/api/).

You can read Reddit's Terms of Service [here](https://www.reddit.com/wiki/api/).

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please contact us at [this email address](mailto:ratelimit@reddit.com?Subject=Blocked).

when contacting us, please include your ip address which is: xxx.xxx.xxx.xxx and reddit account

What to do

1. libreddit fork with Tor support https://git.spec.cat/Nyaaori/libreddit
2. proxy with Cloudflare warp(proxy mode)

https://blog.cloudflare.com/announcing-warp-for-linux-and-proxy-mode/

warp-cli set-mode proxy

and than connect to reddit through socks5://127.0.0.1:40000

So I hope @sigaloid could resolve #841 first.

Or you could use my Temporary solution https://github.com/freedit-dev/libreddit

[maxexcloo]

Can I ask what your solution does @freedit-dev?

[nickelswitte]

I have the same problem as well.. Its really sad how reckless reddit its driving its decisions.

[DuplicantWood]

This is happening to me now too, i've tried some proton vpn servers but they all seem to be blocked.

[freedit-dev]

Can I ask what your solution does @freedit-dev?

You could just build and deploy libreddit fork with Tor support https://git.spec.cat/Nyaaori/libreddit

Or build and deploy my temporary solution https://github.com/freedit-dev/libreddit which add support socks proxy. And then install Cloudfalre warp, enable proxy mode. set env LIBREDDIT_SOCKET_PROXY to socks5://127.0.0.1:40000

[duoi]

@freedit-dev mind releasing a binary?

[freedit-dev]

@duoi Ok, but it's not compatible to the original libreddit. You must offer a socks5 proxy to make it work.

[duoi]

@freedit-dev great, thank you. Yes, entirely aware of the incompatibility. I can offer my Docker configuration after I get it working as a PR to your fork so that it can be standalone.

[maxysoft]

https://git.spec.cat/Nyaaori/libreddit is working fine for me. Can happen that after some time of inactivity appears "impossible to connect to tor" but just reload the page. As for the speed obviously is that of the tor network.

[wadbr]

If you're attemping to access Reddit via a hosting provider or private VPN, please either login at reddit.com/login/ or register/sign in with your developer credentials here.

Apologies if this is a stupid question, but can't we send a dummy/anonymous non-dev account login or cookie so reddit thinks we are logged in?

[duoi]

@freedit-dev posting here as issues aren't enabled on your repo. It seems that I can't execute the binary. The size is quite different from the officially compiled one, not sure if that's playing a role.

-rwxr-xr-x    1 root     root       8503616 Dec 13 22:16 libreddit

vs.

-rwxr-xr-x    1 root     root       6989056 Dec 13 22:17 libreddit

The outcome of execution is exec /usr/local/bin/libreddit: no such file or directory. I originally tried fetching through code, then manually downloaded and unzipped in case there were issues in how I was unzipping it, but the issue persisted. The same block of code fetching from the core libreddit repo binary executes without issue.

[Unmoved38]

Since I also didn't know how to use freedit his modified libreddit, here is another solution.

version: '3'

services:
  warp:
    image: caomingjun/warp
    container_name: warp
    restart: always
    ports:
      - 8080:8080 # libreddit
    environment:
      - WARP_SLEEP=2
    cap_add:
      - NET_ADMIN
    sysctls:
      - net.ipv6.conf.all.disable_ipv6=0
      - net.ipv4.conf.all.src_valid_mark=1
    volumes:
      - ./data:/var/lib/cloudflare-warp
 
  libreddit:
    container_name: libreddit
    restart: unless-stopped
    image: libreddit/libreddit
    network_mode: "service:warp"

https://github.com/cmj2002/warp-docker

[duoi]

@Unmoved38 I forked @freedit-dev's repo here and enabled the build step that the core libreddit use to get a functioning binary. Haven't had time to install warp onto the container but this is will at least fetch the release from my fork and start the container: https://gist.github.com/duoi/5314a768d8815171f77c950aa5a9da08

[yaky-dev]

I suspect reddit started blocking some IP ranges of Linode, regardless of traffic or software like libreddit or teddit. I rent two VPSes, with separate IP addresses, and only one has ever hosted libreddit. However, I get 403 Blocked and "whoa there, pardner!" text from both VPSes.

Out of sheer curiosity, if anyone has a Linode, DigitalOcean, or other rented VPS with a static IP, would you mind confirming that you are also blocked by reddit? If it matters, I used Lynx from the terminal, lynx reddit.com.

[maxexcloo]

Oracle Cloud is definitely blocked also.

[mcanyucel]

I suspect reddit started blocking some IP ranges of Linode, regardless of traffic or software like libreddit or teddit. I rent two VPSes, with separate IP addresses, and only one has ever hosted libreddit. However, I get 403 Blocked and "whoa there, pardner!" text from both VPSes.

Out of sheer curiosity, if anyone has a Linode, DigitalOcean, or other rented VPS with a static IP, would you mind confirming that you are also blocked by reddit? If it matters, I used Lynx from the terminal, lynx reddit.com.

I have a vps from time4vps in lithuania and its ip is blocked, too. Anxiously waiting for a docker image with vpn ability :) .

[Nionor]

I have a vps from time4vps in lithuania and its ip is blocked, too. Anxiously waiting for a docker image with vpn ability :) .

I was hosting fine on Oracle Cloud but it got blocked too, so I throw in gluetun to a vpn and it works fine again, but it might just be a question of time before it gets blocked too.

[maxysoft]

Contabo and Hetzner CIDR seesms to be blocked too. Just tried from a couple of vps that had never connected to reddit servers before.

<p>Your request has been blocked due to a network policy.</p>

The question is: Why don't just use tor? My instance is still working fine after 2 weeks.

[sigaloid]

@wadbr:

Apologies if this is a stupid question, but can't we send a dummy/anonymous non-dev account login or cookie so reddit thinks we are logged in?

Not a stupid question, as it's exactly the method I'm going to use when I fork it in the next few weeks :D

@maxysoft:

The question is: Why don't just use tor? My instance is still working fine after 2 weeks.

Tor is a good alternative, probably one I'll add as well once I get all of this on the front burner (next 2 weeks ish?)

[cronyakatsuki]

Contabo and Hetzner CIDR seesms to be blocked too. Just tried from a couple of vps that had never connected to reddit servers before.

<p>Your request has been blocked due to a network policy.</p>

The question is: Why don't just use tor? My instance is still working fine after 2 weeks.

@maxysoft How did you setup tor for libreddit? I would do it but only for libreddit itself so that my other stuff doesn't use tor traffic.

[cronyakatsuki]

I suspect reddit started blocking some IP ranges of Linode, regardless of traffic or software like libreddit or teddit. I rent two VPSes, with separate IP addresses, and only one has ever hosted libreddit. However, I get 403 Blocked and "whoa there, pardner!" text from both VPSes.

Out of sheer curiosity, if anyone has a Linode, DigitalOcean, or other rented VPS with a static IP, would you mind confirming that you are also blocked by reddit? If it matters, I used Lynx from the terminal, lynx reddit.com.

Both my vps's on vultr and contabo are block rrn, it started about 3 weeks back, then they got block literally the same hour.

[maxysoft]

@maxysoft How did you setup tor for libreddit? I would do it but only for libreddit itself so that my other stuff doesn't use tor traffic.

I've used the modified libreddit by nyaaori

https://git.spec.cat/Nyaaori/libreddit is working fine for me. Can happen that after some time of inactivity appears "impossible to connect to tor" but just reload the page. As for the speed obviously is that of the tor network.

[sigaloid]

For all still impacted by this: I've forked it and added OAuth spoofing which bypasses the rate limiting.

https://github.com/redlib-org/redlib

[jrejaud]

an, or other rented VPS with a static IP, would you mind confirming that you are also blocked by reddit? If it mat

I also use a rented VPS with static IP, also having the same issues with both teddit and libreddit

[jrejaud]

For all still impacted by this: I've forked it and added OAuth spoofing which bypasses the rate limiting.

https://github.com/redlib-org/redlib

Worked for me, thank you.

Also thank you for adding a docker compose file, I've been using NPM + portainer for my self hosted servers so a docker file is very much appreciated.

[sigaloid]

Glad to hear it fixed your issue! 😁

[maxexcloo]

Glad to hear it fixed your issue! 😁

Can I ask if you’ve included any other PRs or tweaks? Looks awesome!

[sigaloid]

You can find a full list here: redlib-org/redlib#2

Tldr, most of the active PRs in the repository that were important changes or fixes. If I missed any someone wants, would be happy to take a look.

[0x09AF]

I myself have moved to redlib.
After deploying even my old subscriptions have been picked up by Redlib.
Awesome work, thanks @sigaloid