[FR]: How to setup the network requests for feeds?

[Dean-Corso]

Brief description of the feature request

Just would like to ask whether there is any setting to setup custom send requests (number of requests)? In some cases I need to limit the requests to prevent error responses 429 (too many requests) or others to get blocked by an server for specific time X. Specially when requesting many feeds to the same server (rss service).

Feature request: Would be nice to have an user option to set custom requests count to send at once for any specific selected feeds (if possible) or else a global option would be also good to have anything. So I don't see such an option in rssguard yet in network setting.

[martinrotter]

There is global feed update interval in app settings -> feeds & articles, also you can select multiple feeds and edit the with right click and set specific intevals to those.

All options are there my friend. Just look better.

[Dean-Corso]

Hi @martinrotter,

could you explain that some more where the setting is? I can't find it. So in Feeds & Article setting window. Just see a timeout when to aboard reading a feed which is 30s but that's all. In the Network setting I also don't see anything to setup for the feeds request times / speed etc. When I do select one more more feeds and open the edit window for them I also don't have anything in Network tab to set / change something about requests etc. Maybe you can tell me exactly where it is or show me an image to see it. 😃

As I said, I have two types of feeds. Normal official feeds and custom feeds using same server and where I want to limit / slow down the request interval to prevent error responses like 429, So the best would be to have a global setting (how much max requests to send at once) and a custom setting for any feeds I want (exclude global request interval setting for them).

Example: I have 2 folder with 10 feeds in one of them. Folder 1 = Official feeds & folder 2 = Custom feeds. When I now select both folders to update them all at once then some same request interval is used (I did not find the settings for this yet in RSS Guard). In folder 1 I get no issues but in folder 2 I get issues (429) and request did fail because they was to much. Now I would like to limit / slower the request interval for all feeds in folder 2 only or just some selected feeds etc. Just to set a custom thread count you know what I mean. So in QuiteRSS tool is such an option in the network tab where you can set the request count value (default is 10) I had to slow down to make it work / request more smooth to prevent error responses. But the option is just global for all feeds only. Just looking for some similar or better setting for this in RSS Guard you know. Maybe you can tell me whether its possible already to set this in RSS Guard or not. Thanks.

PS: Sorry, I don't see it yet. Maybe its well hidden. The options I found for intervals to wait 15 seconds when app was started or when article should get updated / abgeholt is not what I'm looking for. Also not the under 10 seconds interval option to "Schnell-Abhol" is not the right thing. Anyway, if there is any option/s for the requests then show me please or otherwise if there is nothing then it would be nice if you could add such an option is possible. "Timeout X" (you have this option / 30s) - "Number of requests X" (max requests to send at once / threads - you don't have this option or I can't find it). "Number of attempts X" (how often the feed should get requested max before giving up). You know.

[Ac314]

Settings -> Feeds & articles -> Feed fetching -> Auto-fetch articles for all feeds every - this is a global (default) option for all feeds.
To customize it for specific feed: right click on the feed -> Edit selected items -> Articles -> Auto-downloading of articles - this will set the customized option for specific feed.
If you need to customize this option for several feeds just select multiple feeds at once (for example all feeds in folder) and do the same as with the sole feed. The option will be set for all selected feeds.

[Dean-Corso]

Hi @Ac314,

so I don't use any auto feed fetching option just want to update manually when I want it. So I have this options unchecked / disabled. Even that single fetching setting is not what I'm looking for. I want to set a custom request number / value RSS Guard should use to fetch feeds without to run into an Error state like 429 in case its sending to much requests at once without to wait. In QuiteRSS I have that option in the network tab which is in default state set to 10 (10 requests / 10 single threads) but this is too high and I have to lower it to prevent response issues from server. I'm looking for such an option in RSS Guard but don't see it. Do you know what I mean?

I'm not talking about any kind of auto feed checking or something like that. Just want to set some request value. Lets say I want to update all feeds at once but only ONE by ONE (one request only till request is finished and then RSS Guard should handle the next feed) or 5 requests at once maximal etc. Just to set any decent value for the requests to send to an server. Make more sense to have such an option in case when requesting the same server itself in case of custom feeds / feeds services or websites who having many different feeds you can use. So if someone wants to visit you then this person will not ring your bell tons of time (ring, ring, ring, ring etc) at once. Anyway, so I hope you understand now what I mean (network connections).

[Ac314]

So you have several feeds from one server and you want them to be updated one-by-one (or restrict maximum number of simultaneous connections to the server) to avoid http errors like "too many requests", right?

[Dean-Corso]

Yes. Just wanna tell specific feeds how to request them (max X at once only) or if this is not doable to setup specific feeds then a global setting for the requests would be also OK. So QuiteRSS has such an option in network settings. Just missing something like that in RSS Guard. See image...

...from QuiteRSS tool. I did set it to 5 requests maximal to send / handle at once. So that's a limit to prevent sending more requests as it does in default mode (10). Problem in this QuiteRSS is that this setting is global and used for all feeds. Would be nice to have such a option in RSS Guard for specific feeds / selected feeds etc or if not possible then a global option so that I can set a limit to prevent error responses from server. The good thing is that RSS Guard is not sending any HEAD requests what QuiteRSS does first before sending GET request. In this case the request times are twice as much as in RSS Guard. Anyway, but a setting in RSS Guard would be nice to have of course.

PS: Small question - So I did noticed that RSS Guard always hangs / freeze a while `(maybe 5 seconds) when I start it. Why? How to prevent that?

[Ac314]

This setting cannot be feed specific, it is server specific. Feed itself is just updated periodically (or manually) so it uses single connection only. Restrictions could be needed only if we have several feeds with the same web server. So it is application wide setting.
Martin, if you read this, could you please clarify what is happened if feeds are updated and some of the feeds are from the same web server? Are they updated one by one, simultaneously or someway else? This could happen, for example, for sites like reddit - if user is subscribed to, like, 15 subreddit feeds and updating is started, what will happen?

PS: Small question - So I did noticed that RSS Guard always hangs / freeze a while `(maybe 5 seconds) when I start it. Why? How to prevent that?

I never encountered such a thing so I cannot help here.

[martinrotter]

Like I said in the other ticket.

I will love to implement that but I need OPML file or URLs of feeds from same website which do have this limit to actually test the feature.

At this point, the feeds are updated in parallel, feed website/origin is not taken into account at all.

[Ac314]

Well I personally did not encounter such a problem by myself, I just wanted to help the OP. Maybe @Dean-Corso could provide the sample OPML with the bunch of the feeds form the same server to test.

I guess the most apparent way to resolve this is to restrict the amount of simultaneous connections to the same server while updating feeds (the option " Restrict the amount of simultaneous connections to the same server while updating the feeds", default is no restrictions). Like if app is going to update the feeds (automatically or manually), there are multiple feeds to update at once and some of them are from the same server then these specific feeds are updated with some queue with the maximum amount of connections is defined by the above option. All other feeds are updated as before with no restrictions.

In case even this is not enough (for servers with the very severe restrictions) there could be the additional option "Delay between the consecutive connections to the same server while updating the feeds". It works only if maximum connections (from above option) is set to 1. Default delay is 0, but users could set it manually to allow updating from specific servers when even just making the sequential "one after another" connections is not enough.

[martinrotter]

@Ac314 You described at all well, the only thing I need is an actual feed list. Without it, I do not plan to implement.