feat: add proxy rotation (settable per store) #1026
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Doridian
changed the title
|
@Doridian with the great contribution! Thanks for help. I'll take a look at this tonight and get back. |
|
quick question about this. I know format is socks5://username:password@ip, but what if there is no username/password for said socks5 proxies? Also since default port is 1080, is that what it's guessing as it's port or will a format of socks5://ip:port work as well? Thanks for the contribution! |
|
Correct. That format should work. Default port is 1080 indeed.
…On Mon, Nov 30, 2020, at 19:34, Muji Sayed wrote:
quick question about this. I know format is ***@***.***, but what if there is no username/password for said socks5 proxies? Also since default port is 1080, is that what it's guessing as it's port or will a format of socks5://ip:port work as well?
Thanks for the contribution!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#1026 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAE2E4OK5ESLOR2YL2FTQNTSSRP5RANCNFSM4UHDFMTQ>.
|
jef
approved these changes
Dec 2, 2020
| return false; | ||
| } | ||
|
|
||
| const typ = request.resourceType(); |
There was a problem hiding this comment.
Is this because type is a reserved word? Haha 😝
|
Indeed it is.
On Tue, Dec 1, 2020, at 16:11, Jef LeCompte wrote:
***@***.**** approved this pull request.
Amazing. Thank you so much @Doridian <https://github.com/Doridian>. This really needed some clean up. I appreciate the time spent on the refactoring as well!
In src/store/lookup.ts <#1026 (comment)>:
> + store.currentProxyIndex = 0;
+ }
+
+ logger.info(
+ `ℹ [${store.name}] Next proxy index: ${store.currentProxyIndex} / Count: ${store.proxyList.length}`
+ );
+
+ return store.proxyList[store.currentProxyIndex];
+}
+
+async function handleLowBandwidth(request: Request) {
+ if (!config.browser.lowBandwidth) {
+ return false;
+ }
+
+ const typ = request.resourceType();
…
Is this because `type` is a reserved word? Haha 😝
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#1026 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAE2E4PDSXMPZP2BBCLZ3L3SSWA3PANCNFSM4UHDFMTQ>.
|
jef
changed the title
|
@Doridian which socks5 proxies do you use? Looking for suggestions |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Adds ability to add multiple proxies per store, rotated once per fetch.
Proxies are read from a file
STORENAME.proxiesin the format likesocks5://username:password@ipone per line.Adds some require architectural changes for that to ensure adblocker doesn't explode (puppeteer only supports one request listener per instance, so we have to write our own chain there)
Testing
New dependencies