CloudProxy gets stuck on jsch challenge #20
Comments
|
I’ve been a little backlogged lately at work but I’ll get back to reviewing issues and PRs shortly. Thanks for being thing this to my attention 🙏 |
|
@lululombard I don't think this is a bug or issue of the proxy. Most sites will need the proxy to try several times before the challenge is solved. Don't ask me why but this been my experience so far. So for your site, I ran below curl 2 times and got a successful response then. Note that it is important to create a custom session, so that the correct headers and cookies are sent. Request: Response: |
|
@jairoxyz thanks for testing dude! When I run the following command, it will get stuck: I get the following reply: {
"status": "error",
"message": "Navigation timeout of 30000 ms exceeded",
"startTimestamp": 1596805537584,
"endTimestamp": 1596805586296,
"version": "1.0.0"
}And here are the logs: Even with sessions, I can login, browse resources, but not download them: |
|
Ah I see you are using the download parameter in the GET request for downloading the file itself:
I haven't tried that myself yet. Will have a look at the code and see if something occurs to me ;) PS: OK so I added download:true to the request and I get the timeout on first try but on second try I get: I checked in Fiddler and the response is actually successful but I think there is an issue with MIME type or so because response is not processed correctly: Maybe sending a header like this would help?
Or it might even work in your py code because you can handle the bin data there. I think you're giving up after 1st timeout. Run a small loop to rerun the POST with timeout 10secs until response is 200. |
|
So I've been trying to do some retries, I also got the |
|
Update! By adding this to routes.ts I am able to download files: Chrome says it failed and will keep the temporary download name in my downloads folder, but it's the right md5, I can rename the file with the correct extension and it will work, so that's good progress. I'll continue working on this and hopefully have a PR ready in a few hours |
|
Yes, I saw that Page.setDownloadBehavior when googling for Puppeteer file download yesterday. Good job :) |
|
@lululombard Wait so will this download the file to the downloads directory? That's probably better than the base64 encoded buffer solution I was using but I wonder how this would work with the Docker? Tbh I barely even use the Docker file so I might add a pole to remove it if no one else is making use of it. The only reason I've been using ENV variables to pass things to the script is because of Docker. I'd much rather use CLI flags or a config file or something. |
|
@NoahCardoza I'm using the tmp package to set a download directory, it works with Windows, macOS and Linux from my testing. I'll try with Docker once I'm sure that I don't need to make any changes to the code :) |
|
Ah, so you can set the directory! Good. We might be able to work it like this: Save the file to the tmp dir, and then return a url rather than the file in the json. Then they can use that url to download the file from the tmp dir and after serving the file it will be deleted? |
|
Right now I'm returning the base64 from the file like before, and deleting the temporary file and folder when CloudProxy replies, but yes we can do anything we want :) |
|
I think the idea I just purposed would be the best general solution, since it wouldn't be a good idea to encode larger files into JSON. Like if someone wanted to download a video or even a large photo. |
|
Indeed, for now I'll still work like this, I'm downloading files of 3-4 MB and it works, but sure we'll probably hit a limit. I'll continue working on this though :) |
|
Great! |
|
Sorry I kind of got off track. I have something working but not in a final state, and I'll probably not update it because dumping cookies and use them directly in my Python program works. That said, it's available here: https://github.com/lululombard/CloudProxy/tree/feature/download-redirect-support |
So I've been implementing CloudProxy on one of my projects, basically I'm trying to download plugins from SpigotMC.org and everything works fine until I want to download a plugin.
When browsing normally, the first time I download a plugin, I'll get the following screen:
And the download starts.
But with CloudProxy, this happens:
I think it's probably related to the
// TODO: find out why these pages hang sometimesbut no mater how many times I try, it will fail on that link. You can add me on Discord (lululombard#1337) if you require more information, I can even give you the credentials to my account so you can try to run it for yourself to debug :)The text was updated successfully, but these errors were encountered: