Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

User Repo #17

Closed
procount opened this issue Mar 6, 2020 · 9 comments
Closed

User Repo #17

procount opened this issue Mar 6, 2020 · 9 comments
Labels
wontfix This will not be worked on

Comments

@procount
Copy link

procount commented Mar 6, 2020

The imaging Utility does not like to access resources hosted on sourceforge.
I suspect it is because the UserAgent is set to a default value that is not compatible.
I suggest to use "Wget/1.15 (linux-gnu)" which works in PINN.
I notice there is void DownloadThread::setUserAgent(const QByteArray &ua) but I don't know what it gets set to (if at all).

@dividuum
Copy link

dividuum commented Mar 6, 2020

The user agent set is Mozilla/5.0. Full request when using imagingutility --repo http://localhost:9999:

GET / HTTP/1.1
Connection: Keep-Alive
Accept-Encoding: gzip, deflate
Accept-Language: en,*
User-Agent: Mozilla/5.0
Host: localhost:9999

@procount
Copy link
Author

procount commented Mar 6, 2020

That will probably be the cause. Sourceforge will think it is a browser and present a webpage to select download mirrors etc., instead of just serving the file directly.

@maxnet
Copy link
Collaborator

maxnet commented Mar 7, 2020

The user agent set is Mozilla/5.0

Note that the user agent is likely different when downloading images.
(downloading the repository files is done by Qt/qml, downloading the actual images by libcurl, as there were problems with downloading large files with Qt's network code in the past. It would buffer them to memory without limit, causing problems on devices with little memory).

@procount
Copy link
Author

procount commented Mar 7, 2020

I hosted a custom json file on sourceforge and it wouldn't load. I had to move it and the icon file to github for it to work. I had the same issue with PINN which is why I know sourceforge is picky about user agents. But the image won't load from sourceforge either. Do you know the correct URL API to access an asset from a release on GitHub.

I will post my URLS later so you can try it out.

@maxnet
Copy link
Collaborator

maxnet commented Mar 7, 2020

But the image won't load from sourceforge either.

While I have not tested it with the rpi Imager, I recall Berryboot does use the standard libcurl user-agent, and gets its images from sourceforge.
So may want to look if there is not another issue that influences this, like exact URL used (more then one way to link to files on sourceforge).

@procount
Copy link
Author

procount commented Mar 7, 2020

I've just tried sourcing the image from Sourceforge again and it works now, whereas it didn't before. Maybe it was just a temporary internet issue at my end. 🤷‍♂

So the image now loads from there, but the icon or json file still don't.

You can try it with this JSON file.

{
        "name": "PINN",
        "description": "A multi-boot OS installer with OS admin features",
        "icon": "http://raw.githubusercontent.com/procount/pinn-os/master/os/pinn.png",
        "subitems_url": "http://raw.githubusercontent.com/procount/pinn-os/master/os/os_list_imagingutility_pinn.json"
}

The subitem URL contains 2 copies of the same PINN image, one on Github and one on Sourceforge. Both images can now be downloaded, but the icon from sourceforge does not load. Debug says the image is the wrong format - I suspect because sourceforge is serving up a HTML file based on the user-agent.

EDIT: Here is the debug message:
qrc:/main.qml:413:21: QML Image: Error decoding: https://sourceforge.net/projects/pinn/files/Etcher/pinn.png/download?use_mirror=iweb: Unsupported image format

@lurch
Copy link
Contributor

lurch commented Mar 7, 2020

EDIT: Here is the debug message:
qrc:/main.qml:413:21: QML Image: Error decoding: https://sourceforge.net/projects/pinn/files/Etcher/pinn.png/download?use_mirror=iweb: Unsupported image format

If I try to download https://sourceforge.net/projects/pinn/files/Etcher/pinn.png using wget (on my regular Linux box) it follows a whole heap of redirects ( 😢 ) and eventually downloads https://iweb.dl.sourceforge.net/project/pinn/Etcher/pinn.png
(I dunno if this helps, or is info you already knew? 🤷‍♂️ )

....

Ahhh, but I get different behaviour if I use wget --user-agent="Mozilla/5.0" https://sourceforge.net/projects/pinn/files/Etcher/pinn.png - it then downloads an HTML page instead of the PNG image 😕

@maxnet
Copy link
Collaborator

maxnet commented Mar 7, 2020

Even if we change the code to set user-agent now to trick sourceforge, that would mean that users that have downloaded the imager software already would not be able to use your repository if it were added to the main list.
So I don't think there's much point in doing so.

Just find another hoster for your json and icon. And only use them for the actual images.

Keep in mind that there is no update mechanism in place for the imager software.

@procount
Copy link
Author

procount commented Mar 8, 2020

I've resolved the hosting for the icon and json files by putting them on github, and the image loads ok from sourceforge now.
So here is my final json fragment for PINN (as I posted in #21)

{
        "name": "PINN",
        "description": "A multi-boot OS installer with OS admin features",
        "icon": "https://raw.githubusercontent.com/procount/pinn-os/master/iu/pinn.png",
        "subitems_url": "https://raw.githubusercontent.com/procount/pinn-os/master/iu/os_list_iu_pinn.json"
}

@maxnet maxnet added the wontfix This will not be worked on label Mar 9, 2020
@maxnet maxnet closed this as completed Mar 9, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wontfix This will not be worked on
Projects
None yet
Development

No branches or pull requests

4 participants