[Bug]: Get-EvergreenApp Filezilla is currently not returning any content

[AScott-WWF]

What happened?

Get-EvergreenApp Filezilla
Currently returns nothing
Not really sure that this is strictly a bug with Evergreen

I assumed the source that Evergreen checks was not responding or returning nothing, but from the -Verbose output I have discovered that when attempting to open in a web browser the Url listed against the Invoke-Webwrapper command, i.e.:
https://update.filezilla-project.org/update.php?initial=0&manual=1&osarch=64&osversion=10.0&package=1&platform=x86_64-w64-mingw32&updated=0&version=3.47.2.1.
All browsers report an HSTS error, So I suspect this to be an issue with the service hosting the url
N.B. This also occurs using the base URL: https://update.filezilla-project.org/

Firefox helpfully states:
"update.filezilla-project.org has a security policy called HTTP Strict Transport Security (HSTS), which means that Firefox can only connect to it securely. You can’t add an exception to visit this site.

The issue is most likely with the web site, and there is nothing you can do to resolve it."

Only fix I can suggest is some better error handling for this type of response (shrug)

Version

2310.868

What PowerShell edition/s are you running Evergreen on?

PowerShell Core, Windows PowerShell

Which operating system/s are you running Evergreen on?

Windows 10+

Have you reviewed the documentation?

• Troubleshooting at: https://stealthpuppy.com/evergreen/troubleshoot/
• Known issues at: https://stealthpuppy.com/evergreen/issues/

Verbose output

Get-EvergreenApp  -Name FileZilla -Verbose
VERBOSE: Function path: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2310.868\Apps\Get-FileZilla.ps1
VERBOSE: Function exists: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2310.868\Apps\Get-FileZilla.ps1.
VERBOSE: Dot sourcing: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2310.868\Apps\Get-FileZilla.ps1.
VERBOSE: Get-FunctionResource: read application resource strings from [C:\Program Files\WindowsPowerShell\Modules\Evergreen\2310.868\Manifests\FileZilla.json]
VERBOSE: Calling: Get-FileZilla.
VERBOSE: Invoke-WebRequestWrapper: Adding SkipCertificateCheck.
VERBOSE: Invoke-WebRequestWrapper: Using temp file C:\Users\AScott\AppData\Local\Temp\tmpmsgoqf.tmp.
VERBOSE: Invoke-WebRequestWrapper: Invoke-WebRequest parameter: UseBasicParsing: True.
VERBOSE: Invoke-WebRequestWrapper: Invoke-WebRequest parameter: Method: Default.
VERBOSE: Invoke-WebRequestWrapper: Invoke-WebRequest parameter: UserAgent: FileZilla/3.47.2.1.
VERBOSE: Invoke-WebRequestWrapper: Invoke-WebRequest parameter: Uri: https://update.filezilla-project.org/update.php?initial=0&manual=1&osarch=64&osversion=10.0&package=1&platform=x86_64-w64-mingw32&updated=0&version=3.47.2.1.
VERBOSE: Invoke-WebRequestWrapper: Invoke-WebRequest parameter: PassThru: True.
VERBOSE: Invoke-WebRequestWrapper: Invoke-WebRequest parameter: SkipCertificateCheck: True.
VERBOSE: Invoke-WebRequestWrapper: Invoke-WebRequest parameter: OutFile: C:\Users\AScott\AppData\Local\Temp\tmpmsgoqf.tmp.
VERBOSE: Requested HTTP/1.1 GET with 0-byte payload
VERBOSE: Received HTTP/1.1 response of content type text/html of unknown size
VERBOSE: File Name: tmpmsgoqf.tmp
VERBOSE: Invoke-WebRequestWrapper: Response: 200.
VERBOSE: Invoke-WebRequestWrapper: Content type: text/html; charset=UTF-8.
VERBOSE: Invoke-WebRequestWrapper: Returning content of length: 203.

[aaronparker]

I can't get https://update.filezilla-project.org/update.php to return anything other than a redirect to the FileZilla home page. Not sure why at this stage.

[jms-du-se]

Not sure if this helps, but I just noticed that the FileZilla response is inconsistent, sometimes it returns 3.64.0 and sometimes 3.66.1 (which is the latest official release:

3.64.0

VERBOSE: Function path: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2312.878\Apps\Get-filezilla.ps1
VERBOSE: Function exists: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2312.878\Apps\Get-filezilla.ps1.
VERBOSE: Dot sourcing: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2312.878\Apps\Get-filezilla.ps1.
VERBOSE: Get-FunctionResource: read application resource strings from [C:\Program Files\WindowsPowerShell\Modules\Evergreen\2312.878\Manifests\filezilla.json]
VERBOSE: Calling: Get-filezilla.
VERBOSE: Invoke-EvergreenWebRequest: Creating class TrustAllCertsPolicy.
VERBOSE: Invoke-EvergreenWebRequest: Using temp file C:\Users\jms\AppData\Local\Temp\tmpB211.tmp.
VERBOSE: Invoke-EvergreenWebRequest: Invoke-WebRequest parameter: UserAgent: FileZilla/3.66.1.
VERBOSE: Invoke-EvergreenWebRequest: Invoke-WebRequest parameter: UseBasicParsing: True.
VERBOSE: Invoke-EvergreenWebRequest: Invoke-WebRequest parameter: Method: Default.
VERBOSE: Invoke-EvergreenWebRequest: Invoke-WebRequest parameter: Uri: https://update.filezilla-project.org/update.php?osarch=64&platform=x86_64-w64-mingw32&version=3.56.1.
VERBOSE: Invoke-EvergreenWebRequest: Invoke-WebRequest parameter: PassThru: True.
VERBOSE: Invoke-EvergreenWebRequest: Invoke-WebRequest parameter: OutFile: C:\Users\jms\AppData\Local\Temp\tmpB211.tmp.
VERBOSE: GET with 0-byte payload
VERBOSE: received -1-byte response of content type text/plain;charset=UTF-8
VERBOSE: Invoke-EvergreenWebRequest: Response: 200.
VERBOSE: Invoke-EvergreenWebRequest: Content type: text/plain;charset=UTF-8.
VERBOSE: Invoke-EvergreenWebRequest: Returning content of length: 115.

Version Size     Hash                                                                                                                             URI
------- ----     ----                                                                                                                             ---
3.64.0  12140960 30762221e8b41c63383c06e639cab6730603c5d76bbe14cfd0c499bd3ad690bc82d220b4cff215f2c5b74b81087cdcbd6255c5aa598c350bb111a880f6d9a8dd https://download.filezilla-project.org/client/FileZilla_3.64.0_win64-setup.exe

3.66.1

VERBOSE: Function path: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2312.878\Apps\Get-filezilla.ps1
VERBOSE: Function exists: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2312.878\Apps\Get-filezilla.ps1.
VERBOSE: Dot sourcing: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2312.878\Apps\Get-filezilla.ps1.
VERBOSE: Get-FunctionResource: read application resource strings from [C:\Program Files\WindowsPowerShell\Modules\Evergreen\2312.878\Manifests\filezilla.json]
VERBOSE: Calling: Get-filezilla.
VERBOSE: Invoke-EvergreenWebRequest: Creating class TrustAllCertsPolicy.
VERBOSE: Invoke-EvergreenWebRequest: Using temp file C:\Users\jms\AppData\Local\Temp\tmpA8D9.tmp.
VERBOSE: Invoke-EvergreenWebRequest: Invoke-WebRequest parameter: UserAgent: FileZilla/3.66.1.
VERBOSE: Invoke-EvergreenWebRequest: Invoke-WebRequest parameter: UseBasicParsing: True.
VERBOSE: Invoke-EvergreenWebRequest: Invoke-WebRequest parameter: Method: Default.
VERBOSE: Invoke-EvergreenWebRequest: Invoke-WebRequest parameter: Uri: https://update.filezilla-project.org/update.php?osarch=64&platform=x86_64-w64-mingw32&version=3.56.1.
VERBOSE: Invoke-EvergreenWebRequest: Invoke-WebRequest parameter: PassThru: True.
VERBOSE: Invoke-EvergreenWebRequest: Invoke-WebRequest parameter: OutFile: C:\Users\jms\AppData\Local\Temp\tmpA8D9.tmp.
VERBOSE: GET with 0-byte payload
VERBOSE: received -1-byte response of content type text/plain;charset=UTF-8
VERBOSE: Invoke-EvergreenWebRequest: Response: 200.
VERBOSE: Invoke-EvergreenWebRequest: Content type: text/plain;charset=UTF-8.
VERBOSE: Invoke-EvergreenWebRequest: Returning content of length: 150.

Version Size     Hash                                                                                                                             URI
------- ----     ----                                                                                                                             ---
3.66.1  12383864 36c1a15f525800a2b0c03325802b603088d0cd6c137408ed584b8091caaced41b140d16e2de92d0b0782c363ca487379ffa44b80c20e2a947cdf8f5f2c5981f3 https://download.filezilla-project.org/client/FileZilla_3.66.1_win64-setup.exe

[AScott-WWF]

I've done a litle more investigation on this.
It appears that the URL that Evergreen uses to check for updates https://update.filezilla-project.org/update.php?osarch=64&platform=x86_64-w64-mingw32&version=3.56.1
Is served from a site secured with a self signed (untrusted) SSL certificate.
Most browsers bring up a security warning as the SSL cert is untrusted, I'd hazard a guess that this is the same issue that Powershell is encountering as this webrequest is made through the Edge engine.
As I see it there are a couple of possible solutions to this:
Investigate possible fix to Evergreen:

• Check if a later version of FileZilla tries to make a call to a new / different url when checking for updates using Wireshark to do a packet capture (and then if it has changed update the URL Evergreen uses).
  Mark as Vendor issue:
• Requires Filezilla to use a trusted SSL cert on the update url https://update.filezilla-project.org
  N.B. I think the Filezilla Project relies on advertising revenue to pay for their software / software hosting, so it might be difficult to 'persuade' them to allow their software to be downloaded using automation therefore bypassing the advertisments.

In the mean time I have resorted to web scraping to get the version:

$DownloadsUrl = "https://filezilla-project.org/download.php?show_all=1"
    try {
        #Define Headers to ensure FileZilla webpage can be read
        $Headers = @{
            "User-Agent" = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:123.0) Gecko/20100101 Firefox/123.0"
            "Referer" = "https://download.filezilla-project.org/client/"
            "Upgrade-Insecure-Requests" = "1"
        }
        $Content = (Invoke-WebRequest -Uri $DownloadsUrl -Headers $Headers).Content
        $SearchString = "The latest stable version of FileZilla Client is "
        $CaptureStart = $Content.IndexOf($SearchString) + $SearchString.Length
        $EndIndex = $Content.IndexOf("</p>", $CaptureStart) - $CaptureStart
        [version]$LatestVersion = ($Content.Substring($CaptureStart, $EndIndex).Trim())

        $SourceUrl = "https://download.filezilla-project.org/client/FileZilla_$($LatestVersion)_win64-setup.exe"
    }
    catch {
        Throw "Failed to connect to $($Application) Download ($($DownloadsUrl)) with error $_."
        Break
    }
    finally {
        # Success
    }
    

[aaronparker]

Get-FileZilla is configured to skip the certificate check:

evergreen/Evergreen/Apps/Get-FileZilla.ps1

Lines 21 to 22 in 46ee734

	$params = @{
	Uri = $res.Get.Update.Uri

$params = @{
    Uri                 = $res.Get.Update.Uri
    UserAgent           = $res.Get.Update.UserAgent
    SkipCertificateCheck = $true
    Raw                 = $true
}
$Content = Invoke-EvergreenWebRequest @params

[durrante]

Hi guys,

Just wondering if there was an update to this? FileZilla doesn't return any results for me either.

[aaronparker]

No updates. Not likely we're going to be able to find a fix unfortunately.

[durrante]

Understood, thanks for the update 👍

[DanGough]

What's odd is Get-EvergreenApp FileZilla is currently working for me in PS5.1 but not PS7.

Yet in PS7 I can query the URL with Invoke-WebRequest 'https://update.filezilla-project.org/update.php' -SkipCertificateCheck -UserAgent 'FileZilla/3.56.1', but I can't do that in PS5.1 as it doesn't support -SkipCertificateCheck.

Alternative is to just use Invoke-EvergreenApp FileZilla to grab it from your API!

[AScott-WWF]

What's odd is Get-EvergreenApp FileZilla is currently working for me in PS5.1 but not PS7.

I've just tried this, but I get no results for either - I'm suspecting the CDN that Filezilla uses to serve content has differing settings, hence the inconsistencies

Alternative is to just use Invoke-EvergreenApp FileZilla to grab it from your API!

That's a good call - however, it's returning

Version Size     Hash
------- ----     ----
3.66.4  12377344 860354025f8bd60388490c3429df1c2a090ff0bee7ee8afe41ec2bd07891428634e12d36777893cd257ab59df8e2ebf316802…

3.66.4 is not the latest - 3.66.5 is: https://filezilla-project.org/download.php?show_all=1

[AScott-WWF]

TBH I think the time has come that FileZilla might need to be retired / withdrawn from Evergreen?

They have changed their website further and it is seemingly no longer possible to even webscrape the downloads page (https://filezilla-project.org/download.php?show_all=1), (unless @DanGough can work his magic with NeverGreen?)
I believe this is because FileZilla rely on advertising revenue to fund their existence - which this type of automation bypasses

So I have resorted to having this in my download script:

$DownloadsUrl = "https://filezilla-project.org/download.php?show_all=1"
Start-Process $DownloadsUrl

so I can manually check for a new version in a browser each day 👎

and I can then (at least for now) then go to https://download.filezilla-project.org/client/ to download the latest FileZilla_<version>_win64-setup.exe file

[DanGough]

You just need to add the request header accept = */* to talk to FileZilla servers. At least that worked a few weeks ago for downloading the files, I added that as a default in PsDownload.

[aaronparker]

I've tested a few other things today including the Accept header, different parameters on the update URL all with not luck.

To be honest, I'd recommend people use an application that not as anti-customer as FileZilla, e.g. WinSCP.

[AScott-WWF]

FYI:
@DanGough I have just raised a New App request in NeverGreen for this on your Issues page DanGough/Nevergreen#64 which includes my working script 😉