Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Download speed from Insider storage is limited for Download-Artifacts commandlet #3320

Open
RadoArvay opened this issue Jan 31, 2024 · 6 comments
Assignees

Comments

@RadoArvay
Copy link

Description:
When we download an artifact from Insider storage (Get-BCArtifactUrl -storageAccount bcinsider), a download speed is limited in compare to other storages.
But, it happens under specific conditions only. It means it happens on our company's internet connection, on most of computers (unfortunately also on Azure agent's server for pipelines, but, surprisingly, we have found out exceptional one without an issue when we tested it). And, for example, it does not happen on my personal connection at my home.
IT department examined a firewall, packets, setups and did not find anything.

Based on BcContainerHelper function ReplaceCDN (and info from Release Note: Avoid using CDN when downloading from bcinsider as this frequently changes and subsequently fails), I have tried a direct download of the same artifact file from https://bcinsider.azureedge.net/ and https://bcinsider.blob.core.windows.net/. It seems, that Blob storage is limited for us in compare to CDN and the speed from Blob reaches around 30% or less of CDN speed.

Any solution or idea? Does anyone have the similar issue?

Request:
Would it be possible to change a behaviour in Download-File commandlet...

if ($bcContainerHelperConfig.DoNotUseCdnForArtifacts -or $sourceUrl -like 'https://bcinsider.azureedge.net/*') {
    # Do not use CDN when configured or bcinsider
    $sourceUrl = ReplaceCDN -sourceUrl $sourceUrl
}

...to force to use CDN source (maybe a new condition code part before that one above) and in a case of error to continue to Blob? Or something like that?

@freddydk
Copy link
Contributor

freddydk commented Feb 1, 2024

Content Delivery Network (CDN) works poorly with stuff that changes frequently, giving a lot of problems for people.
Performance however is the one problem it would have solved, which was the reason for adding this in the first place.
The problem isn't only failures, it is also that CDN's might not be fully updated until days after the blob update - some blobs are updated (dk, us maybe) and then the platform is missing or like.

@freddydk
Copy link
Contributor

freddydk commented Feb 1, 2024

We are working on testing different mechanisms for artifacts which should solve both issues.

@RadoArvay
Copy link
Author

Ok, let´s wait for it. (Did you mean Nuget?) I think we could close this for now and I will raise it up again if it is needed.

@freddydk
Copy link
Contributor

freddydk commented Feb 1, 2024

For this - NuGet might not be the right choice - NuGet works poorly with packages 200Mb and above...

@RadoArvay
Copy link
Author

Ah, ok, I've seen a discussion on Yammer where it was mentioned, so I thought....

@freddydk
Copy link
Contributor

freddydk commented Feb 1, 2024

NuGet packages will be used for app sharing - but not for full artifacts download.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants