Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement downloads from json file #2

Draft
wants to merge 41 commits into
base: dev
Choose a base branch
from

Conversation

Thomas1664
Copy link

@Thomas1664 Thomas1664 commented Apr 2, 2024

Fixes open-vcpkg/meta#6

See the linked issue for more information.

Benefits

  • Simplified SBOM generation
  • Better experience when installing ports using --only-downloads
  • More localization
  • Better performance during download

Changes in behavior

Applying patches

All patches are now applied within a single git command. Because the output from stdout is always empty, all logs are saved into a single file.


r.optional_object_field(obj, "patches", result.patches, Json::IdentifierArrayDeserializer::instance);

result.urls.emplace_back(fmt::format("https://api.{}/repos/{}/tarball/{}", gh_gost, repo, ref));
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I decided to build the URL here in order to unify and simplify DownloadedFile for all downloading methods.

Optional<type> visit_object(Json::Reader& r, const Json::Object& obj) const override
{
DownloadedFile result;
if (auto value = obj.get("github"))
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using this instead of the id as keys simplifies parsing a lot.

Thomas1664 and others added 20 commits April 2, 2024 15:26
Hopefully fixes: microsoft/vcpkg#37842

Special thanks to @TrueBrain for figuring out the actual problem in microsoft#1378

Co-authored-by: Patric Stout <truebrain@openttd.org>
* should be forward slashes

* add script to check for unused messages

* add messages check to pr workflow

* error if unused messages

* dont print every message every time

* remove bogus comment

* remove bogus comment

* remove bogus comment (again)

* Remove timer; add word boundry to regex;merge format and very messages job in pr.yaml

* Use JSON files instead of REGEX to declare messages

* pass artifact messages as argument

* filter out empty from unused messages

* filter out empty from unused messages

* infer path from PSScriptRoot; reuse configure in format job

* remove unused messages

* add new line at the end of ps script

* remove unused messages

* try something

* add messages.json again

---------

Co-authored-by: Javier Matos <javiermatos@Javiers-Laptop.local>
Co-authored-by: Javier Matos <javiermatos@Javiers-Laptop.lan>
Problems were:
- the only test in e2e_ports folder, everything else in e2e-ports
- tests modify files under $VCPKG_ROOT
- Divided in two files where one enough
- Tests fail if package is already in binary cache
…ft#1367)

In microsoft/vcpkg#37563 (comment) we have an example of versions which meet the semantic versioning format but don't sort like semantic versions, so the message:

```console
PS D:\vcpkg> .\vcpkg.exe x-add-version sqlite-modern-cpp
Use the version scheme "version" rather than "version-string" in port "sqlite-modern-cpp".
Use --skip-version-format-check to disable this check.
```

is too strong. We need to explain to the user that they should only do that if the versions actually are intended to sort that way.
csigs and others added 12 commits April 29, 2024 17:13
* Eliminate PurgeDecompressFailure.

PurgeDecompressFailure causes the files backend for binary caching to delete the zip file if it fails to decompress. The install command set this to false due to a race consideration: it's possible that a partially uploaded zip is observed when the files backend is pointed to a network share, and deleting the file because we failed to decompress it just stomps on another builder sharing that same binary cache.

It seems incorrect that this reasoning would not apply to any other commands, but most of the other commands said PurgeDecompressFailure::Yes.

This change removes the option entirely in favor of the 'No' behavior.

* Do not redundantly set Editable or Use Head to false in manifest mode.

These are already implied by the blockers on line 1075 and 1080.

* While reviewing changes to microsoft#1339 , I observed that it was strange that the abi hash depends on whether download_only is selected. After discussion with @ras0219-msft , he agreed that the ABI hash shouldn't be affected by this setting.

However, given that in download mode, we intentionally build things which do not have all their dependencies present, it would not be safe to attempt to cache or otherwise install anything in that condition.

This change removes download_only from the ABI calculation, and changes installation to only attempt to install built bits whose dependencies are all satisfied.
…crosoft#1397)

* Add support for VS 2022 using VS170COMNTOOLS environment variable

* keep existing pattern

Co-authored-by: Billy O'Neal <bion@microsoft.com>

---------

Co-authored-by: Javier Matos <javiermatos@Javiers-Laptop.local>
Co-authored-by: Billy O'Neal <bion@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
9 participants