Skip to content

2024 Developer Meetings

Jump to bottom
Kate edited this page May 14, 2024 · 49 revisions

Next meeting: 20/05/2024 14:00 CET

2024-05-13

Present: Alpha (@moyodiallo), David (@dra27), Kate (@kit-ty-kate), Raja (@rjbou)

opam 2.2 (kanban board)

  • We briefly discussed the upcoming blog post for Windows ocaml/platform-blog#99
  • #5741 (opam update takes 10mins on Windows) Kate restarted from a new fresh branch to focus on the required API changes first hoping it would make the whole change easier. Per last week's meeting, the issue was moved to the "To keep on hand" column but is still being worked on.
  • #5817 / #5908 (opam install --deps-only performance regression) Raja will review it right after the meeting.
  • #5921 (downloading URLs with invalid filename) We discussed the remaining questions. Raja approved it and we merged it.
  • #5823 (last-env files should be stored outside the switch) Alpha is on it this week.
  • #5835 (improve Git-for-Windows opam init menu) David is on it
  • #5923 (badly quoted PKG_CONFIG_PATH) David is on it
  • #5930 (opam init --cygwin-extra-packages) Raja tested and did a review. Kate would like to test it before merging.
  • #5830 (small tweaks to internal Cygwin setup) This PR was only blocked by the first commit that had to be moved to do for later. David rebased it and marked it ready to review.
  • #5935 / #5925 / #5926 (fix reversal of environment updates) Raja reviewed it and made a branch with fixup commits. David is rebasing it.
  • #5927 (bypass lint warning 41 when under pkg:installed) David is on it

new issues

  • #5945 / #5946 (README: instruction on how to pin opam is not right) We noted that the instruction in the README aren't really clear on what they are trying to achieve (local build vs. install in general)
  • #5947 (opam install --formula don't set root packages) We discussed the purpose of the roots field for packages that are already listed in the switch invariant. Details can be found here.
  • #5948 (lint check for " = " in filters) The idea was generally accepted. Raja noted that we should make sure the strings are indeed constant and do not contain any variable interpolation.
  • #5949 / #5950 (change the value of arch on 32bit kernel in a 64bit machine) The PR fixes the issue as noted but we aren't quite sure this is the behaviour that we want yet and it should not be a regression so it isn't planned to be merged before 2.2.0.
  • #5951 (lint check use of package variables where disallowed) Raja noted that there is a warning that does that already (W29). However this warning is not checking depexts which was the problem encountered and the documentation can still be improved.

2024-05-07

Present: David (@dra27), Kate (@kit-ty-kate), Raja (@rjbou)

The meeting ran longer than usual once again (1h30) and was more free-form than the usual per-issue discussions.

opam 2.2 (kanban board)

  • #5741 (opam update takes 10mins on Windows) Kate did resume the work yesterday and the code is hard deal with with due to the way the code for the VCSs and HTTP repositories are abstracted out. Given how the branch is growing in size and with the relatively small amount of time remaining before the wanted release date of beta3, we're considering moving this item to 2.3 (which thanks to a change in the release schedule of opam should come pretty soon after 2.2). 2.2.1 is not conceivable given the degree of change in the API and the change to the way opam works internally. Kate will continue working on it but it is no longer a hard requirement for 2.2.
  • Users are asking on different platforms about the promised blog post about testing opam 2.2.0~beta2 on Windows. We discussed about reviews on David's pre-draft PR on opam-repository and the blog post will be written by David as soon as the draft PR is updated and opened in the upstream opam-repository.
  • #5835 (improve Git-for-Windows opam init menu) To fix this issue, David would like to change how opam init works a bit more than expected. We discussed the use of the sys-pkg-manager-cmd option, which we want to keep. The discussed changes were received positively.
  • It was asked who would do the CI for Windows for opam-repository and Kate said she would do it during the rc phase.
  • We discussed different preferred target date for the releases. We landed on a tentative 27th of May for beta3 and 10th of June for rc1.

opam 2.1.6

We agreed on starting the release process for opam 2.1.6 (see #5870). Kate will finish the PR up and Raja will review.

2024-05-02

Present: Alpha (@moyodiallo), David (@dra27), Kate (@kit-ty-kate), Raja (@rjbou), Ryan (@RyanGibb)

The meeting ran longer than usual (2 hours). Ryan and Kate met at the mirageOS retreat last week.

opam 2.2 (kanban board)

  • #5741 (opam update takes 10mins on Windows) Kate will be able to resume the work on this on Monday
  • #5755 / #5919 (weird behaviours of opam tree --test --doc) Kate opened a PR to fix this (#5919) a few weeks ago. Kate needs to add a test. Raja will review it.
  • #5800 / #5906 (PowerShell script installer) was moved to finish before the RC as another method of installation (winget) is now available.
  • #5816 / #5904 (jobs variable reset does not work when upgrading from 2.0 via 2.1) Kate noted that her fix isn't quite correct in all cases (see comment), she will update the PR to update the root version as well. Raja will do a review.
  • #5817 / #5908 (opam install --deps-only performance regression) Kate still needs to explain the fix in the PR description, and Raja will review the required PR to merge before (#5909)
  • #5830 (small tweaks to internal Cygwin setup) David is still on it
  • #5838 (env revert issues on Windows) David is still on it
  • #5823 (last-env files should be stored outside the switch) Alpha offered to do it. From the two options offered, the one that uses OPAMROOT instead of /tmp was preferred.
  • #5835 (improve Git-for-Windows opam init menu) David is still on it
  • #5920 (opam tree --{test,doc} --no-switch doesn't work) is a TODO extracted from #5755. Kate had a quick look but couldn't see the problem. Raja will have a look at it.
  • #5921 (downloading URLs with invalid filename) Kate wondered if it could be related to #5939 and if the filename couldn't be static instead. David argued that it all depends on every usage of the function and that it would complicate things a little. Kate approved it on the basis of the current PR being an improvement. Raja will review it.
  • #5923 (badly quoted PKG_CONFIG_PATH) David understands the issue and will work on it.
  • #5930 (opam init --cygwin-extra-packages) Has been reviewed and approved by Kate and David. Raja wants to have a look before merging.
  • #5935 / #5925 / #5926 (fix reversal of environment updates) David is working on it
  • #5941 / #5943 (switch import from 2.1 doesn't work) Is a 2.2 regression and the PR should be ready to go. Raja will review it.
  • #5927 (bypass lint warning 41 when under pkg:installed) This is required to make the compiler PR pass the linter and was added to the beta3 column.

The idea to propose a talk about opam 2.2 at the OCaml 2024 workshop was raised, but no strong commitment yet.

new issues

  • #5914 (self-extracting extra-source) The idea was broadly positively accepted. David wondered if we could also have a similar option to extract other package sources in a directory. Kate wondered if we could also have an option for git submodules.
  • #5918 (--dry-run modifies the file system) Raja had a look at this and subcommands other than opam switch also exhibit this behaviour. She'll have a look at this.
  • #5929 (opam show --field without quotes) Raja noted that the behaviour already exists but is not consistent (see comment). We'd like to fix this at some point.
  • #5931 (missing --update-invariant hint) looks like a UI problem. We'll have a look at it when we have the time.
  • #5932 (add checksum to opam files) Similar to opam admin add-constraint we could add an admin subcommand to add a checksum to a package in a repository.
  • #5933 (opam switch import doesn't set invariant) Is indeed a bug in 2.2 still. Raja will look at it.
  • #5934 (opam upgrade --fixup leaves packages to be upgraded) We wondered if the behaviour still exists in 2.1.5 or 2.2 given the solver was upgraded since 2.1.2.
  • #5936 (opam install doesn't do necessary upgrade of ocaml-system) is an issue that happened during the mirageOS retreat. We believe the code for this in opam upgrade was added in 2.1 but we can't remember which PR did it. We'll have a look at it to see if it can be extended to opam install.
  • #5939 (pin-depends removes part of the content) is an issue encountered by Leandro at the mirageOS retreat. It turned out to be fixed by a 2 years old PR that we haven't merged because it wasn't critical at the time and added a bit of complexity to the code.
  • #5940 (remove use of coreutils on macOS in CI) is just a TODO extracted from discussion in #5938.

new PRs

  • #5922 (package installed state written upon failure) for now is just a PR adding a test showing the faulty behaviour. David will have a look if it's worth doing in 2.2. It was added in the "keep on hand" column.
  • #5928 (relax lint error 31 for non-tautological dependencies) there is a work around in the compiler PR so it is not strictly necessary for 2.2 but David would like to have this fixed at some point.
  • #5942 (NixOS depexts support) Ryan presented his PR and explained the challanges of Nix compared to other mutable system package managers. David mentioned a complementary solution based on opam packages which would require a change in the sandbox to be viable. The PR was tagged as WIP and we'll rediscuss it when the solution has been a little more ironed-out.

2024-04-08

Present: Alpha (@moyodiallo), David (@dra27), Kate (@kit-ty-kate), Raja (@rjbou), Ryan (@RyanGibb)

opam 2.2 (kanban board)

All the remaining issues and PRs for 2.2.0~beta2 have been fixed/merged so we are ready to release it. Kate offered to handle the release.

We triaged the issues that were tagged to do after beta2 to a new beta3 column on the board:

  • #5741 (opam update takes 10mins on Windows) is important for the final release as we do not want Windows users to get annoyed waiting 10mins on each opam update or init, and the amount of work to fix this is fairly substantial so it should be done sooner rather than later to catch any potential bugs introduced by the upcoming fix. This will be done by Kate.
  • #5755 (weird behaviours of opam tree --test --doc) is a bug in a feature introduced in 2.2 and we'd like to fix it before the release. Kate will also be able to mentor Ambre in the upcoming two weeks so we are pretty confident the issue will have a fix fairly soon.
  • #5800 / #5906 (PowerShell script installer) we'd like to provide a way for Windows users an easy way to install the latest beta before the final release of 2.2 and iron out any kinks of it. David will have a look at improving the initial draft script. The choice towards a PowerShell script instead of a windows installer was made during the February 5th dev meeting
  • #5816 / #5904 (jobs variable reset does not work when upgrading from 2.0 via 2.1) Raja will review the PR.
  • #5817 / #5908 (opam install --deps-only performance regression) Kate explained the fix but the code is tricky so she will update the PR description to explain what the PR does.
  • #5830 (small tweaks to internal Cygwin setup) David is planning to finish the PR so it was added to the beta3 column
  • #5838 (env revert issues on Windows) David plans to work on it
  • #5823 (last-env files should be stored outside the switch) David plans to work on it
  • #5835 (improve Git-for-Windows opam init menu) should be easy to do and would remove an annoyance for Windows users.

The rest of the issues in the 2.2 project were deemed non-essential for beta3 and were left in the For RC column for now.

new issues

  • #5910 (rounding issue on the debug timings) The issue is clearly a bug. David noticed it's a rounding error as Printf.printf "%f" does round the number up but in this particular case we want to round down. The fix will be available soon and should be a one-line fix.
  • #5911 (capricious solver with avoid-version) We discussed different solutions and potential causes to this issue. One possible solution could be to add extra weight for the solver to the compiler package so as to avoid downgrades. Another solution could be to detect for each packages preprocessed for the solver, if it is flagged as avoid-version, remove the avoid-version handling if a dependency (e.g. the compiler) also flagged with avoid-version is already installed.

new PRs

  • #5907 (falsely triggered recompilation of unchanged packages) The API change done by the PR isn't satisfactory. After some discussion an alternative could be to not change effective_part but instead make name and version equal if both aren't set already, in effectively_equal. Kate will do the change.
  • #5909 (benchmark opam install --deps-only) is to be merged before #5908 (opam install --deps-only performance regression) to check the PR fixes the issue. This was added to the beta3 column.

2024-04-02

Present: David (@dra27), Kate (@kit-ty-kate), Raja (@rjbou)

opam 2.2 (kanban board)

  • Cygwin 3.5.2 (release of cygwin fixing the regressions making it impossible to compile OCaml) 3.5.2 wasn't released last month. We might send another email to the cygwin mailing-list as locking the version of cygwin for the internal cygwin doesn't seem to work. Kate noticed that we're not really saying the internal version of Cygwin is highly recommended so she opened #5903 to make that clearer.
  • #5832 (find a better place for the Cygwin directory in PATH) is waiting for a review from Raja
  • #5840 ('+' in package names clashes with opam syntax) is currently being rebased by Raja and will be merged after.
  • #5061 (git source --dev changed to use --depth=1) Raja did a review and the PR needs some changes. We wondered if ?full_fetch should be passed explicitly or omitted. No consensus on that was found.

Kate spent most of last week working on #5892 (use ocaml-patch instead of GNU patch). However it is too drastic of a change to be integrated into 2.1.6, so instead we could use #5893 (warn if GNU patch isn't found) as a stop gap for 2.2.0~beta2 and 2.1.6, then think about #5892 again for 2.2 because #5741 (opam update takes 10mins on Windows) (slotted for 2.2) requires an OCaml diff implementation anyway.

new PRs

  • #5900 (benchmark OpamSystem.read) this change is required to show the result of #5896 (speedup OpamSystem.read). This was reviewed and merged right away.
  • #5899 (disable Software Heritage fallback) was discussed in the past and was planned to do before the 2.2 release. The change is a one-liner so it was reviewed and merged right away.
  • #5896 (speedup OpamSystem.read) is required for #5892 (use ocaml-patch instead of GNU patch) so the new diff implementation doesn't loose too much performance compared to GNU diff. We discussed different options as the code in its current form isn't thread-safe. Further performance analysis showed a 10% speedup using the thread-unsafe code, 8% when using a fixed size buffer and no intermediate bytes buffer and no improvement using in_channel_length.

new issues

  • #5898 (no newline at the end of lock files) is an issue also observed in opam-publish. Kate believes it changed with the introduction of opam-file-format's FullPos submodule in #4298 but isn't sure. Raja thought that it could be #4302 that introduced it, as lock uses with preserved format write function. A check need to be done.
  • #5901 (mention log/-vv/--debug on failing commands) is a common ask from us when someone submit a bug-report so mentioning these debug options seem like a good idea, however it should be concise enough to avoid bloating the output.
  • #5902 (a potential issue with git caching to debug) is a split from #5061 to be tracked independently.

misc

Github introduced some mac arm64 machines that we might want to use on the CI matrix. It might also be useful to automate the release process as the current process requires a mac arm64 host. We wondered if those machine can be trusted. We'll think about it again at some point in the future.

2024-03-25

Present: Kate (@kit-ty-kate), Raja (@rjbou), David (@dra27), Alpha (@moyodiallo), Leandro (@leostera)

opam 2.2 (kanban board)

  • #5829 (improve and add missing sys-ocaml-* variables on Windows) Raja applied some changes (test, formatting), David will take a look and it should be good to go.
  • #5832 (find a better place for the Cygwin directory in PATH) David is hoping to have the PR updated in the next couple of days
  • #5840 ('+' in package names clashes with opam syntax) Raja reviewed it, there are a couple of changes to be done. She volunteered to do them.
  • Cygwin 3.5.2 (release of cygwin fixing the regressions making it impossible to compile OCaml) there has been one commit this week. Kate noted that the main Cygwin maintainer seems to release every Tuesday so she's hoping it will be released tomorrow.
  • #5061 (git source --dev changed to use --depth=1) Alpha almost finished, there is just one more test to add and change their order so that the changes can be better seen in the git history, then Raja will review it and it should be good to go.

If all goes well we're on track to release 2.2.0~beta2 this week.

Kate is planning to work on a fix for #5741/#5648 (use ocaml-tar to speedup opam update on Windows) and she's hopping to be able to open a draft PR by next week.

new PRs

  • 3 possible fixes for the issue described in #5891 (revert use of git apply to apply patches):
    • #5893 (detect if the patch command is GNU patch) could be reasonable to have in 2.1.6 but there are concerns on whether or not this is future proof (e.g. different distros could change the name given by patch --version). We also wondered if we should check for the version number to fix #3782 but it doesn't seem worth it as the issue itself is non-critical and it would complexify an already brittle check.
    • #5894 (use git diff instead of the system diff command) uses the standardized output from git diff to feed it to non-GNU patch command. The downside of this is that opam would now require git. However we could fallback on diff if git is not available.
    • #5892 (use ocaml-patch) this requires a release from ocaml-patch but Kate got the necessary rights on the repository from Hannes to do it if needed. While being the most invasive change, this also brings an implementation of diff which is needed to fix #5741/#5648 to diff between two tarballs and between one tarball and one directory. This change is restricted to opam update where the input and output are controlled so the change is actually not as invasive as it first seems. This PR received in the end a moderately more positive approval than the others for now. Kate will push this PR further this week.

Since we want some kind of fix for this for 2.1.6, we would like to release it in 2.2.0~beta2 so that it gets as much time for testing as possible. Kate will work on finishing up #5892 (use ocaml-patch) this week and release a new version of ocaml-patch.

2024-03-19

Present: Kate (@kit-ty-kate), Raja (@rjbou), David (@dra27), Alpha (@moyodiallo)

opam 2.2 (kanban board)

  • #5829 (improve and add missing sys-ocaml-* variables on Windows) Raja did a review and will do the modifications during the week.
  • #5832 (find a better place for the Cygwin directory in PATH) David is hoping to finish it soon.
  • #5840 ('+' in package names clashes with opam syntax) Raja will do a review this week.
  • Cygwin 3.5.2 (release of cygwin fixing the regressions making it impossible to compile OCaml) there was a commit in the 3.5 branch this week and we're all hoping for a release sometime next week. If that doesn't happen we can always specifically require an older version of Cygwin when installing cygwin internally and do an announcement for users who will use an external cygwin.
  • #5061 (git source --dev changed to use --depth=1) Alpha is almost done and asked a few questions regarding the new expected behaviour of opam source and the internals of the opam-repository library.

new issues

  • #5881 (record git commit of pin-depends in lockfiles) it looks like a genuine bug and this could be the new behaviour without even having to add an option.
  • #5884 (gradually cleanup build directory) Kate will implement this at some point in the future for opam 2.3. We discussed what should <pkg>:build become and we're not sure what its original intent was given its current behaviour. We could implement a check to see if this variable is used and not delete the directory if that's the case, but users could still use opam var <pkg>:build somewhere in the build. Removing each directories sequentially after the install could be a solution to not have to deal with this variable.
  • #5886 (check patches during opam lint --check-upstream) the feature request sounds good on its own, however after some discussion we discovered #5400 caused a lot more issues than expected. Kate will look into it urgently as it is central for both 2.2.0~beta2, 2.1.6 and the future changes to opam-repository.

new PRs

  • #5879 (support avoid-version with 0install) needs someone to have a look at the linked PR in opam-0install again. This is not urgent and can be done later.
  • #5883 (UI improvement for Windows) At least part of this PR would be nice to have for opam 2.2, as extraction of tarballs is slower on Windows and showing what is happening would make so that users do not wonder if opam crashed every 5 seconds. This was added to list of things to do before the 2.2.0~rc1.

Misc

Raja wondered if we should enforce a code indentation to avoid unrelated diffs in external contributors' PRs and make the indentation more coherent for ourselves. She opened #5889 to track that. She also wondered if the OpamClient module should be split at some point since that module is particularly big and contains different sections that could be separated. She opened #5890 to track that.

2024-03-11

Present: Kate (@kit-ty-kate), Raja (@rjbou), David (@dra27), Alpha (@moyodiallo)

opam 2.2

  • #5829 (improve and add missing sys-ocaml-* variables on Windows) is pretty much ready. David and Raja want to move the definitions to OpamEnv before merging. Kate will do it.
  • #5832 (find a better place for the Cygwin directory in PATH) David is actively working on it.
  • #5840 ('+' in package names clashes with opam syntax) is good to go for David. Raja would like to review it before merge.
  • #5861 (unicode characters makes dune&opam fail) the release of dune 3.14.1 is on its way and should happen some time this week.
  • Cygwin 3.5.2 (release of cygwin fixing the regressions making it impossible to compile OCaml) Kate sent an email to the Cygwin maintainers to ask when they think the next release of Cygwin would be, but realized too late it should have been sent instead to the mailing-list. She'll send an email to the mailing-list promptly.
  • #5061 (git source --dev changed to use --depth=1) Alpha had a chat with Kate last week asking for tasks to do that could help learn about the opam codebase. Kate looked around in tasks to do before the RC and #5061 seemed like an easy enough task which also happens to be reasonably deep inside the codebase to help learn it. Alpha agreed, assigned himself the issue and started working on it. Kate and Raja emphasized that any questions or requests are most welcome.

Modulo Dune 3.14.1 and Cygwin 3.5.2 everything seems to be mostly ready for 2.2.0~beta2 so we anticipate a release soon. Kate wondered if we should start planning for beta3, but after reflection it seems slightly too early for that. David wondered if the release should be the first one to get rid of opam-repository-mingw. Instead we could point to the branch on which the PR adding the compilers support for Windows to opam-repository is, to make sure the PR is usable for everyone and that we are not missing anything, in hope to avoid as little changes as possible for the compiler packages (which would sadly require a rebuild for everyone, even if you're not using Windows)

new issues

  • #5875 (bad gateway on opam.ocaml.org over the weekend) this issue has been fixed by the ocaml.org team over the weekend. Kate opened #5876 to make sure people know where to send bugreports next time there is an issue on the website.
  • #5872 (having ocamlopt in /usr/bin breaks when installing a bytecode-only compiler) Kate and Arthur talked about it previously. It is mostly a build-system specific issue, however Kate wondered if a warning could be issued by opam if it detects ocamlopt in PATH but only ocamlc in the switch. Raja pointed out that this would be against the idea of opam being OCaml agnostic.
  • #5873 (Failure trying to install OCaml 4.11 on Windows) Kate asked David if the plan for the compiler includes 4.11 and the answer was yes but later. David answered to the ticket.
  • #5874 (should opam deprecate use of git://) Raja made the case for showing a warning during lint check. Kate argued that git:// are still used by smaller git servers and work just fine. David pointed out that we could simply test if the dev-repo is available by simply cloning it when using opam lint --check-upstream. That idea made consensus.

new PRs

  • #5870 (opam 2.1.6) The proposed changes look good. However we should give a little more time for the change to use git apply to be more wildly tested. It was proposed to wait one week after the release of 2.2.0~beta2 to release 2.1.6. That idea made consensus.
  • #5877 (speedup opam using domainslib) The proposed changes look good and are very welcome. Kate would like to double-check the 40% speedup claim as she got bitten in the past when benchmarking opam built with 4.14 and 5.0 in #5518. Since most of the PR consists of passing and adding ~task_pool parameters, Raja wondered if instead the thread pool could be moved as part of OpamStateConfig's global variable. This way the amount of change to the API would be reduced by quite a lot. Kate wondered by how much the changes to dose3 speeds up opam. Kate would also like for someone more familiar with multicore-ocaml to review this work when it's ready.

Misc

Kate and Raja continued last week's discussion after the meeting about #5857 (wishlist for opam lock). We discussed about introducing the list of repositories as a field in lock files. One of the problems with that is that https repositories do not currently have a set state contrary to git. We wondered if we could use the stamp: "<hash>" field from opam.ocaml.org in combination with the upstream field to recreate the source git repository at a set commit, that would fix the issue for opam.ocaml.org at the very least and for everyone else they would need to add this field or switch to a git repository, which sounds reasonable.

One other issue is that installing several locked packages would end up with a proliferation of repository and make opam update very slow. We wondered if we could leverage git. If we clone the source repository, all of its git commits should be available, so opam could call git show <revision>:path/to/opam and get the state of the packages at a given revision and make them an opam pin when installing the locked package whose value would be the opam file at said hash. However there needs to be a way in the pinned package to retrace the source of the package for opam lock to be reproducible.

Use of a future revision field has been discussed too, however it would not be useful in this context because there would be no programmatic enforcement of the use of revisions when modifying a package in an opam-repository.

The opam switch export format could be a place of experiment and test a new repositories field as the current lock file format reuses the full opam file format and it's harder to change.

Overall we agreed that opam lock is not strictly reproducible, it would need to store the list of the current repositories at their current hashes for that to be the case. We explored the several possibilities to perform that using a new format for opam lock in the future.

2024-03-04

Present: Kate (@kit-ty-kate), Raja (@rjbou), Leandro (@leostera), Ambre (@ElectreAAS), David (@dra27)

opam 2.2

  • #5683 (MSYS2 support) was tested by Kate and merged right away.
  • #5741 & #5648 (tar.gz repos support) Kate hasn't had time to work on it this week again. Was moved to to before the RC.
  • #5802 (fix fish integration) Ambre split the original PR into several ones #5864 was reviewed right away
  • #5829 (improve and add missing sys-ocaml-* variables on Windows) David said this is ready to be reviewed and just need to update the tests. Was undrafted and marked for review.
  • #5832 (find a better place for the Cygwin directory in PATH) David says the code is missing some parts. We'll discuss it again next meeting.
  • #5834 (pass extra packages to cygwin's setup.exe) was moved to do before the RC
  • #5835 (Git-for-Windows warning displayed several times) was moved to do before the RC
  • #5400 (patch --> git apply) David reviewed it and it was marked as ready to be merged.
  • #5840 ('+' in package names clashes with opam syntax) The idea was presented to David who was onboard with the idea.
  • #5862 (rm deprecated function SHGetFolderPath on Windows) David reviewed it and it was marked as ready to merge.
  • #5861 (unicode characters makes dune&opam fail) Kate spent half of last week debug 3 different critical bugs in Cygwin and Dune/Spawn (ocaml/dune#10180 and cygwin#255561) and one of the output was this PR that gets rid of a deprecated function on Windows. A note was also added in the project to wait for dune 3.14.1 to release opam 2.2.0~beta2 as the number of people with unicode characters in their name is quite a bit higher on Windows and the current version of dune is corrupting some of the Windows internal environment which in turns makes opam fail.

new issues

  • #5857 (wishlist for opam lock) a discussion between Kate and Raja was started at the end of the meeting on the meaning/goals of lock files, however we didn't have enough time to arrive at a conclusion. The discussion will restart at the next meeting.
  • #5859 (make lib-pkg fails) the lib-pkg target isn't being used and is broken. Kate made a case for #5494 (the fix, removing the *-pkg targets) to be merged before the RC, David agreed.
  • #5860 (melange specific flag/depopt) Kate had already answered. David pointed out that with the depopt solution, the variable isn't necessary anymore.
  • #5863 (failure when using the temporary opam-repository for windows) David replied.

new PRs

  • #5858 (improve CONTRIBUTING.md) is an idea from Leandro to make it simpler for new or external contributors to contribute to the opam codebase. Raja did a review, Kate will look at it and try to improve it further.
  • #5865 (don't silence errors in shell init scripts) Kate isn't sure we want that. For example if a syntax gets deprecated, every prompt would output a warning. The PR was left open to leave more time to think about it.

2024-02-26

Present: Kate (@kit-ty-kate), Alpha (@moyodiallo), Raja (@rjbou)

opam 2.2

  • #5683 (MSYS2 support): Raja hasn't had time to work on it this week.
  • #5741 & #5648 (tar.gz repos support) Kate hasn't had time to work on it this week again. The WIP branch was shown and the changes needed and TODOs were exposed. Kate should have time to work on it in a couple weeks again so this might have to be moved to beta3.
  • #5802 (fix fish integration) Raja reviewed it, added some tests and discovered more bugs with fish.
  • #5829 (improve and add missing sys-ocaml-* variables on Windows) Raja will ask David what to do about this. This is required for beta2.
  • #5832 (find a better place for the Cygwin directory in PATH) Raja will ask David what to do about this. This is required for beta2.
  • #5834 (pass extra packages to cygwin's setup.exe) might be moved to beta3
  • #5835 (Git-for-Windows warning displayed several times) might be moved to beta3
  • #5400 (patch --> git apply) was discussed at the future of opam-repository meeting and it is needed as soon as possible in 2.1.6 and beta2 so that as much possible as possible get it and do not get an error if packages get removed from opam-repository. Looks good for Raja but would like David to give a review.
  • #5061 (opam source --dev uses shallow clone) is actually a 2.2 regression and was moved to be done before the RC. There might also be an issue with caching – where the repositories might not be cached at all – that we need to check.
  • #5840 ('+' in package names clashes with opam syntax) Kate had an idea to get filters working. She exposed it to Raja who agreed it might be one of the only solution. Kate hopes to expose it to David next week.

new issues

  • #5851 (ideas to improve the documentation around repositories) the documentation isn't clear on what is optional for a repository to be a repository and could be improved.
  • #5853 (shadowing uname breaks opam) while opam could create a custom C stub to use uname(2) (such as in #4274) to fix this specific errors with uname, this wouldn't be the only breakage in such an environment and we expect users to provide a consistent environment via PATH. Kate will reply to them and close the issue.
  • #5854 (fish is broken on windows) is one of the issue previously discussed while reviewing #5802
  • #5855 (autopin should have ?with_version) is a hole in the API that Kate discovered while working on opam-build. She isn't using autopin anymore and went the more lower-level way instead using OpamSwitchState.update_pin and OpamSwitchAction.write_selections which seems to be much faster and what she wanted anyway.
  • #5856 (make cold does not honour -j) -j without argument is used by default and it might be better to use the equivalent of -j$(nproc) to avoid stressing smaller machines too much. However for the issue at hand of not honouring -j given in argument this should probably be done before the release of 2.2 as more people might use make cold to build opam themselves on Windows. This was slotted to do before 2.2.0~rc.

2024-02-19

Present: Kate (@kit-ty-kate), Alpha (@moyodiallo), Raja (@rjbou), Leandro (@leostera)

opam 2.2

  • #5683 (MSYS2 support): Raja opened the PR. Kate tested it briefly and it seemed to work. Kate suggested to detect msys2 automatically when giving e.g. C:\msys64 and use <prefix>\usr instead of simply <prefix>. It was also suggested to automatically populate the package manager command option when msys2 is detected. Raja will do the changes.
  • #5741 & #5648 (tar.gz repos support) Kate hasn't had time to work on it this week again. If this continues the work could be handed over to Raja next week or otherwise moved to a future 2.2.0~beta3.
  • #5793 (native Windows symlinks) Kate rewrote it. It needs some testing but it otherwise looks fine. Raja offered to have a look.
  • #5841 (Windows paths appear in Cygwin's .bashrc) Kate opened the PR, Raja modified to use cygpath and Kate made it use Lazy. Kate is wondering if cygpath was doing any transformations that could clash with the single quotes added here. Raja will have a look.
  • #5802 (fix fish integration) Raja hasn't had time to review it.
  • #5829 (improve and add missing sys-ocaml-* variables on Windows) is a draft PR by David who isn't there today so we'll talk about it next week.
  • #5832 (find a better place for the Cygwin directory in PATH) is a draft PR by David who isn't there today so we'll talk about it next week.
  • #5834 (pass extra packages to cygwin's setup.exe) no-one is assigned to it but this is fairly trivial work and will get done when one of us has some spare time.
  • #5835 (Git-for-Windows warning displayed several times) no-one is assigned to it but this is fairly trivial work and will get done when one of us has some spare time.
  • #5848 (fix handling of git-location in opamrc) looks good and was reviewed on the spot by Kate. Raja will rebase it and change uses of init config file by opamrc.

new issues

  • #2715 (buggy behaviour when disk is full) a new comment was added and none of us understand how the file descriptors ended up being tangled this way.
  • #5836 (Package variables in depexts) looks like reasonable feature request to do after 2.2.0.
  • #5838 (env revert not working on Windows) looks annoying and we'll wait for David's return next week to discuss that again
  • #5839 (cygwin depexts do not fail on missing pkgs) we might have to parse the output of the package manager. Kate wondered if this also happens if case of failure to download the package or if it's just for missing package. In which case Raja wondered if we could try again to detect all available packages in advance. This was slotted to do before 2.2.0~rc

new PRs

  • #5840 (packages with + in their name should not have been allowed) this is a fairly tricky issue brought up by David in opam-file-format#59 in which the + character should not have been allowed in the opam syntax as it clashes with the {pkg1+pkg2:var} syntax. + is forbidden as name in the documentation. Kate wondered if instead we could allow + in the name but make the filter syntax parseable somehow. Alpha proposed to use some sort of URL-like formatting. Kate and Raja will try to test what is possible and will report back next week.

2024-02-12

Present: Kate (@kit-ty-kate), Alpha (@moyodiallo), Ambre (@ElectreAAS), David (@dra27), Raja (@rjbou)

David won't be available next week and might not be able to make it to the meeting until the 11th of March. Meetings on the 26th and 4th might be moved to accommodate him. TBD.

opam 2.2

  • #5683 (MSYS2 support): Raja hasn't had time to work on it this week. Kate and David proposed to offload to them in a Draft PR as the code is done but only live tests are missing.
  • #5741 & #5648 (tar.gz repos support) Kate hasn't had time to work on it this week. She was busy with testing the first alpha of OCaml 5.2.
  • #5796 (disable Cygwin ACL) Kate hasn't had time to apply the review.
  • #5793 (native Windows symlinks) Kate hasn't had time to apply the review.
  • #5797 (removing env.exe from the list of shells) was approved by everyone and just needs a rebase. It was then merged.
  • #5794 (don't check for cppo) was reviewed by Raja and merged.
  • #5804 (Windows paths appear in Cygwin's .bashrc) nobody was still assigned but there are bigger things to do before beta2 and it looks straightforward to fix.
  • #5755 (weird behaviours of opam tree --test --doc) Ambre hasn't had time to work on it. She was busy working on Dune.
  • David gave an update on the opam-repository work to have ocaml-base-compiler working on Windows and it shouldn't be long before the PR is open. It is confirmed that ocaml-base-compiler will be supporting Windows (some preview version of the fix used ocaml-variants instead) and noted that some conf-* packages will have 3 versions instead of 1 due to some windows specific bits (e.g. g++ has an x86_64 version and a i686 version). This could be improved in the future when cross-compilation is properly supported.

new issues

  • #5821 (stop magic from happening in OpamFilename) is from a discussion that happened at the last meeting concerning the internal OpamFilename API which we might want to explicitly handle ~/ instead of the opposite currently. This is not slotted for 2.2 at all but we hope to clear this at a future time.
  • #5823 (global last-env) opam 2.2.0~alpha introduced the last-env file in each switch in #5417, however when the user removes a switch the last-env file is removed with it, which is an issue. To remedy this last-env would be global instead. Kate was concerned about the proposal to place it in /tmp as this directory is writeable and might be a security risk if a package were to write in it. David points out that the content should match the name (checksum) however this is probably md5 so this is still a security risk. This was slotted before 2.2.0~rc1 as last-env was added in 2.2.
  • #5824 (partial reload on opam update) opam update already diffs between the preview state and the new one so it would be a big performance improvement to only reload the content of the files that have changed instead of re-reading the repository once again, this is relevant on Windows in particular. While this is not strictly to be done before 2.2.0, this was slotted to do before 2.2.0~rc1 because David pointed out that we should at least show a message indicating that opam is doing something as some users might misunderstand the time this takes to be opam crashing.
  • #5825 (speed-up git+file://) looks to be a nice feature request for some performance improvement and it doesn't seems to be critical to the 2.2.0 release, so it was tagged as such.
  • #5826 (opam admin add-constraint unwanted edits) is a straightforward format preservation bug but doesn't make the feature unusable. It is not critical for 2.2.0 so it was left as a feature request to do afterwards.
  • #5827 (opam switch remove instead of rm -rf to remove broken switches) non-critical feature request.
  • #5833 (run cygwin's setup.exe once) is a small feature request for a performance improvement on opam init on Windows. This was deemed to be not critical and thus left for a 2.2.x point release.
  • #5834 (pass extra packages to cygwin's setup.exe) David made a case for this to be added to the 2.2.0~beta2 slot because this would be quite useful to have for setup-ocaml or other CIs and reasonably easy for us to do.
  • #5835 (Git-for-Windows warning displayed several times) the git-for-windows warning added in the previous 2.2.0~beta1 release shows up several times and should be removed when the user has done what we asked. This was added to the 2.2.0~beta2 slot for now.

new PRs

  • #5829 (improve and add missing sys-ocaml-* variables on Windows) the sys-ocaml-* variables were added a long time ago for the Windows support and are using cygwin shell to parse the output of ocamlc -config. However cygwin shell is fairly slow and using cmd instead is much faster. Instead of using a pipe with Cmd it is easier to use ocamlc -config-var directly. However this is an OCaml >= 4.08 feature but David points out that older compilers work very poorly on Windows anyway so this is totally fine to do. David also needed the system variable for his work on the opam-repository support so it was added there. Kate points-out that the system variable before OCaml 5.0 is badly defined and prone to issues, however we might be able to unify the values afterwards in opam for our purpose. Kate also noted that we could add all the config variables directly in the future. This is required for the opam-repository support thus it was slotted for beta2.
  • #5830 (improve cygwin setup.exe) is a straightforward small tweak to the internal cygwin install. It is not urgent but it is already done so it was slotted to do before 2.2.0~rc1.
  • #5832 (find a better place for the Cygwin directory in PATH) opam currently always adds the path to Cygwin at the end of PATH, however some tools in native windows have the same name than their Unix counterpart and cause problems. To remedy this we need to move the cygwin path at the correct place in the PATH so that there are as little problems as possible. This was deemed critical and was slotted to beta2.
  • #5828 (do not let Windows show a windowed dialog on missing dlls) by default Windows tries to display a dialog box when an error appears such as missing dlls. This stops the execution of the program. To remedy this we disable this dialog on windows as soon as the opam binary starts. The PR was simple enough to be reviewed and merged on the spot.
  • #5831 (recommend developer mode on Windows) is linked to #5782 and was slotted for beta2.

2024-02-05

Present: Kate (@kit-ty-kate), Alpha (@moyodiallo), Ambre (@ElectreAAS), Etienne (@emillon), David (@dra27), Raja (@rjbou)

The meeting ran longer than usual (2 hours) Raja asked Kate (current note taker) to change the format of the meeting notes to include the title or summary of the issues/PRs to make it easier to read.

opam 2.2

  • #5683 (MSYS2 support) by Raja, is its testing phase
  • #5741 & #5648 (tar.gz repos support) by Kate is still in development. It touches a good chunk of the code and will take some more time to finish. Hannes proposed to do the diffing between two tar.gz archives as it is required for conex.
  • #5796 (disable Cygwin ACL) by Kate was reviewed by David who would prefer to use pure OCaml instead of Cygwin sed.
  • #5793 (native Windows symlinks) by Kate was reviewed by David and both agreed on merging this more-or-less as-is (which means Admin Privileges would be recommended) as a stop-gap and to open a new ticket later to make sure to have a path that does not require Admin Privileges.
  • #5797 (removing env.exe from the list of shells) looks good but requires some testing to make sure to understand the issue reported by Jonah about cmdliner. This will be done by Kate.
  • #5794 (don't check for cppo) is straightforward and needs a review. Raja is on it.
  • #5804 (Windows paths appear in Cygwin's .bashrc) looks straightforward to fix. Nobody was assigned but there are bigger things to do for beta2 before next week's meeting.
  • #5755 (weird behaviours of opam tree --test --doc) Ambre is still on it and made progress understanding opam's codebase and test system with help from Kate and Raja.

new issues

  • #5791 (opam env --bare) looks fine and a straightforward feature request for after 2.2.0
  • #5795 (why pin-depends does not imply depends) is a fair question and many people fell for it in the past. pin-depends are pinned but not installed by default so users can use either in depends or depopts. Raja suggested that a lint check could be added. Everyone agreed.
  • #5798 (no opam repo but sucessful opam init) puzzled Kate and requires more debug. It was tagged as a non-critical bug.
  • #5800 (windows installer) we wondered if we should use a batch/powershell script or a windows installer and David suggested that a Powershell script would be much easier to maintain (and is used by Microsoft itself for some it its tools), the rest of the people agreed.
  • #5805 (opam admin check --obsolete not working) after the public meeting for the future of opam-repository the need for a tool to detect packages that aren't required anymore arose and currently this command exists but is way too slow and gives erroneous results.
  • #5806 (4.14.1+mscv64c can't link) David has a fix and would like to comment on and close the issue when it's fixed opam-repository/opam-repository-mingw.
  • #5807 (broken setup on macOS) is the usual macOS shenanigans but the person who opened the issue doesn't seem to be experienced in environment management to fix this broken setup themselve. Kate started a list of this kind of issues in the opam doctor feature request and will close this issue next time if the person who opened the issue either don't reply or says it's fixed.
  • #5809 (VCS pins include untracked files) is indeed a bug to look into bug as it is a 2.1 regression it seems that it is not urgent and can wait until after 2.2.0.
  • #5810 (VCS pins use rsync) is a minor issue that we can look into later.
  • #5811 (stop support extra-files) is an opam 3.0 type feature request that arose when Kate was coding #5648 as scanning a tar.gz file would be infinitely easier and faster if only 1 file were to be read. David also agreed that is it a problem for opam-repository where duplicated files accumulates. Kate noted that for smaller repositories would could instead have a separate directory that extra-source would target. Raja was concerned that it would make maintenance more complicated but David noted there are ways around it using git for instance.
  • #5814 (random recompilation) requires some testing but Kate wondered if the issue could be that Docker rounds up the modification time (see #3997) in some cases. She asked Louis.
  • #5815 (perf regression in opam init --bare) should be straightforward to fix but is not critical.
  • #5816 (jobs option not reset when upgrading from 2.1 to 2.2) seems straightforward to fix and everybody agreed doing it regardless of the version number to upgrade to is an ok fix. This was scheduled for 2.2.0 as it is a 2.2 regression.
  • #5817 (performance regression when installing lock files) might be related to the recent change in the handling of --deps-only and might not be trivial to fix.
  • #5819 (should opam init change .profile or .bashrc) Kate pointed out that opam currently doesn't differenciate between POSIX shell and bash and it's a problem as if we were to change the .bashrc it wouldn't be POSIX compatible anymore. Supporting bash separately would be a good idea to understand the problem better. Kate also pointed out that she had to do something extra on top of opam init for both login mode and command mode to work with bash when doing the draft for the official Docker image for OCaml

new PRs

  • #5799 (add the list of all repos to opam config report) from Kate is just something that would help debug. Kate wants to list all the repositories explicitly instead of just counting them which is fairly useless for us but this isn't planned for 2.2.0 so this will wait.
  • #5820 (support doas in install.sh) is straightforward and was instantly reviewed by Raja.
  • #5818 (fix doc of opam lint) is straightforward and was also instantly reviewed by Raja.
  • #5803 (handle ~/ in opam init) by Ambre requires a review and the devs are currently unsure what is the best thing to do so it was not slotted for 2.2.0. Then Kate went on tangent about how OpamFilename.Dir.of_string shouldn't handle ~/ by default and such thing should be explicit as she went through this code last week and was annoyed by it. However changing this would require thorough checking as this function basically hasn't changed for the past 11 years and is used everywhere in the code, so it's not something we can do immediately so #5821 was opened to remind ourself to look into it in the future.
  • #5802 (fix fish integration) by Ambre was slotted for 2.2.0 and just needs a review. Raja is on it.

2024-01-29

Present: Kate (@kit-ty-kate), Alpha (@moyodiallo), Ambre (@ElectreAAS), Etienne (@emillon)

David and Raja could not make it, so the usual core team's "let's organize and see what's left to do and what's new together" has been moved to next week's meeting.

Instead, Ambre asked if she could help with any of the issues, and Kate thought that #5755 being an issue that does not involve Windows [1] but still slotted for the release, would be a great fit to get introduced to the opam internals while helping for the 2.2 release. Ambre was assigned the issue after roughly describing the core issues and possible ways to fix it.

[1]: setting up a Windows machine and learning about the problematic brought by Windows on top of learning about the opam internals would take too long

2024-01-22

Present: Kate (@kit-ty-kate), David (@dra27), Alpha (@moyodiallo), Ambre (@ElectreAAS)

Side note: Kate was sick last week and still recovering, Raja is sick this week.

opam 2.2

  • 2.2.0~beta1 is released, the blog post is finally up after 8 hours of waiting for ocaml/infrastructure#92. Left to do are the Discuss post, ocaml.org changelog and Discord post. Kate is on it.
  • For beta2:
  • David is writting up a proper TODO list for what's left to be done for the compiler packages on Windows. We should have a clearer view next week.

new issues

  • #5787: was discussed among opam dev already. This is planned for the next overall of the opam syntax
  • #5790: just a TODO for the release script. Using the -vanilla images might reduce the disk space required. This will require some experementation.

new PRs

  • #5788: Straightforward, looks good.
  • #5789: The work is still in progress. Several issues are still to be fixed:
    • Disable unnecessary processes quadrupling the time it takes for the build to finish (Windows Defender, File Indexing, Edge Updater, …)
    • Disable Windows Update for good
    • Windows 10 22H2 removed the possibility to create local accounts on Windows Home, so the installer guide needs to be updated to use Windows Pro instead
    • Re-test again and make sure this is reliable

2024-01-15 + 2024-01-17

Notes to be written down later. Note taker got sick

2024-01-10

Present: Kate (@kit-ty-kate), David (@dra27), Alpha (@moyodiallo), Etienne (@emillon)

opam 2.2

  • #5715: David wants to do a last minute change (creating a new function in OpamConsole meant for outputs to a pipe instead of to the console would be a better API than adding an optional argument to OpamConsole.msg) and it should be ready to merge tonight
  • #5743: CI is green and it should be ready to go once rebased above master once #5715 is merge
  • #5718: Kate still needs to rename the rest of the internal names for this new git-location field (previously named gitbinfield/gitbinpath/gitbin internally and externally) and it should be good to go once this is done, rebased and CI is fixed. Kate chose the new name in the absence of a consensus
  • #5783: this issue while not critical per say for the release, might require some non-trivial work to get it done like renaming of the cygwin focused options towards a more future proof approach (e.g. --win32-distribution-location instead of --cygwin-location). This needs more scrutiny so we’ve slotted it to beta2 in the meantime. We leave it to the next meeting on Monday to decide if this is needed for beta1 or for beta2
  • #5641: this issue while not critical per say for the release, might require some non-trivial internal work to get it fixed (see #5713) and does not seem suited for a point release. So we’ve moved it to be done in beta2 as well

Ran out of time to talk about new issues/PRs

2024-01-02

Present: Kate (@kit-ty-kate), Raja (@rjbou)

opam 2.2

  • #5743 is still queued on #5715
  • #5715 needs to be remade using dup(2) instead of threads
  • #5718 should be good to go once the minor typos have been fixed. We mostly disagree with David's review as Cygwin bin path is always prepended so displaying a warning would not be fixable for the user
  • #5683 is being worked on by Raja
  • #5680's tests are too complicated to all implement. A minimum specification could be to just check that the MinGW binary is static. Kate is on it.

new issues

  • #5770 is a bug in opam 2.1.5 requiring to do a new 2.1.6 release and a bug in master as well.
  • #5771 is a bug on a not officially supported platform and fixing it properly will take time. This is slotted for 2.2.1 for now. In the meantime we have several workarounds that we can try.
  • #5772 is a minor bug. We’ll fix it when we have time.
  • #5773 looks fixed with master.
Clone this wiki locally