Skip to content

2022 Developer Meetings

Jump to bottom
David Allsopp edited this page Oct 3, 2022 · 25 revisions

26-Sep-2022

Agenda

  • Previous week
  • 2.1.4 backports issue
  • 2.2 release plan
    • Remove the warning in #5163?
    • Remaining issues in "To do" are all tied in to the Windows depext support in some way or another!
    • Status of s/--with-tools/--for-dev/ (#5214)
    • Mergeability of Software Heritage PR (#4859) and status of documentation requests from @avsm
    • #5297 looks good to go?
    • Not-yet-tracked issue list
      • Windows depext support (@dra27)
      • opam-repository packages for Windows (@dra27)
  • New issues
    • #5298 - JSON output for opam tree
    • #5294 - work needed?
    • #5288 - menus?
    • #5283 - more ways of ignoring constraints?

Notes

  • Empty issue for 2.1.4 and add #5293
  • #5163 - remove the warning completely; it's only a warning so shouldn't be recognisable impact
  • #5214 - comment in #4659 - rename to --with-dev-setup
  • #4859 - add a small chunk of text to the manual with the technical background; opam option to control globally turning it off (this implies opam root fixes becoming more urgent; IPFS is about distribution SWH is about persistence; work forthcoming for the deployment through opam-publish/dune-release + opam-repository switch-over + CI for opam-repository checking that they're given.
  • Other items:
    • Solution for opam-state not depending on opam-solver
    • opam-root fixes
  • #5283 - consider adding --ignore-availability and possibly ignore-conflicts-with

1-Aug-2022

  • Raja: compiler package changes almost there. Tried the idea from last week with having the compiler but ignoring it - however, this was too invasive. Returned to the original idea - the compiler was the dependency cone of the compiler packages, but it's now - more correctly - the dependency cone of the invariant. That's used when we load the switch state and it's recomputed on each potential change of up-to-date (on install/remove, etc.). Internally, the base packages set is gone, as it's unused (and removing it gets rid of all temptation to use). opam list has a --base option - it now gives the full dependency cone of the invariant and has been renamed. The switch synopsis has been altered for when it's not set so that it's now the switch invariant formula. The changes are all interconnected, so the changes are sat in one PR (ocaml/opam#5208). There's then a small follow-up for opam switch to have the invariant column. For the system polling PR, the assert failure comes on Windows because the os has been manually overridden which means it ignores the bypass. Fix will be to use a non-overridden version of os when polling those variables. Software Heritage is ready to be reviewed (ocaml/opam#4859).
  • Kate: merge fest! opam-repo-ci is now testing opam master by default (with 2.0 and 2.1 as an additional test).

25-Jul-2022

  • Kate: mainly reviewing. Got maintainer rights to cudf, so made a Dune-compatible and 5.0-compatible release.
  • Raja: Looking at where compiler+base packages are used in opam. There's just one place in the switch state for the check where the invariant needs to be recorded. Proposing to remove the compiler field from the switch-state file. @dra27 vaguely remembers there being a problem with base packages beforehand (either in opam itself or in a plugin)

19-Jul-2022

  • David: Fixed AppVeyor for 2.1 branch
  • Raja: looking at tests for the 2.1.3 backport PR. It's just waiting on ocaml/opam#5176, which itself could just do with a test. Fixed the issue with switch import (ocaml/opam#5181). Updated and finalised software heritage PR (ocaml/opam#4859)
  • Kate: Added a test for #4501 (ocaml/opam#5183). Reviewing and answering questions (which led to another test in ocaml/opam#5179 showing that 2.2 fixes #5178)!

8-Jul-2022

Agenda

  • ocaml/opam#5002 - removing the "cute" logic in switch invariant resolution.
    • One idea: remove it, but use the idea as a hint when opam upgrade fails
    • Better idea: I think that the n <= 1 case exactly catches opam-system (and definitely won't catch ocaml-base-compiler). If that's the case, why don't we keep the n <= 1 which fixes the actual issue (upgrading a system compiler) but remove the >= version step which is what's causing the problem? That seems viable for 2.1 - I think it's reasonable to see it as a bug... the invariant was supposed to fix ocaml-system upgrading
    • From meeting The second proposal doesn't fix Kate's original issue - ocaml-base-compiler because of the pin. Further up the file in compute_available_packages the pinned packages are filtered - this can be flipped into two which should then solve the problem? This can also be tested against our Docker base images by loading in a new opam binary.
  • ocaml/opam#5163 - an old warning to do with spaces in commands. Added a very long time ago. Definitely propose we suppress it on Windows, but this shouldn't be an issue on Linux either - I expect it's to do with the worry of passing it to Sys.command (or the higher-level Unix process functions which go via the shell), but opam doesn't do that anymore.
  • ocaml/opam#5173 - repro in place, does this want fixing with 2.2?
  • Steps to 2.2
    • Clear issues which don't have to be addressed right now (including some of mine)
    • Is everything on the board - i.e. do we have tracking issues for everything which needs to be done

This Week

  • Raja: getting back into the swing of things, reviewing the opam tree PR (ocaml/opam#5171). Looking at switch import/export issues - Daniel's issue and also ocaml/opam#5173.
  • David: re-checked the compiler-related patches in opam-repository-mingw (last done in April, I think)
    • Default init repo to be git+https://github.com/ocaml/opam-repository.git
      • Can't do this in a filtered way in opamInitDefaults.ml but we could do this "hackily" (so Windows just reports a different default)
      • Can the redirect in a repository change the backend - so can we redirect to git+https://github.com/ocaml/opam-repository.git in the opam.ocaml.org tarball for Windows only? At present the redirection is for the URL only? What happens with a redirect which includes an opam git+ style URL? Actually, this should work.

28-Apr-2022

  • #5042 - ready to go with one minor tweak; @dra27 proposes a separate PR to rip out the lib-pkg system, because it's wasting too much time maintaining it
  • There's an issue with the depext menu which is temporarily delaying #5057
  • #4816 - find a half-way house where the C stubs library. @dra27 to push suggestions for the reset
  • #4817 rebased and ready to merge
  • 2.1.3 - back-port PR in #5125. Just wants #5117 and #5027 added to it.
  • Caching idemptoency for the Cygwin cache - it's definitely happening where it's either uploaded or downloaded a zero bytes cache
  • Turn off AppVeyor on master - disable after #4817 is done
  • Messages for the experimental CLI flags in Element to review
  • #4981 - let's use the Discussions feature to create a URL, rather than the issue tracker. Main idea being to ensure that new issues aren't constantly opened for an identical issue.
  • #4932 - this is the half-way solution where we detect that the program has disappeared, but not necessarily changed. This doesn't really fix the original issue, though.

11-Mar-2022

Notes

Louis: Discussions last week with Armael on the solver work and looking at cycles. PR reviews. Kate: Not too much opam this week - BSD fix and then the follow-up on shebang with opam-repository David: Going through updates on the GHA PR

04-Mar-2022

Notes

Issues

  • #5078 @louis: it's a blind spot from the beginning
  • #5077 @rjbou to close
  • #5074 @rjbou assigned
  • #5073 look for more information
  • On pin & install, proposal to have homogeneous options: add pin options to install, to apply only of directory targets. Maybe for 2.3 / 2.4
  • #5071 use git am when autogenerated patch. On repos, maybe handle it via git completely
  • #5063 (last week): easy to add a better error message
  • Add json -> try with jsonm, it adds 2 new dependencies (uutf & uchar)

PRs

  • #5075 good, to be continued
  • #5069 We need to check synopsis & description usage. synopsis must not be empty, and description is optional. it's descr that is optional -> add a deprectaed message when encountered (in order to remove)
  • #5045 to test on mac & good to go
  • #5043 to review
  • #5053 merge and change afterwards for messages to change, to test on mac
  • #4796 someone to take it over, maybe @rjbou for simple modifs

This week

  • Louis: discussions around & pr review
  • Raja: split of multi purpose PRs, last rework of shared fetch and merge \o/
  • Kate: mainly review, and issues
  • all: discussion with Armael on cycle detection in package graph

25-Feb-2022

@kit-ty-kate @rjbou

Overview of some issues / PRs

18-Feb-2022

Notes

On depext ui

  • add command display
  • check if need to display sudo need
  • ote: homebrew don't ask for confirmation

New issues

  • #5058: cwe can add a test target, for home we already do the expansion in opam , & dose w already have it in 2.2
  • #5059: @raja try to add a test, why the reinstall is not triggered
  • #5060: @raja pr is coming
  • #5061: add in config or as a label of the fetch function (invasive)

Prs

  • #5055: There is a comment with the test of the test to ensure that it works. @raja want to test all cases to be able to switch the new system directly. @kate can't we drop win 32 ?
  • #5002: "la magie c'est bien, mais ça peut être embêtant" (adage des sables)
  • #5024: is there a reason to not merge ? comment in issue
  • #4975: to review

This week

  • Louis: Finalise depext & init ui, PR reviews, & new experimental version of opam-custom-install that have a better internal handling of opam files (pin, update deps)
  • Kate: review shared fetch & finalisation opam opam sitwch -
  • Raja: Some help on opam lib use in marracheck, rework of shared fetch, working dir fixes (#5060), some tests of gha PR #5055

11-Feb-2022

Agenda

  • New Issue
  • opam 2.1.3 (#5000)
  • opam 2.2.0 release criteria
    • Review of features in-progress
    • Fix feature requirements / release criteria
    • Focus to alpha
  • sigstore

4-Feb-2022

Agenda

  • opam 2.1.3 (#5000)
  • opam 2.2.0 release criteria
    • Review of features in-progress
    • Fix feature requirements / release criteria
    • Focus to alpha
  • sigstore

New(ish) issues (inc. last week's)

This week

  • Raja: worked on unsetting variables rather than setting empty strings and reverting opam env. Some writing for a blog post about opam. More work on the disabling of repo tarring #5015.
  • Louis: cleaning up existing PRs - all PRs ready to green. Popped back to previous stack, therefore - looking at --deps-only-handling and installing with --formula. Given up using an identical code path for each, as it's simply not the same! Rewrite worked well, but in the process the reasons for actions got lost - recovered those. In fact, that fix could be on master now as it involves keeping more information and filtering out the irrelevant parts earlier on. Question on handling of --with-doc, etc. at opam install. At the moment, they're not applied to the --formula. Some more work on the opam-custom-install plugin which is now waiting for feedback from the flambda team.
  • Kate: Mostly busy around OCaml 5 / CI other things. Some time spent helping out and testing #5015!
  • David: CI work, but not as much as wanted.

28-Jan-2022

Notes

  • Various needed discussions! 2.1.3/2.2.0 and issue sweep to next week, as we ran out of time.

This week

  • Louis: opam list optimisation fixes are done and 2.1/2.2 is back down to 2.0's speeds. Tidied up code and accidentally broke the optimisation, but that's now fixed again (all part of #4172). Some time also on opam-custom-install (https://gitlab.ocamlpro.com/louis/opam-custom-install/-/issues/1).
  • Raja: Fixes for opam var and opam option (springing from #5025) - rather more critical bug found which needs back-porting. Some more work on --with-dev - need to be clear in documentation which of dev and with-dev to use.
  • Kate: Looked at state of opam in Linux distros; it's depressing! Alpine is easy to contribute to - now the maintainer for Alpine!
  • David: hot-fixes to make cold for base image builder for sigaltstack. Windows GitHub Actions!

Agenda

  • #5030 and the need for full PR descriptions (@dra27)
  • #5031, #5022, #4901 and the need to talk to each other
  • opam 2.1.3 (#5000)
  • opam 2.2.0 release criteria
    • Review of features added today
    • Fix feature requirements by 4 Feb
    • Focus to alpha

New issues (inc. last week's)

21-Jan-2022

Agenda

  • @NathanReb: Downloading and caching opam repositories
  • @dra27: opam-dev team visibility, MAINTAINERS, etc.
  • @dra27: who goes in AUTHORS!
  • @dra27: opam-depext release needed for 5.00.0 compatibility (see the horror of ocaml-opam/opam-depext#147
  • @dra27: Any other "definitely not going to do now issues" to bump to 2.3?

New issues

Notes

opam-monorepo repository caching

  • opam-monorepo repository caching (@NathanReb). opam-monorepo uses your switch to get the repository information from the current switch. This first step is an issue, since configuration of the switch can vary between users. Idea is to be able to include a list of repositories in the lockfile and some opam configuration variables and have this as metadata in the lockfile.
  • @AltGr: there is a cache for git objects, but it's very hacky (no nice OCaml modules!). It uses gits plumbing to share commits between everything.
  • @dra27: it should be possible to plumb in additional repositories manually to the user's state (given that the opam libraries are already being used) - i.e. load the user's state, but then link in additional repositories which are maintained by opam-monorepo. In this case, you get the benefit of opam's caching, without exposing the extra repositories to the user and so forth

This week

  • Raja: cleaned up and finalised PRs from last week (recpin, lock and sw heritage ID) - all ready for review. PR and issue sweeping.
  • Louis: Also more work on last week's PRs :) Worked on some more optimisations in the solver - computing reverse dependencies it turns out that the graphs were too big! Looking at general costs of installability and co-installability checks for Kate - working assumption that 2.0 was better because the trimming was being done at a higher level. More information is now being propagated down which allows this to be improved (back to 2.0 levels). Requests which had started taking 5 minutes now back to 16 seconds!
  • David: sigaltstack stuff about to be merged! Windows CI next!

14-Jan-2022

Agenda

Towards a 2.2 release plan

Sweep through these issues (and others in the "Bump to 2.3?" project column, as necessary)

  • #4518, #4519, #4520, #4521 (deprecated fields - this was gated on there being a visible format upgrade in opam 2.2)
  • #4543 (non-recursive --with-test / --with-doc in opam list)
  • #4727 - status?
  • #4649 - status?
  • #4686 (linting on pinning) - status? previous discussion?
  • #4419 and #4207 - status?
  • #4311 - status?
  • #4272 - status?
  • #4541 - status?
  • #3722 - status?
  • #4762 - status?
  • #4890 - status?
  • #4959 - someone happy to implement?

New issues

  • #4992 - opam root message in opam init
  • #4991 - doas support (esp. for Alpine)

7-Jan-2022

Agenda

New Issues

  • #4977 - opam pin add <archive url> should read package descriptions from archive
  • #4984 - tracking git branches following opam switch create .
  • #4985 - permit opam switch export --freeze with path pins (adding --force, --permissive or something)

Notes added on issues

Last week

  • Louis / Raja / Kate code walk-through of the solver
  • Kate: following this, deep into a rabbit hole looking at the empty conflict resolutions. Looking at pulling in 0install's conflict resolution solver instead.
  • Raja: updated software heritage PRs - fallback mechanism should be ready later today. Had another look at #4967: several issues are masked by E3 ("format error") where the actual error is then just displayed as a string. Can look at propagating the precise error from OpamFile. For opam repo set-url, implemented the restoring of the backup if the new URL fails. Worked on tests for lock file / rec-pin (which are strangely intertwined!). Spotted that opam upgrade completely ignores lock files - branch in progress to fix that.
  • Louis: adding some (more) tests. Finally merged opam pin --current and the refactoring (#4973 and #4969) and so coming back to opam install --formula (#4975). Been looking at the issues discussed at the previous meeting with the --deps-only issues - various solutions explored - arrived at a working branch which needs cleaning up and then PR opening
  • David: deployment of glibc 2.34 opam-repository patches should be finalised for next week; MSVC patches are now merged upstream and deployed to Docker images (so our CI can adopt them as well)
Clone this wiki locally