Revising conditionals (discussion) [WAS: Breakages caused by packages missing bounds against bytestring-0.11]

[hvr]

No description provided.

[phadej]

I made revisions for (let these be my last revisions as trustee):

• aeson (here more breakges can become visible, I didn't use too much --allow-newer).
• attoparsec
• binary
• blaze-builder
• text

base16-bytestring was also affected, but maintainer made revisions themselves.

I only looked, not tested whether there are compilation errors, but:

• network (older releases, without upper bound on bytestring)
• double-conversion (all releases without upper bounds)
• zlib (older releases are without upper bound on bytestring)
• memory (three or so latest releases have base >=4.9, so are fine)

use PS constructor as well.

Most breakages are invisible, as usage breaks due renaming of (internal) ByteString constructor from PS to BS. However on GHC >=8.0 there is pattern synonym, so things seems to work, until you force bytestring >=0.11 on GHC-7.x.

Note, breakages may become visible before releases with bytestring-0.11 support are made. I have tested locally with patched attoparsec, hashable (just relaxed bounds) and text (where latter restricts most if not all of build plans today, due my revisions).

There were just a handful of older releases which broke due removed inlinePerformIO.

False positives are for example cryptohash-sha256, it uses PS constructor, but there is a bound. People might omit with --allow-newer, but that bound is indeed strict (but again, breaks only with older GHCs).

[phadej]

See also haskell/bytestring#287 (comment)

[sjakobi]

Most breakages are invisible, as usage breaks due renaming of (internal) ByteString constructor from PS to BS. However on GHC >=8.0 there is pattern synonym, so things seems to work, until you force bytestring >=0.11 on GHC-7.x.

In principle such issues can still be addressed with metadata edits where some dependency bounds are within if impl(ghc) conditionals. I ended up doing this in cborg. Unfortunately adding or editing such conditionals isn't allowed in Hackage revisions. What's the reason for this limitation?

I also regret my opposition to removing the support for GHC 7.x in bytestring-0.11.0.0 now. That would have simplified the whole compatibility situation. :/

[gbaz]

the general framework for what are and aren't allowed as revisions is given here: https://github.com/haskell-infra/hackage-trustees/blob/master/revisions-information.md

I think adding flags is discouraged because conditional exposure of modules could modify the semantic meaning of the version assigned to a package. The rules aren't perfect, but are designed to attempt to be conservative in enforcing this property. There might be a way to loosen them while preserving it, but that's the reason at least, as I understand it.

[phadej]

Unfortunately adding or editing such conditionals isn't allowed in Hackage revisions. What's the reason for this limitation?

Technical. The revision comparison is very naive. We (me and Herbert) discussed that adding conditional blocks with dependencies which are already present "on top level" would be useful, i.e. from

build-depends: bytestring

to

build-depends: bytestring
if impl(ghc <8)
  bytestring: <0.11

The important property to maintain is transitivity (you can make one revision instead of small steps) and symmetry of revisions (they can be reverted).

Packages like https://hackage.haskell.org/package/semigroups with dozen flags make naive implementations non-feasible, check whether revision is valid should be relatively fast.

---

An idea which I haven't explored properly is to separate GenericPackageDescription structure into build-depends & other-extensions trees + everything else. Everything else should be strictly equal. build-depends tree can be compared for "implication" (there are no extra dependencies on either side).

I.e. I believe it can be done.

So. Someone have to put few days of coding work (and poke people to deploy that to running hackage-server).

[phadej]

And as a bonus, if we can improve the summary of what revisions have done, that would be great. E.g. when you edit dependencies in conditional block, the summary is not very descriptive.