haddock-project support public sublibraries #9821

coot · 2024-03-19T19:24:23Z

Added argLibraryName to HaddockArgs
haddock-project: fixed main library name
haddock-project support for sublibraries

QA Notes

Running cabal haddock-project on a package with multiple public sublibraries should create ./haddocks directory which contains the package:sublibrary haddocks in ./haddocks/package-sublibrary directory. All cross-references will be resolved to files inside ./haddocks directory. ./haddocks directory should also contain both indexes: html and quick-jump index. Both indexes should contain references to identifiers from all public sublibraries.

When using cabal haddock-project --hackage the cross-references between sublibraries will be resolved to hackage (as described in #9852).

coot · 2024-03-21T18:02:41Z

I am getting a lot of test failures of this sort:

+Warning: Both <ROOT>/cabal.dist/home/.cabal and /home/coot/.config/cabal/config exist - ignoring the former.
+It is advisable to remove one of them. In that case, we will use the remaining one by default (unless '$CABAL_DIR' is explicitly set).

on my dev machine, I use cabal's support for xdg directory layout; is this related?

ulysses4ever · 2024-03-22T12:59:32Z

Try removing ~/.cabal maybe.

coot · 2024-03-22T17:24:57Z

I think the ~/.cabal file comes from the test suite (I suspect it sets HOME variable in some particular way). We probably need to set some XDG env vars in the tests so cabal doesn't find the user's configuration.

coot · 2024-03-22T17:26:17Z

btw, I don't have ~/.cabal file in my HOME directory.

Kleidukos

I'd appreciate if you wrote QA notes to ensure that this PR does for others what you expect it to.

Otherwise I don't see anything blocking. :)

Kleidukos · 2024-04-02T20:18:05Z

Cabal/src/Distribution/Simple/BuildPaths.hs

+    LSubLibName sublib_name ->
+      prettyShow (packageName pkg_descr) ++ "-" ++ prettyShow sublib_name
+    _ -> prettyShow (packageName pkg_descr)
+-- TODO: how Hackage should deal with sublibraries?


How do we resolve this question? Do you want to ask gbaz?

Yes, this needs to be considered.

I see two choices:

each sublibrary is available from it's own page;

All public sublibraries are available on the main library page

The former might be cumbersome (since making a revision of each sublibrary will need to make a revision of all of them).

For the latter one might be able to use cabal haddock-project --hackage maybe with some modifications to write a directory containing haddocks of all sublibraries which can be served by hackage. This might be not only more ergonomic but also the simplest way to support public sublibraries on hackage.

desclaimer: I have never contributed to hackage so please take my word with a grain of salt 😄.

^ ping @gbaz

The former might be cumbersome (since making a revision of each sublibrary will need to make a revision of all of them).

What do you mean by "revision" in this sentence? A cabal file revision on Hackage is a package-level thing so it would affect all package's libraries.

I mean package revisions Hackage. I was considering if a sublibrary is exposed on its own, e.g. hackage.com/package/mypackage-sublibrary while the main library is served from hackage.com/package/mypackage.

But if we publish sublibraries as part of the package, e.g. hackage.com/package/mypackage/sublibrary then it will be more consistent. Also finding sublibraries of a package will be simpler if links to them will be rendered on the package page.

I think mypackage/sublibrary is the correct option here.

coot · 2024-04-02T21:43:32Z

@Kleidukos I added the QA Notes: and checked that it works as expected 😄. Do you know who else could review this PR?

coot · 2024-04-02T21:53:40Z

It might be worth noting that in an earlier version of this PR I experimented using package:sublibrary as path in ./haddocks, but there are two problems with it:

: is an invalid character on Windows
: when used in URLs, requires either relative or absolute paths (which can be done, even without haddock changes)

Maybe it's worth to use the : syntax as we are used to do cabal build package:sublibrary, and letting Windows users use - instead, even though it slightly complicates things. Anyway, for now, I left it outside of this PR. The drawback is that one cannot have some-package:sublib and some:package-sublib in a single project.

Haddock should know the sublibrary name rather than just the package. For sublibraries `package_name:sublib_name` is used. The name is recorded in the `.haddock` file and used when haddock creates the `index.html` file, e.g. when called by `haddock-project` command.

The main library must use the package name rather than `UnitId`, otherwise the links from other sublibraries will not work.

This commit makes haddock-project handle sublibraries. Now `haddock-project` is using `haddock` command to install package (and its sublibraries) documentation in the output directory. The commit also changes how `cabal haddock` works, changing the layout in the `dist-newstyle` folder. With this change `haddock` subcommand will install `package:sublib` component in a directory `package-sublib` under `l/sublib/doc/html/`.

coot · 2024-04-06T19:50:41Z

Ups, I forced pushed after rebasing, and then I noticed some people committed to the branch.

coot · 2024-04-06T20:05:21Z

@andreasabel are you sure you pushed to the right branch?

andreabedini · 2024-04-10T09:23:19Z

Public sublibraries are a package-level concept, but here we are changing haddock-project. Why is that?

Does cabal haddock properly support public sublibraries?

Here is my quick investigation:

λ cabal haddock
Build profile: -w ghc-9.6.4 -O1
In order, the following will be built (use -v for more details):
 - tmp-pkg-0.1.0.0 (lib) (configuration changed)
 - tmp-pkg-0.1.0.0 (lib:b) (configuration changed)
Configuring library for tmp-pkg-0.1.0.0..
Configuring library 'b' for tmp-pkg-0.1.0.0..
...
Documentation created:
/home/andrea/Scratchpad/tmp-pkg/dist-newstyle/build/x86_64-linux/ghc-9.6.4/tmp-pkg-0.1.0.0/l/b/doc/html/tmp-pkg/
Documentation created:
/home/andrea/Scratchpad/tmp-pkg/dist-newstyle/build/x86_64-linux/ghc-9.6.4/tmp-pkg-0.1.0.0/doc/html/tmp-pkg/

This seems reasonable, but

λ cabal haddock --haddock-for-hackage
Build profile: -w ghc-9.6.4 -O1
In order, the following will be built (use -v for more details):
 - tmp-pkg-0.1.0.0 (lib) (configuration changed)
 - tmp-pkg-0.1.0.0 (lib:b) (configuration changed)
Configuring library for tmp-pkg-0.1.0.0..
Configuring library 'b' for tmp-pkg-0.1.0.0..
...
Documentation created:
Documentation created:
/home/andrea/Scratchpad/tmp-pkg/dist-newstyle/build/x86_64-linux/ghc-9.6.4/tmp-pkg-0.1.0.0/l/b/doc/html/tmp-pkg-0.1.0.0-docs/,
/home/andrea/Scratchpad/tmp-pkg/dist-newstyle/build/x86_64-linux/ghc-9.6.4/tmp-pkg-0.1.0.0/doc/html/tmp-pkg-0.1.0.0-docs/,
/home/andrea/Scratchpad/tmp-pkg/dist-newstyle/build/x86_64-linux/ghc-9.6.4/tmp-pkg-0.1.0.0/l/b/doc/html/tmp-pkg-0.1.0.0-docs/tmp-pkg.txt
/home/andrea/Scratchpad/tmp-pkg/dist-newstyle/build/x86_64-linux/ghc-9.6.4/tmp-pkg-0.1.0.0/doc/html/tmp-pkg-0.1.0.0-docs/tmp-pkg.txt
Documentation tarball created:
/home/andrea/Scratchpad/tmp-pkg/dist-newstyle/tmp-pkg-0.1.0.0-docs.tar.gz
Documentation tarball created:
/home/andrea/Scratchpad/tmp-pkg/dist-newstyle/tmp-pkg-0.1.0.0-docs.tar.gz

The documentation for each component's haddock is written to tarballs with the same name, racing each other.

The above is still cabal-install at work, what about Cabal?

λ cabal act-as-setup --build-type=Simple -- configure
Configuring tmp-pkg-0.1.0.0...

~/Scratchpad/tmp-pkg
λ cabal act-as-setup --build-type=Simple -- haddock --for-hackage
Preprocessing library 'b' for tmp-pkg-0.1.0.0..
Running Haddock on library 'b' for tmp-pkg-0.1.0.0..
Warning: --source-* options are ignored when --hyperlinked-source is enabled.
   0% (  0 /  2) in 'MyLibB'
  Missing documentation for:
    Module header
    someFunc (src/MyLibB.hs:3)
Documentation created: dist/doc/html/tmp-pkg-0.1.0.0-docs/,
dist/doc/html/tmp-pkg-0.1.0.0-docs/tmp-pkg.txt
Preprocessing library for tmp-pkg-0.1.0.0..
Running Haddock on library for tmp-pkg-0.1.0.0..
Warning: --source-* options are ignored when --hyperlinked-source is enabled.
   0% (  0 /  2) in 'MyLib'
  Missing documentation for:
    Module header
    someFunc (src/MyLib.hs:3)
Documentation created: dist/doc/html/tmp-pkg-0.1.0.0-docs/,
dist/doc/html/tmp-pkg-0.1.0.0-docs/tmp-pkg.txt

This also seems to write to the same directory, overwriting each components files.

From #9852:

When cabal invokes haddock it does so by calling act-as-setup, which is spawning a new process. In particular, all the information gathered in EllaboratedConfiguredPackage is lost, and when haddock is invoked it only has access to InstalledPackageInfos. This means that we have no information whether a dependency is a local component to the project or not, which is needed to correctly pass --read-interface haddock options.

I see we have Distribution.Simple.Haddock.fromPackageDescription, does that not have the information you need?

coot · 2024-04-11T17:07:44Z

Public sublibraries are a package-level concept, but here we are changing haddock-project. Why is that?

haddock-project aims to create haddocks for all packages in a given project, including all public sublibraries.

Does cabal haddock properly support public sublibraries?

It does, but writes sublibraries to a directory which shares the same name as the main library, e.g. in your cabal haddock --for-hackage output:

/home/andrea/Scratchpad/tmp-pkg/dist-newstyle/build/x86_64-linux/ghc-9.6.4/tmp-pkg-0.1.0.0/l/b/doc/html/tmp-pkg-0.1.0.0-docs/,
/home/andrea/Scratchpad/tmp-pkg/dist-newstyle/build/x86_64-linux/ghc-9.6.4/tmp-pkg-0.1.0.0/doc/html/tmp-pkg-0.1.0.0-docs/

both components tmp-pkg:tmp-pkg and tmp-pkg:b are written to tmp-pkg-0.1.0.0-docs. The overall path differs, but this is not good enough for cabal haddock-package, for which the last directory name must be different since the prefix is the same (e.g. ./haddocks by default).

This PR changes this behaviour. When cabal haddock-package will be invoked it will write the haddocks to ./haddocks/tmp-pkg and ./haddocks/tmp-pkg-b.

The Distribution.Simple.Haddock.haddock is using:

fromPackageDescription
fromLibrary
fromExecutable

and a few other methods; fromPackageDescription is used to construct common args which are merged with args returned by the other calls when building documentation of some components of the package. I had to modify how fromLibrary works.

Mikolaj · 2024-05-01T10:22:10Z

Dear all, could you continue the discussion? It would be a shame to let this PR bitrot!

coot added cabal-install: cmd/haddock cabal-install: cmd/haddock-project labels Mar 19, 2024

coot force-pushed the coot/haddock-project-sublibs branch 4 times, most recently from f9874c3 to 898ff64 Compare March 20, 2024 19:16

coot force-pushed the coot/haddock-project-sublibs branch 2 times, most recently from 31b34fa to 0bd944e Compare March 22, 2024 09:15

coot force-pushed the coot/haddock-project-sublibs branch 2 times, most recently from 291e98b to 223839d Compare March 22, 2024 16:49

coot marked this pull request as ready for review March 22, 2024 17:27

coot force-pushed the coot/haddock-project-sublibs branch 4 times, most recently from 7ed8b05 to f13915c Compare March 23, 2024 12:40

carbolymer mentioned this pull request Mar 26, 2024

Fix haddocks references to libraries IntersectMBO/cardano-cli#445

Draft

3 tasks

coot mentioned this pull request Mar 28, 2024

haddock-project links documentation of local packages to hackage #9852

Open

coot requested a review from andreabedini March 30, 2024 10:53

coot changed the title ~~coot/haddock project sublibs~~ haddock project sublibs Apr 1, 2024

coot changed the title ~~haddock project sublibs~~ haddock-project support public sublibraries Apr 1, 2024

coot requested a review from Kleidukos April 2, 2024 19:23

Kleidukos requested changes Apr 2, 2024

View reviewed changes

coot force-pushed the coot/haddock-project-sublibs branch from f13915c to 9a5b6a7 Compare April 6, 2024 19:43

coot added 3 commits April 6, 2024 21:48

haddock-project: fixed main library name

a9775e2

The main library must use the package name rather than `UnitId`, otherwise the links from other sublibraries will not work.

Added changelog.d entry

320b408

coot force-pushed the coot/haddock-project-sublibs branch from 9a5b6a7 to 320b408 Compare April 6, 2024 19:48

Mikolaj added the attention: needs-review label May 1, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

haddock-project support public sublibraries #9821

haddock-project support public sublibraries #9821

coot commented Mar 19, 2024 •

edited

coot commented Mar 21, 2024

ulysses4ever commented Mar 22, 2024

coot commented Mar 22, 2024

coot commented Mar 22, 2024

Kleidukos left a comment

Kleidukos Apr 2, 2024

coot Apr 2, 2024

coot Apr 3, 2024

andreabedini Apr 5, 2024

coot Apr 6, 2024

andreabedini Apr 10, 2024

coot commented Apr 2, 2024 •

edited

coot commented Apr 2, 2024

coot commented Apr 6, 2024

coot commented Apr 6, 2024

andreabedini commented Apr 10, 2024

coot commented Apr 11, 2024

Mikolaj commented May 1, 2024

haddock-project support public sublibraries #9821

Are you sure you want to change the base?

haddock-project support public sublibraries #9821

Conversation

coot commented Mar 19, 2024 • edited

QA Notes

coot commented Mar 21, 2024

ulysses4ever commented Mar 22, 2024

coot commented Mar 22, 2024

coot commented Mar 22, 2024

Kleidukos left a comment

Choose a reason for hiding this comment

Kleidukos Apr 2, 2024

Choose a reason for hiding this comment

coot Apr 2, 2024

Choose a reason for hiding this comment

coot Apr 3, 2024

Choose a reason for hiding this comment

andreabedini Apr 5, 2024

Choose a reason for hiding this comment

coot Apr 6, 2024

Choose a reason for hiding this comment

andreabedini Apr 10, 2024

Choose a reason for hiding this comment

coot commented Apr 2, 2024 • edited

coot commented Apr 2, 2024

coot commented Apr 6, 2024

coot commented Apr 6, 2024

andreabedini commented Apr 10, 2024

coot commented Apr 11, 2024

Mikolaj commented May 1, 2024

coot commented Mar 19, 2024 •

edited

coot commented Apr 2, 2024 •

edited