Skip to content

Updating Ports

Jump to bottom
PÁLI Gábor János edited this page Jul 20, 2014 · 11 revisions

Preparations

Remember to integrate with the Ports Collection before attempting to do updates to the ports in the development repository. The integration step should be done anyway, but it could be much more tedious to do after the ports have been updated.

Once it is done, have the "Haskell Porter" tool (see the hsporter repository) ready and working. In order to do so, clone all the required repositories (note that the ports repository may be already present, everything is described from scratch here).

$ git clone http://github.com/freebsd-haskell/hsporter
$ git clone http://github.com/freebsd-haskell/ports

The Tools

Despite its name, the hsporter repository accommodates two tools:

  • hsporter, which is a standalone command-line tool for creating and adding new hs- ports. See the corresponding page for more information on how to use it.

  • hsupdater, which is to update the ports. It can be run either as an non-interactive standalone tool from the command line or as an interactive tool from GHCi. Running from GHCi is recommended and detailed below.

Configuring hsupdater

Before launching hsupdater in GHCi, it has to be configured by creating a hsupdater.conf file. The skeleton of this file should be already present as hsporter/hsupdater.conf.sample, so it could be just used as a template.

Most of the configurable settings (such as dbdir for the temporary package description database, threads for setting the number of concurrent download threads, or updatesdir for storing the updated version of the ports) are just fine. That is, only portsdir should be modified. So, replace root_of_freebsd-haskell_tree with the actual path of the ports clone.

$ cd hsporter
$ cat hsupdater.conf.sample | sed 's|root_of_freebsd-haskell_tree|../ports|' > hsupdater.conf

Create the updates and db directories, hsupdater may use them later.

$ mkdir db updates

Updating with hsupdater

After everything is set, use GHCi to launch hsupdater. GHCi itself already implements a "shell-like" interface, so the interactive part of hsupdater is simply just a pack of Haskell functions that may be invoked from there.

$ ghci src-hsupdater/Main.hs

All the available commands start with the cmd prefix, so it is enough to type cmd and then press the Tab key to see a list of them. (The λ? symbol here presents the GHCi prompt.)

λ? cmd<TAB>
cmdDownloadUpdates           cmdPrintUpdateLogs
cmdFetchCabal                cmdPrintUpdates
cmdFetchLatestCabal          cmdPruneBy
cmdGetLatestHackageVersions  cmdShowPruneableBy
cmdIsVersionAllowed          cmdUpdatePortVersions
cmdPrintCabalVersions

The commands themselves do not offer much help, rather their names and their types are trying to document what they are up to. (Well, they are in Haskell, why do not use it for our advantage? :-)) For example, check the type of the UpdatePortVersions command.

λ? :type cmdUpdatePortVersions
cmdUpdatePortVersions :: IO ()

This is IO (), which means it is free to call it right away in the GHCi. All the commands are "IO actions" in the Haskell terminology, an such IO actions may be evaluated inside the GHCi directly.

Cache the Port Versions

So, first, cache the port versions with this command. This will create a portversions file that contains the current versions of the ports in the configured tree. The command may be invoked any time later when the ports in tree change.

λ? cmdUpdatePortVersions

Build the Package Description Database

Next, the db directory has to be populated with the latest version of Cabal package descriptions for each of the ported packages.

λ? cmdGetLatestHackageVersions
Initializing...
Fetching new versions...
Fetched: ALUT-2.3.0.1
Fetched: BNFC-2.6.0.3
[...]

Learn the New Updates

After the package database has been built up, there is a command for finding out which packages have new versions available.

λ? cmdPrintUpdates

This command will give a list of all potentially outdated ports and show if moving to their currently known future version may be safe or not. Based on this, the list items may have different variations:

  • "Safe" updates. Nothing special has to be done, everything is fine.

    Boolean (devel)                  0.2          ---> 0.2.1

    That is, the devel/hs-Boolean port is at version 0.2, but it could be safely updated to 0.2.1.

  • "Unsafe" updates. Some of the port cannot be reliably moved to a newer version. They may be tagged by U (for unsatisfied) or R (for restricts).

    base64-conduit (devel)           1.0.0        -/-> 1.0.0.1      (U: conduit)

    The devel/hs-base64-conduit port is at version 1.0.0 and version 1.0.0.1 is available, but it is not recommended to update, because the latest known version of conduit is not supported by base64-conduit.

    Similarly, the latest known version of conduit is not recommended as it restricts the update of base64-conduit.

    conduit (devel)                  1.0.11.1     -/-> 1.1.0.2      (R: base64-conduit)

Hence the update of conduit is considered "unsafe" because the currently available version of base64-conduit wants conduit of version greater than (or equal to) 1.0 but lesser than 1.1. (This restriction is encoded in the corresponding package description.)

Resolve Update Conflicts

There are multiple solutions to this problem.

  • Do not care about the conflicts, but patch the package descriptions in the port, therefore force that version of the package to be used. This may work in some cases, as certain packages can work with dependencies not satisfying the version requirements strictly. If the port has been already patched to work like this, the same strategy may be maintained for the later updates.

  • Try finding a future version that satisfy the version requirements manually.

    λ? cmdPrintCabalVersions "conduit"
[...] 1.0.17.1, [...] 1.1.0, 1.1.0.1, 1.1.0.2

    The PrintCabalVersions can print all the past and future versions for the given Cabal package, so an appropriate version may be selected. In the example, 1.0.17.1 appears to be a right version for conduit. The FetchCabal command may be used to step back to that version instead.

    λ? cmdFetchCabal "conduit" "1.0.17.1"
Fetching conduit-1.0.17.1...done.

    Now updating base64-conduit becomes safe.

    base64-conduit (devel)           1.0.0        ---> 1.0.0.1

  • Try finding a configuration of versions without conflicts with the aid of the tool. The ShowPruneableBy and the PruneBy commands can respectively show and prune the updates along the version restrictions automatically.

    λ? cmdShowPruneableBy allPruneableUpdates
[...]
conduit-1.0.11.1
[...]
λ? cmdPruneBy allPruneableUpdates 
Initializing...
[...]
Pruned: conduit-1.0.11.1
[...]

    This may be repeated multiple times in order to reach a configuration where there are no update conflicts at all. Note that the final configuration may not contain any updates as a potential result. Thus the automatic pruning of updates gives a more or the less reliable but also a very conservative recommendation on how to update the ports.

  • Experiment with finding the latest and greatest but non-conflicting updates, with some automated help. In this case, the PruneBy command should be used with the pruneableFor function instead. There, a package and its version should be specified and the tool prunes the versions of all the dependencies. This is especially handy for ports with a numerous dependencies.

    For example, the updates for the latest version of yesod-platform can be ironed out in a single moment.

    λ? cmdPruneBy (pruneableFor "yesod-platform" "1.2.10")

Downloading Port Updates

Once the future port versions have been set, hsporter could be used to download the updated versions and generate the corresponding FreeBSD port files for them.

λ? cmdDownloadUpdates
Update starts.
Downloaded: base64-conduit (devel) (1.0.0 -> 1.0.0.1)
[...]
Update finished.

The resulting files are placed to the updates directory, configured earlier. Note that they are not ready to be used directly, although they could be used for doing the updates, as it will be shown below. The GHCi session is over for now, let us quit.

λ? :quit
cd ..

Merging the Updates

As the next step, the updates shall be merged back to the development repository. For doing so, the merge-trees-sdiff.sh script can be used from the hsmtk repository.

$ git clone http://github.com/freebsd-haskell/hsmtk

Note that this script can be also useful for integrating with the Ports Collection. Actually, a similar action is needed, but the direction is the reverse this time. That is why the PORTSDIR environment variable has to be set accordingly.

$ cd ports
$ env PORTSDIR=../hsporter/updates sh ../hsmtk/merge-trees-sdiff.sh

Be careful with the merges, though. The hsporter tool will not always be able to provide complete conversion of Cabal packages, some reviewing and manual post-processing is often recommended. But the combination of these two tools can greatly reduce the complexity of the updates.

Committing the Updates

After the updates have been merged into the ports tree, use git to commit each of the ports updates individually. This is highly recommended in order to keep the log clear. Thanks to git, the updates will be atomic as result of pushing anyway.

Before committing, it is recommended to check if the distinfo and the dependencies are correct, together with checking the changes to be committed.

# make checksum -C devel/hs-base64-conduit
$ git diff devel/hs-base64-conduit

If everything looks all right, commit the changes.

$ git commit -m "devel/hs-base64-conduit: 1.0.0 --> 1.0.0.1" devel/hs-base64-conduit

Although, note that, with the help of hsupdater, the commit logs could be conveniently generated from the list of updates. They could be then copied and pasted to the command line when committing the updates with git.

λ? cmdPrintUpdateLogs
devel/hs-base64-conduit: 1.0.0 --> 1.0.0.1
[...]
Clone this wiki locally