Paludis 0.32.1 Released

Paludis 0.32.1 has been released:

• VDB entries not containing an IUSE file would give incorrect behaviour. This is now fixed.

Paludis 0.32.0 Released

Paludis 0.32.0 has been released:

• accerso now handles unmirrorable files sensibly.
• Sets would sometimes not end up being added to world. This is now fixed.

Recursive Lambdas in Ruby using Object#tap

Paludis represents an ebuild’s homepage as a dependency-style heirarchy, since PMS allows use-conditional blocks like:

HOMEPAGE="http://example.org/foo gtk? ( http://example.org/foo-gtk )"

Given this, we want a quick way of extracting the URLs using the Ruby bindings. One could of course use a function:

def extract_homepage_recursively(spec)
    case spec
    when AllDepSpec, ConditionalDepSpec
        spec.each { | child | extract_homepage_recursively(child) }
    when SimpleURIDepSpec
        puts spec
    end
end

extract_homepage_recursively(id.homepage_key.value) if id.homepage_key

But that’s rather crude. It would be much nicer to use a lambda, since we don’t need a new name for something we’re only using once.

Unfortunately, recursive lambdas are sometimes rather pesky. There are various solutions, most of which involve passing the lambda as a parameter to itself. In the general case, there’s the infamous Y combinator, but since Ruby has language-level recognition for recursion there’s no need to resort to that kind of silliness. We could just use a variable:

recurse = lambda do | recurse, spec |
    case spec
    when AllDepSpec, ConditionalDepSpec
        spec.each { | child | recurse.call(recurse, child) }
    when SimpleURIDepSpec
        puts spec
    end
end

recurse.call(recurse, id.homepage_key.value) if id.homepage_key

But that’s still a pointless waste of a name. We can do better than that.

Ruby 1.9 adds an Object#tap method, which is rather nifty. Ruby 1.8 doesn’t have it, but we can provide it easily:

if not Object.respond_to? :tap
    class Object
        def tap
            yield self
            self
        end
    end
end

Then, we don’t need a variable or a horrid untyped lambda calculus construct at all:

lambda do | recurse, spec |
    case spec
    when AllDepSpec, ConditionalDepSpec
        spec.each { | child | recurse.call(recurse, child) }
    when SimpleURIDepSpec
        puts spec
    end
end.tap { | r | r.call(r, id.homepage_key.value) } if id.homepage_key

Paludis 0.32.0_alpha1 Released

Paludis 0.32.0_alpha1 has been released:

• Support for packages that haven’t been written yet.
• --debug-build and --checks are gone, replaced by the special build_options: choice that can be configured in a similar way to use flags.
• Clients using NoConfigEnvironment now use --extra-repository-dir (possibly multiple times) and --master-repository-name rather than --master-repository-dir.
• The contrarius client has been removed.
• metadata.xml support.

Why must all XML APIs suck?

Given the following:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE pkgmetadata SYSTEM "http://www.gentoo.org/dtd/metadata.dtd">
<pkgmetadata>
    <herd>blah</herd>
    <herd>foo</herd>
    <maintainer>
        <email>foo@bar</email>
        <name>Foo Bar</name>
    </maintainer>
    <maintainer>
        <email>bar@baz</email>
    </maintainer>
    <use>
        <flag name="foo">Adds support for foo. Needs <pkg>cat/fooplugin</pkg> to be useful.</flag>
        <flag name="bar">Adds support for bar.</flag>
    </use>
    <longdescription><![CDATA[
        A giant space monkey has eaten my shorts.
        ]]></longdescription>
    <longdescription lang="fr"><![CDATA[
        Un singe géant de l'espace a mangé mes shorts.
        ]]></longdescription>
</pkgmetadata>

I want the following:

• A set of strings called herds, containing blah and foo.
• A set of pair(string, string) called maintainers, containing ("foo@bar", "Foo Bar") and ("bar@baz", "").
• A map from string to string called use, containing ("foo" => "Adds support for foo. Needs cat/fooplugin to be useful.") and ("bar" => "Adds support for bar").
• A string called longdescription containing "A giant space monkey has eaten my shorts.".

What’s the least painful way of doing it? Why can’t there be a solution concise enough to fit into a comment? Why must XML blow so many goats?

MYOPTIONS: It’s like IUSE with Candy

The exheres-0 package format, used primarily by Exherbo, calls what Gentoo calls ‘USE flags’ ‘options’. What PMS EAPIs call IUSE, exheres-0 calls MYOPTIONS.

Up until recently, this has just been a differently named variable, minus support for IUSE defaults because we hateses them. But now that Paludis has the choices API we’re not stuck using that format. The first extension is fairly simple:

MYOPTIONS="foo bar baz linguas: en en_GB fr"

This is much nicer than having to write out linguas:en linguas:en_GB etc., and is especially important for exheres-0 because SUBOPTIONS (USE_EXPAND) values must be explicitly listed.

The next step is flag descriptions. use.local.desc is rather crude, and XML is horrible, so we thought about re-using annotations:

MYOPTIONS="
    X [[ description = [ Build a graphical user interface ] ]]
    python [[ description = [ Build Python language bindings ] ]]
    nls
    linguas: en en_GB fr"

Any undescribed flag falls back to the global description — general consensus is to keep those, because they make it easier for a user to set up a fresh options.conf.

Whilst we’re at it, we might as well solve the conflicting options problem. In the good old days, use flags were used only when something was optional — that is, if support for foo also needed support for bar, USE="-foo bar" would just compile without bar. Unfortunately, a few people didn’t really like that, and even more unfortunately developers started doing horrible die calls in pkg_setup rather than coming up with a proper solution.

With half of the pkg_setup die calls eliminated by use dependencies, it seems a shame not to fix the other not-quite-half-because-of-a-few-obscure-things. There’s already pkg_pretend for exheres-0, which is a big improvement, but moving handling of the common cases into the package manager is cleaner.

So, we start with the simplest case: flags requiring other flags.

MYOPTIONS="
        X
        python
        gtk [[ requires = [ X python ] ]]
        qt [[ requires = X ]]
        motif [[ requires = X ]]
        "

We might like SUBOPTIONS and negatives too:

MYOPTIONS="
        minimal
        python [[ requires = [ -minimal ] ]]
        linguas:
            en
            en_GB [[ requires = [ linguas: en ] ]]
        "

There might be a case for “if blah is not enabled then …” requirements:

MYOPTIONS="
        X
        -ncurses [[ requires = [ X ] ]]
        "

although we have a nicer solution this particular case. Note that it’s ok to list the same flag multiple times, so the above can be written as:

MYOPTIONS="
        X
        ncurses
        -ncurses [[ requires = [ X ] ]]
        "

For convenience, we’d like to be able to apply the same requires annotation to multiple items:

MYOPTIONS="
        X
        python
        (
            gtk [[ requires = python ]]
            qt
            motif
        ) [[ requires = X ]]
        "

Here, gtk requires both X and python (although excessive mixing of things is discouraged for style reasons).

Sometimes, you have to select one of a number of flags:

MYOPTIONS="
        (
            gtk
            qt
            motif
        ) [[ number-selected = exactly-one ]]
        "

Also allowed are at-least-one and at-most-one.

Sometimes requirements are conditional, too:

MYOPTIONS="
        X
        python
        X? (
            (
                gtk [[ requires = python ]]
                qt
                motif
            ) [[
                number-selected = exactly-one
                requires = X
            ]]
        )
        "

Although, for style reasons, this would end up looking more like:

MYOPTIONS="
        X [[ description = [ Include a GUI ] ]]
        python [[ description = [ Build Python language bindings ] ]]
        gtk
        qt
        motif

        gtk [[ requires = python ]]
        ( gtk qt motif ) [[ requires = X ]]
        X? ( ( gtk qt motif ) [[ number-selected = exactly-one ]] )
        "

As for how these are verified… They’re checked at --pretend --install time, right before pkg_pretend is run. Even if a requirement fails, though, pkg_pretend is still run, allowing us to show as many notices as necessary at the same time.

The failure is indicated to the user by the pkg_bad_options function. This probably won’t be overridden by very many packages, and those that do will almost certainly call default. The default output looks like this:

These packages will be installed:

* test-cat/test-pkg::ciaranm :1 [N 1] <target>
    X gtk -motif -python qt build_options: recommended_tests split strip
    "Dummy test package"

Total: 1 package (1 new)

 * The following option requirements are unmet for test-cat/test-pkg-1:
 *     Enabling option 'gtk' requires option 'python' to be enabled
 *     Exactly one of options ( gtk, qt, motif ) must be met

* Cannot continue with install due to the errors indicated above

And just think, all that without resorting to convoluted and incomprehensible set theory.