There’s something horribly wrong with Gentoo

After a couple of months of it being switched off, the PSU in my Gentoo box decided to let its magic smoke escape. I made the mistake of replacing it, and now I see this:

* virtual/rubygems
    ::gentoo                  1 {:ruby18} 2 {:jruby} 3* {:ree18} (4)KR {:ruby19} (5)K {:rbx}
    virtual/rubygems-3:ree18::gentoo
    Description               Virtual ebuild for rubygems
    Homepage                  
    Herds                     ruby
    Use flags                 
        ruby_targets          
            (ree18)           Build with Ruby Enterprise Edition 1.8.x

That someone thought that that was in any way a good idea pretty much sums up everything that’s wrong with Gentoo. I’m seriously considering just abandoning Gentoo support now — there comes a point when the accumulated bad design decisions would cost more to fix than the total worth of the product, and this plus the Python eclass are suggesting to me that Gentoo has passed that point.

Why GLEP 55 is a Good Idea and thus a Bad Thing

As it seems that ill-thought-out GLEP 55 bashing is back in fashion again, I thought I’d explain why GLEP 55 is a good idea, and thus a bad thing for Gentoo.

In short, GLEP 55 proposes moving the EAPI from being ordinary ebuild metadata to being encoded as part of the filename (e.g. .ebuild-4). The advantages of this are:

• It allows global scope changes that affect metadata generation to be made. Right now, it is impossible for EAPIs to add new global scope functions, and it is impossible for EAPIs to change the behaviour of existing functions, since all ebuilds must be able to be sourced and have at least the EAPI part of their metadata generated by a package manager that does not support newer EAPIs.
• It allows changes to the versioning rules. New EAPIs cannot introduce new package version formats or fix arbitrary limitations with existing formats, since as soon as an older package manager encounters what it thinks is an ill-formatted version, it will produce a noisy, user-visible warning. With GLEP 55, instead these new versions will be invisible.
• It allows ebuilds to use newer bash features in ebuilds without breaking older package managers.
• It makes a package’s EAPI consistently defined throughout the ebuild. Right now it’s legal to set EAPI deep inside a nested chain of eclasses (so long as doing so doesn’t break metadata invariance rules), which means any code encountered before EAPI is set will have a false impression of what the eventual EAPI will be.

Because it’s currently impossible to add per-package eclasses (due to the first item), ebuilds are doing all sorts of stupid things to get the same effect. Because the second item means it’s impossible to add sane versioning for SCM packages, maintainers are making do with lots of 1.2.9999 versions that don’t quite work with dependencies properly. Due to a combination of the first two, I had to write the hideous abomination that is the versionator eclass because it’s impossible to let Portage support versions like 1.23-alpha1 directly, and impossible to add package manager provided version parsing functions that can be used in global scope.

The third is a continual source of problems, as developers frequently accidentally use newer bash features, thus screwing up the upgrade path for anyone who hasn’t updated their box for a few months.

As for the fourth, it’s highly discouraged from a QA perspective these days, but people have set EAPI via an eclass in the past. This one’s more an illustration of icky design than anything else.

Because GLEP 55’s author once tried using a package manager that isn’t Portage, and is thus irrevocably tainted, some alternative ways of handling these deficiencies being proposed.

Alternative One: Repository Capabilities

First, it is suggested that repositories specify, at the repository level, their requirements (presumably via layout.conf). The implications of this are:

• We can’t start using anything new until we’re sure that everyone is using a compliant package manager. That means yet another two year wait before any of this goes anywhere.
• New capabilities can’t be used in the main tree for at least a year after their introduction, because users must be able to upgrade their package manager on a box they haven’t touched for a while.

This means that the cost of introducing a change is extremely high, and so developers will be encouraged to continue using bad solutions rather than fixing things properly.

Also, if repository capabilities replace rather than supplement EAPIs, as some have suggested, then it becomes impossible to change a repository’s capabilities without rewriting every single ebuild to the new standard. For example, one could not turn on a strict-s-checking feature without first checking every single existing package and fixing any that rely upon the older “S doesn’t have to exist” behaviour. Certain developers have proposed branching the tree once a year and rewriting everything to do this; quite how they plan to find the manpower to pull this off has yet to be answered.

Thus, assuming a mass rewrite is out of the question, repository capabilities must be combined with another solution to be of practical use, which brings us to:

Alternative Two: Magic Markers

Second, it is suggested that ebuilds include some kind of magic marker to allow their EAPI to be determined without going through the usual sourcing process. This does absolutely nothing to fix the version formats problem, and so must be combined with repository capabilities anyway. On top of that, none of the proposed magic marker methods are particularly pleasant.

Magic via Fixed EAPI Strings and Parsing Not Using Bash

The first kind of magic marker proposal is to require that ebuilds specify the EAPI string in a particular, fixed format. Proposals are typically along the lines of “it must be specified within the first N lines, and it must be exactly in the form EAPI=X”. So, depending upon the exact wording chosen, none of the following would be legal:

# Copyright 1999-2010 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $

EAPI='4'

(Single quotes might not be legal)

# Copyright 1999-2010 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $

EAPI="4"

(Nor double quotes)

# Copyright 2010 Joe Bloggs
# Derived from foo-1.23.ebuild from Gentoo, which is:
#     Copyright 1999-2010 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $

EAPI=4

(Too many comment lines at the start)

# Copyright 1999-2010 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $

    EAPI=4

(Indenting isn’t allowed)

# Copyright 1999-2010 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $

inherit eclass-that-sets-eapi

(Eclasses can set every metadata variable except EAPI)

# Copyright 1999-2010 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $

SLOT=1
EAPI=4

(EAPI must be first)

To make matters worse, this exception would only apply to the EAPI variable, even though in every other way EAPI would remain a normal metadata variable. This restriction would not necessarily be enforceable by the package manager either, so sometimes screwups would lead to weird results rather than errors.

Also, existing ebuilds don’t follow this format — all of the above examples have occurred in real code. Thus, before switching to this, the entire tree and every single overlay everywhere would have to be fixed.

Magic via Special Comment Strings

The second magic marker variant is to shove a special comment line at the start of every single ebuild containing EAPI information. Again, this would require updating every single existing ebuild, and would also require yet another huge wait before it becomes usable.

This sort of resembles what’s done in certain fixed binary formats. However, ebuilds aren’t fixed binary formats; ebuilds are scripts. In addition, fixed binary formats that do this were designed from the start with different versions in mind; the ebuild format has instead been gradually grown over time, often by people who actively oppose any kind of standardisation.

Magic via Extended Attributes

The third magic marker variant is to store the marker in extended attributes, rather than in the file itself. This means:

• It becomes impossible to store ebuilds anywhere that doesn’t support extended attributes, such as in most version control systems, on pastebins, on bugzilla and on pastebins.
• The EAPI is effectively invisible except to people using obscure tools.

And, again, a huge wait.

Alternative Three: eapi Function

Third, make eapi a function. This will allow early exit for unsupported EAPIs, so global scope changes can be made so long as they occur after the eapi function is called. Again, this means a huge wait, and again, this requires an additional proposal to fix version formats.

Why none of this matters anyway

In other words, the choices come down to:

• Put the EAPI in the filename. Be able to use new features straight away, including in the main repository.
• Introduce repository capability support to package managers. Wait a year or more. Every time you want to use a new capability, release package manager support for it, then wait a year before you can use the new feature. In addition, either rewrite the entire tree every time you change capabilities, or also introduce a second solution.
• Introduce some kind of magic marker. Rewrite every existing ebuild, including the ones in overlays, to use this marker. Wait a year. Still be unable to change version format rules, unless a second solution is also introduced.
• Move eapi to be a function. Wait a year. Still be unable to change version format rules, etc.

Or, of course, the usual Gentoo solution can be applied: do nothing. Continue to be unable to support per-package eclasses, and continue adding extensive convoluted hacks into ebuilds to try sourcing things in FILESDIR to get the same result. Continue to use silly 9999 version numbers, which can’t be used in dependencies correctly, to support version branches of scm packages. Continue to arbitrarily not support certain upstream version formats just because they use -alpha instead of _alpha, and require ebuilds to carry on doing transformations to deal with this. Continue to reject proposals based upon the author rather than technical merit.

It’s obvious that doing nothing is the way best suited to Gentoo’s needs. Gentoo’s main priority is to avoid making any change that is in any way controversial or that could in any way be associated with the wrong people. Delivering a better product to users or making things easier for developers is irrelevant. Thus, maintaining the status quo whilst vaguely mentioning alternatives that can’t and won’t ever happen is the way forward. If things get desperate, and it starts to become impossible to fire any developer who asks for better tools, then at that point one of the “wait several years” solutions can be pulled out as proof that “something is being done”.

At this point, all that switching to a solution that would enable changes to be made would do is illustrate that Gentoo can’t deliver even the simplest of the changes developers need. By repeatedly hovering around proposals that will require long waits before their introduction, Gentoo has a plausible-looking excuse to mask an almost complete lack of progress in Portage; if GLEP 55 were to be introduced, developers might start asking why they still don’t have per-package eclasses, less arbitrarily restricted version numbers, package manager provided version parsing helpers or the ability to use newer bash features in ebuilds.

Paludis is Going Into a Cave

The following applies to Gentoo, not Exherbo. Exherbo developers (Exherbo has no users) already know what’s going on there. I figure it’s worth having a clear source of information on this for Gentoo users, though, rather than making people rely upon rumours and third hand transcriptions of what’s been said on IRC.

The original Paludis client was more or less designed to be a less perverse version of emerge, with additional options for things like querying. On the plus side, it made things slightly easier for Gentoo users to pick up. However:

• Users would be confused as to why things like --show-use-descriptions, which is an option for the --install action, would not also do something with --query. Funnily enough, users don’t seem to wonder why emerge --sync --changelog --pretend doesn’t work.
• We had to be extremely careful allocating short options. Thus, we only allocated short options when we were sure an option would regularly be used both enabled and disabled, since otherwise it could just go in PALUDIS_OPTIONS. This worked well enough for users who were paying attention, but seemed to scare off certain people who jumped in without understanding what they were doing.
• It meant we regularly had to decide between breaking familiarity or doing something dumb just to make Portage converts comfortable.

I’ve never especially liked the result. It’s not close enough to emerge to make learning zero cost, and it’s too close to emerge to be a pleasant user interface. Fortunately, the UI and the library are mostly nicely separated, so we can fix this. The plan is as follows:

• A new client, named cave, will be provided. All pretence of feeling anything like emerge has been abandoned for cave. Instead, the interface is roughly based upon git and similar tools.
• Depending upon whether or not we can be bothered, a second new client named egress may or may not be produced. If it is, it will be command-line-compatible with emerge, and with a reasonably similar output format (within reason). I’m not convinced something like this would be worth the effort to write, although there are some Gentoo developers who insist that something that behaves almost identically to emerge is a critical requirement for any official Gentoo package manager…
• The paludis client will be deprecated and phased out.

To make things simpler, we’ve also decided to use the introduction of cave to switch to the shiny new resolver. The shiny new resolver is a lot more flexible, a lot more powerful and a heck of a lot easier to maintain and modify than the old resolver. However, making the paludis client use it would be quite a bit of work, so we’re not going to backport it.

There is no specific timeframe for any of this, and no estimates will be provided. There are, however, lists of things that need to be done.

• The Basic Functionality milestone contains all of the things that have to be done before we consider cave to be ‘more or less usable’. Until this milestone is complete, we won’t be enabling cave in any of the ebuilds on Gentoo, and we don’t recommend users sneakily enabling it themselves. Note in particular that some of the things not yet implemented include “displaying a user-readable error message rather than a big fat “UI not implemented!” when attempting an unsafe package uninstall, and “handle virtual blockers”.
• The Useful Functionality milestone contains all of the things that have to be done before we remove the paludis client. We will be making cave easily available before it includes every bit of functionality present in paludis, but we won’t be forcing a switch until we’ve either implemented equivalents for every feature or decided we’re not going to support a particular feature at all.
• The Long Term Extras milestone contains things we’ll be doing at some point.

These milestones are probably incomplete, and are definitely open to arbitrary additions, removals, changes and being ignored based upon developer whims, available development time, bribery, patches, lack of interest, features looking fun or boring to code and the phase of the moon.

I realise this is probably futile, but I should stress that users should not attempt to enable or use cave on Gentoo until we pass the Basic Functionality milestone. This isn’t taking away your fun — some of the not yet completed Basic Functionality items are necessary for basic usability and correct operation on Gentoo.

Paludis kdebuild-1 Removal

kdebuild-1 was an EAPI used by the Gentoo KDE team and the Genkdesvn project for various live ebuilds in overlays. It was created following a request from the Gentoo KDE project, who needed certain features that Portage had not yet implemented. It was a documented, public standard, in the same way that EAPIs 0, 1 and 2 are.

The Gentoo Council has decided that all mention of the kdebuild-1 EAPI is to be removed from the Package Manager Specification immediately. A request that this be done only after a deprecation period to give users more time to catch up was rejected. As such, support for this EAPI, including support for uninstalling packages with that EAPI, will be removed from Paludis in version 0.44.

Users should verify that they have no kdebuild-1 packages installed. If Paludis was built with the ‘inquisitio’ use flag enabled, you can use:

$ inquisitio -K installed -k EAPI kdebuild-1

Otherwise, use:

$ paludis -q '*/*::/[.EAPI=kdebuild-1]'

and, if any such packages are installed, they must be uninstalled before Paludis is upgraded.

Good News Everyone!

Good news everyone! Portage no longer corrupts mtimes when merging, which hopefully means the Council won’t force us to screw up EAPI 3 any more. Now Portage will either exactly preserve mtimes to the degree supported by the filesystem, or drop the nanosecond part to 0 (which some filesystems do anyway, so programs have to be aware of that possibility), which is what the sane people have been after all along.

Next up: see how long it takes everyone to see the light and realise that the package mangler should only be preserving mtimes that are after the start of the build process, so you don’t end up with lots of files with mtimes in the past getting merged.

EAPI 3 to Specify “Auto Space like Word 95”

When Microsoft was trying to get OOXML standardised, it received all kinds of flak for including a feature described as “auto space like Word 95”. Unfortunately, the Gentoo Council has just done the same thing for EAPI 3.

Historically, Portage would always clobber a package’s mtimes when merging. This wasn’t really a problem, because nothing relied upon it. Paludis carefully copied this behaviour.

Later on, Portage was changed to more or less preserve file mtimes, but it would sometimes corrupt those mtimes thanks to deficiencies in Python. This is where things started to get messy.

People began writing ebuilds that relied upon mtimes being preserved exactly. Because corruption of the seconds part of mtimes was relatively rare, and because most programs don’t yet make use of nanosecond-resolution timestamps (which were introduced in POSIX 2008), and because invalid bytecode-compiled files are usually silently ignored, this would usually work most of the time, except when it didn’t, or when people were using older Portage versions, or when people were using Paludis.

To solve this, mtime preservation was proposed for EAPI 3. Unfortunately, what ended up being voted in was to “make PMS specify exactly what Portage does”. Assuming “what Portage does” refers to what Portage did at the time the Council vote was held, this means that ebuild authors should note the following:

• The seconds part of mtimes will be preserved, except if the build filesystem supports nanosecond-resolution timestamps, in which case the seconds part of mtimes may sometimes be one higher than they should be.
• The nanoseconds part of mtimes may either be preserved exactly, or set to zero, or rounded to some arbitrary number of digits, with some of those digits being wrong.
• It will now be your responsibility to ensure that packages don’t merge files with silly timestamps (such as when packages don’t use a Makefile to install things, but just cp an extracted tarball’s contents to DESTDIR).

Or, in other words, you still can’t rely upon mtime preservation, but the package manager must guarantee that your code will only sometimes break horribly, and that such breakages will be extremely difficult to reproduce consistently (unless your code uses st_mtim instead of st_mtime, as per newer POSIX, in which case it will break most times but not all of the time).

Today In PMS FUD: PMS Shot JFK

Patrick Lauer is complaining that one of his patches to PMS was rejected. What he is not telling you is why it was rejected, so I shall explain here:

The version of bash to be required by ebuilds was decided by the Council, as was the specification defining it. The PMS project quite rightly doesn’t have the authority to override a Council decision. Patrick was told this, and was told that the process for getting EAPI definitions retroactively changed was to go to the gentoo-dev list and from there to the Council. Such changes have been made in the past, and they’re open to being made again in the future.

Unfortunately it looks like he’s not interested in getting his concern fixed, and is instead just trying to cause trouble.

Ten Ways PMS Raped your Baby

Since hating PMS seems to be back in fashion again this week, I thought I’d list ten of the stupidest claims that I’ve seen of late in the hope that some of the FUD might die down:

1. PMS slows down new features and prevents innovation

Actually, once new ebuild-usable features end up in Portage, they very quickly end up in a published EAPI. The reason you can’t use all the fancy new features in EAPI 3 that you’ve all been waiting for for so long is that Portage still hasn’t implemented them. In addition, we’ve gone from “EAPI 3 will be ready for Portage within a month” three months ago to “there’s an 80% chance EAPI 3 support will be ready in Portage by the end of the year“.

Time-wise, EAPI 3 has been waiting for Portage for six months and before that, for the Council to come to decisions for two months. The total overhead imposed by PMS was around four days, and those four days weren’t holding up anything else anyway.

Still, at least there’s a long way to go before EAPI 3 takes as long as it took Portage to get use dependencies…

Similarly, PMS isn’t to blame for profiles not being able to make use of new features. People who are telling you this are probably thinking about an undocumented Portage feature that isn’t in PMS and that isn’t supported by other package managers. This feature could very easily be in PMS, but there’s been no interest from the Council in retroactively adding it to EAPI 3 or in doing an EAPI 2.1 just to include that feature. The feature almost certainly will be in EAPI 4, but work on EAPI 4 isn’t going to start until Portage is done with EAPI 3. So again, PMS isn’t the reason you can’t use it.

2. PMS or EAPI is about ebuilds, not profiles

This one’s from people who haven’t bothered to read the opening of PMS, or who haven’t been paying attention to the Council.

PMS covers ebuilds and the tree format, including things like profiles. The aim is to cover everything necessary to produce a package manager that can use ebuilds (except possibly VDB, which probably shouldn’t be necessary for ebuild support but currently is…).

EAPI is used to indicate the rules used to handle ebuilds, and also profiles following the Council accepting Zac’s proposal last year.

3. PMS imposes an Exherbo agenda upon Gentoo

Exherbo doesn’t use PMS or Gentoo EAPIs.

4. PMS imposes a Paludis agenda upon Gentoo

Again, no. There’s no feature in any EAPI that’s there because of Paludis. Every feature in EAPIs 1, 2 and 3 was either requested by a Gentoo developer or included to make things easier for Portage. To get into an EAPI, features merely have to be vetted by the Portage team and the Council.

5. PMS is only used by Paludis anyway

Nope. PMS is used by both Portage and Paludis, as well as a number of third party libraries and utilities which don’t support full package management operations (things that need to compare two versions, for example, need to use PMS), and it was also used by Pkgcore.

Saying that PMS is only of use for third party package managers is like saying that the HTTP specification is irrelevant for Internet Explorer.

6. PMS stops other distributions from doing things

Again, no. Other distributions can ignore PMS entirely. Doing so would of course be a horrible idea, as all the people who wrote websites to work on Internet Explorer 5 found out, but that’s their decision to make. A much better option would be for those distributions to roll their own derived EAPIs, which, as happened for the Gentoo KDE project’s kdebuild-1, could be added to PMS. That way they are guaranteed that any undocumented features they rely upon won’t break with the next release, as well as avoiding complaints from users who want to use a different package manager, thus avoiding the problems people who wrote Internet Explorer 5 specific code rather than following the HTML specification encountered.

7. PMS stops Gentoo from deciding its own features

PMS is run by a Gentoo developer and approval of new EAPI features is handled by the Portage team and the Gentoo council. For that matter, the PMS team has never rejected a single feature for inclusion in a future EAPI.

8. PMS introduces red tape

No, the previous Council introduced red tape, primarily because a couple of Council members refused to read any submissions more than five minutes before a meeting. Had the Council used the two weeks between meetings to read over and state their opinions on the EAPI 3 feature list, EAPI 3 would have been approved within two meetings rather than dragging on for months.

Unfortunately the current Council doesn’t seem to have improved, with at least one member showing up to a meeting having not read the agenda beforehand.

9. PMS imposed stupid ordering on metadata files

There’s a tendency amongst certain people to blame PMS for stupid or arbitrary rules. A typical example is to moan about PMS because it says that EAPI has to be on line 15 of metadata cache files. Quite how PMS is to blame for a decision that was made before PMS existed, and that was made because line 15 was the first available cache line when EAPI was introduced as a metadata key, is completely beyond me. Similarly, the rules for handling leading 0s in version numbers is Portage’s fault (although ultimately it’s Perl’s fault), as is any other format gripe you care to name.

10. PMS will stop me from using my favourite feature in configuration files

PMS doesn’t discuss user configuration at all. Handling of user configuration is left entirely to the package manager.

What’s Happened to EAPI 3?

EAPI 3 was effectively finished three months ago; some of you are no doubt wondering what happened to it and why there hasn’t been any news.

EAPI 3’s approval was conditional upon Portage support. This isn’t unreasonable, at least for main tree usage, although it’s arguably less relevant for overlays. Unfortunately, Portage is suffering from a severe lack of maintainers.

You can track Portage’s EAPI 3 implementation progress via Gentoo’s bugzilla. As you can see, some progress has been made, but it’s slow going, and hasn’t met the “we’ll be ready within a month” goal given at the Council meeting a month ago.

In the mean time, Paludis 0.38.0 has EAPI 3 support present but not enabled at install time (unlike Portage, we don’t hardcode EAPI numbers into the source, so supporting a feature doesn’t involve any kind of treatment for the EAPI that provides it). Anyone wishing to play with EAPI 3 support for personal use can just copy the EAPI 3 definition file from the source tree.

It’s a shame that Gentoo has to hold back on delivering a better user experience and making good ebuilds easier to write because of Portage. I’d like to encourage anyone who can handle Portage’s codebase to give Gentoo the help it so badly needs.

As for future EAPIs… The current Council has expressed an interest in handing off the EAPI approval process to a separate group. This is either good news if that group is going to put in the work necessary to get things done and deliver practical results, or terrible news if that group is going to perpetuate the worst habits of the previous Council.

What’s in EAPI 3?

EAPI 3’s feature list has been provisionally approved by the Gentoo Council, subject to nothing going wrong when Portage implements it. It’s not yet usable in ebuilds.

EAPI 3 is defined based upon EAPI 2. The changes are:

• pkg_pretend
• Slot operator dependencies
• Use dependency defaults
• DEFINED_PHASES and PROPERTIES are now mandatory
• New default src_install
• Controllable compression
• dodoc -r
• doins symlink support
• dohard and dosed banned
• unpack xz support
• New default econf arguments
• pkg_info changes
• Profile-controlled IUSE injection
• AA and KV removed
• Replace version variables
• The S to WORKDIR fallback is sometimes gone
• RDEPEND=DEPEND is gone
• Most utility commands now die on failure, unless prefixed with nonfatal

Full and precise definitions will be provided in PMS (and can currently be found in a draft PMS branch); an overview of each feature will appear in subsequent posts.