Today In PMS FUD: PMS Shot JFK

Patrick Lauer is complaining that one of his patches to PMS was rejected. What he is not telling you is why it was rejected, so I shall explain here:

The version of bash to be required by ebuilds was decided by the Council, as was the specification defining it. The PMS project quite rightly doesn’t have the authority to override a Council decision. Patrick was told this, and was told that the process for getting EAPI definitions retroactively changed was to go to the gentoo-dev list and from there to the Council. Such changes have been made in the past, and they’re open to being made again in the future.

Unfortunately it looks like he’s not interested in getting his concern fixed, and is instead just trying to cause trouble.

Funtoo’s Website is Designed for Internet Explorer

Taken from the Funtoo FAQ:

“Is Mozilla Firefox compatible with funtoo.org?”

Mozilla Firefox does not appear to be compatible with the Funtoo.org website, unfortunately. It does not support the insertAdjacentHTML function, and also does not support XML data islands. These features are supported by Internet Explorer and we use both of these features. I think that insertAdjacentHTML is a good idea, and I also think it’s a good idea for the browser to support XML data islands, which are particularly useful for formatting text. Funtoo.org uses its own documentation management method, and these documents are easier to maintain if insertAdjacentHTML is correctly supported by the browser, and XML data islands are a great help to us as well.

We could maintain patched versions of Mozilla Firefox that would be usable with the Funtoo.org website, but I don’t want to get into the business of supporting a non-standard browser as the upstream project does not seem very supportive of getting these compatibility issues resolved, and thus this could turn into a maintenance burden for Funtoo. So I’d rather just not support Mozilla Firefox for now.

I guess I expected open source projects to be more widely aware of the importance of having open standards, coding to a standard rather than an implementation and driving innovation through new, public standards rather than abusing undocumented proprietary extensions.

Update: This post seems to have been exposed to a wider audience than my usual readership. For those who don’t feel like reading the comments, and who don’t understand what’s being discussed here, you can compare the quote above to the Funtoo FAQ entry introduced by this commit (unfortunately there’s no way to link to individual Funtoo FAQ entries).

What’s Happened to EAPI 3?

EAPI 3 was effectively finished three months ago; some of you are no doubt wondering what happened to it and why there hasn’t been any news.

EAPI 3’s approval was conditional upon Portage support. This isn’t unreasonable, at least for main tree usage, although it’s arguably less relevant for overlays. Unfortunately, Portage is suffering from a severe lack of maintainers.

You can track Portage’s EAPI 3 implementation progress via Gentoo’s bugzilla. As you can see, some progress has been made, but it’s slow going, and hasn’t met the “we’ll be ready within a month” goal given at the Council meeting a month ago.

In the mean time, Paludis 0.38.0 has EAPI 3 support present but not enabled at install time (unlike Portage, we don’t hardcode EAPI numbers into the source, so supporting a feature doesn’t involve any kind of treatment for the EAPI that provides it). Anyone wishing to play with EAPI 3 support for personal use can just copy the EAPI 3 definition file from the source tree.

It’s a shame that Gentoo has to hold back on delivering a better user experience and making good ebuilds easier to write because of Portage. I’d like to encourage anyone who can handle Portage’s codebase to give Gentoo the help it so badly needs.

As for future EAPIs… The current Council has expressed an interest in handing off the EAPI approval process to a separate group. This is either good news if that group is going to put in the work necessary to get things done and deliver practical results, or terrible news if that group is going to perpetuate the worst habits of the previous Council.

New Gentoo EAPI Things: Secure Working Directories

This is the third post in a series about new Gentoo EAPI things.

Consider an ebuild that does something like this:

pkg_setup() {
    python -m foo -c bar
}

Here, python will first look in the current working directory for the foo module. If the superuser calling the package manager is doing so from a directory writeable by a malicious user, that user can create a foo.py in that directory that does something naughty, and it will be executed with the privileges of the superuser.

Historically, it has been the ebuild’s responsibility to work around this kind of behaviour. This is tricky, however, and most ebuilds have never bothered — most developers aren’t aware of this kind of attack, and may not even realise that their code eventually calls some program that has this behaviour (Python isn’t the only culprit). Thus, it was decided in Gentoo bug 239560 to make it the package manager’s responsibility to ensure a trusted working directory for any ebuild calls.

New Gentoo EAPI Things: Profile eapi Files

This is the second post in a series about new Gentoo EAPI things.

EAPIs as originally conceived applied only to individual package versions. However, EAPI dependent things like package dependency specifications show up in other places too, such as in various profiles files. PMS previously restricted such specifications to EAPI 0 rules, since that was all that could safely be used, but this means no slot dependencies.

A better option would be to let profile files specify which EAPI’s rules to use when parsing the files. This is what the newly-defined profile eapi files are for. Any directory under profiles/, including profiles/ itself, may now contain a file named eapi, consisting of a single line with an EAPI name in it. If present, rather than using EAPI 0 rules for files in that directory the package manager shall use that EAPI. If a package manager doesn’t support that EAPI, it must refuse to use the profile in question — or, for the top level profiles/ directory, the entire repository.

Since this is a new introduction, people must still wait until compliant package managers are ‘universally’ available (that is to say, in release stages) before starting to use EAPI 1 features elsewhere. But this is now a future possibility, rather than something that can’t happen at all.

Incidentally, you might think you’ve thought of something clever you could do with profiles if only you could use use dependencies there. Your clever idea has already occurred to others, and it doesn’t work. There is no legitimate use for any of EAPI 2’s new features in profiles.

Paludis has profile eapi file support in the next release (0.32.4 or 0.34.0). Portage doesn’t yet support this.

New Gentoo EAPI Things: DEFINED_PHASES

This is the first post in a series about new Gentoo EAPI things.

DEFINED_PHASES is a new magic metadata variable that we’ve retroactively added to all existing EAPIs. As such, package managers can’t assume that it’s supported, but that’s not really a problem. DEFINED_PHASES is purely for package manager use; it may or may not be exported in the environment to ebuilds, and ebuilds mustn’t touch it.

If a package manager supports it, at metadata generation time, the package manager will generate a list of phase names whose phase functions are defined by an ebuild (or an eclass inherited by that ebuild). The DEFINED_PHASES metadata cache key contains this list, space separated and in no particular order. For example, the current value for app-editors/vim-7.2.021 would be (possibly not in this order):

DEFINED_PHASES="setup unpack compile install test postinst postrm"

In this case, all of these functions come from eclasses.

As a special rule, if an ebuild defines no phase functions (for Gentoo, this is mostly just metapackages and new style virtuals, since the default src_install is useless), the value is a single hyphen rather than an empty string. This lets package managers tell the difference between “cache generated by an old package manager” and “no phases defined”.

Note that even the name DEFINED_PHASES is largely meaningless. The current cache format is a flat list, rather than a key=value format, so DEFINED_PHASES doesn’t show up somewhere. It has a name because at some point the switch to a key=value format cache might be made, and to avoid any future name collisions in ebuilds which might otherwise use DEFINED_PHASES as a variable.

So what’s the point?

With current 0-based EAPIs, DEFINED_PHASES has a few uses:

• It can be used to save a bit of time during the build process by ignoring phases that have an empty default implementation and no ebuild implementation, but this isn’t really a huge deal. Package managers could get this benefit anyway by regenerating metadata locally up-front.
• For interactive packages, it lets the package manager avoid having to go into ‘interactive’ mode for most phases.
• It lets the package manager parallelise pkg_ phases if there’s no implementation. Normally pkg_ phases have to be run exclusively because they can modify the live filesystem.

DEFINED_PHASES really comes into its own with various things that will probably be in future EAPIs (some of which are already in exheres-0):

• The proposed pkg_pretend function, which will be run at pretend-install time, would require about 0.1s per package to be changed. When doing a large upgrade, this can result in the user having to sit there waiting for tens of seconds whilst the package manager runs lots of mostly useless ebuild phase functions.
• Similarly, src_fetch_extra (to handle fetching scm things, rather than using src_unpack) shouldn’t be run except where necessary.

It might be possible to use this to get pkg_nofetch information upfront at pretend-install time too. I’m still thinking about that.

Paludis has had DEFINED_PHASES support since 0.32.2, although no cache line had been standardised so it was only used for self-generated caches. The next release (0.32.4 or 0.34.0) will use the agreed-upon line. Portage doesn’t yet support this, so Gentoo-generated caches won’t include a value yet anyway.

What’s New in Gentoo EAPIs?

Three moderately interesting new things have been approved by the Gentoo Council:

• DEFINED_PHASES
• Profile eapi files
• Secure working directories

Full details can be found in a series of patches which will be merged to PMS shortly; an overview of each will appear in subsequent posts.