Ciaran McCreesh’s Blag

Now with 17% more caffeine

EAPI 3 to Specify “Auto Space like Word 95″

When Microsoft was trying to get OOXML standardised, it received all kinds of flak for including a feature described as “auto space like Word 95″. Unfortunately, the Gentoo Council has just done the same thing for EAPI 3.

Historically, Portage would always clobber a package’s mtimes when merging. This wasn’t really a problem, because nothing relied upon it. Paludis carefully copied this behaviour.

Later on, Portage was changed to more or less preserve file mtimes, but it would sometimes corrupt those mtimes thanks to deficiencies in Python. This is where things started to get messy.

People began writing ebuilds that relied upon mtimes being preserved exactly. Because corruption of the seconds part of mtimes was relatively rare, and because most programs don’t yet make use of nanosecond-resolution timestamps (which were introduced in POSIX 2008), and because invalid bytecode-compiled files are usually silently ignored, this would usually work most of the time, except when it didn’t, or when people were using older Portage versions, or when people were using Paludis.

To solve this, mtime preservation was proposed for EAPI 3. Unfortunately, what ended up being voted in was to “make PMS specify exactly what Portage does”. Assuming “what Portage does” refers to what Portage did at the time the Council vote was held, this means that ebuild authors should note the following:

  • The seconds part of mtimes will be preserved, except if the build filesystem supports nanosecond-resolution timestamps, in which case the seconds part of mtimes may sometimes be one higher than they should be.
  • The nanoseconds part of mtimes may either be preserved exactly, or set to zero, or rounded to some arbitrary number of digits, with some of those digits being wrong.
  • It will now be your responsibility to ensure that packages don’t merge files with silly timestamps (such as when packages don’t use a Makefile to install things, but just cp an extracted tarball’s contents to DESTDIR).

Or, in other words, you still can’t rely upon mtime preservation, but the package manager must guarantee that your code will only sometimes break horribly, and that such breakages will be extremely difficult to reproduce consistently (unless your code uses st_mtim instead of st_mtime, as per newer POSIX, in which case it will break most times but not all of the time).

About these ads

5 responses to “EAPI 3 to Specify “Auto Space like Word 95″

  1. Jason December 12, 2009 at 9:28 am

    Why did they not do it the right way?

    • Ciaran McCreesh December 12, 2009 at 9:53 am

      Because they were convinced that what Portage did was enough when they first voted (despite having being told how it wasn’t, but apparently purely theoretical proofs don’t count), and then because they’re too stubborn to change their minds.

      • Jason December 12, 2009 at 10:49 pm

        Is this something extremely difficult to fix in portage? As I understand from your previous post about python time stamp mangling, this seems to be a core python problem. Are there easy workarounds available to portage?

        • Ciaran McCreesh December 13, 2009 at 2:33 pm

          No, it’s not hard. All you have to do is set the nanosecond part of mtimes to 0 (which programs have to support, because not all filesystems handle nanosecond resolution timestamps). But for reasons completely lost to me, it is apparently considered better to set the nanosecond part to a value that is approximately but not exactly correct, and that may be rounded up or down, even though there’s practically no chance whatsoever that anyone would write a program that accepts that behaviour.

  2. Pingback: Good News Everyone! « Ciaran McCreesh’s Blag

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.