Solving the server / headers / libs USE flag problem

There’s been some discussion on the paludis-user mailing list about embedded package management. This got me thinking about the server / headers / libs problem, which is roughly as follows:

In some situations, we’d like to be able to split a package up into different ‘parts’. A part is something like ‘headers’, ‘libraries’, ‘server’, ‘client’, ‘binaries’ and ‘docs’. Some distributions do this by having separate packages. This is nasty:

• It means either building gcc five times or introducing new hacks to make multiple packages from a single build.
• It hides information from the package manager.
• It means the user has to have lots of packages installed to represent one real ‘package’. This in turn leads to complications like “if I uninstall foo-libs, should foo-headers go automatically too?”.
• It’s a hack introduced to get around package manager limitations.

Use flags for parts, as use flags are currently, aren’t the solution either, since packages need to be able to depend upon “foo with at least libs and headers enabled”. Theoretically USE dependencies (which are in kdebuild-1 and hopefully future 0-derived EAPIs) solve this. But it’s a mess — every package DEPENDing upon foo would have to do DEPEND="blah/foo[libs][headers]" RDEPEND="blah/foo[libs]". This doesn’t scale well.

I used to think that solving this required package manager knowledge of something we could call ‘parts’, which aren’t use flags. But it occurs to me that parts really are just use flags, with a few extra requirements:

• Enabling some parts might require enabling others, and enabling no parts at all might not make sense. So we need some way of restricting valid combinations of USE flags.
• Dependencies upon packages with parts need to be simpler. Rather than saying “I need foo with this part and foo with that part at DEPEND time, and this part at RDEPEND time”, we’d like to be able to say “I need foo” and only specify parts in special cases.
• The package has to be able to do the splitting and partial builds as appropriate or possible. This one needs per-ebuild support, since packages vary wildly in how they support partial builds.

The first bullet point is something we need to solve for future EAPIs anyway. The second is more interesting, and I think there might be a fairly simple solution…

One thing we’ve been playing with for consideration for future EAPIs or exheres is tweaking IUSE (or MYOPTIONS for exheres) syntax somewhat:

IUSE="foo bar linguas: en fr userland: gnu"

instead of:

IUSE="foo bar linguas_en linguas_fr userland_gnu"

This gets especially important when one learns that there is talk of removing the “expanded values don’t need to go in IUSE” special case that current EAPIs have.

When thinking about implementing this for Paludis, I started considering whether anything else useful could go in IUSE. And it seems to me that solving the parts problem could be as simple as this:

IUSE="foo bar libs(+DEPEND,+RDEPEND) headers(+DEPEND)"

What this means is “unless explicitly stated otherwise, when I resolve a DEPEND, I need to be built with libs and headers enabled, and when I resolve a RDEPEND I need to be built with headers enabled”. Then we’d need a new kind of USE dependency meaning “I don’t care about this flag” for special cases.

This also solves another problem:

IUSE="minimal(-*)"

Which means “if I’m built with minimal enabled, I don’t satisfy any dependencies unless they explicitly say that minimal is ok”.

Syntax could change. And I might drop the entire idea if it turns out it doesn’t do its job. But it’s interesting enough to write up so that I don’t forget about it entirely, at least…