Assorted C++ Linkage

• A talk by Bjarne Stroustrup on the design of C++0x.
• Range based for loops in C++0x without concepts, good to know this hasn’t been dropped.
• Making std::list::size() O(1) [PDF], a shame to see this hasn’t been dropped.
• And on a less technical note, filtering compiler optimisation flags is not a solution from Diego. Unfortunately, he leaves out some of the worst offenders for flags that users think they can use that do change the meaning of programs: -fvisibility-inlines-hidden and -Wl,--as-needed.

C++ Const Curiosity

GCC will accept the following (look closely at S::f’s signature in both places):

struct T
{
    void foo()
    {
    }
};

struct S
{
    void f(const T);
};

void
S::f(T t)
{
    t.foo();
}

int main(int, char *[])
{
    T t;
    S s;
    s.f(t);
}

The question is, should it?

Update: and the answer is, yes, it should, according to [dcl.fct] in the standard. This is both useful and annoying.

Programming – Principles and Practice using C++

Programming — Principles and Practice Using C++ is the new book by Bjarne Stroustrup, the daddy of C++. It’s an introduction to programming rather than an advanced book; I’ve been holding off writing up my impressions of it because I’m not entirely sure what to say.

The overriding theme of this book seems to be “there are lots of complications, special cases and obscure things”. This is of course true, and it’s a refreshing change from most introductory books that go out of their way to construct highly contrived examples that conveniently ignore any obscurity. But I suspect it goes too far — pretty much every example is twice as long as it probably should be. There’s so much focus on dealing with complexities that the underlying “what’s going on?” is lost.

Partly this is down to C++. A language designed to handle real world, large scale problems and provide for maintainability over decades isn’t going to be the most elegant. On the other hand, purely teaching languages that dismiss the real world entirely are of no practical use. The question is whether C++ as a first language is a sensible idea, and I’m not in the least bit convinced that it is.

Partly, though, this is down to the choice of projects. An example: two chapters are devoted to writing a calculator program. These chapters cover lexers, parsers, grammars and error recovery. This isn’t one of those cop-out calculator programs where syntax is carefully selected to hide any kind of mess, either — it almost looks like the book is going to end up implementing a compiler… Unfortunately, there’s nothing in the final program that really needs any of this complexity; a simple “tokenise into a list, then replace all the multiplications with their result, then replace all the divisions with their result and so on” would work just as well for the requirements, and wouldn’t have most of the mess.

The scope of the book is impressive, though. It doesn’t gloss over classes, templates, pointers, exceptions or even dealing with code written in C. It’s extremely comprehensive, even in places where it probably shouldn’t be.

Finally, a note on writing style. The word ‘basically’ appears on average once per page, and sometimes three or four times in a single paragraph. This gets very annoying very quickly. Stroustrup’s other books don’t suffer from this.

I suppose my conclusion is: if you have to learn C++ as a first language, this is the book to use. If you have a choice, though, learn one of the monkey languages first, and then pick up The C++ Programming Language and The C++ Standard Library.

Stopping Pythons from eating your Rams

As various people have found out the hard way, and much to my annoyance because my laptop is memory starved, building Paludis can sometimes take lots and lots of RAM.

Originally, we didn’t do anything about this. But unfortunately lots of users have silly things like MAKEOPTS="-j9", which can result in the build process wanting somewhere in the region of eight gigs of RAM, which in turn leads to users whining about gcc internal errors or random processes being OOMed. So we stuck a nasty hack in the ebuild that would reduce MAKEOPTS based upon how much free memory you had — all very well, but it screws over distcc users and isn’t even necessary for many combinations of USE flags and CXXFLAGS.

It’s worth working out exactly what makes the compiler memory usage so high. It’s fairly easy to figure out that it’s only an issue when building the Python bindings. We use Boost.Python for these, and unfortunately Boost will quite happily use horrible preprocessor hacks that result in huge generated source files and all sorts of nasty workarounds just to get code to work on ancient unsupported Microsoft compilers. It’s enough of a problem for enough people that there’s a tutorial section on reducing memory consumption for Boost.Python, but we already do those things.

There’s something else interesting, though. With debugging turned on (which autotools does by default), we need something like 800MBytes to compile one particular Python binding file. With it turned off, we only need 300MBytes, which is much less likely to be a problem (and more to the point it won’t make my laptop start swapping). It turns out that building the Python bindings with -g isn’t even useful — gdb doesn’t give particularly useful backtraces on the Python interpreter, and there are better ways of tracking down problems there.

So it looks like it makes sense to add -g0 (after checking that the compiler supports it) to CXXFLAGS for the Python bindings. Easy enough, right?

Wrong. As with everything else involving autotools, we have to jump through all sorts of convoluted hoops to do this. CXXFLAGS is a user variable (so we aren’t supposed to change it), and it takes precedence over AM_CXXFLAGS (which we can change). There’s no ‘more important than the user variable’ variable, and we can’t sensibly override CXXCOMPILE, so this gets messy. We have to abuse configure.ac to remove the debugging options from the user’s CXXFLAGS and move them into something that ends up in AM_CXXFLAGS, which we can then override. Horrid.

The next Paludis release will include this voodoo, which should improve things considerably and let us avoid the nasty MAKEOPTS mangling. But it’s still not ideal.

Most Gentoo users have USE="python" set, either from profiles or explicitly. Most of these users do not want to build the Python bindings. Some of these users don’t think to look at the dependencies before moaning that Paludis requires Boost, so they don’t even realise it’s only because they’re using a USE flag they probably don’t want enabled. So what can we do about this?

We can’t use IUSE defaults, since we don’t really want to use anything later than EAPI 0 for package manager ebuilds. We could turn off python selectively in package.use, but lots of users still have USE="python" set explicitly. So we use a different USE flag name. We’ve gone for python-bindings, along with a use description that makes it clear that thinking “well I have some things that use Python so I probably want this flag on” is wrong.

For consistency, we’ve also renamed ruby to ruby-bindings. These are a lot more useful than the Python bindings (playman is written in Ruby), and a lot faster to build thanks to Ruby having a reasonably sane API, so we might end up having to mess with profiles to turn them on by default.

I might end up reverting all of this if it turns out it does more harm than good. We’ll see.

C++ Explicit Template Instantiation Hate

In part two of a never ending series of why I hate C++ but have to use it anyway because there’s nothing else, we come to explicit template instantiation.

Explicit template instantiation is a nuisance that only exists because all build systems suck. Unfortunately, doing everything implicitly makes compiles take way too long, so explicit instantiation is a pragmatic nuisance. So let’s have a header that doesn’t slow down the compiler too much:

#include <string>

template <typename Item_>
struct ItemMaker
{
    Item_ make_me_an_item() const;
};

typedef ItemMaker<int> IntMaker;
typedef ItemMaker<std::string> StringMaker;

And then an implementation:

template <typename Item_>
Item_
ItemMaker<Item_>::make_me_an_item() const
{
    return Item_();
}

But then we need explicit instantiation. Should be no problem, right?

template class IntMaker;
template class StringMaker;

Wrong! Explicit instantiation only works for declarations, not typedefs, so we have to copy things out all over again:

template class ItemMaker<int>;
template class ItemMaker<std::string>;

Yes, thanks for that, Standard guys.

C++ Overload Resolution Hate

Sometimes, C++’s overload resolution rules are a pain in the ass.

Let’s say we have the following:

#include <tr1/memory>

struct Bar
{
    Bar()
    {
    }
};

struct Baz
{
    Baz()
    {
    }
};

struct Foo
{
    explicit Foo(const std::tr1::shared_ptr<const Bar> &)
    {
    }

    explicit Foo(const std::tr1::shared_ptr<const Baz> &)
    {
    }
};

Then the following is ambiguous, and won’t compile:

Foo foo(std::tr1::shared_ptr<Bar>(new Bar));

Here’s why: Neither constructor exactly matches the argument given, so the compiler falls back to construction and type conversions. std::tr1::shared_ptr<T_> has an implicit constructor template <typename U_> shared_ptr(const shared_ptr<U_> &), which is good because it lets you use a shared pointer to a derived class when a shared pointer to a base class is expected. But that conversion can take place for all U_, which means the compiler doesn’t know whether you want to convert to a shared pointer to const Bar or const Baz — it isn’t until the constructor body is instantiated that the compiler finds that only one of the two conversions will compile successfully.

So, one has to be explicit when creating the shared pointer:

Foo foo(std::tr1::shared_ptr<const Bar>(new Bar));

Except, usually we create shared pointers using a helper function, to avoid specifying the type name twice:

template <typename T_>
std::tr1::shared_ptr<T_> make_shared_ptr(T_ * const t)
{
    return std::tr1::shared_ptr<T_>(t);
}

So we’re stuck having to use a slightly weird looking allocation:

Foo foo(make_shared_ptr(new const Bar));

Incidentally, C++0x has a std::make_shared which is a lot better than this, but it requires rvalue references and std::forward to work. It would look like this:

Foo foo(std::make_shared<Bar>());

Or, if we still need the const:

Foo foo(std::make_shared<const Bar>());

Why might we not need the const? The current C++0x draft standard includes the wording “[the template] constructor shall not participate in the overload resolution unless U_ * is implicitly convertible to T_ *“, which presumably means implementations have to solve the problem using concepts to restrict the template constructor.

And, of course, there’s one final gotcha. The new const Foo form is only legal if Foo has a user defined constructor. I have no idea why, but C++03 explicitly says so.

Making Paludis Compile with C++0x

I managed to get gcc 4.4 svn to compile, so I decided to see just how badly the experimental C++0x support would break Paludis. Turns out, not too badly. Firstly, things caught by increased strictness or general rearrangement of headers:

• We had a few extra semicolons lying around. These now generate warnings, so we might as well shut them up. [fix]
• We weren’t including <stdint.h> to get uintptr_t. Things were working by fluke because other headers were including it. [fix]
• We were using ::rename rather than std::rename. [fix]

Then, the real issues:

• n2246 adds a std::next. Paludis has a paludis::next. ADL means this sometimes causes confusion. To keep compatibility with non-0x compilers, we use using to get std::next into paludis:: where necessary. [fix]
• std::list<>::push_back is now overloaded on rvalue references, so we can no longer easily get a PMF. If we were only interested in 0x, we’d use a lambda, but for backwards compatibility we write a wrapper function instead. (Or we could use the static_cast hack, but that’s horribly unreadable.) [fix]

All in all, not too bad. I suspect things will get a bit messier if a concept-enabled standard library makes it into the final proposal, but that can be dealt with later…

Bedtime Reading III

• An interview with Bjarne Stroustrup
• Google C++ Style Guide
• Sustainable Energy – Without the Hot Air, David J.C. MacKay

On-Demand Loading using Smart Pointers

Previously, I explained how to implement something like the Active Object thread pattern using smart pointers. Next we’ll use the same trick to implement on-demand, lazy construction.

There’s nothing difficult here, once we realise we can reuse the ‘return a different pointer’ trick. We make use of std::tr1::function rather than a raw function pointer so that parameter values can be bound at pointer-construction time. Again, we parameterise on pointer type, not raw type.

template <typename T_>
class DeferredConstructionPtr
{
    private:
        mutable T_ _ptr;
        std::tr1::function<T_ ()> _f;
        mutable bool _done;

    public:
        DeferredConstructionPtr(const std::tr1::function<T_ ()> & f) :
            _ptr(),
            _f(f),
            _done(false)
        {
        }

        DeferredConstructionPtr(const DeferredConstructionPtr & other) :
            _ptr(other._ptr),
            _f(other._f),
            _done(other._done)
        {
        }

        DeferredConstructionPtr &
        operator= (const DeferredConstructionPtr & other)
        {
            if (this != &other)
            {
                _ptr = other._ptr;
                _f = other._f;
                _done = other._done;
            }
            return *this;
        }

        T_ operator-> () const
        {
            if (! _done)
            {
                _ptr = _f();
                _done = true;
            }

            return _ptr;
        }
};

Again, some caveats:

• Dealing with const is left to the reader.
• Dealing with thread safety is left to the next article.
• If the constructor in question can throw exceptions, the exception will be thrown at what could be a rather unobvious place. This may or may not be a problem.

Bedtime Reading II

• Source Code Optimisation, Felix von Leitner. PDF.
• Open Multi-Methods for C++, Peter Pirkelbauer, Yuriy Solodkyy, and Bjarne Stroustrup. PDF.
• Floating point arithmetic in C++ templates. Heh.