Exherbo Development Workflow

In answer to what appears to be becoming a frequently asked question in #exherbo, my development workflow (and by extension, the one true workflow, any deviation from which is clearly heresy) is as follows:

For any repository I consider interesting, I have a local copy (but not a clone, because git clone is Satan’s work) in my home directory.

For Paludis, every repository it sees has location under /var somewhere. Paludis is never pointed at a repository that is modified by anything other than itself.

For syncing, any repository I consider interesting is synced using sync = git+file:///home/users/ciaranm/repos/blah, with sync_options = --reset (as previously described). Others are synced as normal.

Before syncing normally, I pull all of the interesting repositories I have checked out in my home directory, so that Paludis ends up with everything up to date. The shell one-liner to do this is in my history, so it’s no additional work thanks to the wonder that is reverse-i-search.

On those rare occasions when I have to do some work on Exherbo that I can’t either just yell about until someone fixes it for me or force to be fixed by making Paludis reject it, I work as follows:

• Changes are made and committed in my home directory copy of the repository.
• Paludis is synced, picking up those changes.
• Testing is done.
• More changes are made and committed, since things never work as expected the first time.
• Paludis is synced, picking up those changes.
• And so on.
• When things finally work, git rebase -i is used to turn all my messy work-in-progress commits into something suitable for pushing. Given that other people are often working on the repositories in question, this also rebases my changes against current master.
• Things are pushed.
• When syncing again, the --reset ensures that Paludis ends up with the history-rewritten result, not some horrible automatic merge of the end result and previous works in progress.

Fortunately, Git is easily powerful enough to handle this kind of thing, meaning Exherbo development workflows are designed around what works best, not around what is possible.

On a related note, I am still strongly considering making --reset the default one of these days. Anyone using paludis --sync on a repository they themselves modify should quickly justify their iniquity or risk being horribly surprised when the default changes.

Changes to the Paludis Git Syncer

I’ve just committed two changes to the Paludis Git syncer.

The first allows you to do sync_options = --branch=foo to specify a particular branch. I don’t expect this to be widely used, since branching for repositories usually means you’re not taking advantage of the better facilities available for that kind of thing. Still, it has its uses.

The second is sync_options = --reset. Unlike with, say, rsync, syncing via Git will merge any changes you’ve made to the repository with new upstream changes (it uses git pull). With --reset, it will instead discard any local changes and just become equal to whatever you’re syncing against (using instead git fetch and git reset --hard).

It’s a matter of considerable debate as to whether the reset behaviour is the right thing to do.

On the one hand, some people like working directly on a checkout to which Paludis is pointed, but still want cave sync to bring in updates.

On the other hand, doing that is horrible and evil, and a much better workflow is this:

• Have a local checkout for development work. Commit your changes to it.
• Have a separate checkout that is synced against your local checkout for Paludis use. After making changes, commit them to your local checkout and sync.
• When happy with your changes, rebase and squash them in your local checkout, and then push them upstream.

Doing that requires --reset, since otherwise the checkout Paludis sees will end up as some horrible auto-merged mess that includes changes you thought you’d discarded ages ago.

I’m strongly considering making --reset by default sometime in the future. However, this will make syncing a destructive operation for anyone who hates puppies enough to be doing work on a Paludis-synced checkout (in the same way that it’s already destructive for rsync).

Hot Fresh Shiny Config Files

I’ve moved my configuration files repository from Subversion to Git, since I’m sick of trying to decide whether my laptop or desktop should hold the ‘master’ copy. For the lulz, I shall also push to GitHub at irregular intervals, so you shall no longer need to pester me to upload newer versions of my bashrc / vimrc / whatever to webspace somewhere. Instead, you can just:

git clone git://github.com/ciaranm/dotfiles-ciaranm.git

Or browse them online.

We’ll see whether this lasts…

Git and Subversion information in the Bash Prompt

Here’s my variation on the “Git and Subversion status in the bash prompt” thing:

ps_scm_f() {
    local s=
    if [[ -d ".svn" ]] ; then
        local r=$(svn info | sed -n -e '/^Revision: \([0-9]*\).*$/s//\1/p' )
        s="(r$r$(svn status | grep -q -v '^?' && echo -n "*" ))"
    else
        local d=$(git rev-parse --git-dir 2>/dev/null ) b= r= a=
        if [[ -n "${d}" ]] ; then
            if [[ -d "${d}/../.dotest" ]] ; then
                if [[ -f "${d}/../.dotest/rebase" ]] ; then
                    r="rebase"
                elif [[ -f "${d}/../.dotest/applying" ]] ; then
                    r="am"
                else
                    r="???"
                fi
                b=$(git symbolic-ref HEAD 2>/dev/null )
            elif [[ -f "${d}/.dotest-merge/interactive" ]] ; then
                r="rebase-i"
                b=$(<${d}/.dotest-merge/head-name)
            elif [[ -d "${d}/../.dotest-merge" ]] ; then
                r="rebase-m"
                b=$(<${d}/.dotest-merge/head-name)
            elif [[ -f "${d}/MERGE_HEAD" ]] ; then
                r="merge"
                b=$(git symbolic-ref HEAD 2>/dev/null )
            elif [[ -f "${d}/BISECT_LOG" ]] ; then
                r="bisect"
                b=$(git symbolic-ref HEAD 2>/dev/null )"???"
            else
                r=""
                b=$(git symbolic-ref HEAD 2>/dev/null )
            fi

            if git status | grep -q '^# Changed but not updated:' ; then
                a="${a}*"
            fi

            if git status | grep -q '^# Changes to be committed:' ; then
                a="${a}+"
            fi

            if git status | grep -q '^# Untracked files:' ; then
                a="${a}?"
            fi

            b=${b#refs/heads/}
            b=${b// }
            [[ -n "${r}${b}${a}" ]] && s="(${r:+${r}:}${b}${a:+ ${a}})"
        fi
    fi
    s="${s}${ACTIVE_COMPILER}"
    s="${s:+${s} }"
    echo -n "$s"
}

This is added to PS1 as normal.

For Subversion, it gives us the revision, and a * if anything’s been modified.

For Git, we get quite a bit more. If we’re in the middle of a merge, interactive rebase or am, we get told so. If we’re in the middle of a bisect, we get question marks, since I’ve not had to bisect anything since I wrote that code and I haven’t implemented anything fancy for it. We also get the branch, and a combination of * for changed but not updated files, + for changed and committed files and ? for untracked files.

It’s generally sufficiently fast to not be annoying, although the initial cd into a large project can take a few seconds. It’s probably possible to drop to a single ‘git status’ call if performance matters, although the kernel does a pretty good job of speeding up subsequent runs.

Yay for git

Linux 2.6.26-rc2 wouldn’t boot on my desktop. Linux 2.6.25 worked. In the good old days, tracking down why would be a major pain in the ass. But now, a quick git bisect and fifteen reboots later, I have the exact commit: 3def3d6ddf43dbe20c00c3cbc38dfacc8586998f, also known as:

Author: Yinghai Lu
Date:   Fri Feb 22 17:07:16 2008 -0800

    x86: clean up e820_reserve_resources on 64-bit

    e820_resource_resources could use insert_resource instead of request_resource
    also move code_resource, data_resource, bss_resource, and crashk_res
    out of e820_reserve_resources.

    Signed-off-by: Yinghai Lu
    Signed-off-by: Ingo Molnar

Verifying that this really is the offender is equally easy — a quick ‘git revert’ on head and another reboot and the kernel’s working again. Now, I know nothing about what an e820 is beyond what Google tells me, but hopefully someone else will.

As much as I hate to say it, if this were Subversion I’d still be tracking down the bug. And if it were CVS, I wouldn’t've bothered.