Blag

He's not dead, he's resting

Tag Archives: git

Exherbo Development Workflow, Version 2

My original Exherbo Development Workflow post seems to have become the standard way of doing things. However, it does rather assume that you are developing on most repositories most of the time. When that’s not the case, a new feature named “sync suffixes” may be of use. With sync suffixes, a typical workflow now looks like this:

Repositories are configured as normal, with their sync set to point to the usual remote location. In addition, for any repository you are going to work on, you use a sync suffix to specify a local path too. For example:

sync = git://git.exherbo.org/arbor.git local: git+file:///home/users/ciaranm/repos/arbor

where /home/users/ciaranm/repos/arbor is a personal copy of the repository that is entirely unrelated to the checkout Paludis uses.

Normally, when you sync, you’ll be syncing against upstream. But when you want to do some work:

  • Update your personal copy of the repository.
  • Work on and commit your changes.
  • Use cave sync --suffix local arbor to sync just that repository, and against your local checkout rather than upstream.
  • Test your changes.
  • Make fixes, commit, sync using the suffix etc until everything works.
  • Use the wonders of git rebase -i to tidy up your work into nice friendly pushable commits.
  • Push or submit a git format-patch for your changes.
  • Go back to syncing without the suffix.

Some things to note:

  • This only really works with Git, and only when using the default ‘reset’ sync mode.
  • You’re never manually modifying any repository which Paludis also modifies.
  • Unlike the original version of this workflow, you only need to keep your personal copies of repositories up to date when you work on them.
  • The suffix things work on sync_options too, if you need it. Thus, for branches, you can use sync_options = --branch=branch-on-upstream local: --branch=my-local-branch.
Advertisements

Exherbo Development Workflow

Update: also see Exherbo Development Workflow, Version 2.

In answer to what appears to be becoming a frequently asked question in #exherbo, my development workflow (and by extension, the one true workflow, any deviation from which is clearly heresy) is as follows:

For any repository I consider interesting, I have a local copy (but not a clone, because git clone is Satan’s work) in my home directory.

For Paludis, every repository it sees has location under /var somewhere. Paludis is never pointed at a repository that is modified by anything other than itself.

For syncing, any repository I consider interesting is synced using sync = git+file:///home/users/ciaranm/repos/blah, with sync_options = --reset (as previously described). Others are synced as normal.

Before syncing normally, I pull all of the interesting repositories I have checked out in my home directory, so that Paludis ends up with everything up to date. The shell one-liner to do this is in my history, so it’s no additional work thanks to the wonder that is reverse-i-search.

On those rare occasions when I have to do some work on Exherbo that I can’t either just yell about until someone fixes it for me or force to be fixed by making Paludis reject it, I work as follows:

  • Changes are made and committed in my home directory copy of the repository.
  • Paludis is synced, picking up those changes.
  • Testing is done.
  • More changes are made and committed, since things never work as expected the first time.
  • Paludis is synced, picking up those changes.
  • And so on.
  • When things finally work, git rebase -i is used to turn all my messy work-in-progress commits into something suitable for pushing. Given that other people are often working on the repositories in question, this also rebases my changes against current master.
  • Things are pushed.
  • When syncing again, the --reset ensures that Paludis ends up with the history-rewritten result, not some horrible automatic merge of the end result and previous works in progress.

Fortunately, Git is easily powerful enough to handle this kind of thing, meaning Exherbo development workflows are designed around what works best, not around what is possible.

On a related note, I am still strongly considering making --reset the default one of these days. Anyone using paludis --sync on a repository they themselves modify should quickly justify their iniquity or risk being horribly surprised when the default changes.

Changes to the Paludis Git Syncer

I’ve just committed two changes to the Paludis Git syncer.

The first allows you to do sync_options = --branch=foo to specify a particular branch. I don’t expect this to be widely used, since branching for repositories usually means you’re not taking advantage of the better facilities available for that kind of thing. Still, it has its uses.

The second is sync_options = --reset. Unlike with, say, rsync, syncing via Git will merge any changes you’ve made to the repository with new upstream changes (it uses git pull). With --reset, it will instead discard any local changes and just become equal to whatever you’re syncing against (using instead git fetch and git reset --hard).

It’s a matter of considerable debate as to whether the reset behaviour is the right thing to do.

On the one hand, some people like working directly on a checkout to which Paludis is pointed, but still want cave sync to bring in updates.

On the other hand, doing that is horrible and evil, and a much better workflow is this:

  • Have a local checkout for development work. Commit your changes to it.
  • Have a separate checkout that is synced against your local checkout for Paludis use. After making changes, commit them to your local checkout and sync.
  • When happy with your changes, rebase and squash them in your local checkout, and then push them upstream.

Doing that requires --reset, since otherwise the checkout Paludis sees will end up as some horrible auto-merged mess that includes changes you thought you’d discarded ages ago.

I’m strongly considering making --reset by default sometime in the future. However, this will make syncing a destructive operation for anyone who hates puppies enough to be doing work on a Paludis-synced checkout (in the same way that it’s already destructive for rsync).

Hot Fresh Shiny Config Files

I’ve moved my configuration files repository from Subversion to Git, since I’m sick of trying to decide whether my laptop or desktop should hold the ‘master’ copy. For the lulz, I shall also push to GitHub at irregular intervals, so you shall no longer need to pester me to upload newer versions of my bashrc / vimrc / whatever to webspace somewhere. Instead, you can just:

git clone git://github.com/ciaranm/dotfiles-ciaranm.git

Or browse them online.

We’ll see whether this lasts…

Git and Subversion information in the Bash Prompt

Here’s my variation on the “Git and Subversion status in the bash prompt” thing:

ps_scm_f() {
    local s=
    if [[ -d ".svn" ]] ; then
        local r=$(svn info | sed -n -e '/^Revision: \([0-9]*\).*$/s//\1/p' )
        s="(r$r$(svn status | grep -q -v '^?' && echo -n "*" ))"
    else
        local d=$(git rev-parse --git-dir 2>/dev/null ) b= r= a=
        if [[ -n "${d}" ]] ; then
            if [[ -d "${d}/../.dotest" ]] ; then
                if [[ -f "${d}/../.dotest/rebase" ]] ; then
                    r="rebase"
                elif [[ -f "${d}/../.dotest/applying" ]] ; then
                    r="am"
                else
                    r="???"
                fi
                b=$(git symbolic-ref HEAD 2>/dev/null )
            elif [[ -f "${d}/.dotest-merge/interactive" ]] ; then
                r="rebase-i"
                b=$(<${d}/.dotest-merge/head-name)
            elif [[ -d "${d}/../.dotest-merge" ]] ; then
                r="rebase-m"
                b=$(<${d}/.dotest-merge/head-name)
            elif [[ -f "${d}/MERGE_HEAD" ]] ; then
                r="merge"
                b=$(git symbolic-ref HEAD 2>/dev/null )
            elif [[ -f "${d}/BISECT_LOG" ]] ; then
                r="bisect"
                b=$(git symbolic-ref HEAD 2>/dev/null )"???"
            else
                r=""
                b=$(git symbolic-ref HEAD 2>/dev/null )
            fi

            if git status | grep -q '^# Changed but not updated:' ; then
                a="${a}*"
            fi

            if git status | grep -q '^# Changes to be committed:' ; then
                a="${a}+"
            fi

            if git status | grep -q '^# Untracked files:' ; then
                a="${a}?"
            fi

            b=${b#refs/heads/}
            b=${b// }
            [[ -n "${r}${b}${a}" ]] && s="(${r:+${r}:}${b}${a:+ ${a}})"
        fi
    fi
    s="${s}${ACTIVE_COMPILER}"
    s="${s:+${s} }"
    echo -n "$s"
}

This is added to PS1 as normal.

For Subversion, it gives us the revision, and a * if anything’s been modified.

For Git, we get quite a bit more. If we’re in the middle of a merge, interactive rebase or am, we get told so. If we’re in the middle of a bisect, we get question marks, since I’ve not had to bisect anything since I wrote that code and I haven’t implemented anything fancy for it. We also get the branch, and a combination of * for changed but not updated files, + for changed and committed files and ? for untracked files.

It’s generally sufficiently fast to not be annoying, although the initial cd into a large project can take a few seconds. It’s probably possible to drop to a single ‘git status’ call if performance matters, although the kernel does a pretty good job of speeding up subsequent runs.

Yay for git

Linux 2.6.26-rc2 wouldn’t boot on my desktop. Linux 2.6.25 worked. In the good old days, tracking down why would be a major pain in the ass. But now, a quick git bisect and fifteen reboots later, I have the exact commit: 3def3d6ddf43dbe20c00c3cbc38dfacc8586998f, also known as:

Author: Yinghai Lu
Date:   Fri Feb 22 17:07:16 2008 -0800

    x86: clean up e820_reserve_resources on 64-bit
    
    e820_resource_resources could use insert_resource instead of request_resource
    also move code_resource, data_resource, bss_resource, and crashk_res
    out of e820_reserve_resources.
    
    Signed-off-by: Yinghai Lu
    Signed-off-by: Ingo Molnar

Verifying that this really is the offender is equally easy — a quick ‘git revert’ on head and another reboot and the kernel’s working again. Now, I know nothing about what an e820 is beyond what Google tells me, but hopefully someone else will.

As much as I hate to say it, if this were Subversion I’d still be tracking down the bug. And if it were CVS, I wouldn’t’ve bothered.