drewdevault.com

how-to-fuck-up-releases.md (4543B)

---
date: 2019-10-12
title: How to fuck up software releases
layout: post
tags: [practices]
---
I manage releases for a bunch of free & open-source software. Just about every
time I ship a release, I find a novel way to fuck it up. Enough of these
fuck-ups have accumulated now that I wanted to share some of my mistakes and how
I (try to) prevent them from happening twice.
At first, I did everything manually. This is fine enough for stuff with simple
release processes - stuff that basically amounts to tagging a commit, pushing
it, and calling it a day. But even this gets tedious, and I'd often make a
mistake when picking the correct version number. So, I wrote a small script:
[semver](https://git.sr.ht/~sircmpwn/dotfiles/tree/master/bin/semver). `semver
patch` bumps the patch version, `semver minor` bumps the minor version, and
`semver major` bumps the major version, based on semantic versioning. I got into
the habit of using this script instead of making the tags manually. The next
fuckup soon presented itself: when preparing the
[shortlog](https://git-scm.com/docs/git-shortlog), I would often feed it the
wrong commits, and the changelog would be messed up. So, I updated the script to
run the appropriate shortlog command and pre-populate the annotated tag with it,
launching the editor to adjust the changelog as necessary.
Soon I wanted to apply this script to other projects, but not all of them used
semantic versioning. I updated it to work for projects which just use
`major.minor` versions as well. However, another problem arose: some projects
have the version number specified in the Makefile or meson.build. I would
frequently fuck this up in many creative ways: forgetting it entirely; updating
it but not committing it; updating it and committing it, but tagging the wrong
commit; etc. [wlroots](https://github.com/swaywm/wlroots) in particular was
difficult because I also had to update the soversion, which had special
requirements. To address these issues, I added a custom `.git/_incr_version`
script which can add additional logic on a per-repo basis, and updated semver to
call this script if present.[^1]
Eventually, I went on vacation and shipped a release while I was there. The
`_incr_version` script I had put into `.git` on my home workstation wasn't
checked into version control and didn't come with me on vacation, leading to yet
another fucked up release. I moved it from `.git/_incr_version` to
`contrib/_incr_version`. I made the mistake, however, of leaving the old path in
as a fallback, which meant that I never noticed that *another* project's script
was still in `.git` until I went on another vacation and fucked up another
release. Add a warning which detects if the script is at the old path...
Some of my projects don't use semantic versioning at all, but still have all of
these other gotchas, so I added an option to just override the automatic version
increment with a user-specified override. For a while, this worked well. But,
inevitably, no matter how much I scripted away my mistakes I would always find a
new and novel way of screwing up. The next one came when I shipped a release
while on an Alpine Linux machine, which ships Busybox instead of GNU tools.
Turns out Busybox gzip produces output which does not match the GNU output,
which means the tarballs I signed locally differed from the ones generated by
Github. Update the signing script to save the tarball to disk (previously,
it lived in a pipe) and upload these alongside the releases...[^2]
Surely, there are no additional ways to fuck it up at this point. I must have
every base covered, right? Wrong. Dead wrong. On the very next release I
shipped, I mistakenly did everything from a feature branch, and shipped
experimental, incomplete code in a stable release. Update the script to warn if
the master branch isn't checked out... Then, of course, another fuckup: I tagged
a release without pulling first, and when I pushed, git happily rejected my
branch and accepted the tag - shipping an outdated commit as the release. Update
the script to `git pull` first...
I am doomed to creatively outsmart my tools in releases. If you'd like to save
yourself from some of the mistakes I've made, you can [find my semver script
here](https://git.sr.ht/~sircmpwn/dotfiles/tree/master/bin/semver).
[^1]: Each of these `_incr_version` scripts proved to have many bugs of their own, of course.
[^2]: Eli Schwartz of Arch Linux also sent a patch to Busybox which made their gzip implementation consistent with GNU's.