bootstrap-initrd

README.md (7525B)

# bootstrap-initrd
```
Copyright 2017 Haelwenn (lanodan) Monnier <contact@hacktivis.me>
SPDX-License-Identifier: MPL-2.0
```
A initrd generator for Linux to bootstrap a POSIX-ish system from a reasonably small binary seed.
## Status
- [x] Small binary seed of only TCC and musl (~1.2MB of binaries, depending on the architecture), enforced by [deblob](https://hacktivis.me/projects/deblob)
- [x] Capable of bootstrapping <https://git.sr.ht/~mcf/oasis>
- [ ] Self-replicating: Able to reassemble the initrd with Alpine apks
- [ ] Self-hosting: Able to recompile all the software used from source (including tcc and musl)
- [x] Code compiled by `init.sh` is reviewable, mostly thanks to a lack of autotools
## Explorations
- Reduce the binary seed even further, without compromising on keeping the bootstrapping simple and reasonably fast
## Non-Goals
- GNU Environment, there is already <https://github.com/fosslinux/live-bootstrap> for this and it's a massive effort
- Become a distro, at worst it should end up deferring to an existing ports system like <https://www.pkgsrc.org/>
## Architectures
You can set `ALPINE_ARCH` to change the architecture, by default it's set to `x86_64`.
- `aarch64`: Works except for extras (tested on HoneyComb LX2k with linux 6.12.21-gentoo)
- `armhf`: Untested
- `armv7`: Untested
- `riscv64`: Works except for extras (tested on VisionFive2 with linux-lts from Alpine)
- `x86`: Works
- `x86_64` (aka amd64): Works
## Rationales
Each of those roughly by their build order.
### TCC (binary)
Need to start somewhere and TCC allows to interpret C, allowing to avoid seeding another interpreter.  
The reason to not pick Guile+MesCC instead is because I find Guile Scheme to be less well-known than amd64 itself.
### musl (binary)
Need a libc to start somewhere, tryhards could maybe compile it from source with TCC to reduce the seed further but this hasn't been tried.  
It's `libc.a` is removed, to shave off 9.1MB from the binary seed, going from 11MB total to 1.2MB total, you're likely going to rebuild musl anyway since this is a sort of stage0.
### TCC as ar(1) implementation
Meanwhile:
- Binutils: 300+ MiB repository of sources, generated artifacts frequently over 1MB, and test fixtures also often over 1MB. Nope.
### OpenBSD ksh
Portable version done by Dr. Brian Robert Callahan aka ibara@, of OpenBSD.
Buildable with only a C compiler while being nicely complete (maybe even too much).
Builtin commands (as of 7.6):
- defined in c_sh.c: ., :, [, break, builtin, continue, eval, exec, exit, false, return, set, shift, times, trap, wait, read, test, true, ulimit, umask, unset, suspend
- defined in c_ksh.c: alias, cd, command, echo, export, fc, getopts, jobs, kill, let, print, pwd, readonly, type, typeset, unalias, whence, bg, fg (and bind when emacs-mode is enabled)
Meanwhile:
- loksh: similar port but header names overrides the libc ones
- mrsh: Too incomplete for actual use, but trivial to build with only a C Compiler
- heirloom-sh: Also buildable with only a C Compiler but fails to allocate memory at launch
- bash, yash: autoconf, so needs an existing shell
- dash: autoconf and generator scripts, so needs an existing shell
- mksh: ./Build.sh, so needs an existing shell
- AT&T ksh: Way too big to even try, might as well try Perl
### OpenBSD yacc
Portable version done by Dr. Brian Robert Callahan aka ibara@, of OpenBSD.
Uses a BSD-style configure script, trivial to diff between versions and adjust hardcoded compiler flags (no utilities to run said script yet).
Meanwhile:
- bison: autoconf, way too early
- byacc: autoconf
- [yacc from compilertools.net](https://web.archive.org/web/20220511161030/http://dinosaur.compilertools.net/): Domain expired years ago and turns out Gentoo was the only distro still providing it
### utils-std
My own software, which explicitly allows to bootstrap with an incomplete POSIX environment such as this one.
Meanwhile:
- GNU coreutils: Autoconf, therefore needs the utilities it's building plus some extras to already be available
- Busybox and Toybox: Heavy dependency on GNU Make, bash, …, which can only come much later on in the bootstrapping process
## sbase
Simple `ed` and `sed` implementations which uses `regex.h` instead of some broken regex engine.
Lack of support of `-i` option on sed(1) is a bit annoying, might patch this.
Meanwhile:
- minised: Broken regex engine which is too limited to pass `./configure` checks like the one in GNU make
- GNU sed: `./configure` script requires an existing sed (*why*), doesn't seems like there's a workaround.
### (One True) awk
Reference implementation and AFAIK the one used in BSDs, can be built with only a Yacc implementation and a C Compiler.
### pdpmake
Public Domain POSIX make, trivial to compile without an existing make, features future POSIX additions and common extensions.
Meanwhile:
- bmake (NetBSD make port): autotools configure script, simpler than the one in GNU make but still unreviewable
- OpenBSD make, port at <https://github.com/ibara/make> is incomplete and likely outdated
- (OpenOffice) dmake: Horribly massive
### bzip2
Builds with C Compiler + POSIX make.
Meanwhile:
- pbzip2: Requires C++ compiler
### zlib
Simple albeit slightly broken `./configure` script (see patches).
### pigz
Simple, needs make + c compiler + zlib.  
Note: Vendored zopfli got removed, so no compression beyond -9
Meanwhile:
- GNU gzip: Autotools
### Heirloom-devtools lex
Slightly messy Makefiles but only has few dependencies.
Meanwhile:
- flex: Autotools
### Heirloom-devtools m4
Slightly messy Makefiles but only has few dependencies.
Meanwhile:
- GNU m4: Not even close
- OpenBSD m4 aka om4: Requires a more modern lex than heirloom-devtools provides
### Heirloom (toolchest) diff, sort
Slightly messy Makefiles but only has few dependencies.
### Heirloom (toolchest) tar
Slightly messy Makefiles but only has few dependencies.
Meanwhile:
- libarchive aka bsdtar: Autotools
- GNU tar: [No.](https://www.roguelazer.com/blog/surprising-behavior-in-gnu-tar/)
### extras as packed tarballs
- GNU make: Required by the other extras
- muxzcat: To decompress qbe tarball
- lua: For oasis
- muon: For pkgconf and git without autotools
- pkgconf: For iproute2
- iproute2: Basic networking configuration
- bearssl: small TLS library
- tiny-curl: Still pretty big but hopefully close enough to the usual one
- cacert (from curl.haxx.se): Maybe could be reduced to a handful of CAs (like maybe even just Let's Encrypt)
- git: For oasis
- e2fsprogs: In case you need to format some extra storage
You can automatically build all of the above with `/extras-build-all.sh` but be warned that some use pre-generated autotools blobs which harms reviewability.
You can also exclude those tarballs from the initrd with setting the `EXCLUDE_EXTRAS` environment variable to any value.
## Launching in QEMU
* You need a Linux kernel, so far no known version limitations
* Combination of `panic=1` and `-no-reboot` allows to exit+relaunch
```
$ ./make-initrd.sh && qemu-system-x86_64 -enable-kvm -m 512 -kernel /boot/vmlinuz-6.6.21-gentoo -initrd bootstrap-initrd/x86_64.cpio.gz -append 'init=/init console=ttyS0 panic=1' -nographic -no-reboot
```
## Dev setup via bubblewrap
```
$ ./make-root.sh && bwrap --clearenv --unshare-all --bind bootstrap-initrd/x86_64/ / --proc /proc --dev /dev --uid 0 --gid 0 /init
```
Useful as it allows to manipulate the environment externally, for example to edit files in an editor more comfortable than `ed(1)`.