bootstrap-initrd

README.md (7516B)

# bootstrap-initrd
```
Copyright 2017 Haelwenn (lanodan) Monnier <contact@hacktivis.me>
SPDX-License-Identifier: MPL-2.0
```
A initrd generator for Linux to bootstrap a POSIX-ish system from a reasonably small binary seed.
## Status
- [x] Small binary seed of only TCC and musl (~1.2MB of binaries, depending on the architecture), enforced by [deblob](https://hacktivis.me/projects/deblob)
- [x] Capable of bootstrapping <https://git.sr.ht/~mcf/oasis>
- [ ] Self-replicating: Able to reassemble the initrd with Alpine apks
- [ ] Self-hosting: Able to recompile all the software used from source (including tcc and musl)
- [x] Code compiled by `init.sh` is reviewable, mostly thanks to a lack of autotools
## Explorations
- Reduce the binary seed even further, without compromising on keeping the bootstrapping simple and reasonably fast
## Non-Goals
- GNU Environment, there is already <https://github.com/fosslinux/live-bootstrap> for this and it's a massive effort
- Become a distro, at worst it should end up deferring to an existing ports system like <https://www.pkgsrc.org/>
## Architectures
You can set `ALPINE_ARCH` to change the architecture, by default it's set to `x86_64`.
- `aarch64`: Untested
- `armhf`: Untested
- `armv7`: Untested
- `riscv64`: Broken.
- `x86`: Works
- `x86_64` (aka amd64): Works
## Rationales
Each of those roughly by their build order.
### TCC (binary)
Need to start somewhere and TCC allows to interpret C, allowing to avoid seeding another interpreter.  
The reason to not pick Guile+MesCC instead is because I find Guile Scheme to be less well-known than amd64 itself.
### musl (binary)
Need a libc to start somewhere, tryhards could maybe compile it from source with TCC to reduce the seed further but this hasn't been tried.  
It's `libc.a` is removed, to shave off 9.1MB from the binary seed, going from 11MB total to 1.2MB total, you're likely going to rebuild musl anyway since this is a sort of stage0.
### TCC as ar(1) implementation
Meanwhile:
- Binutils: 300+ MiB repository of sources, generated artifacts frequently over 1MB, and test fixtures also often over 1MB. Nope.
### OpenBSD ksh
Portable version done by Dr. Brian Robert Callahan aka ibara@, of OpenBSD.
Buildable with only a C compiler while being nicely complete (maybe even too much).
Builtin commands (as of 7.6):
- defined in c_sh.c: ., :, [, break, builtin, continue, eval, exec, exit, false, return, set, shift, times, trap, wait, read, test, true, ulimit, umask, unset, suspend
- defined in c_ksh.c: alias, cd, command, echo, export, fc, getopts, jobs, kill, let, print, pwd, readonly, type, typeset, unalias, whence, bg, fg (and bind when emacs-mode is enabled)
Meanwhile:
- loksh: similar port but header names overrides the libc ones
- mrsh: Too incomplete for actual use, but trivial to build with only a C Compiler
- heirloom-sh: Also buildable with only a C Compiler but fails to allocate memory at launch
- bash, yash: autoconf, so needs an existing shell
- dash: autoconf and generator scripts, so needs an existing shell
- mksh: ./Build.sh, so needs an existing shell
- AT&T ksh: Way too big to even try, might as well try Perl
### OpenBSD yacc
Portable version done by Dr. Brian Robert Callahan aka ibara@, of OpenBSD.
Uses a BSD-style configure script, trivial to diff between versions and adjust hardcoded compiler flags (no utilities to run said script yet).
Meanwhile:
- bison: autoconf, way too early
- byacc: autoconf
- [yacc from compilertools.net](https://web.archive.org/web/20220511161030/http://dinosaur.compilertools.net/): Domain expired years ago and turns out Gentoo was the only distro still providing it
### utils-std
My own software, which explicitly allows to bootstrap with an incomplete POSIX environment such as this one.
Meanwhile:
- GNU coreutils: Autoconf, therefore needs the utilities it's building plus some extras to already be available
- Busybox and Toybox: Heavy dependency on GNU Make, bash, …, which can only come much later on in the bootstrapping process
## sbase
Simple `ed` and `sed` implementations which uses `regex.h` instead of some broken regex engine.
Lack of support of `-i` option on sed(1) is a bit annoying, might patch this.
Meanwhile:
- minised: Broken regex engine which is too limited to pass `./configure` checks like the one in GNU make
- GNU sed: `./configure` script requires an existing sed (*why*), doesn't seems like there's a workaround.
### (One True) awk
Reference implementation and AFAIK the one used in BSDs, can be built with only a Yacc implementation and a C Compiler.
### pdpmake
Public Domain POSIX make, trivial to compile without an existing make, features future POSIX additions and common extensions.
Meanwhile:
- bmake (NetBSD make port): autotools configure script, simpler than the one in GNU make but still unreviewable
- OpenBSD make, port at <https://github.com/ibara/make> is incomplete and likely outdated
- (OpenOffice) dmake: Horribly massive
### bzip2
Builds with C Compiler + POSIX make.
Meanwhile:
- pbzip2: Requires C++ compiler
### zlib
Simple albeit slightly broken `./configure` script (see patches).
### pigz
Simple, needs make + c compiler + zlib.  
Note: Vendored zopfli got removed, so no compression beyond -9
Meanwhile:
- GNU gzip: Autotools
### Heirloom-devtools lex
Slightly messy Makefiles but only has few dependencies.
Meanwhile:
- flex: Autotools
### Heirloom-devtools m4
Slightly messy Makefiles but only has few dependencies.
Meanwhile:
- GNU m4: Not even close
- OpenBSD m4 aka om4: Requires a more modern lex than heirloom-devtools provides
### Heirloom (toolchest) diff, sort
Slightly messy Makefiles but only has few dependencies.
### Heirloom (toolchest) tar
Slightly messy Makefiles but only has few dependencies.
Meanwhile:
- libarchive aka bsdtar: Autotools
- GNU tar: [No.](https://www.roguelazer.com/blog/surprising-behavior-in-gnu-tar/)
### extras as packed tarballs
- GNU make: Required by the other extras
- xz: Not for anything yet, but requires working around libtool to build so including it
- lua: For oasis
- pkgconf: For iproute2
- iproute2: Basic networking configuration
- bearssl: small TLS library
- tiny-curl: Still pretty big but hopefully close enough to the usual one
- cacert (from curl.haxx.se): Maybe could be reduced to a handful of CAs (like maybe even just Let's Encrypt)
- gettext-tiny: Much smaller replacement to GNU gettext, for git
- git: For oasis
- e2fsprogs: In case you need to format some extra storage
You can automatically build all of the above with `/extras-build-all.sh` but be warned that those aren't as curated and due to lack of deblob, they might contain binaries and some use autotools which harms reviewability.
You can also exclude those tarballs from the initrd with setting the `EXCLUDE_EXTRAS` environment variable to any value.
## Launching in QEMU
* You need a Linux kernel, so far no known version limitations
* Combination of `panic=1` and `-no-reboot` allows to exit+relaunch
```
$ ./make-initrd.sh && qemu-system-x86_64 -enable-kvm -m 512 -kernel /boot/vmlinuz-6.6.21-gentoo -initrd bootstrap-initrd/x86_64.cpio.gz -append 'init=/init console=ttyS0 panic=1' -nographic -no-reboot
```
## Dev setup via bubblewrap
```
$ ./make-root.sh && bwrap --clearenv --unshare-all --bind bootstrap-initrd/x86_64/ / --proc /proc --dev /dev --uid 0 /init
```
Useful as it allows to manipulate the environment externally, for example to edit files in an editor more comfortable than `ed(1)`.