logo

drewdevault.com

[mirror] blog and personal website of Drew DeVault git clone https://hacktivis.me/git/mirror/drewdevault.com.git

hare-ssh.md (32420B)


  1. ---
  2. title: Implementing an SSH agent in Hare
  3. author: Drew DeVault
  4. date: 2022-05-09
  5. ---
  6. *Cross-posted from [the Hare blog](https://harelang.org/blog/2022-05-09-hare-ssh/)*
  7. In the process of writing an SSH agent for [Himitsu], I needed to implement many
  8. SSH primitives from the ground up in Hare, now available via [hare-ssh]. Today,
  9. I'm going to show you how it works!
  10. [Himitsu]: https://sr.ht/~sircmpwn/himitsu
  11. [hare-ssh]: https://sr.ht/~sircmpwn/hare-ssh
  12. **Important**: This blog post deals with cryptography-related code. The code
  13. you're going to see today is incomplete, unaudited, and largely hasn't even seen
  14. any code review. Let me begin with a quote from the "crypto" module's
  15. documentation in the Hare standard library:
  16. > Cryptography is a difficult, high-risk domain of programming. The life and
  17. > well-being of your users may depend on your ability to implement cryptographic
  18. > applications with due care. Please carefully read all of the documentation,
  19. > double-check your work, and seek second opinions and independent review of
  20. > your code. Our documentation and API design aims to prevent easy mistakes from
  21. > being made, but it is no substitute for a good background in applied
  22. > cryptography.
  23. Do your due diligence before repurposing anything you see here.
  24. ## Decoding SSH private keys
  25. Technically, you do not need to deal with OpenSSH private keys when implementing
  26. an SSH agent. However, my particular use-case includes dealing with this format,
  27. so I started here. Unlike much of SSH, the OpenSSH private key format (i.e. the
  28. format of the file at ~/.ssh/id_ed25519) is, well, private. It's not
  29. documented and I had to get most of the details from reverse-engineering the
  30. OpenSSH C code. The main area of interest is sshkey.c. I'll spare you from
  31. reading it yourself and just explain how it works.
  32. First of all, let's just consider what an SSH private key looks like:
  33. ```
  34. -----BEGIN OPENSSH PRIVATE KEY-----
  35. b3BlbnNzaC1rZXktdjEAAAAACmFlczI1Ni1jdHIAAAAGYmNyeXB0AAAAGAAAABDTIm/zSI
  36. 7zeHAs4rIXaOD1AAAAEAAAAAEAAAAzAAAAC3NzaC1lZDI1NTE5AAAAIE7qq/pMk9VrRupn
  37. 9j4/tNHclJnKgJAE1pfUecRNT1fAAAAAoEcx6mnJmFlYXx1eYztw6SZ5yuL6T1LWfj+bpg
  38. 7zNQBoqJW1j+Q17PUMtXj9wDDOQx+6OE7JT/RrK3Vltp4oXmFI4FgsYbE9RbNXSC2xvLaX
  39. fplmx+eAOir9UTZGTIbOGy1cVho8LzDLLo4WiGYbpxtIvkJE72f0YdTm8RrNVkLlAy7ayV
  40. uFcoq1JBrjIAa7UtqIr9SG8b76ALJZb9jPc3A=
  41. -----END OPENSSH PRIVATE KEY-----
  42. ```
  43. We can immediately tell that this is a [PEM] file ([RFC 7468]). The first step
  44. to read this file was to implement a decoder for the PEM format, which has been
  45. on our to-do list for a while now, and is also needed for many other use-cases.
  46. Similar to many other formats provided in the standard library, you can call
  47. [pem::newdecoder] to create a PEM decoder for an arbitrary I/O source, returning
  48. the decoder state on the stack. We can then call [pem::next] to find the next
  49. PEM header (`-----BEGIN...`), which returns a decoder for that specific PEM blob
  50. (this design accommodates PEM files which have several PEM segments concatenated
  51. together, or intersperse other data in the file alongside the PEM bits. This is
  52. common for other PEM use-cases). With this secondary decoder, we can simply read
  53. from it like any other I/O source and it decodes the base64-encoded data and
  54. returns it to us as bytes.
  55. [PEM]: https://docs.harelang.org/encoding/pem
  56. [RFC 7468]: https://www.rfc-editor.org/rfc/rfc7468
  57. [pem::newdecoder]: https://docs.harelang.org/encoding/pem#newdecoder
  58. [pem::next]: https://docs.harelang.org/encoding/pem#next
  59. Based on this, we can examine the contents of this key with a simple program.
  60. ```hare
  61. use encoding::hex;
  62. use encoding::pem;
  63. use fmt;
  64. use io;
  65. use os;
  66. export fn main() void = {
  67. const dec = pem::newdecoder(os::stdin);
  68. defer pem::finish(&dec);
  69. for (true) {
  70. const reader = match (pem::next(&dec)) {
  71. case let reader: (str, pem::pemdecoder) =>
  72. yield reader;
  73. case io::EOF =>
  74. break;
  75. };
  76. const name = reader.0, stream = reader.1;
  77. defer io::close(&stream)!;
  78. fmt::printfln("PEM data '{}':", name)!;
  79. const bytes = io::drain(&stream)!;
  80. defer free(bytes);
  81. hex::dump(os::stdout, bytes)!;
  82. };
  83. };
  84. ```
  85. Running this program on our sample key yields the following:
  86. ```
  87. PEM data 'OPENSSH PRIVATE KEY':
  88. 00000000 6f 70 65 6e 73 73 68 2d 6b 65 79 2d 76 31 00 00 |openssh-key-v1..|
  89. 00000010 00 00 0a 61 65 73 32 35 36 2d 63 74 72 00 00 00 |...aes256-ctr...|
  90. 00000020 06 62 63 72 79 70 74 00 00 00 18 00 00 00 10 d3 |.bcrypt.........|
  91. 00000030 22 6f f3 48 8e f3 78 70 2c e2 b2 17 68 e0 f5 00 |"o.H..xp,...h...|
  92. 00000040 00 00 10 00 00 00 01 00 00 00 33 00 00 00 0b 73 |..........3....s|
  93. 00000050 73 68 2d 65 64 32 35 35 31 39 00 00 00 20 4e ea |sh-ed25519... N.|
  94. 00000060 ab fa 4c 93 d5 6b 46 ea 67 f6 3e 3f b4 d1 dc 94 |..L..kF.g.>?....|
  95. 00000070 99 ca 80 90 04 d6 97 d4 79 c4 4d 4f 57 c0 00 00 |........y.MOW...|
  96. 00000080 00 a0 47 31 ea 69 c9 98 59 58 5f 1d 5e 63 3b 70 |..G1.i..YX_.^c;p|
  97. 00000090 e9 26 79 ca e2 fa 4f 52 d6 7e 3f 9b a6 0e f3 35 |.&y...OR.~?....5|
  98. 000000a0 00 68 a8 95 b5 8f e4 35 ec f5 0c b5 78 fd c0 30 |.h.....5....x..0|
  99. 000000b0 ce 43 1f ba 38 4e c9 4f f4 6b 2b 75 65 b6 9e 28 |.C..8N.O.k+ue..(|
  100. 000000c0 5e 61 48 e0 58 2c 61 b1 3d 45 b3 57 48 2d b1 bc |^aH.X,a.=E.WH-..|
  101. 000000d0 b6 97 7e 99 66 c7 e7 80 3a 2a fd 51 36 46 4c 86 |..~.f...:*.Q6FL.|
  102. 000000e0 ce 1b 2d 5c 56 1a 3c 2f 30 cb 2e 8e 16 88 66 1b |..-\V.</0.....f.|
  103. 000000f0 a7 1b 48 be 42 44 ef 67 f4 61 d4 e6 f1 1a cd 56 |..H.BD.g.a.....V|
  104. 00000100 42 e5 03 2e da c9 5b 85 72 8a b5 24 1a e3 20 06 |B.....[.r..$.. .|
  105. 00000110 bb 52 da 88 af d4 86 f1 be fa 00 b2 59 6f d8 cf |.R..........Yo..|
  106. 00000120 73 70 |sp|
  107. ```
  108. OpenSSH private keys begin with a magic string, "openssh-key-v1\0", which we can
  109. see here. Following this are a number of binary encoded fields which are
  110. represented in a manner similar to the SSH wire protocol, most often as strings
  111. prefixed by their length, encoded as a 32-bit big-endian integer. In order, the
  112. fields present here are:
  113. - Cipher name (aes256-ctr)
  114. - KDF name (bcrypt)
  115. - KDF data
  116. - Public key data
  117. - Private key data (plus padding)
  118. We parse this information like so:
  119. ```hare
  120. export type sshprivkey = struct {
  121. cipher: str,
  122. kdfname: str,
  123. kdf: []u8,
  124. pubkey: []u8,
  125. privkey: []u8,
  126. };
  127. export fn decodesshprivate(in: io::handle) (sshprivkey | error) = {
  128. const pem = pem::newdecoder(in);
  129. const dec = match (pem::next(&pem)?) {
  130. case io::EOF =>
  131. return invalid;
  132. case let dec: (str, pem::pemdecoder) =>
  133. if (dec.0 != "OPENSSH PRIVATE KEY") {
  134. return invalid;
  135. };
  136. yield dec.1;
  137. };
  138. let magicbuf: [15]u8 = [0...];
  139. match (io::readall(&dec, magicbuf)?) {
  140. case size => void;
  141. case io::EOF =>
  142. return invalid;
  143. };
  144. if (!bytes::equal(magicbuf, strings::toutf8(magic))) {
  145. return invalid;
  146. };
  147. let key = sshprivkey { ... };
  148. key.cipher = readstr(&dec)?;
  149. key.kdfname = readstr(&dec)?;
  150. key.kdf = readslice(&dec)?;
  151. let buf: [4]u8 = [0...];
  152. match (io::readall(&dec, buf)?) {
  153. case size => void;
  154. case io::EOF =>
  155. return invalid;
  156. };
  157. const nkey = endian::begetu32(buf);
  158. if (nkey != 1) {
  159. // OpenSSH currently hard-codes the number of keys to 1
  160. return invalid;
  161. };
  162. key.pubkey = readslice(&dec)?;
  163. key.privkey = readslice(&dec)?;
  164. // Add padding bytes
  165. append(key.privkey, io::drain(&dec)?...);
  166. return key;
  167. };
  168. ```
  169. However, to get at the actual private key &mdash; so that we can do
  170. cryptographic operations with it &mdash; we first have to decrypt this inner
  171. data. Those three fields &mdash; cipher name, KDF name, and KDF data &mdash; are
  172. our hint. In essence, this data is encrypted by OpenSSH by using a variant of
  173. [bcrypt] as a [key derivation function], which turns your password (plus a salt)
  174. into a symmetric encryption key. Then it uses [AES 256] in [CTR mode] with this
  175. symmetric key to encrypt the private key data. With the benefit of hindsight, I
  176. might question these primitives, but that's what they use so we'll have to work
  177. with it.
  178. [bcrypt]: https://en.wikipedia.org/wiki/Bcrypt
  179. [key derivation function]: https://en.wikipedia.org/wiki/Key_derivation_function
  180. [AES 256]: https://en.wikipedia.org/wiki/Advanced_Encryption_Standard
  181. [CTR mode]: https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Counter_(CTR)
  182. Prior to starting this work, Hare already had support for [AES] and [CTR],
  183. though they gained some upgrades during the course of this work, since using an
  184. interface for real-world code is the best way to evaluate its design. This
  185. leaves us to implement bcrypt.
  186. [AES]: https://docs.harelang.org/crypto/aes
  187. [CTR]: https://docs.harelang.org/crypto/cipher#ctr
  188. bcrypt is a password hashing algorithm invented by OpenBSD based on the
  189. [Blowfish] cipher, and it is pretty badly designed. However, Blowfish was fairly
  190. straightforward to implement. I'll spare you the details, but here's the
  191. [documentation](https://docs.harelang.org/crypto/blowfish) and
  192. [implementation](https://git.sr.ht/~sircmpwn/hare/tree/master/crypto/blowfish)
  193. for your consideration. I also implemented the standard bcrypt hash at
  194. [crypto::bcrypt], whose implementation is
  195. [here](https://git.sr.ht/~sircmpwn/hare/tree/master/crypto/bcrypt) (for now).
  196. This isn't especially relevant for us, however, since OpenSSH uses a modified
  197. form of bcrypt as a key derivation function.
  198. [Blowfish]: https://en.wikipedia.org/wiki/Blowfish_(cipher)
  199. [crypto::bcrypt]: https://docs.harelang.org/crypto/bcrypt
  200. The [implementation](https://git.sr.ht/~sircmpwn/hare-ssh/tree/master/item/format/ssh/bcrypt.ha)
  201. the bcrypt KDF in Hare is fairly straightforward. To write it, I referenced
  202. OpenSSH portable's vendored OpenBSD implementation at
  203. `openbsd-compat/bcrypt_pbkdf.c`, as well as the Go implementation in
  204. `golang.org/x/crypto`. Then, with these primitives done, we can implement the
  205. actual key decryption.
  206. First, not all keys are encrypted with a passphrase, so a simple function tells
  207. us if this step is required:
  208. ```hare
  209. // Returns true if this private key is encrypted with a passphrase.
  210. export fn isencrypted(key: *sshprivkey) bool = {
  211. return key.kdfname != "none";
  212. };
  213. ```
  214. The "decrypt" function is used to perform the actual decryption. It begins by
  215. finding the symmetric key, like so:
  216. ```hare
  217. export fn decrypt(key: *sshprivkey, pass: []u8) (void | error) = {
  218. assert(isencrypted(key));
  219. const cipher = getcipher(key.cipher)?;
  220. let ckey: []u8 = alloc([0...], cipher.keylen + cipher.ivlen);
  221. defer {
  222. bytes::zero(ckey);
  223. free(ckey);
  224. };
  225. let kdfbuf = bufio::fixed(key.kdf, io::mode::READ);
  226. switch (key.kdfname) {
  227. case "bcrypt" =>
  228. const salt = readslice(&kdfbuf)?;
  229. defer free(salt);
  230. const rounds = readu32(&kdfbuf)?;
  231. bcrypt_pbkdf(ckey, pass, salt, rounds);
  232. case =>
  233. return badcipher;
  234. };
  235. ```
  236. The "KDF data" field I mentioned earlier uses a format private to each KDF mode,
  237. though at the present time the only supported KDF is this bcrypt one. In this
  238. case, it serves as the salt. The "getcipher" function returns some data from a
  239. [static table of supported ciphers][ciphers], which provides us with the
  240. required size of the cipher's key and IV parameters. We allocate sufficient
  241. space to store these, create a [bufio reader][bufio::fixed] from the KDF field,
  242. read out the salt and hashing rounds, and hand all of this over to the bcrypt
  243. function to produce our symmetric key (and I/V) in the "ckey" variable.
  244. [ciphers]: https://git.sr.ht/~sircmpwn/hare-ssh/tree/master/item/format/ssh/cipher.ha
  245. [bufio::fixed]: https://docs.harelang.org/bufio#fixed
  246. We may then use these parameters to decrypt the private key area.
  247. ```hare
  248. let secretbuf = bufio::fixed(key.privkey, io::mode::READ);
  249. const cipher = cipher.init(&secretbuf,
  250. ckey[..cipher.keylen], ckey[cipher.keylen..]);
  251. defer cipher_free(cipher);
  252. let buf: []u8 = alloc([0...], len(key.privkey));
  253. defer free(buf);
  254. io::readall(cipher, buf)!;
  255. const a = endian::begetu32(buf[..4]);
  256. const b = endian::begetu32(buf[4..8]);
  257. if (a != b) {
  258. return badpass;
  259. };
  260. key.privkey[..] = buf[..];
  261. free(key.kdf);
  262. free(key.kdfname);
  263. free(key.cipher);
  264. key.kdfname = strings::dup("none");
  265. key.cipher = strings::dup("none");
  266. key.kdf = [];
  267. };
  268. ```
  269. The "cipher.init" function is an abstraction that allows us to support more
  270. ciphers in the future. For this particular cipher mode, it's implemented fairly
  271. simply:
  272. ```hare
  273. type aes256ctr = struct {
  274. st: cipher::ctr_stream,
  275. block: aes::ct64_block,
  276. buf: [aes::CTR_BUFSIZE]u8,
  277. };
  278. fn aes256ctr_init(handle: io::handle, key: []u8, iv: []u8) *io::stream = {
  279. let state = alloc(aes256ctr {
  280. block = aes::ct64(),
  281. ...
  282. });
  283. aes::ct64_init(&state.block, key);
  284. state.st = cipher::ctr(handle, &state.block, iv, state.buf);
  285. return state;
  286. };
  287. ```
  288. Within this private key data section, once decrypted, are several fields. First
  289. is a random 32-bit integer which is written twice &mdash; comparing that these
  290. are equal to one another allows us to verify the user's password. Once verified,
  291. we overwrite the private data field in the key structure with the decrypted
  292. data, and update the cipher and KDF information to indicate that the key is
  293. unencrypted. We could decrypt it directly into the existing private key buffer,
  294. without allocating a second buffer, but this would overwrite the encrypted data
  295. with garbage if the password was wrong &mdash; you'd have to decode the key all
  296. over again if the user wants to try again.
  297. So, what does this private key blob look like once decrypted? The hare-ssh
  298. repository includes a little program at `cmd/sshkey` which dumps all of the
  299. information stored in an SSH key, and it provides us with this peek at the
  300. private data:
  301. ```
  302. 00000000 fb 15 e6 16 fb 15 e6 16 00 00 00 0b 73 73 68 2d |............ssh-|
  303. 00000010 65 64 32 35 35 31 39 00 00 00 20 4e ea ab fa 4c |ed25519... N...L|
  304. 00000020 93 d5 6b 46 ea 67 f6 3e 3f b4 d1 dc 94 99 ca 80 |..kF.g.>?.......|
  305. 00000030 90 04 d6 97 d4 79 c4 4d 4f 57 c0 00 00 00 40 17 |.....y.MOW....@.|
  306. 00000040 bf 87 74 0b 2a 74 d5 29 d0 14 10 3f 04 5d 88 c6 |..t.*t.)...?.]..|
  307. 00000050 32 fa 21 9c e9 97 b0 5a e7 7e 5c 02 72 35 72 4e |2.!....Z.~\.r5rN|
  308. 00000060 ea ab fa 4c 93 d5 6b 46 ea 67 f6 3e 3f b4 d1 dc |...L..kF.g.>?...|
  309. 00000070 94 99 ca 80 90 04 d6 97 d4 79 c4 4d 4f 57 c0 00 |.........y.MOW..|
  310. 00000080 00 00 0e 73 69 72 63 6d 70 77 6e 40 74 61 69 67 |...sircmpwn@taig|
  311. 00000090 61 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f |a...............|
  312. ```
  313. We can see upfront these two 32-bit verification numbers I mentioned, and
  314. following this are several fields in a similar format to earlier &mdash;
  315. length-prefixed strings. The fields are:
  316. - Key type ("ssh-ed25519" in this case)
  317. - Public key (in a format specific to each key type)
  318. - Private key (in a format specific to each key type)
  319. - Comment
  320. - Padding up to the cipher's block size (16)
  321. This is a little bit weird in my opinion &mdash; the public key field is
  322. redundant with the unencrypted data in this file, and the comment field is
  323. probably not so secret as to demand encryption. I think these are just
  324. consequences of the file format being private to OpenSSH's implementation; not
  325. much thought has gone into it and implementation details (like the ability to
  326. call the same "dump private key" function here as OpenSSH uses elsewhere) have
  327. probably leaked through.
  328. We can decode this data with the following Hare code:
  329. ```hare
  330. export fn decodeprivate(src: *sshprivkey) (key | error) = {
  331. assert(!isencrypted(src));
  332. const buf = bufio::fixed(src.privkey, io::mode::READ);
  333. let verify: [8]u8 = [0...];
  334. io::read(&buf, verify)!;
  335. const a = endian::begetu32(verify[..4]);
  336. const b = endian::begetu32(verify[4..8]);
  337. if (a != b) {
  338. return badpass;
  339. };
  340. const keytype = readstr(&buf)?;
  341. defer free(keytype);
  342. switch (keytype) {
  343. case "ssh-ed25519" =>
  344. let key = ed25519key { ... };
  345. decode_ed25519_sk(&key, &buf)?;
  346. return key;
  347. case =>
  348. // TODO: Support additional key types
  349. return badcipher;
  350. };
  351. };
  352. // An ed25519 key pair.
  353. export type ed25519key = struct {
  354. pkey: ed25519::publickey,
  355. skey: ed25519::privatekey,
  356. comment: str,
  357. };
  358. fn decode_ed25519_pk(key: *ed25519key, buf: io::handle) (void | error) = {
  359. const l = readu32(buf)?;
  360. if (l != ed25519::PUBLICKEYSZ) {
  361. return invalid;
  362. };
  363. io::readall(buf, key.pkey)?;
  364. };
  365. fn decode_ed25519_sk(key: *ed25519key, buf: io::handle) (void | error) = {
  366. decode_ed25519_pk(key, buf)?;
  367. const l = readu32(buf)?;
  368. if (l != ed25519::PRIVATEKEYSZ) {
  369. return invalid;
  370. };
  371. io::readall(buf, key.skey)?;
  372. // Sanity check
  373. const pkey = ed25519::skey_getpublic(&key.skey);
  374. if (!bytes::equal(pkey, key.pkey)) {
  375. return invalid;
  376. };
  377. key.comment = readstr(buf)?;
  378. };
  379. ```
  380. Fairly straightforward! Finally, we have extracted the actual private key from
  381. the file. For this SSH key, in base64, the cryptographic keys are:
  382. ```
  383. Public key: Tuqr+kyT1WtG6mf2Pj+00dyUmcqAkATWl9R5xE1PV8A=
  384. Private key: F7+HdAsqdNUp0BQQPwRdiMYy+iGc6ZewWud+XAJyNXJO6qv6TJPVa0bqZ/Y+P7TR3JSZyoCQBNaX1HnETU9XwA==
  385. ```
  386. ## Signing and verification with ed25519
  387. Using these private keys, implementing signatures and signature verification are
  388. pretty straightforward. We can stop reading the OpenSSH code at this point
  389. &mdash; [RFC 8709] standardizes this format for ed25519 signatures.
  390. [RFC 8709]: https://datatracker.ietf.org/doc/html/rfc8709
  391. ```hare
  392. use crypto::ed25519;
  393. use io;
  394. // Signs a message using the provided key, writing the message signature in the
  395. // SSH format to the provided sink.
  396. export fn sign(
  397. sink: io::handle,
  398. key: *key,
  399. msg: []u8,
  400. ) (void | io::error) = {
  401. const signature = ed25519::sign(&key.skey, msg);
  402. writestr(sink, "ssh-ed25519")?;
  403. writeslice(sink, signature)?;
  404. };
  405. // Reads an SSH wire signature from the provided I/O handle and verifies that it
  406. // is a valid signature for the given message and key. If valid, void is
  407. // returned; otherwise [[badsig]] is returned.
  408. export fn verify(
  409. source: io::handle,
  410. key: *key,
  411. msg: []u8,
  412. ) (void | error) = {
  413. const sigtype = readstr(source)?;
  414. defer free(sigtype);
  415. if (sigtype != keytype(key)) {
  416. return badsig;
  417. };
  418. const sig = readslice(source)?;
  419. defer free(sig);
  420. assert(sigtype == "ssh-ed25519"); // TODO: other key types
  421. if (len(sig) != ed25519::SIGNATURESZ) {
  422. return badsig;
  423. };
  424. const sig = sig: *[*]u8: *[ed25519::SIGNATURESZ]u8;
  425. if (!ed25519::verify(&key.pkey, msg, sig)) {
  426. return badsig;
  427. };
  428. };
  429. ```
  430. This implementation writes and reads signatures in the SSH wire format, which is
  431. generally how they will be most useful in this context. This code will be
  432. expanded in the future with additional keys, such as RSA, once the necessary
  433. primitives are implemented for Hare's standard library.
  434. ## The SSH agent protocol
  435. The agent protocol is also standardized (albeit in draft form), so we refer to
  436. [draft-miller-ssh-agent-11] from this point onwards. It's fairly
  437. straightforward. The agent communicates over an unspecified protocol (Unix
  438. sockets in practice) by sending messages in the SSH wire format, which, again,
  439. mainly comes in the form of strings prefixed by their 32-bit length in network
  440. order.
  441. [draft-miller-ssh-agent-11]: https://tools.ietf.org/id/draft-miller-ssh-agent-11.html
  442. The first step for implementing net::ssh::agent starts with adding types for all
  443. of the data structures and enums for all of the constants, which you can find in
  444. [types.ha]. Each message begins with its length, then a message type (one byte)
  445. and a message payload; the structure of the latter varies with the message type.
  446. [types.ha]: https://git.sr.ht/~sircmpwn/hare-ssh/tree/master/item/net/ssh/agent/types.ha
  447. I started to approach this by writing some functions which, given a byte buffer
  448. that contains an SSH agent message, either parses it or asks for more data.
  449. ```hare
  450. export fn parse(msg: []u8) (message | size | invalid) = {
  451. if (len(msg) < 5) {
  452. return 5 - len(msg);
  453. };
  454. const ln = endian::begetu32(msg[..4]);
  455. if (len(msg) < 4 + ln) {
  456. return 4 + ln - len(msg);
  457. };
  458. const mtype = msg[4];
  459. const buf = bufio::fixed(msg[5..], io::mode::READ);
  460. switch (mtype) {
  461. case messagetype::REQUEST_IDENTITIES =>
  462. return request_identities;
  463. case messagetype::SIGN_REQUEST =>
  464. return parse_sign_request(&buf)?;
  465. case messagetype::ADD_IDENTITY =>
  466. return parse_add_identity(&buf)?;
  467. // ...trimmed for brevity, and also because it's full of TODOs...
  468. case =>
  469. return invalid.
  470. };
  471. };
  472. ```
  473. Each individual message payload includes its own parser, except for some
  474. messages (such as `REQUEST_IDENTITIES`), which have no payload. Here's what the
  475. parser for `SIGN_REQUEST` looks like:
  476. ```hare
  477. fn parse_sign_request(src: io::handle) (sign_request | invalid) = {
  478. return sign_request {
  479. key = readslice(src)?,
  480. data = readslice(src)?,
  481. flags = readu32(src)?: sigflag,
  482. };
  483. };
  484. ```
  485. Pretty straightforward! A more complex one is `ADD_IDENTITY`:
  486. ```hare
  487. fn parse_add_identity(src: io::handle) (add_identity | invalid) = {
  488. const keytype = readstr(src)?;
  489. // TODO: Support more key types
  490. const key: ssh::key = switch (keytype) {
  491. case "ssh-ed25519" =>
  492. let key = ssh::ed25519key { ... };
  493. const npub = readu32(src)?;
  494. if (npub != len(key.pkey)) {
  495. return invalid;
  496. };
  497. io::readall(src, key.pkey)!;
  498. const npriv = readu32(src)?;
  499. if (npriv != len(key.skey)) {
  500. return invalid;
  501. };
  502. io::readall(src, key.skey)!;
  503. yield key;
  504. case =>
  505. return invalid;
  506. };
  507. return add_identity {
  508. keytype = keytype,
  509. key = key,
  510. comment = readstr(src)?,
  511. };
  512. };
  513. ```
  514. One thing I'm not thrilled with in this code is memory management. In Hare,
  515. libraries like this one are not supposed to allocate memory if they can get away
  516. with it, and if they must, they should do it as conservatively as possible. This
  517. implementation does a lot of its own allocations, which is unfortunate. I might
  518. refactor it in the future to avoid this. A more subtle issue here is the memory
  519. leaks on errors &mdash; each of the readslice/readstr functions allocates data
  520. for its return value, but if they return an error, the ? operator will return
  521. immediately without freeing them. This is a known problem with Hare's language
  522. design, and while we have some ideas for addressing it, we have not completed
  523. any of them yet. This is one of a small number of goals for Hare which will
  524. likely require language changes prior to 1.0.
  525. We have a [little bit more code](https://git.sr.ht/~sircmpwn/hare-ssh/tree/master/item/net/ssh/agent/agent.ha)
  526. in net::ssh::agent, which you can check out if you like, but this covers most of
  527. it &mdash; time to move onto the daemon implementation.
  528. ## Completing our SSH agent
  529. The [ssh-agent] command in the hare-ssh tree is a simple (and non-production)
  530. implementation of an SSH agent based on this work. Let's go over its code to see
  531. how this all comes together to make it work.
  532. [ssh-agent]: https://git.sr.ht/~sircmpwn/hare-ssh/tree/master/item/cmd/ssh-agent/main.ha
  533. First, we set up a Unix socket, and somewhere to store our application state.
  534. ```hare
  535. let running: bool = true;
  536. type identity = struct {
  537. comment: str,
  538. privkey: ssh::key,
  539. pubkey: []u8,
  540. };
  541. type state = struct {
  542. identities: []identity,
  543. };
  544. export fn main() void = {
  545. let state = state { ... };
  546. const sockpath = "./socket";
  547. const listener = unix::listen(sockpath)!;
  548. defer {
  549. net::shutdown(listener);
  550. os::remove(sockpath)!;
  551. };
  552. os::chmod(sockpath, 0o700)!;
  553. log::printfln("Listening at {}", sockpath);
  554. ```
  555. We also need a main loop, but we need to clean up that Unix socket when we
  556. terminate, so we'll also set up some signal handlers.
  557. ```hare
  558. signal::handle(signal::SIGINT, &handle_signal);
  559. signal::handle(signal::SIGTERM, &handle_signal);
  560. for (running) {
  561. // ...stay tuned...
  562. };
  563. for (let i = 0z; i < len(state.identities); i += 1) {
  564. const ident = state.identities[i];
  565. ssh::key_finish(&ident.privkey);
  566. free(ident.pubkey);
  567. free(ident.comment);
  568. };
  569. log::printfln("Terminated.");
  570. };
  571. // ...elsewhere...
  572. fn handle_signal(sig: int, info: *signal::siginfo, ucontext: *void) void = {
  573. running = false;
  574. };
  575. ```
  576. The actual clean-up is handled by our "defer" statement at the start of "main".
  577. The semantics of signal handling on Unix are complex (and bad), and beyond the
  578. scope of this post, so hopefully you already grok them. Our stdlib [provides
  579. docs](https://docs.harelang.org/unix/signal), if you care to learn more, but
  580. also includes this warning:
  581. > Signal handling is stupidly complicated and easy to get wrong. The standard
  582. > library makes little effort to help you deal with this. Consult your local man
  583. > pages, particularly signal-safety(7) on Linux, and perhaps a local priest as
  584. > well. We advise you to get out of the signal handler as soon as possible, for
  585. > example via the "self-pipe trick".
  586. We also provide signalfds on platforms that support them (such as Linux), which
  587. is less fraught with issues. Good luck.
  588. Next: the main loop. This code accepts new clients, prepares an agent for them,
  589. and hands them off to a second function:
  590. ```hare
  591. const client = match (net::accept(listener)) {
  592. case errors::interrupted =>
  593. continue;
  594. case let err: net::error =>
  595. log::fatalf("Error: accept: {}", net::strerror(err));
  596. case let fd: io::file =>
  597. yield fd;
  598. };
  599. const agent = agent::new(client);
  600. defer agent::agent_finish(&agent);
  601. run(&state, &agent);
  602. ```
  603. This is a really simple event loop for a network daemon, and comes with one
  604. major limitation: no support for serving multiple clients connecting at once. If
  605. you're curious what a more robust network daemon looks like in Hare, consult the
  606. [Himitsu] code.
  607. [Himitsu]: https://git.sr.ht/~sircmpwn/himitsu/tree/master/item/cmd/himitsud/socket.ha
  608. The "run" function simply reads SSH agent commands and processes them, until the
  609. client disconnects.
  610. ```hare
  611. fn run(state: *state, agent: *agent::agent) void = {
  612. for (true) {
  613. const msg = match (agent::readmsg(agent)) {
  614. case (io::EOF | agent::error) =>
  615. break;
  616. case void =>
  617. continue;
  618. case let msg: agent::message =>
  619. yield msg;
  620. };
  621. defer agent::message_finish(&msg);
  622. const res = match (msg) {
  623. case agent::request_identities =>
  624. yield handle_req_ident(state, agent);
  625. case let msg: agent::add_identity =>
  626. yield handle_add_ident(state, &msg, agent);
  627. case let msg: agent::sign_request =>
  628. yield handle_sign_request(state, &msg, agent);
  629. case agent::extension =>
  630. const answer: agent::message = agent::extension_failure;
  631. agent::writemsg(agent, &answer)!;
  632. case => abort();
  633. };
  634. match (res) {
  635. case void => yield;
  636. case agent::error => abort();
  637. };
  638. };
  639. };
  640. ```
  641. Again, this is non-production code, and, among other things, is missing good
  642. error handling. The handlers for each message are fairly straightforward,
  643. however. Here's the handler for `REQUEST_IDENTITIES`:
  644. ```hare
  645. fn handle_req_ident(
  646. state: *state,
  647. agent: *agent::agent,
  648. ) (void | agent::error) = {
  649. let idents: agent::identities_answer = [];
  650. defer free(idents);
  651. for (let i = 0z; i < len(state.identities); i += 1) {
  652. const ident = &state.identities[i];
  653. append(idents, agent::identity {
  654. pubkey = ident.pubkey,
  655. comment = ident.comment,
  656. });
  657. };
  658. const answer: agent::message = idents;
  659. agent::writemsg(agent, &answer)!;
  660. };
  661. ```
  662. The first one to do something interesting is `ADD_IDENTITY`, which allows the
  663. user to supply SSH private keys to the agent to work with:
  664. ```hare
  665. fn handle_add_ident(
  666. state: *state,
  667. msg: *agent::add_identity,
  668. agent: *agent::agent,
  669. ) (void | agent::error) = {
  670. let sink = bufio::dynamic(io::mode::WRITE);
  671. ssh::encode_pubkey(&sink, &msg.key)!;
  672. append(state.identities, identity {
  673. comment = strings::dup(msg.comment),
  674. privkey = msg.key,
  675. pubkey = bufio::buffer(&sink),
  676. });
  677. const answer: agent::message = agent::agent_success;
  678. agent::writemsg(agent, &answer)?;
  679. log::printfln("Added key {}", msg.comment);
  680. };
  681. ```
  682. With these two messages, we can start to get the agent to do something
  683. relatively interesting: accepting and listing keys.
  684. ```
  685. $ hare run cmd/ssh-agent/
  686. [2022-05-09 17:39:12] Listening at ./socket
  687. ^Z[1]+ Stopped hare run cmd/ssh-agent/
  688. $ bg
  689. [1] hare run cmd/ssh-agent/
  690. $ export SSH_AUTH_SOCK=./socket
  691. $ ssh-add -l
  692. The agent has no identities.
  693. $ ssh-add ~/.ssh/id_ed25519
  694. Enter passphrase for /home/sircmpwn/.ssh/id_ed25519:
  695. Identity added: /home/sircmpwn/.ssh/id_ed25519 (sircmpwn@homura)
  696. 2022-05-09 17:39:31] Added key sircmpwn@homura
  697. $ ssh-add -l
  698. 256 SHA256:kPr5ZKTNE54TRHGSaanhcQYiJ56zSgcpKeLZw4/myEI sircmpwn@homura (ED25519)
  699. ```
  700. With the last message handler, we can upgrade from something "interesting" to
  701. something "useful":
  702. ```hare
  703. fn handle_sign_request(
  704. state: *state,
  705. msg: *agent::sign_request,
  706. agent: *agent::agent,
  707. ) (void | agent::error) = {
  708. let key: nullable *identity = null;
  709. for (let i = 0z; i < len(state.identities); i += 1) {
  710. let ident = &state.identities[i];
  711. if (bytes::equal(ident.pubkey, msg.key)) {
  712. key = ident;
  713. break;
  714. };
  715. };
  716. const key = match (key) {
  717. case let key: *identity =>
  718. yield key;
  719. case null =>
  720. const answer: agent::message = agent::agent_failure;
  721. agent::writemsg(agent, &answer)?;
  722. return;
  723. };
  724. let buf = bufio::dynamic(io::mode::WRITE);
  725. defer io::close(&buf)!;
  726. ssh::sign(&buf, &key.privkey, msg.data)!;
  727. const answer: agent::message = agent::sign_response {
  728. signature = bufio::buffer(&buf),
  729. };
  730. agent::writemsg(agent, &answer)?;
  731. log::printfln("Signed challenge with key {}", key.comment);
  732. };
  733. ```
  734. For performance reasons, it may be better to use a hash map in a production Hare
  735. program (and, as many commenters will be sure to point out, Hare does not
  736. provide a built-in hash map or generics). We select the desired key with a
  737. linear search, sign the provided payload, and return the signature to the
  738. client. Finally, the big pay-off:
  739. ```
  740. $ ssh git@git.sr.ht
  741. [2022-05-09 17:41:42] Signed challenge with key sircmpwn@homura
  742. PTY allocation request failed on channel 0
  743. Hi sircmpwn! You've successfully authenticated, but I do not provide an interactive shell. Bye!
  744. Connection to git.sr.ht closed.
  745. ```
  746. ## Incorporating it into Himitsu
  747. [Himitsu] was the motivation for all of this work, and I have yet to properly
  748. introduce it to the public. I will go into detail later, but in essence, Himitsu
  749. is a key-value store that stores some keys in plaintext and some keys encrypted,
  750. and acts as a more general form of a password manager. One of the things it can
  751. do (at least as of this week) is store your SSH private keys and act as an SSH
  752. agent, via a helper called [himitsu-ssh]. The user can import their private key
  753. from OpenSSH's private key format via the "hissh-import" tool, and then the
  754. "hissh-agent" daemon provides agent functionality via the Himitsu key store.
  755. [himitsu-ssh]: https://git.sr.ht/~sircmpwn/himitsu-ssh
  756. The user can import their SSH key like so:
  757. ```
  758. $ hissh-import < ~/.ssh/id_ed25519
  759. Enter SSH key passphrase:
  760. key proto=ssh type=ssh-ed25519 pkey=pF7SljE25sVLdWvInO4gfqpJbbjxI6j+tIUcNWzVTHU= skey! comment=sircmpwn@homura
  761. # Query the key store for keys matching proto=ssh:
  762. $ hiq proto=ssh
  763. proto=ssh type=ssh-ed25519 pkey=pF7SljE25sVLdWvInO4gfqpJbbjxI6j+tIUcNWzVTHU= skey! comment=sircmpwn@homura
  764. ```
  765. Then, when running the agent:
  766. <video src="https://mirror.drewdevault.com/hissh-agent.webm" controls muted></video>
  767. (Yes, I know that the GUI has issues. I slapped it together in C in an afternoon
  768. and it needs a lot of work. [Help wanted!][hiprompt-gtk])
  769. [hiprompt-gtk]: https://git.sr.ht/~sircmpwn/hiprompt-gtk
  770. Ta-da!
  771. ## What's next?
  772. I accomplished my main goal, which was getting my SSH setup working with
  773. Himitsu. The next steps for expanding hare-ssh are:
  774. 1. Expanding the supported key types and ciphers (RSA, DSA, etc), which first
  775. requires implementing the primitives in the standard library
  776. 2. Implement the SSH connection protocol, which requires primitives like ECDH in
  777. the standard library. Some required primitives, like ChaCha, are already
  778. supported.
  779. 3. Improve the design of the networking code. hare-ssh is one of a very small
  780. number of network-facing Hare libraries, and it's treading new design ground
  781. here.
  782. SSH is a relatively small target for a cryptography implementation to aim for.
  783. I'm looking forward to using it as a testbed for our cryptographic suite. If
  784. you're interested in helping with any of these, [please get in
  785. touch][community]! If you're curious about Hare in general, check out the
  786. [language introduction][intro] to get started. Good luck!
  787. [intro]: https://harelang.org/tutorials/introduction/
  788. [community]: https://harelang.org/community/