logo

oasis-root

Compiled tree of Oasis Linux based on own branch at <https://hacktivis.me/git/oasis/> git clone https://anongit.hacktivis.me/git/oasis-root.git

xz.1 (77549B)


  1. '\" t
  2. .\" SPDX-License-Identifier: 0BSD
  3. .\"
  4. .\" Authors: Lasse Collin
  5. .\" Jia Tan
  6. .\"
  7. .TH XZ 1 "2025-03-08" "Tukaani" "XZ Utils"
  8. .
  9. .SH NAME
  10. xz, unxz, xzcat, lzma, unlzma, lzcat \- Compress or decompress .xz and .lzma files
  11. .
  12. .SH SYNOPSIS
  13. .B xz
  14. .RI [ option... ]
  15. .RI [ file... ]
  16. .
  17. .SH COMMAND ALIASES
  18. .B unxz
  19. is equivalent to
  20. .BR "xz \-\-decompress" .
  21. .br
  22. .B xzcat
  23. is equivalent to
  24. .BR "xz \-\-decompress \-\-stdout" .
  25. .br
  26. .B lzma
  27. is equivalent to
  28. .BR "xz \-\-format=lzma" .
  29. .br
  30. .B unlzma
  31. is equivalent to
  32. .BR "xz \-\-format=lzma \-\-decompress" .
  33. .br
  34. .B lzcat
  35. is equivalent to
  36. .BR "xz \-\-format=lzma \-\-decompress \-\-stdout" .
  37. .PP
  38. When writing scripts that need to decompress files,
  39. it is recommended to always use the name
  40. .B xz
  41. with appropriate arguments
  42. .RB ( "xz \-d"
  43. or
  44. .BR "xz \-dc" )
  45. instead of the names
  46. .B unxz
  47. and
  48. .BR xzcat .
  49. .
  50. .SH DESCRIPTION
  51. .B xz
  52. is a general-purpose data compression tool with
  53. command line syntax similar to
  54. .BR gzip (1)
  55. and
  56. .BR bzip2 (1).
  57. The native file format is the
  58. .B .xz
  59. format, but the legacy
  60. .B .lzma
  61. format used by LZMA Utils and
  62. raw compressed streams with no container format headers
  63. are also supported.
  64. In addition, decompression of the
  65. .B .lz
  66. format used by
  67. .B lzip
  68. is supported.
  69. .PP
  70. .B xz
  71. compresses or decompresses each
  72. .I file
  73. according to the selected operation mode.
  74. If no
  75. .I files
  76. are given or
  77. .I file
  78. is
  79. .BR \- ,
  80. .B xz
  81. reads from standard input and writes the processed data
  82. to standard output.
  83. .B xz
  84. will refuse (display an error and skip the
  85. .IR file )
  86. to write compressed data to standard output if it is a terminal.
  87. Similarly,
  88. .B xz
  89. will refuse to read compressed data
  90. from standard input if it is a terminal.
  91. .PP
  92. Unless
  93. .B \-\-stdout
  94. is specified,
  95. .I files
  96. other than
  97. .B \-
  98. are written to a new file whose name is derived from the source
  99. .I file
  100. name:
  101. .IP \(bu 3
  102. When compressing, the suffix of the target file format
  103. .RB ( .xz
  104. or
  105. .BR .lzma )
  106. is appended to the source filename to get the target filename.
  107. .IP \(bu 3
  108. When decompressing, the
  109. .BR .xz ,
  110. .BR .lzma ,
  111. or
  112. .B .lz
  113. suffix is removed from the filename to get the target filename.
  114. .B xz
  115. also recognizes the suffixes
  116. .B .txz
  117. and
  118. .BR .tlz ,
  119. and replaces them with the
  120. .B .tar
  121. suffix.
  122. .PP
  123. If the target file already exists, an error is displayed and the
  124. .I file
  125. is skipped.
  126. .PP
  127. Unless writing to standard output,
  128. .B xz
  129. will display a warning and skip the
  130. .I file
  131. if any of the following applies:
  132. .IP \(bu 3
  133. .I File
  134. is not a regular file.
  135. Symbolic links are not followed,
  136. and thus they are not considered to be regular files.
  137. .IP \(bu 3
  138. .I File
  139. has more than one hard link.
  140. .IP \(bu 3
  141. .I File
  142. has setuid, setgid, or sticky bit set.
  143. .IP \(bu 3
  144. The operation mode is set to compress and the
  145. .I file
  146. already has a suffix of the target file format
  147. .RB ( .xz
  148. or
  149. .B .txz
  150. when compressing to the
  151. .B .xz
  152. format, and
  153. .B .lzma
  154. or
  155. .B .tlz
  156. when compressing to the
  157. .B .lzma
  158. format).
  159. .IP \(bu 3
  160. The operation mode is set to decompress and the
  161. .I file
  162. doesn't have a suffix of any of the supported file formats
  163. .RB ( .xz ,
  164. .BR .txz ,
  165. .BR .lzma ,
  166. .BR .tlz ,
  167. or
  168. .BR .lz ).
  169. .PP
  170. After successfully compressing or decompressing the
  171. .IR file ,
  172. .B xz
  173. copies the owner, group, permissions, access time,
  174. and modification time from the source
  175. .I file
  176. to the target file.
  177. If copying the group fails, the permissions are modified
  178. so that the target file doesn't become accessible to users
  179. who didn't have permission to access the source
  180. .IR file .
  181. .B xz
  182. doesn't support copying other metadata like access control lists
  183. or extended attributes yet.
  184. .PP
  185. Once the target file has been successfully closed, the source
  186. .I file
  187. is removed unless
  188. .B \-\-keep
  189. was specified.
  190. The source
  191. .I file
  192. is never removed if the output is written to standard output
  193. or if an error occurs.
  194. .PP
  195. Sending
  196. .B SIGINFO
  197. or
  198. .B SIGUSR1
  199. to the
  200. .B xz
  201. process makes it print progress information to standard error.
  202. This has only limited use since when standard error
  203. is a terminal, using
  204. .B \-\-verbose
  205. will display an automatically updating progress indicator.
  206. .
  207. .SS "Memory usage"
  208. The memory usage of
  209. .B xz
  210. varies from a few hundred kilobytes to several gigabytes
  211. depending on the compression settings.
  212. The settings used when compressing a file determine
  213. the memory requirements of the decompressor.
  214. Typically the decompressor needs 5\ % to 20\ % of
  215. the amount of memory that the compressor needed when
  216. creating the file.
  217. For example, decompressing a file created with
  218. .B xz \-9
  219. currently requires 65\ MiB of memory.
  220. Still, it is possible to have
  221. .B .xz
  222. files that require several gigabytes of memory to decompress.
  223. .PP
  224. Especially users of older systems may find
  225. the possibility of very large memory usage annoying.
  226. To prevent uncomfortable surprises,
  227. .B xz
  228. has a built-in memory usage limiter, which is disabled by default.
  229. While some operating systems provide ways to limit
  230. the memory usage of processes, relying on it
  231. wasn't deemed to be flexible enough (for example, using
  232. .BR ulimit (1)
  233. to limit virtual memory tends to cripple
  234. .BR mmap (2)).
  235. .PP
  236. The memory usage limiter can be enabled with
  237. the command line option \fB\-\-memlimit=\fIlimit\fR.
  238. Often it is more convenient to enable the limiter
  239. by default by setting the environment variable
  240. .\" TRANSLATORS: Don't translate the uppercase XZ_DEFAULTS.
  241. .\" It's a name of an environment variable.
  242. .BR XZ_DEFAULTS ,
  243. for example,
  244. .BR XZ_DEFAULTS=\-\-memlimit=150MiB .
  245. It is possible to set the limits separately
  246. for compression and decompression by using
  247. .BI \-\-memlimit\-compress= limit
  248. and \fB\-\-memlimit\-decompress=\fIlimit\fR.
  249. Using these two options outside
  250. .B XZ_DEFAULTS
  251. is rarely useful because a single run of
  252. .B xz
  253. cannot do both compression and decompression and
  254. .BI \-\-memlimit= limit
  255. (or
  256. .B \-M
  257. .IR limit )
  258. is shorter to type on the command line.
  259. .PP
  260. If the specified memory usage limit is exceeded when decompressing,
  261. .B xz
  262. will display an error and decompressing the file will fail.
  263. If the limit is exceeded when compressing,
  264. .B xz
  265. will try to scale the settings down so that the limit
  266. is no longer exceeded (except when using
  267. .B \-\-format=raw
  268. or
  269. .BR \-\-no\-adjust ).
  270. This way the operation won't fail unless the limit is very small.
  271. The scaling of the settings is done in steps that don't
  272. match the compression level presets, for example, if the limit is
  273. only slightly less than the amount required for
  274. .BR "xz \-9" ,
  275. the settings will be scaled down only a little,
  276. not all the way down to
  277. .BR "xz \-8" .
  278. .
  279. .SS "Concatenation and padding with .xz files"
  280. It is possible to concatenate
  281. .B .xz
  282. files as is.
  283. .B xz
  284. will decompress such files as if they were a single
  285. .B .xz
  286. file.
  287. .PP
  288. It is possible to insert padding between the concatenated parts
  289. or after the last part.
  290. The padding must consist of null bytes and the size
  291. of the padding must be a multiple of four bytes.
  292. This can be useful, for example, if the
  293. .B .xz
  294. file is stored on a medium that measures file sizes
  295. in 512-byte blocks.
  296. .PP
  297. Concatenation and padding are not allowed with
  298. .B .lzma
  299. files or raw streams.
  300. .
  301. .SH OPTIONS
  302. .
  303. .SS "Integer suffixes and special values"
  304. In most places where an integer argument is expected,
  305. an optional suffix is supported to easily indicate large integers.
  306. There must be no space between the integer and the suffix.
  307. .TP
  308. .B KiB
  309. Multiply the integer by 1,024 (2^10).
  310. .BR Ki ,
  311. .BR k ,
  312. .BR kB ,
  313. .BR K ,
  314. and
  315. .B KB
  316. are accepted as synonyms for
  317. .BR KiB .
  318. .TP
  319. .B MiB
  320. Multiply the integer by 1,048,576 (2^20).
  321. .BR Mi ,
  322. .BR m ,
  323. .BR M ,
  324. and
  325. .B MB
  326. are accepted as synonyms for
  327. .BR MiB .
  328. .TP
  329. .B GiB
  330. Multiply the integer by 1,073,741,824 (2^30).
  331. .BR Gi ,
  332. .BR g ,
  333. .BR G ,
  334. and
  335. .B GB
  336. are accepted as synonyms for
  337. .BR GiB .
  338. .PP
  339. The special value
  340. .B max
  341. can be used to indicate the maximum integer value
  342. supported by the option.
  343. .
  344. .SS "Operation mode"
  345. If multiple operation mode options are given,
  346. the last one takes effect.
  347. .TP
  348. .BR \-z ", " \-\-compress
  349. Compress.
  350. This is the default operation mode when no operation mode option
  351. is specified and no other operation mode is implied from
  352. the command name (for example,
  353. .B unxz
  354. implies
  355. .BR \-\-decompress ).
  356. .IP ""
  357. .\" The DESCRIPTION section already says this but it's good to repeat it
  358. .\" here because the default behavior is a bit dangerous and new users
  359. .\" in a hurry may skip reading the DESCRIPTION section.
  360. After successful compression, the source file is removed
  361. unless writing to standard output or
  362. .B \-\-keep
  363. was specified.
  364. .TP
  365. .BR \-d ", " \-\-decompress ", " \-\-uncompress
  366. Decompress.
  367. .\" The DESCRIPTION section already says this but it's good to repeat it
  368. .\" here because the default behavior is a bit dangerous and new users
  369. .\" in a hurry may skip reading the DESCRIPTION section.
  370. After successful decompression, the source file is removed
  371. unless writing to standard output or
  372. .B \-\-keep
  373. was specified.
  374. .TP
  375. .BR \-t ", " \-\-test
  376. Test the integrity of compressed
  377. .IR files .
  378. This option is equivalent to
  379. .B "\-\-decompress \-\-stdout"
  380. except that the decompressed data is discarded instead of being
  381. written to standard output.
  382. No files are created or removed.
  383. .TP
  384. .BR \-l ", " \-\-list
  385. Print information about compressed
  386. .IR files .
  387. No uncompressed output is produced,
  388. and no files are created or removed.
  389. In list mode, the program cannot read
  390. the compressed data from standard
  391. input or from other unseekable sources.
  392. .IP ""
  393. The default listing shows basic information about
  394. .IR files ,
  395. one file per line.
  396. To get more detailed information, use also the
  397. .B \-\-verbose
  398. option.
  399. For even more information, use
  400. .B \-\-verbose
  401. twice, but note that this may be slow, because getting all the extra
  402. information requires many seeks.
  403. The width of verbose output exceeds
  404. 80 characters, so piping the output to, for example,
  405. .B "less\ \-S"
  406. may be convenient if the terminal isn't wide enough.
  407. .IP ""
  408. The exact output may vary between
  409. .B xz
  410. versions and different locales.
  411. For machine-readable output,
  412. .B \-\-robot \-\-list
  413. should be used.
  414. .
  415. .SS "Operation modifiers"
  416. .TP
  417. .BR \-k ", " \-\-keep
  418. Don't delete the input files.
  419. .IP ""
  420. Since
  421. .B xz
  422. 5.2.6,
  423. this option also makes
  424. .B xz
  425. compress or decompress even if the input is
  426. a symbolic link to a regular file,
  427. has more than one hard link,
  428. or has the setuid, setgid, or sticky bit set.
  429. The setuid, setgid, and sticky bits are not copied
  430. to the target file.
  431. In earlier versions this was only done with
  432. .BR \-\-force .
  433. .TP
  434. .BR \-f ", " \-\-force
  435. This option has several effects:
  436. .RS
  437. .IP \(bu 3
  438. If the target file already exists,
  439. delete it before compressing or decompressing.
  440. .IP \(bu 3
  441. Compress or decompress even if the input is
  442. a symbolic link to a regular file,
  443. has more than one hard link,
  444. or has the setuid, setgid, or sticky bit set.
  445. The setuid, setgid, and sticky bits are not copied
  446. to the target file.
  447. .IP \(bu 3
  448. When used with
  449. .B \-\-decompress
  450. .B \-\-stdout
  451. and
  452. .B xz
  453. cannot recognize the type of the source file,
  454. copy the source file as is to standard output.
  455. This allows
  456. .B xzcat
  457. .B \-\-force
  458. to be used like
  459. .BR cat (1)
  460. for files that have not been compressed with
  461. .BR xz .
  462. Note that in future,
  463. .B xz
  464. might support new compressed file formats, which may make
  465. .B xz
  466. decompress more types of files instead of copying them as is to
  467. standard output.
  468. .BI \-\-format= format
  469. can be used to restrict
  470. .B xz
  471. to decompress only a single file format.
  472. .RE
  473. .TP
  474. .BR \-c ", " \-\-stdout ", " \-\-to\-stdout
  475. Write the compressed or decompressed data to
  476. standard output instead of a file.
  477. This implies
  478. .BR \-\-keep .
  479. .TP
  480. .B \-\-single\-stream
  481. Decompress only the first
  482. .B .xz
  483. stream, and
  484. silently ignore possible remaining input data following the stream.
  485. Normally such trailing garbage makes
  486. .B xz
  487. display an error.
  488. .IP ""
  489. .B xz
  490. never decompresses more than one stream from
  491. .B .lzma
  492. files or raw streams, but this option still makes
  493. .B xz
  494. ignore the possible trailing data after the
  495. .B .lzma
  496. file or raw stream.
  497. .IP ""
  498. This option has no effect if the operation mode is not
  499. .B \-\-decompress
  500. or
  501. .BR \-\-test .
  502. .IP ""
  503. Since
  504. .B xz
  505. 5.7.1alpha,
  506. .B \-\-single\-stream
  507. implies
  508. .BR \-\-keep .
  509. .TP
  510. .B \-\-no\-sparse
  511. Disable creation of sparse files.
  512. By default, if decompressing into a regular file,
  513. .B xz
  514. tries to make the file sparse if the decompressed data contains
  515. long sequences of binary zeros.
  516. It also works when writing to standard output
  517. as long as standard output is connected to a regular file
  518. and certain additional conditions are met to make it safe.
  519. Creating sparse files may save disk space and speed up
  520. the decompression by reducing the amount of disk I/O.
  521. .TP
  522. \fB\-S\fR \fI.suf\fR, \fB\-\-suffix=\fI.suf
  523. When compressing, use
  524. .I .suf
  525. as the suffix for the target file instead of
  526. .B .xz
  527. or
  528. .BR .lzma .
  529. If not writing to standard output and
  530. the source file already has the suffix
  531. .IR .suf ,
  532. a warning is displayed and the file is skipped.
  533. .IP ""
  534. When decompressing, recognize files with the suffix
  535. .I .suf
  536. in addition to files with the
  537. .BR .xz ,
  538. .BR .txz ,
  539. .BR .lzma ,
  540. .BR .tlz ,
  541. or
  542. .B .lz
  543. suffix.
  544. If the source file has the suffix
  545. .IR .suf ,
  546. the suffix is removed to get the target filename.
  547. .IP ""
  548. When compressing or decompressing raw streams
  549. .RB ( \-\-format=raw ),
  550. the suffix must always be specified unless
  551. writing to standard output,
  552. because there is no default suffix for raw streams.
  553. .TP
  554. \fB\-\-files\fR[\fB=\fIfile\fR]
  555. Read the filenames to process from
  556. .IR file ;
  557. if
  558. .I file
  559. is omitted, filenames are read from standard input.
  560. Filenames must be terminated with the newline character.
  561. A dash
  562. .RB ( \- )
  563. is taken as a regular filename; it doesn't mean standard input.
  564. If filenames are given also as command line arguments, they are
  565. processed before the filenames read from
  566. .IR file .
  567. .TP
  568. \fB\-\-files0\fR[\fB=\fIfile\fR]
  569. This is identical to \fB\-\-files\fR[\fB=\fIfile\fR] except
  570. that each filename must be terminated with the null character.
  571. .
  572. .SS "Basic file format and compression options"
  573. .TP
  574. \fB\-F\fR \fIformat\fR, \fB\-\-format=\fIformat
  575. Specify the file
  576. .I format
  577. to compress or decompress:
  578. .RS
  579. .TP
  580. .\" TRANSLATORS: Don't translate bold string B<auto>.
  581. .B auto
  582. This is the default.
  583. When compressing,
  584. .B auto
  585. is equivalent to
  586. .BR xz .
  587. When decompressing,
  588. the format of the input file is automatically detected.
  589. Note that raw streams (created with
  590. .BR \-\-format=raw )
  591. cannot be auto-detected.
  592. .TP
  593. .B xz
  594. Compress to the
  595. .B .xz
  596. file format, or accept only
  597. .B .xz
  598. files when decompressing.
  599. .TP
  600. .BR lzma ", " alone
  601. Compress to the legacy
  602. .B .lzma
  603. file format, or accept only
  604. .B .lzma
  605. files when decompressing.
  606. The alternative name
  607. .B alone
  608. is provided for backwards compatibility with LZMA Utils.
  609. .TP
  610. .B lzip
  611. Accept only
  612. .B .lz
  613. files when decompressing.
  614. Compression is not supported.
  615. .IP ""
  616. The
  617. .B .lz
  618. format version 0 and the unextended version 1 are supported.
  619. Version 0 files were produced by
  620. .B lzip
  621. 1.3 and older.
  622. Such files aren't common but may be found from file archives
  623. as a few source packages were released in this format.
  624. People might have old personal files in this format too.
  625. Decompression support for the format version 0 was removed in
  626. .B lzip
  627. 1.18.
  628. .IP ""
  629. .B lzip
  630. 1.4 and later create files in the format version 1.
  631. The sync flush marker extension to the format version 1 was added in
  632. .B lzip
  633. 1.6.
  634. This extension is rarely used and isn't supported by
  635. .B xz
  636. (diagnosed as corrupt input).
  637. .TP
  638. .B raw
  639. Compress or uncompress a raw stream (no headers).
  640. This is meant for advanced users only.
  641. To decode raw streams, you need use
  642. .B \-\-format=raw
  643. and explicitly specify the filter chain,
  644. which normally would have been stored in the container headers.
  645. .RE
  646. .TP
  647. \fB\-C\fR \fIcheck\fR, \fB\-\-check=\fIcheck
  648. Specify the type of the integrity check.
  649. The check is calculated from the uncompressed data and
  650. stored in the
  651. .B .xz
  652. file.
  653. This option has an effect only when compressing into the
  654. .B .xz
  655. format; the
  656. .B .lzma
  657. format doesn't support integrity checks.
  658. The integrity check (if any) is verified when the
  659. .B .xz
  660. file is decompressed.
  661. .IP ""
  662. Supported
  663. .I check
  664. types:
  665. .RS
  666. .TP
  667. .\" TRANSLATORS: Don't translate the bold strings B<none>, B<crc32>,
  668. .\" B<crc64>, and B<sha256>. The command line option --check accepts
  669. .\" only the untranslated strings.
  670. .B none
  671. Don't calculate an integrity check at all.
  672. This is usually a bad idea.
  673. This can be useful when integrity of the data is verified
  674. by other means anyway.
  675. .TP
  676. .B crc32
  677. Calculate CRC32 using the polynomial from IEEE-802.3 (Ethernet).
  678. .TP
  679. .B crc64
  680. Calculate CRC64 using the polynomial from ECMA-182.
  681. This is the default, since it is slightly better than CRC32
  682. at detecting damaged files and the speed difference is negligible.
  683. .TP
  684. .B sha256
  685. Calculate SHA-256.
  686. This is somewhat slower than CRC32 and CRC64.
  687. .RE
  688. .IP ""
  689. Integrity of the
  690. .B .xz
  691. headers is always verified with CRC32.
  692. It is not possible to change or disable it.
  693. .TP
  694. .B \-\-ignore\-check
  695. Don't verify the integrity check of the compressed data when decompressing.
  696. The CRC32 values in the
  697. .B .xz
  698. headers will still be verified normally.
  699. .IP ""
  700. .B "Do not use this option unless you know what you are doing."
  701. Possible reasons to use this option:
  702. .RS
  703. .IP \(bu 3
  704. Trying to recover data from a corrupt .xz file.
  705. .IP \(bu 3
  706. Speeding up decompression.
  707. This matters mostly with SHA-256 or
  708. with files that have compressed extremely well.
  709. It's recommended to not use this option for this purpose
  710. unless the file integrity is verified externally in some other way.
  711. .RE
  712. .TP
  713. .BR \-0 " ... " \-9
  714. Select a compression preset level.
  715. The default is
  716. .BR \-6 .
  717. If multiple preset levels are specified,
  718. the last one takes effect.
  719. If a custom filter chain was already specified, setting
  720. a compression preset level clears the custom filter chain.
  721. .IP ""
  722. The differences between the presets are more significant than with
  723. .BR gzip (1)
  724. and
  725. .BR bzip2 (1).
  726. The selected compression settings determine
  727. the memory requirements of the decompressor,
  728. thus using a too high preset level might make it painful
  729. to decompress the file on an old system with little RAM.
  730. Specifically,
  731. .B "it's not a good idea to blindly use \-9 for everything"
  732. like it often is with
  733. .BR gzip (1)
  734. and
  735. .BR bzip2 (1).
  736. .RS
  737. .TP
  738. .BR "\-0" " ... " "\-3"
  739. These are somewhat fast presets.
  740. .B \-0
  741. is sometimes faster than
  742. .B "gzip \-9"
  743. while compressing much better.
  744. The higher ones often have speed comparable to
  745. .BR bzip2 (1)
  746. with comparable or better compression ratio,
  747. although the results
  748. depend a lot on the type of data being compressed.
  749. .TP
  750. .BR "\-4" " ... " "\-6"
  751. Good to very good compression while keeping
  752. decompressor memory usage reasonable even for old systems.
  753. .B \-6
  754. is the default, which is usually a good choice
  755. for distributing files that need to be decompressible
  756. even on systems with only 16\ MiB RAM.
  757. .RB ( \-5e
  758. or
  759. .B \-6e
  760. may be worth considering too.
  761. See
  762. .BR \-\-extreme .)
  763. .TP
  764. .B "\-7 ... \-9"
  765. These are like
  766. .B \-6
  767. but with higher compressor and decompressor memory requirements.
  768. These are useful only when compressing files bigger than
  769. 8\ MiB, 16\ MiB, and 32\ MiB, respectively.
  770. .RE
  771. .IP ""
  772. On the same hardware, the decompression speed is approximately
  773. a constant number of bytes of compressed data per second.
  774. In other words, the better the compression,
  775. the faster the decompression will usually be.
  776. This also means that the amount of uncompressed output
  777. produced per second can vary a lot.
  778. .IP ""
  779. The following table summarises the features of the presets:
  780. .RS
  781. .RS
  782. .PP
  783. .TS
  784. tab(;);
  785. c c c c c
  786. n n n n n.
  787. Preset;DictSize;CompCPU;CompMem;DecMem
  788. \-0;256 KiB;0;3 MiB;1 MiB
  789. \-1;1 MiB;1;9 MiB;2 MiB
  790. \-2;2 MiB;2;17 MiB;3 MiB
  791. \-3;4 MiB;3;32 MiB;5 MiB
  792. \-4;4 MiB;4;48 MiB;5 MiB
  793. \-5;8 MiB;5;94 MiB;9 MiB
  794. \-6;8 MiB;6;94 MiB;9 MiB
  795. \-7;16 MiB;6;186 MiB;17 MiB
  796. \-8;32 MiB;6;370 MiB;33 MiB
  797. \-9;64 MiB;6;674 MiB;65 MiB
  798. .TE
  799. .RE
  800. .RE
  801. .IP ""
  802. Column descriptions:
  803. .RS
  804. .IP \(bu 3
  805. DictSize is the LZMA2 dictionary size.
  806. It is waste of memory to use a dictionary bigger than
  807. the size of the uncompressed file.
  808. This is why it is good to avoid using the presets
  809. .BR \-7 " ... " \-9
  810. when there's no real need for them.
  811. At
  812. .B \-6
  813. and lower, the amount of memory wasted is
  814. usually low enough to not matter.
  815. .IP \(bu 3
  816. CompCPU is a simplified representation of the LZMA2 settings
  817. that affect compression speed.
  818. The dictionary size affects speed too,
  819. so while CompCPU is the same for levels
  820. .BR \-6 " ... " \-9 ,
  821. higher levels still tend to be a little slower.
  822. To get even slower and thus possibly better compression, see
  823. .BR \-\-extreme .
  824. .IP \(bu 3
  825. CompMem contains the compressor memory requirements
  826. in the single-threaded mode.
  827. It may vary slightly between
  828. .B xz
  829. versions.
  830. .IP \(bu 3
  831. DecMem contains the decompressor memory requirements.
  832. That is, the compression settings determine
  833. the memory requirements of the decompressor.
  834. The exact decompressor memory usage is slightly more than
  835. the LZMA2 dictionary size, but the values in the table
  836. have been rounded up to the next full MiB.
  837. .RE
  838. .IP ""
  839. Memory requirements of the multi-threaded mode are
  840. significantly higher than that of the single-threaded mode.
  841. With the default value of
  842. .BR \-\-block\-size ,
  843. each thread needs 3*3*DictSize plus CompMem or DecMem.
  844. For example, four threads with preset
  845. .B \-6
  846. needs 660\(en670\ MiB of memory.
  847. .TP
  848. .BR \-e ", " \-\-extreme
  849. Use a slower variant of the selected compression preset level
  850. .RB ( \-0 " ... " \-9 )
  851. to hopefully get a little bit better compression ratio,
  852. but with bad luck this can also make it worse.
  853. Decompressor memory usage is not affected,
  854. but compressor memory usage increases a little at preset levels
  855. .BR \-0 " ... " \-3 .
  856. .IP ""
  857. Since there are two presets with dictionary sizes
  858. 4\ MiB and 8\ MiB, the presets
  859. .B \-3e
  860. and
  861. .B \-5e
  862. use slightly faster settings (lower CompCPU) than
  863. .B \-4e
  864. and
  865. .BR \-6e ,
  866. respectively.
  867. That way no two presets are identical.
  868. .RS
  869. .RS
  870. .PP
  871. .TS
  872. tab(;);
  873. c c c c c
  874. n n n n n.
  875. Preset;DictSize;CompCPU;CompMem;DecMem
  876. \-0e;256 KiB;8;4 MiB;1 MiB
  877. \-1e;1 MiB;8;13 MiB;2 MiB
  878. \-2e;2 MiB;8;25 MiB;3 MiB
  879. \-3e;4 MiB;7;48 MiB;5 MiB
  880. \-4e;4 MiB;8;48 MiB;5 MiB
  881. \-5e;8 MiB;7;94 MiB;9 MiB
  882. \-6e;8 MiB;8;94 MiB;9 MiB
  883. \-7e;16 MiB;8;186 MiB;17 MiB
  884. \-8e;32 MiB;8;370 MiB;33 MiB
  885. \-9e;64 MiB;8;674 MiB;65 MiB
  886. .TE
  887. .RE
  888. .RE
  889. .IP ""
  890. For example, there are a total of four presets that use
  891. 8\ MiB dictionary, whose order from the fastest to the slowest is
  892. .BR \-5 ,
  893. .BR \-6 ,
  894. .BR \-5e ,
  895. and
  896. .BR \-6e .
  897. .TP
  898. .B \-\-fast
  899. .PD 0
  900. .TP
  901. .B \-\-best
  902. .PD
  903. These are somewhat misleading aliases for
  904. .B \-0
  905. and
  906. .BR \-9 ,
  907. respectively.
  908. These are provided only for backwards compatibility
  909. with LZMA Utils.
  910. Avoid using these options.
  911. .TP
  912. .BI \-\-block\-size= size
  913. When compressing to the
  914. .B .xz
  915. format, split the input data into blocks of
  916. .I size
  917. bytes.
  918. The blocks are compressed independently from each other,
  919. which helps with multi-threading and
  920. makes limited random-access decompression possible.
  921. This option is typically used to override the default
  922. block size in multi-threaded mode,
  923. but this option can be used in single-threaded mode too.
  924. .IP ""
  925. In multi-threaded mode about three times
  926. .I size
  927. bytes will be allocated in each thread for buffering input and output.
  928. The default
  929. .I size
  930. is three times the LZMA2 dictionary size or 1 MiB,
  931. whichever is more.
  932. Typically a good value is 2\(en4 times
  933. the size of the LZMA2 dictionary or at least 1 MiB.
  934. Using
  935. .I size
  936. less than the LZMA2 dictionary size is waste of RAM
  937. because then the LZMA2 dictionary buffer will never get fully used.
  938. In multi-threaded mode,
  939. the sizes of the blocks are stored in the block headers.
  940. This size information is required for multi-threaded decompression.
  941. .IP ""
  942. In single-threaded mode no block splitting is done by default.
  943. Setting this option doesn't affect memory usage.
  944. No size information is stored in block headers,
  945. thus files created in single-threaded mode
  946. won't be identical to files created in multi-threaded mode.
  947. The lack of size information also means that
  948. .B xz
  949. won't be able decompress the files in multi-threaded mode.
  950. .TP
  951. .BI \-\-block\-list= items
  952. When compressing to the
  953. .B .xz
  954. format, start a new block with an optional custom filter chain after
  955. the given intervals of uncompressed data.
  956. .IP ""
  957. The
  958. .I items
  959. are a comma-separated list.
  960. Each item consists of an optional filter chain number
  961. between 0 and 9 followed by a colon
  962. .RB ( : )
  963. and a required size of uncompressed data.
  964. Omitting an item (two or more consecutive commas) is a
  965. shorthand to use the size and filters of the previous item.
  966. .IP ""
  967. If the input file is bigger than the sum of
  968. the sizes in
  969. .IR items ,
  970. the last item is repeated until the end of the file.
  971. A special value of
  972. .B 0
  973. may be used as the last size to indicate that
  974. the rest of the file should be encoded as a single block.
  975. .IP ""
  976. An alternative filter chain for each block can be
  977. specified in combination with the
  978. .BI \-\-filters1= filters
  979. \&...\&
  980. .BI \-\-filters9= filters
  981. options.
  982. These options define filter chains with an identifier
  983. between 1\(en9.
  984. Filter chain 0 can be used to refer to the default filter chain,
  985. which is the same as not specifying a filter chain.
  986. The filter chain identifier can be used before the uncompressed
  987. size, followed by a colon
  988. .RB ( : ).
  989. For example, if one specifies
  990. .B \-\-block\-list=1:2MiB,3:2MiB,2:4MiB,,2MiB,0:4MiB
  991. then blocks will be created using:
  992. .RS
  993. .IP \(bu 3
  994. The filter chain specified by
  995. .B \-\-filters1
  996. and 2 MiB input
  997. .IP \(bu 3
  998. The filter chain specified by
  999. .B \-\-filters3
  1000. and 2 MiB input
  1001. .IP \(bu 3
  1002. The filter chain specified by
  1003. .B \-\-filters2
  1004. and 4 MiB input
  1005. .IP \(bu 3
  1006. The filter chain specified by
  1007. .B \-\-filters2
  1008. and 4 MiB input
  1009. .IP \(bu 3
  1010. The default filter chain and 2 MiB input
  1011. .IP \(bu 3
  1012. The default filter chain and 4 MiB input for every block until
  1013. end of input.
  1014. .RE
  1015. .IP ""
  1016. If one specifies a size that exceeds the encoder's block size
  1017. (either the default value in threaded mode or
  1018. the value specified with \fB\-\-block\-size=\fIsize\fR),
  1019. the encoder will create additional blocks while
  1020. keeping the boundaries specified in
  1021. .IR items .
  1022. For example, if one specifies
  1023. .B \-\-block\-size=10MiB
  1024. .B \-\-block\-list=5MiB,10MiB,8MiB,12MiB,24MiB
  1025. and the input file is 80 MiB,
  1026. one will get 11 blocks:
  1027. 5, 10, 8, 10, 2, 10, 10, 4, 10, 10, and 1 MiB.
  1028. .IP ""
  1029. In multi-threaded mode the sizes of the blocks
  1030. are stored in the block headers.
  1031. This isn't done in single-threaded mode,
  1032. so the encoded output won't be
  1033. identical to that of the multi-threaded mode.
  1034. .TP
  1035. .BI \-\-flush\-timeout= timeout
  1036. When compressing, if more than
  1037. .I timeout
  1038. milliseconds (a positive integer) has passed since the previous flush and
  1039. reading more input would block,
  1040. all the pending input data is flushed from the encoder and
  1041. made available in the output stream.
  1042. This can be useful if
  1043. .B xz
  1044. is used to compress data that is streamed over a network.
  1045. Small
  1046. .I timeout
  1047. values make the data available at the receiving end
  1048. with a small delay, but large
  1049. .I timeout
  1050. values give better compression ratio.
  1051. .IP ""
  1052. This feature is disabled by default.
  1053. If this option is specified more than once, the last one takes effect.
  1054. The special
  1055. .I timeout
  1056. value of
  1057. .B 0
  1058. can be used to explicitly disable this feature.
  1059. .IP ""
  1060. This feature is not available on non-POSIX systems.
  1061. .IP ""
  1062. .\" FIXME
  1063. .B "This feature is still experimental."
  1064. Currently
  1065. .B xz
  1066. is unsuitable for decompressing the stream in real time due to how
  1067. .B xz
  1068. does buffering.
  1069. .TP
  1070. .B \-\-no\-sync
  1071. Do not synchronize the target file and its directory
  1072. to the storage device before removing the source file.
  1073. This can improve performance if compressing or decompressing
  1074. many small files.
  1075. However, if the system crashes soon after the deletion,
  1076. it is possible that the target file was not written
  1077. to the storage device but the delete operation was.
  1078. In that case neither the original source file
  1079. nor the target file is available.
  1080. .IP ""
  1081. This option has an effect only when
  1082. .B xz
  1083. is going to remove the source file.
  1084. In other cases synchronization is never done.
  1085. .IP ""
  1086. The synchronization and
  1087. .B \-\-no\-sync
  1088. were added in
  1089. .B xz
  1090. 5.7.1alpha.
  1091. .TP
  1092. .BI \-\-memlimit\-compress= limit
  1093. Set a memory usage limit for compression.
  1094. If this option is specified multiple times,
  1095. the last one takes effect.
  1096. .IP ""
  1097. If the compression settings exceed the
  1098. .IR limit ,
  1099. .B xz
  1100. will attempt to adjust the settings downwards so that
  1101. the limit is no longer exceeded and display a notice that
  1102. automatic adjustment was done.
  1103. The adjustments are done in this order:
  1104. reducing the number of threads,
  1105. switching to single-threaded mode
  1106. if even one thread in multi-threaded mode exceeds the
  1107. .IR limit ,
  1108. and finally reducing the LZMA2 dictionary size.
  1109. .IP ""
  1110. When compressing with
  1111. .B \-\-format=raw
  1112. or if
  1113. .B \-\-no\-adjust
  1114. has been specified,
  1115. only the number of threads may be reduced
  1116. since it can be done without affecting the compressed output.
  1117. .IP ""
  1118. If the
  1119. .I limit
  1120. cannot be met even with the adjustments described above,
  1121. an error is displayed and
  1122. .B xz
  1123. will exit with exit status 1.
  1124. .IP ""
  1125. The
  1126. .I limit
  1127. can be specified in multiple ways:
  1128. .RS
  1129. .IP \(bu 3
  1130. The
  1131. .I limit
  1132. can be an absolute value in bytes.
  1133. Using an integer suffix like
  1134. .B MiB
  1135. can be useful.
  1136. Example:
  1137. .B "\-\-memlimit\-compress=80MiB"
  1138. .IP \(bu 3
  1139. The
  1140. .I limit
  1141. can be specified as a percentage of total physical memory (RAM).
  1142. This can be useful especially when setting the
  1143. .B XZ_DEFAULTS
  1144. environment variable in a shell initialization script
  1145. that is shared between different computers.
  1146. That way the limit is automatically bigger
  1147. on systems with more memory.
  1148. Example:
  1149. .B "\-\-memlimit\-compress=70%"
  1150. .IP \(bu 3
  1151. The
  1152. .I limit
  1153. can be reset back to its default value by setting it to
  1154. .BR 0 .
  1155. This is currently equivalent to setting the
  1156. .I limit
  1157. to
  1158. .B max
  1159. (no memory usage limit).
  1160. .RE
  1161. .IP ""
  1162. For 32-bit
  1163. .B xz
  1164. there is a special case: if the
  1165. .I limit
  1166. would be over
  1167. .BR "4020\ MiB" ,
  1168. the
  1169. .I limit
  1170. is set to
  1171. .BR "4020\ MiB" .
  1172. On MIPS32
  1173. .B "2000\ MiB"
  1174. is used instead.
  1175. (The values
  1176. .B 0
  1177. and
  1178. .B max
  1179. aren't affected by this.
  1180. A similar feature doesn't exist for decompression.)
  1181. This can be helpful when a 32-bit executable has access
  1182. to 4\ GiB address space (2 GiB on MIPS32)
  1183. while hopefully doing no harm in other situations.
  1184. .IP ""
  1185. See also the section
  1186. .BR "Memory usage" .
  1187. .TP
  1188. .BI \-\-memlimit\-decompress= limit
  1189. Set a memory usage limit for decompression.
  1190. This also affects the
  1191. .B \-\-list
  1192. mode.
  1193. If the operation is not possible without exceeding the
  1194. .IR limit ,
  1195. .B xz
  1196. will display an error and decompressing the file will fail.
  1197. See
  1198. .BI \-\-memlimit\-compress= limit
  1199. for possible ways to specify the
  1200. .IR limit .
  1201. .TP
  1202. .BI \-\-memlimit\-mt\-decompress= limit
  1203. Set a memory usage limit for multi-threaded decompression.
  1204. This can only affect the number of threads;
  1205. this will never make
  1206. .B xz
  1207. refuse to decompress a file.
  1208. If
  1209. .I limit
  1210. is too low to allow any multi-threading, the
  1211. .I limit
  1212. is ignored and
  1213. .B xz
  1214. will continue in single-threaded mode.
  1215. Note that if also
  1216. .B \-\-memlimit\-decompress
  1217. is used,
  1218. it will always apply to both single-threaded and multi-threaded modes,
  1219. and so the effective
  1220. .I limit
  1221. for multi-threading will never be higher than the limit set with
  1222. .BR \-\-memlimit\-decompress .
  1223. .IP ""
  1224. In contrast to the other memory usage limit options,
  1225. .BI \-\-memlimit\-mt\-decompress= limit
  1226. has a system-specific default
  1227. .IR limit .
  1228. .B "xz \-\-info\-memory"
  1229. can be used to see the current value.
  1230. .IP ""
  1231. This option and its default value exist
  1232. because without any limit the threaded decompressor
  1233. could end up allocating an insane amount of memory with some input files.
  1234. If the default
  1235. .I limit
  1236. is too low on your system,
  1237. feel free to increase the
  1238. .I limit
  1239. but never set it to a value larger than the amount of usable RAM
  1240. as with appropriate input files
  1241. .B xz
  1242. will attempt to use that amount of memory
  1243. even with a low number of threads.
  1244. Running out of memory or swapping
  1245. will not improve decompression performance.
  1246. .IP ""
  1247. See
  1248. .BI \-\-memlimit\-compress= limit
  1249. for possible ways to specify the
  1250. .IR limit .
  1251. Setting
  1252. .I limit
  1253. to
  1254. .B 0
  1255. resets the
  1256. .I limit
  1257. to the default system-specific value.
  1258. .TP
  1259. \fB\-M\fR \fIlimit\fR, \fB\-\-memlimit=\fIlimit\fR, \fB\-\-memory=\fIlimit
  1260. This is equivalent to specifying
  1261. .BI \-\-memlimit\-compress= limit
  1262. .BI \-\-memlimit-decompress= limit
  1263. \fB\-\-memlimit\-mt\-decompress=\fIlimit\fR.
  1264. .TP
  1265. .B \-\-no\-adjust
  1266. Display an error and exit if the memory usage limit cannot be
  1267. met without adjusting settings that affect the compressed output.
  1268. That is, this prevents
  1269. .B xz
  1270. from switching the encoder from multi-threaded mode to single-threaded mode
  1271. and from reducing the LZMA2 dictionary size.
  1272. Even when this option is used the number of threads may be reduced
  1273. to meet the memory usage limit as that won't affect the compressed output.
  1274. .IP ""
  1275. Automatic adjusting is always disabled when creating raw streams
  1276. .RB ( \-\-format=raw ).
  1277. .TP
  1278. \fB\-T\fR \fIthreads\fR, \fB\-\-threads=\fIthreads
  1279. Specify the number of worker threads to use.
  1280. Setting
  1281. .I threads
  1282. to a special value
  1283. .B 0
  1284. makes
  1285. .B xz
  1286. use up to as many threads as the processor(s) on the system support.
  1287. The actual number of threads can be fewer than
  1288. .I threads
  1289. if the input file is not big enough
  1290. for threading with the given settings or
  1291. if using more threads would exceed the memory usage limit.
  1292. .IP ""
  1293. The single-threaded and multi-threaded compressors produce different output.
  1294. Single-threaded compressor will give the smallest file size but
  1295. only the output from the multi-threaded compressor can be decompressed
  1296. using multiple threads.
  1297. Setting
  1298. .I threads
  1299. to
  1300. .B 1
  1301. will use the single-threaded mode.
  1302. Setting
  1303. .I threads
  1304. to any other value, including
  1305. .BR 0 ,
  1306. will use the multi-threaded compressor
  1307. even if the system supports only one hardware thread.
  1308. .RB ( xz
  1309. 5.2.x
  1310. used single-threaded mode in this situation.)
  1311. .IP ""
  1312. To use multi-threaded mode with only one thread, set
  1313. .I threads
  1314. to
  1315. .BR +1 .
  1316. The
  1317. .B +
  1318. prefix has no effect with values other than
  1319. .BR 1 .
  1320. A memory usage limit can still make
  1321. .B xz
  1322. switch to single-threaded mode unless
  1323. .B \-\-no\-adjust
  1324. is used.
  1325. Support for the
  1326. .B +
  1327. prefix was added in
  1328. .B xz
  1329. 5.4.0.
  1330. .IP ""
  1331. If an automatic number of threads has been requested and
  1332. no memory usage limit has been specified,
  1333. then a system-specific default soft limit will be used to possibly
  1334. limit the number of threads.
  1335. It is a soft limit in sense that it is ignored
  1336. if the number of threads becomes one,
  1337. thus a soft limit will never stop
  1338. .B xz
  1339. from compressing or decompressing.
  1340. This default soft limit will not make
  1341. .B xz
  1342. switch from multi-threaded mode to single-threaded mode.
  1343. The active limits can be seen with
  1344. .BR "xz \-\-info\-memory" .
  1345. .IP ""
  1346. Currently the only threading method is to split the input into
  1347. blocks and compress them independently from each other.
  1348. The default block size depends on the compression level and
  1349. can be overridden with the
  1350. .BI \-\-block\-size= size
  1351. option.
  1352. .IP ""
  1353. Threaded decompression only works on files that contain
  1354. multiple blocks with size information in block headers.
  1355. All large enough files compressed in multi-threaded mode
  1356. meet this condition,
  1357. but files compressed in single-threaded mode don't even if
  1358. .BI \-\-block\-size= size
  1359. has been used.
  1360. .IP ""
  1361. The default value for
  1362. .I threads
  1363. is
  1364. .BR 0 .
  1365. In
  1366. .B xz
  1367. 5.4.x and older the default is
  1368. .BR 1 .
  1369. .
  1370. .SS "Custom compressor filter chains"
  1371. A custom filter chain allows specifying
  1372. the compression settings in detail instead of relying on
  1373. the settings associated to the presets.
  1374. When a custom filter chain is specified,
  1375. preset options
  1376. .RB ( \-0
  1377. \&...\&
  1378. .B \-9
  1379. and
  1380. .BR \-\-extreme )
  1381. earlier on the command line are forgotten.
  1382. If a preset option is specified
  1383. after one or more custom filter chain options,
  1384. the new preset takes effect and
  1385. the custom filter chain options specified earlier are forgotten.
  1386. .PP
  1387. A filter chain is comparable to piping on the command line.
  1388. When compressing, the uncompressed input goes to the first filter,
  1389. whose output goes to the next filter (if any).
  1390. The output of the last filter gets written to the compressed file.
  1391. The maximum number of filters in the chain is four,
  1392. but typically a filter chain has only one or two filters.
  1393. .PP
  1394. Many filters have limitations on where they can be
  1395. in the filter chain:
  1396. some filters can work only as the last filter in the chain,
  1397. some only as a non-last filter, and some work in any position
  1398. in the chain.
  1399. Depending on the filter, this limitation is either inherent to
  1400. the filter design or exists to prevent security issues.
  1401. .PP
  1402. A custom filter chain can be specified in two different ways.
  1403. The options
  1404. .BI \-\-filters= filters
  1405. and
  1406. .BI \-\-filters1= filters
  1407. \&...\&
  1408. .BI \-\-filters9= filters
  1409. allow specifying an entire filter chain in one option using the
  1410. liblzma filter string syntax.
  1411. Alternatively, a filter chain can be specified by using one or more
  1412. individual filter options in the order they are wanted in the filter chain.
  1413. That is, the order of the individual filter options is significant!
  1414. When decoding raw streams
  1415. .RB ( \-\-format=raw ),
  1416. the filter chain must be specified in the same order as
  1417. it was specified when compressing.
  1418. Any individual filter or preset options specified before the full
  1419. chain option
  1420. (\fB\-\-filters=\fIfilters\fR)
  1421. will be forgotten.
  1422. Individual filters specified after the full chain option will reset the
  1423. filter chain.
  1424. .PP
  1425. Both the full and individual filter options take filter-specific
  1426. .I options
  1427. as a comma-separated list.
  1428. Extra commas in
  1429. .I options
  1430. are ignored.
  1431. Every option has a default value, so
  1432. specify those you want to change.
  1433. .PP
  1434. To see the whole filter chain and
  1435. .IR options ,
  1436. use
  1437. .B "xz \-vv"
  1438. (that is, use
  1439. .B \-\-verbose
  1440. twice).
  1441. This works also for viewing the filter chain options used by presets.
  1442. .TP
  1443. .BI \-\-filters= filters
  1444. Specify the full filter chain or a preset in a single option.
  1445. Each filter can be separated by spaces or two dashes
  1446. .RB ( \-\- ).
  1447. .I filters
  1448. may need to be quoted on the shell command line so it is
  1449. parsed as a single option.
  1450. To denote
  1451. .IR options ,
  1452. use
  1453. .B :
  1454. or
  1455. .BR = .
  1456. A preset can be prefixed with a
  1457. .B \-
  1458. and followed with zero or more flags.
  1459. The only supported flag is
  1460. .B e
  1461. to apply the same options as
  1462. .BR \-\-extreme .
  1463. .TP
  1464. \fB\-\-filters1\fR=\fIfilters\fR ... \fB\-\-filters9\fR=\fIfilters
  1465. Specify up to nine additional filter chains that can be used with
  1466. .BR \-\-block\-list .
  1467. .IP ""
  1468. For example, when compressing an archive with executable files
  1469. followed by text files, the executable part could use a filter
  1470. chain with a BCJ filter and the text part only the LZMA2 filter.
  1471. .TP
  1472. .B \-\-filters-help
  1473. Display a help message describing how to specify presets and
  1474. custom filter chains in the
  1475. .B \-\-filters
  1476. and
  1477. .BI \-\-filters1= filters
  1478. \&...\&
  1479. .BI \-\-filters9= filters
  1480. options, and exit successfully.
  1481. .TP
  1482. \fB\-\-lzma1\fR[\fB=\fIoptions\fR]
  1483. .PD 0
  1484. .TP
  1485. \fB\-\-lzma2\fR[\fB=\fIoptions\fR]
  1486. .PD
  1487. Add LZMA1 or LZMA2 filter to the filter chain.
  1488. These filters can be used only as the last filter in the chain.
  1489. .IP ""
  1490. LZMA1 is a legacy filter,
  1491. which is supported almost solely due to the legacy
  1492. .B .lzma
  1493. file format, which supports only LZMA1.
  1494. LZMA2 is an updated
  1495. version of LZMA1 to fix some practical issues of LZMA1.
  1496. The
  1497. .B .xz
  1498. format uses LZMA2 and doesn't support LZMA1 at all.
  1499. Compression speed and ratios of LZMA1 and LZMA2
  1500. are practically the same.
  1501. .IP ""
  1502. LZMA1 and LZMA2 share the same set of
  1503. .IR options :
  1504. .RS
  1505. .TP
  1506. .\" TRANSLATORS: Don't translate bold strings like B<preset>, B<dict>,
  1507. .\" B<mode>, B<nice>, B<fast>, or B<normal> because those are command line
  1508. .\" options. On the other hand, do translate the italic strings like
  1509. .\" I<preset>, I<size>, and I<mode>, because such italic strings are
  1510. .\" placeholders which a user replaces with an actual value.
  1511. .BI preset= preset
  1512. Reset all LZMA1 or LZMA2
  1513. .I options
  1514. to
  1515. .IR preset .
  1516. .I Preset
  1517. consist of an integer, which may be followed by single-letter
  1518. preset modifiers.
  1519. The integer can be from
  1520. .B 0
  1521. to
  1522. .BR 9 ,
  1523. matching the command line options
  1524. .B \-0
  1525. \&...\&
  1526. .BR \-9 .
  1527. The only supported modifier is currently
  1528. .BR e ,
  1529. which matches
  1530. .BR \-\-extreme .
  1531. If no
  1532. .B preset
  1533. is specified, the default values of LZMA1 or LZMA2
  1534. .I options
  1535. are taken from the preset
  1536. .BR 6 .
  1537. .TP
  1538. .BI dict= size
  1539. Dictionary (history buffer)
  1540. .I size
  1541. indicates how many bytes of the recently processed
  1542. uncompressed data is kept in memory.
  1543. The algorithm tries to find repeating byte sequences (matches) in
  1544. the uncompressed data, and replace them with references
  1545. to the data currently in the dictionary.
  1546. The bigger the dictionary, the higher is the chance
  1547. to find a match.
  1548. Thus, increasing dictionary
  1549. .I size
  1550. usually improves compression ratio, but
  1551. a dictionary bigger than the uncompressed file is waste of memory.
  1552. .IP ""
  1553. Typical dictionary
  1554. .I size
  1555. is from 64\ KiB to 64\ MiB.
  1556. The minimum is 4\ KiB.
  1557. The maximum for compression is currently 1.5\ GiB (1536\ MiB).
  1558. The decompressor already supports dictionaries up to
  1559. one byte less than 4\ GiB, which is the maximum for
  1560. the LZMA1 and LZMA2 stream formats.
  1561. .IP ""
  1562. Dictionary
  1563. .I size
  1564. and match finder
  1565. .RI ( mf )
  1566. together determine the memory usage of the LZMA1 or LZMA2 encoder.
  1567. The same (or bigger) dictionary
  1568. .I size
  1569. is required for decompressing that was used when compressing,
  1570. thus the memory usage of the decoder is determined
  1571. by the dictionary size used when compressing.
  1572. The
  1573. .B .xz
  1574. headers store the dictionary
  1575. .I size
  1576. either as
  1577. .RI "2^" n
  1578. or
  1579. .RI "2^" n " + 2^(" n "\-1),"
  1580. so these
  1581. .I sizes
  1582. are somewhat preferred for compression.
  1583. Other
  1584. .I sizes
  1585. will get rounded up when stored in the
  1586. .B .xz
  1587. headers.
  1588. .TP
  1589. .BI lc= lc
  1590. Specify the number of literal context bits.
  1591. The minimum is 0 and the maximum is 4; the default is 3.
  1592. In addition, the sum of
  1593. .I lc
  1594. and
  1595. .I lp
  1596. must not exceed 4.
  1597. .IP ""
  1598. All bytes that cannot be encoded as matches
  1599. are encoded as literals.
  1600. That is, literals are simply 8-bit bytes
  1601. that are encoded one at a time.
  1602. .IP ""
  1603. The literal coding makes an assumption that the highest
  1604. .I lc
  1605. bits of the previous uncompressed byte correlate
  1606. with the next byte.
  1607. For example, in typical English text, an upper-case letter is
  1608. often followed by a lower-case letter, and a lower-case
  1609. letter is usually followed by another lower-case letter.
  1610. In the US-ASCII character set, the highest three bits are 010
  1611. for upper-case letters and 011 for lower-case letters.
  1612. When
  1613. .I lc
  1614. is at least 3, the literal coding can take advantage of
  1615. this property in the uncompressed data.
  1616. .IP ""
  1617. The default value (3) is usually good.
  1618. If you want maximum compression, test
  1619. .BR lc=4 .
  1620. Sometimes it helps a little, and
  1621. sometimes it makes compression worse.
  1622. If it makes it worse, test
  1623. .B lc=2
  1624. too.
  1625. .TP
  1626. .BI lp= lp
  1627. Specify the number of literal position bits.
  1628. The minimum is 0 and the maximum is 4; the default is 0.
  1629. .IP ""
  1630. .I Lp
  1631. affects what kind of alignment in the uncompressed data is
  1632. assumed when encoding literals.
  1633. See
  1634. .I pb
  1635. below for more information about alignment.
  1636. .TP
  1637. .BI pb= pb
  1638. Specify the number of position bits.
  1639. The minimum is 0 and the maximum is 4; the default is 2.
  1640. .IP ""
  1641. .I Pb
  1642. affects what kind of alignment in the uncompressed data is
  1643. assumed in general.
  1644. The default means four-byte alignment
  1645. .RI (2^ pb =2^2=4),
  1646. which is often a good choice when there's no better guess.
  1647. .IP ""
  1648. When the alignment is known, setting
  1649. .I pb
  1650. accordingly may reduce the file size a little.
  1651. For example, with text files having one-byte
  1652. alignment (US-ASCII, ISO-8859-*, UTF-8), setting
  1653. .B pb=0
  1654. can improve compression slightly.
  1655. For UTF-16 text,
  1656. .B pb=1
  1657. is a good choice.
  1658. If the alignment is an odd number like 3 bytes,
  1659. .B pb=0
  1660. might be the best choice.
  1661. .IP ""
  1662. Even though the assumed alignment can be adjusted with
  1663. .I pb
  1664. and
  1665. .IR lp ,
  1666. LZMA1 and LZMA2 still slightly favor 16-byte alignment.
  1667. It might be worth taking into account when designing file formats
  1668. that are likely to be often compressed with LZMA1 or LZMA2.
  1669. .TP
  1670. .BI mf= mf
  1671. Match finder has a major effect on encoder speed,
  1672. memory usage, and compression ratio.
  1673. Usually Hash Chain match finders are faster than Binary Tree
  1674. match finders.
  1675. The default depends on the
  1676. .IR preset :
  1677. 0 uses
  1678. .BR hc3 ,
  1679. 1\(en3
  1680. use
  1681. .BR hc4 ,
  1682. and the rest use
  1683. .BR bt4 .
  1684. .IP ""
  1685. The following match finders are supported.
  1686. The memory usage formulas below are rough approximations,
  1687. which are closest to the reality when
  1688. .I dict
  1689. is a power of two.
  1690. .RS
  1691. .TP
  1692. .B hc3
  1693. Hash Chain with 2- and 3-byte hashing
  1694. .br
  1695. Minimum value for
  1696. .IR nice :
  1697. 3
  1698. .br
  1699. Memory usage:
  1700. .br
  1701. .I dict
  1702. * 7.5 (if
  1703. .I dict
  1704. <= 16 MiB);
  1705. .br
  1706. .I dict
  1707. * 5.5 + 64 MiB (if
  1708. .I dict
  1709. > 16 MiB)
  1710. .TP
  1711. .B hc4
  1712. Hash Chain with 2-, 3-, and 4-byte hashing
  1713. .br
  1714. Minimum value for
  1715. .IR nice :
  1716. 4
  1717. .br
  1718. Memory usage:
  1719. .br
  1720. .I dict
  1721. * 7.5 (if
  1722. .I dict
  1723. <= 32 MiB);
  1724. .br
  1725. .I dict
  1726. * 6.5 (if
  1727. .I dict
  1728. > 32 MiB)
  1729. .TP
  1730. .B bt2
  1731. Binary Tree with 2-byte hashing
  1732. .br
  1733. Minimum value for
  1734. .IR nice :
  1735. 2
  1736. .br
  1737. Memory usage:
  1738. .I dict
  1739. * 9.5
  1740. .TP
  1741. .B bt3
  1742. Binary Tree with 2- and 3-byte hashing
  1743. .br
  1744. Minimum value for
  1745. .IR nice :
  1746. 3
  1747. .br
  1748. Memory usage:
  1749. .br
  1750. .I dict
  1751. * 11.5 (if
  1752. .I dict
  1753. <= 16 MiB);
  1754. .br
  1755. .I dict
  1756. * 9.5 + 64 MiB (if
  1757. .I dict
  1758. > 16 MiB)
  1759. .TP
  1760. .B bt4
  1761. Binary Tree with 2-, 3-, and 4-byte hashing
  1762. .br
  1763. Minimum value for
  1764. .IR nice :
  1765. 4
  1766. .br
  1767. Memory usage:
  1768. .br
  1769. .I dict
  1770. * 11.5 (if
  1771. .I dict
  1772. <= 32 MiB);
  1773. .br
  1774. .I dict
  1775. * 10.5 (if
  1776. .I dict
  1777. > 32 MiB)
  1778. .RE
  1779. .TP
  1780. .BI mode= mode
  1781. Compression
  1782. .I mode
  1783. specifies the method to analyze
  1784. the data produced by the match finder.
  1785. Supported
  1786. .I modes
  1787. are
  1788. .B fast
  1789. and
  1790. .BR normal .
  1791. The default is
  1792. .B fast
  1793. for
  1794. .I presets
  1795. 0\(en3 and
  1796. .B normal
  1797. for
  1798. .I presets
  1799. 4\(en9.
  1800. .IP ""
  1801. Usually
  1802. .B fast
  1803. is used with Hash Chain match finders and
  1804. .B normal
  1805. with Binary Tree match finders.
  1806. This is also what the
  1807. .I presets
  1808. do.
  1809. .TP
  1810. .BI nice= nice
  1811. Specify what is considered to be a nice length for a match.
  1812. Once a match of at least
  1813. .I nice
  1814. bytes is found, the algorithm stops
  1815. looking for possibly better matches.
  1816. .IP ""
  1817. .I Nice
  1818. can be 2\(en273 bytes.
  1819. Higher values tend to give better compression ratio
  1820. at the expense of speed.
  1821. The default depends on the
  1822. .IR preset .
  1823. .TP
  1824. .BI depth= depth
  1825. Specify the maximum search depth in the match finder.
  1826. The default is the special value of 0,
  1827. which makes the compressor determine a reasonable
  1828. .I depth
  1829. from
  1830. .I mf
  1831. and
  1832. .IR nice .
  1833. .IP ""
  1834. Reasonable
  1835. .I depth
  1836. for Hash Chains is 4\(en100 and 16\(en1000 for Binary Trees.
  1837. Using very high values for
  1838. .I depth
  1839. can make the encoder extremely slow with some files.
  1840. Avoid setting the
  1841. .I depth
  1842. over 1000 unless you are prepared to interrupt
  1843. the compression in case it is taking far too long.
  1844. .RE
  1845. .IP ""
  1846. When decoding raw streams
  1847. .RB ( \-\-format=raw ),
  1848. LZMA2 needs only the dictionary
  1849. .IR size .
  1850. LZMA1 needs also
  1851. .IR lc ,
  1852. .IR lp ,
  1853. and
  1854. .IR pb .
  1855. .TP
  1856. \fB\-\-x86\fR[\fB=\fIoptions\fR]
  1857. .PD 0
  1858. .TP
  1859. \fB\-\-arm\fR[\fB=\fIoptions\fR]
  1860. .TP
  1861. \fB\-\-armthumb\fR[\fB=\fIoptions\fR]
  1862. .TP
  1863. \fB\-\-arm64\fR[\fB=\fIoptions\fR]
  1864. .TP
  1865. \fB\-\-powerpc\fR[\fB=\fIoptions\fR]
  1866. .TP
  1867. \fB\-\-ia64\fR[\fB=\fIoptions\fR]
  1868. .TP
  1869. \fB\-\-sparc\fR[\fB=\fIoptions\fR]
  1870. .TP
  1871. \fB\-\-riscv\fR[\fB=\fIoptions\fR]
  1872. .PD
  1873. Add a branch/call/jump (BCJ) filter to the filter chain.
  1874. These filters can be used only as a non-last filter
  1875. in the filter chain.
  1876. .IP ""
  1877. A BCJ filter converts relative addresses in
  1878. the machine code to their absolute counterparts.
  1879. This doesn't change the size of the data
  1880. but it increases redundancy,
  1881. which can help LZMA2 to produce 0\(en15\ % smaller
  1882. .B .xz
  1883. file.
  1884. The BCJ filters are always reversible,
  1885. so using a BCJ filter for wrong type of data
  1886. doesn't cause any data loss, although it may make
  1887. the compression ratio slightly worse.
  1888. The BCJ filters are very fast and
  1889. use an insignificant amount of memory.
  1890. .IP ""
  1891. These BCJ filters have known problems related to
  1892. the compression ratio:
  1893. .RS
  1894. .IP \(bu 3
  1895. Some types of files containing executable code
  1896. (for example, object files, static libraries, and Linux kernel modules)
  1897. have the addresses in the instructions filled with filler values.
  1898. These BCJ filters will still do the address conversion,
  1899. which will make the compression worse with these files.
  1900. .IP \(bu 3
  1901. If a BCJ filter is applied on an archive,
  1902. it is possible that it makes the compression ratio
  1903. worse than not using a BCJ filter.
  1904. For example, if there are similar or even identical executables
  1905. then filtering will likely make the files less similar
  1906. and thus compression is worse.
  1907. The contents of non-executable files in the same archive can matter too.
  1908. In practice one has to try with and without a BCJ filter to see
  1909. which is better in each situation.
  1910. .RE
  1911. .IP ""
  1912. Different instruction sets have different alignment:
  1913. the executable file must be aligned to a multiple of
  1914. this value in the input data to make the filter work.
  1915. .RS
  1916. .RS
  1917. .PP
  1918. .TS
  1919. tab(;);
  1920. l n l
  1921. l n l.
  1922. Filter;Alignment;Notes
  1923. x86;1;32-bit or 64-bit x86
  1924. ARM;4;
  1925. ARM-Thumb;2;
  1926. ARM64;4;4096-byte alignment is best
  1927. PowerPC;4;Big endian only
  1928. IA-64;16;Itanium
  1929. SPARC;4;
  1930. RISC-V;2;
  1931. .TE
  1932. .RE
  1933. .RE
  1934. .IP ""
  1935. Since the BCJ-filtered data is usually compressed with LZMA2,
  1936. the compression ratio may be improved slightly if
  1937. the LZMA2 options are set to match the
  1938. alignment of the selected BCJ filter.
  1939. Examples:
  1940. .RS
  1941. .IP \(bu 3
  1942. IA-64 filter has 16-byte alignment so
  1943. .B pb=4,lp=4,lc=0
  1944. is good
  1945. with LZMA2 (2^4=16).
  1946. .IP \(bu 3
  1947. RISC-V code has 2-byte or 4-byte alignment
  1948. depending on whether the file contains
  1949. 16-bit compressed instructions (the C extension).
  1950. When 16-bit instructions are used,
  1951. .B pb=2,lp=1,lc=3
  1952. or
  1953. .B pb=1,lp=1,lc=3
  1954. is good.
  1955. When 16-bit instructions aren't present,
  1956. .B pb=2,lp=2,lc=2
  1957. is the best.
  1958. .B readelf \-h
  1959. can be used to check if "RVC"
  1960. appears on the "Flags" line.
  1961. .IP \(bu 3
  1962. ARM64 is always 4-byte aligned so
  1963. .B pb=2,lp=2,lc=2
  1964. is the best.
  1965. .IP \(bu 3
  1966. The x86 filter is an exception.
  1967. It's usually good to stick to LZMA2's defaults
  1968. .RB ( pb=2,lp=0,lc=3 )
  1969. when compressing x86 executables.
  1970. .RE
  1971. .IP ""
  1972. All BCJ filters support the same
  1973. .IR options :
  1974. .RS
  1975. .TP
  1976. .BI start= offset
  1977. Specify the start
  1978. .I offset
  1979. that is used when converting between relative
  1980. and absolute addresses.
  1981. The
  1982. .I offset
  1983. must be a multiple of the alignment of the filter
  1984. (see the table above).
  1985. The default is zero.
  1986. In practice, the default is good; specifying a custom
  1987. .I offset
  1988. is almost never useful.
  1989. .RE
  1990. .TP
  1991. \fB\-\-delta\fR[\fB=\fIoptions\fR]
  1992. Add the Delta filter to the filter chain.
  1993. The Delta filter can be only used as a non-last filter
  1994. in the filter chain.
  1995. .IP ""
  1996. Currently only simple byte-wise delta calculation is supported.
  1997. It can be useful when compressing, for example, uncompressed bitmap images
  1998. or uncompressed PCM audio.
  1999. However, special purpose algorithms may give significantly better
  2000. results than Delta + LZMA2.
  2001. This is true especially with audio,
  2002. which compresses faster and better, for example, with
  2003. .BR flac (1).
  2004. .IP ""
  2005. Supported
  2006. .IR options :
  2007. .RS
  2008. .TP
  2009. .BI dist= distance
  2010. Specify the
  2011. .I distance
  2012. of the delta calculation in bytes.
  2013. .I distance
  2014. must be 1\(en256.
  2015. The default is 1.
  2016. .IP ""
  2017. For example, with
  2018. .B dist=2
  2019. and eight-byte input A1 B1 A2 B3 A3 B5 A4 B7, the output will be
  2020. A1 B1 01 02 01 02 01 02.
  2021. .RE
  2022. .
  2023. .SS "Other options"
  2024. .TP
  2025. .BR \-q ", " \-\-quiet
  2026. Suppress warnings and notices.
  2027. Specify this twice to suppress errors too.
  2028. This option has no effect on the exit status.
  2029. That is, even if a warning was suppressed,
  2030. the exit status to indicate a warning is still used.
  2031. .TP
  2032. .BR \-v ", " \-\-verbose
  2033. Be verbose.
  2034. If standard error is connected to a terminal,
  2035. .B xz
  2036. will display a progress indicator.
  2037. Specifying
  2038. .B \-\-verbose
  2039. twice will give even more verbose output.
  2040. .IP ""
  2041. The progress indicator shows the following information:
  2042. .RS
  2043. .IP \(bu 3
  2044. Completion percentage is shown
  2045. if the size of the input file is known.
  2046. That is, the percentage cannot be shown in pipes.
  2047. .IP \(bu 3
  2048. Amount of compressed data produced (compressing)
  2049. or consumed (decompressing).
  2050. .IP \(bu 3
  2051. Amount of uncompressed data consumed (compressing)
  2052. or produced (decompressing).
  2053. .IP \(bu 3
  2054. Compression ratio, which is calculated by dividing
  2055. the amount of compressed data processed so far by
  2056. the amount of uncompressed data processed so far.
  2057. .IP \(bu 3
  2058. Compression or decompression speed.
  2059. This is measured as the amount of uncompressed data consumed
  2060. (compression) or produced (decompression) per second.
  2061. It is shown after a few seconds have passed since
  2062. .B xz
  2063. started processing the file.
  2064. .IP \(bu 3
  2065. Elapsed time in the format M:SS or H:MM:SS.
  2066. .IP \(bu 3
  2067. Estimated remaining time is shown
  2068. only when the size of the input file is
  2069. known and a couple of seconds have already passed since
  2070. .B xz
  2071. started processing the file.
  2072. The time is shown in a less precise format which
  2073. never has any colons, for example, 2 min 30 s.
  2074. .RE
  2075. .IP ""
  2076. When standard error is not a terminal,
  2077. .B \-\-verbose
  2078. will make
  2079. .B xz
  2080. print the filename, compressed size, uncompressed size,
  2081. compression ratio, and possibly also the speed and elapsed time
  2082. on a single line to standard error after compressing or
  2083. decompressing the file.
  2084. The speed and elapsed time are included only when
  2085. the operation took at least a few seconds.
  2086. If the operation didn't finish, for example, due to user interruption,
  2087. also the completion percentage is printed
  2088. if the size of the input file is known.
  2089. .TP
  2090. .BR \-Q ", " \-\-no\-warn
  2091. Don't set the exit status to 2
  2092. even if a condition worth a warning was detected.
  2093. This option doesn't affect the verbosity level, thus both
  2094. .B \-\-quiet
  2095. and
  2096. .B \-\-no\-warn
  2097. have to be used to not display warnings and
  2098. to not alter the exit status.
  2099. .TP
  2100. .B \-\-robot
  2101. Print messages in a machine-parsable format.
  2102. This is intended to ease writing frontends that want to use
  2103. .B xz
  2104. instead of liblzma, which may be the case with various scripts.
  2105. The output with this option enabled is meant to be stable across
  2106. .B xz
  2107. releases.
  2108. See the section
  2109. .B "ROBOT MODE"
  2110. for details.
  2111. .TP
  2112. .B \-\-info\-memory
  2113. Display, in human-readable format, how much physical memory (RAM)
  2114. and how many processor threads
  2115. .B xz
  2116. thinks the system has and the memory usage limits for compression
  2117. and decompression, and exit successfully.
  2118. .TP
  2119. .BR \-h ", " \-\-help
  2120. Display a help message describing the most commonly used options,
  2121. and exit successfully.
  2122. .TP
  2123. .BR \-H ", " \-\-long\-help
  2124. Display a help message describing all features of
  2125. .BR xz ,
  2126. and exit successfully
  2127. .TP
  2128. .BR \-V ", " \-\-version
  2129. Display the version number of
  2130. .B xz
  2131. and liblzma in human readable format.
  2132. To get machine-parsable output, specify
  2133. .B \-\-robot
  2134. before
  2135. .BR \-\-version .
  2136. .
  2137. .SH "ROBOT MODE"
  2138. The robot mode is activated with the
  2139. .B \-\-robot
  2140. option.
  2141. It makes the output of
  2142. .B xz
  2143. easier to parse by other programs.
  2144. Currently
  2145. .B \-\-robot
  2146. is supported only together with
  2147. .BR \-\-list ,
  2148. .BR \-\-filters\-help ,
  2149. .BR \-\-info\-memory ,
  2150. and
  2151. .BR \-\-version .
  2152. It will be supported for compression and
  2153. decompression in the future.
  2154. .
  2155. .SS "List mode"
  2156. .B "xz \-\-robot \-\-list"
  2157. uses tab-separated output.
  2158. The first column of every line has a string
  2159. that indicates the type of the information found on that line:
  2160. .TP
  2161. .\" TRANSLATORS: The bold strings B<name>, B<file>, B<stream>, B<block>,
  2162. .\" B<summary>, and B<totals> are produced by the xz tool for scripts to
  2163. .\" parse, thus the untranslated strings must be included in the translated
  2164. .\" man page. It may be useful to provide a translated string in parenthesis
  2165. .\" without bold, for example: "B<name> (nimi)"
  2166. .B name
  2167. This is always the first line when starting to list a file.
  2168. The second column on the line is the filename.
  2169. .TP
  2170. .B file
  2171. This line contains overall information about the
  2172. .B .xz
  2173. file.
  2174. This line is always printed after the
  2175. .B name
  2176. line.
  2177. .TP
  2178. .B stream
  2179. This line type is used only when
  2180. .B \-\-verbose
  2181. was specified.
  2182. There are as many
  2183. .B stream
  2184. lines as there are streams in the
  2185. .B .xz
  2186. file.
  2187. .TP
  2188. .B block
  2189. This line type is used only when
  2190. .B \-\-verbose
  2191. was specified.
  2192. There are as many
  2193. .B block
  2194. lines as there are blocks in the
  2195. .B .xz
  2196. file.
  2197. The
  2198. .B block
  2199. lines are shown after all the
  2200. .B stream
  2201. lines; different line types are not interleaved.
  2202. .TP
  2203. .B summary
  2204. This line type is used only when
  2205. .B \-\-verbose
  2206. was specified twice.
  2207. This line is printed after all
  2208. .B block
  2209. lines.
  2210. Like the
  2211. .B file
  2212. line, the
  2213. .B summary
  2214. line contains overall information about the
  2215. .B .xz
  2216. file.
  2217. .TP
  2218. .B totals
  2219. This line is always the very last line of the list output.
  2220. It shows the total counts and sizes.
  2221. .PP
  2222. The columns of the
  2223. .B file
  2224. lines:
  2225. .PD 0
  2226. .RS
  2227. .IP 2. 4
  2228. Number of streams in the file
  2229. .IP 3. 4
  2230. Total number of blocks in the stream(s)
  2231. .IP 4. 4
  2232. Compressed size of the file
  2233. .IP 5. 4
  2234. Uncompressed size of the file
  2235. .IP 6. 4
  2236. Compression ratio, for example,
  2237. .BR 0.123 .
  2238. If ratio is over 9.999, three dashes
  2239. .RB ( \-\-\- )
  2240. are displayed instead of the ratio.
  2241. .IP 7. 4
  2242. Comma-separated list of integrity check names.
  2243. The following strings are used for the known check types:
  2244. .\" TRANSLATORS: Don't translate the bold strings B<None>, B<CRC32>,
  2245. .\" B<CRC64>, B<SHA-256>, or B<Unknown-> here. In robot mode, xz produces
  2246. .\" them in untranslated form for scripts to parse.
  2247. .BR None ,
  2248. .BR CRC32 ,
  2249. .BR CRC64 ,
  2250. and
  2251. .BR SHA\-256 .
  2252. For unknown check types,
  2253. .BI Unknown\- N
  2254. is used, where
  2255. .I N
  2256. is the Check ID as a decimal number (one or two digits).
  2257. .IP 8. 4
  2258. Total size of stream padding in the file
  2259. .RE
  2260. .PD
  2261. .PP
  2262. The columns of the
  2263. .B stream
  2264. lines:
  2265. .PD 0
  2266. .RS
  2267. .IP 2. 4
  2268. Stream number (the first stream is 1)
  2269. .IP 3. 4
  2270. Number of blocks in the stream
  2271. .IP 4. 4
  2272. Compressed start offset
  2273. .IP 5. 4
  2274. Uncompressed start offset
  2275. .IP 6. 4
  2276. Compressed size (does not include stream padding)
  2277. .IP 7. 4
  2278. Uncompressed size
  2279. .IP 8. 4
  2280. Compression ratio
  2281. .IP 9. 4
  2282. Name of the integrity check
  2283. .IP 10. 4
  2284. Size of stream padding
  2285. .RE
  2286. .PD
  2287. .PP
  2288. The columns of the
  2289. .B block
  2290. lines:
  2291. .PD 0
  2292. .RS
  2293. .IP 2. 4
  2294. Number of the stream containing this block
  2295. .IP 3. 4
  2296. Block number relative to the beginning of the stream
  2297. (the first block is 1)
  2298. .IP 4. 4
  2299. Block number relative to the beginning of the file
  2300. .IP 5. 4
  2301. Compressed start offset relative to the beginning of the file
  2302. .IP 6. 4
  2303. Uncompressed start offset relative to the beginning of the file
  2304. .IP 7. 4
  2305. Total compressed size of the block (includes headers)
  2306. .IP 8. 4
  2307. Uncompressed size
  2308. .IP 9. 4
  2309. Compression ratio
  2310. .IP 10. 4
  2311. Name of the integrity check
  2312. .RE
  2313. .PD
  2314. .PP
  2315. If
  2316. .B \-\-verbose
  2317. was specified twice, additional columns are included on the
  2318. .B block
  2319. lines.
  2320. These are not displayed with a single
  2321. .BR \-\-verbose ,
  2322. because getting this information requires many seeks
  2323. and can thus be slow:
  2324. .PD 0
  2325. .RS
  2326. .IP 11. 4
  2327. Value of the integrity check in hexadecimal
  2328. .IP 12. 4
  2329. Block header size
  2330. .IP 13. 4
  2331. Block flags:
  2332. .B c
  2333. indicates that compressed size is present, and
  2334. .B u
  2335. indicates that uncompressed size is present.
  2336. If the flag is not set, a dash
  2337. .RB ( \- )
  2338. is shown instead to keep the string length fixed.
  2339. New flags may be added to the end of the string in the future.
  2340. .IP 14. 4
  2341. Size of the actual compressed data in the block (this excludes
  2342. the block header, block padding, and check fields)
  2343. .IP 15. 4
  2344. Amount of memory (in bytes) required to decompress
  2345. this block with this
  2346. .B xz
  2347. version
  2348. .IP 16. 4
  2349. Filter chain.
  2350. Note that most of the options used at compression time
  2351. cannot be known, because only the options
  2352. that are needed for decompression are stored in the
  2353. .B .xz
  2354. headers.
  2355. .RE
  2356. .PD
  2357. .PP
  2358. The columns of the
  2359. .B summary
  2360. lines:
  2361. .PD 0
  2362. .RS
  2363. .IP 2. 4
  2364. Amount of memory (in bytes) required to decompress
  2365. this file with this
  2366. .B xz
  2367. version
  2368. .IP 3. 4
  2369. .B yes
  2370. or
  2371. .B no
  2372. indicating if all block headers have both compressed size and
  2373. uncompressed size stored in them
  2374. .PP
  2375. .I Since
  2376. .B xz
  2377. .I 5.1.2alpha:
  2378. .IP 4. 4
  2379. Minimum
  2380. .B xz
  2381. version required to decompress the file
  2382. .RE
  2383. .PD
  2384. .PP
  2385. The columns of the
  2386. .B totals
  2387. line:
  2388. .PD 0
  2389. .RS
  2390. .IP 2. 4
  2391. Number of streams
  2392. .IP 3. 4
  2393. Number of blocks
  2394. .IP 4. 4
  2395. Compressed size
  2396. .IP 5. 4
  2397. Uncompressed size
  2398. .IP 6. 4
  2399. Average compression ratio
  2400. .IP 7. 4
  2401. Comma-separated list of integrity check names
  2402. that were present in the files
  2403. .IP 8. 4
  2404. Stream padding size
  2405. .IP 9. 4
  2406. Number of files.
  2407. This is here to
  2408. keep the order of the earlier columns the same as on
  2409. .B file
  2410. lines.
  2411. .PD
  2412. .RE
  2413. .PP
  2414. If
  2415. .B \-\-verbose
  2416. was specified twice, additional columns are included on the
  2417. .B totals
  2418. line:
  2419. .PD 0
  2420. .RS
  2421. .IP 10. 4
  2422. Maximum amount of memory (in bytes) required to decompress
  2423. the files with this
  2424. .B xz
  2425. version
  2426. .IP 11. 4
  2427. .B yes
  2428. or
  2429. .B no
  2430. indicating if all block headers have both compressed size and
  2431. uncompressed size stored in them
  2432. .PP
  2433. .I Since
  2434. .B xz
  2435. .I 5.1.2alpha:
  2436. .IP 12. 4
  2437. Minimum
  2438. .B xz
  2439. version required to decompress the file
  2440. .RE
  2441. .PD
  2442. .PP
  2443. Future versions may add new line types and
  2444. new columns can be added to the existing line types,
  2445. but the existing columns won't be changed.
  2446. .
  2447. .SS "Filters help"
  2448. .B "xz \-\-robot \-\-filters-help"
  2449. prints the supported filters in the following format:
  2450. .PP
  2451. \fIfilter\fB:\fIoption\fB=<\fIvalue\fB>,\fIoption\fB=<\fIvalue\fB>\fR...
  2452. .TP
  2453. .I filter
  2454. Name of the filter
  2455. .TP
  2456. .I option
  2457. Name of a filter specific option
  2458. .TP
  2459. .I value
  2460. Numeric
  2461. .I value
  2462. ranges appear as
  2463. \fB<\fImin\fB\-\fImax\fB>\fR.
  2464. String
  2465. .I value
  2466. choices are shown within
  2467. .B "< >"
  2468. and separated by a
  2469. .B |
  2470. character.
  2471. .PP
  2472. Each filter is printed on its own line.
  2473. .
  2474. .SS "Memory limit information"
  2475. .B "xz \-\-robot \-\-info\-memory"
  2476. prints a single line with multiple tab-separated columns:
  2477. .IP 1. 4
  2478. Total amount of physical memory (RAM) in bytes.
  2479. .IP 2. 4
  2480. Memory usage limit for compression in bytes
  2481. .RB ( \-\-memlimit\-compress ).
  2482. A special value of
  2483. .B 0
  2484. indicates the default setting
  2485. which for single-threaded mode is the same as no limit.
  2486. .IP 3. 4
  2487. Memory usage limit for decompression in bytes
  2488. .RB ( \-\-memlimit\-decompress ).
  2489. A special value of
  2490. .B 0
  2491. indicates the default setting
  2492. which for single-threaded mode is the same as no limit.
  2493. .IP 4. 4
  2494. Since
  2495. .B xz
  2496. 5.3.4alpha:
  2497. Memory usage for multi-threaded decompression in bytes
  2498. .RB ( \-\-memlimit\-mt\-decompress ).
  2499. This is never zero because a system-specific default value
  2500. shown in the column 5
  2501. is used if no limit has been specified explicitly.
  2502. This is also never greater than the value in the column 3
  2503. even if a larger value has been specified with
  2504. .BR \-\-memlimit\-mt\-decompress .
  2505. .IP 5. 4
  2506. Since
  2507. .B xz
  2508. 5.3.4alpha:
  2509. A system-specific default memory usage limit
  2510. that is used to limit the number of threads
  2511. when compressing with an automatic number of threads
  2512. .RB ( \-\-threads=0 )
  2513. and no memory usage limit has been specified
  2514. .RB ( \-\-memlimit\-compress ).
  2515. This is also used as the default value for
  2516. .BR \-\-memlimit\-mt\-decompress .
  2517. .IP 6. 4
  2518. Since
  2519. .B xz
  2520. 5.3.4alpha:
  2521. Number of available processor threads.
  2522. .PP
  2523. In the future, the output of
  2524. .B "xz \-\-robot \-\-info\-memory"
  2525. may have more columns, but never more than a single line.
  2526. .
  2527. .SS Version
  2528. .B "xz \-\-robot \-\-version"
  2529. prints the version number of
  2530. .B xz
  2531. and liblzma in the following format:
  2532. .PP
  2533. .\" TRANSLATORS: Don't translate the uppercase XZ_VERSION or LIBLZMA_VERSION.
  2534. .BI XZ_VERSION= XYYYZZZS
  2535. .br
  2536. .BI LIBLZMA_VERSION= XYYYZZZS
  2537. .TP
  2538. .I X
  2539. Major version.
  2540. .TP
  2541. .I YYY
  2542. Minor version.
  2543. Even numbers are stable.
  2544. Odd numbers are alpha or beta versions.
  2545. .TP
  2546. .I ZZZ
  2547. Patch level for stable releases or
  2548. just a counter for development releases.
  2549. .TP
  2550. .I S
  2551. Stability.
  2552. 0 is alpha, 1 is beta, and 2 is stable.
  2553. .I S
  2554. should be always 2 when
  2555. .I YYY
  2556. is even.
  2557. .PP
  2558. .I XYYYZZZS
  2559. are the same on both lines if
  2560. .B xz
  2561. and liblzma are from the same XZ Utils release.
  2562. .PP
  2563. Examples: 4.999.9beta is
  2564. .B 49990091
  2565. and
  2566. 5.0.0 is
  2567. .BR 50000002 .
  2568. .
  2569. .SH "EXIT STATUS"
  2570. .TP
  2571. .B 0
  2572. All is good.
  2573. .TP
  2574. .B 1
  2575. An error occurred.
  2576. .TP
  2577. .B 2
  2578. Something worth a warning occurred,
  2579. but no actual errors occurred.
  2580. .PP
  2581. Notices (not warnings or errors) printed on standard error
  2582. don't affect the exit status.
  2583. .
  2584. .SH ENVIRONMENT
  2585. .B xz
  2586. parses space-separated lists of options
  2587. from the environment variables
  2588. .\" TRANSLATORS: Don't translate the uppercase XZ_DEFAULTS or XZ_OPT.
  2589. .\" They are names of environment variables.
  2590. .B XZ_DEFAULTS
  2591. and
  2592. .BR XZ_OPT ,
  2593. in this order, before parsing the options from the command line.
  2594. Note that only options are parsed from the environment variables;
  2595. all non-options are silently ignored.
  2596. Parsing is done with
  2597. .BR getopt_long (3)
  2598. which is used also for the command line arguments.
  2599. .PP
  2600. .B Warning:
  2601. By setting these environment variables,
  2602. one is effectively modifying programs and scripts that run
  2603. .BR xz .
  2604. Most of the time it is safe to set memory usage limits, number of threads,
  2605. and compression options via the environment variables.
  2606. However, some options can break scripts.
  2607. An obvious example is
  2608. .B \-\-help
  2609. which makes
  2610. .B xz
  2611. show the help text instead of compressing or decompressing a file.
  2612. More subtle examples are
  2613. .B \-\-quiet
  2614. and
  2615. .BR \-\-verbose .
  2616. In many cases it works well to enable the progress indicator using
  2617. .BR \-\-verbose ,
  2618. but in some situations the extra messages create problems.
  2619. The verbosity level also affects the behavior of
  2620. .BR \-\-list .
  2621. .TP
  2622. .B XZ_DEFAULTS
  2623. User-specific or system-wide default options.
  2624. Typically this is set in a shell initialization script to enable
  2625. .BR xz 's
  2626. memory usage limiter by default or set the default number of threads.
  2627. Excluding shell initialization scripts
  2628. and similar special cases, scripts should never set or unset
  2629. .BR XZ_DEFAULTS .
  2630. .TP
  2631. .B XZ_OPT
  2632. This is for passing options to
  2633. .B xz
  2634. when it is not possible to set the options directly on the
  2635. .B xz
  2636. command line.
  2637. This is the case when
  2638. .B xz
  2639. is run by a script or tool, for example, GNU
  2640. .BR tar (1):
  2641. .RS
  2642. .RS
  2643. .PP
  2644. .nf
  2645. .ft CR
  2646. XZ_OPT=\-2v tar caf foo.tar.xz foo
  2647. .ft R
  2648. .fi
  2649. .RE
  2650. .RE
  2651. .IP ""
  2652. Scripts may use
  2653. .BR XZ_OPT ,
  2654. for example, to set script-specific default compression options.
  2655. It is still recommended to allow users to override
  2656. .B XZ_OPT
  2657. if that is reasonable.
  2658. For example, in
  2659. .BR sh (1)
  2660. scripts one may use something like this:
  2661. .RS
  2662. .RS
  2663. .PP
  2664. .nf
  2665. .ft CR
  2666. XZ_OPT=${XZ_OPT\-"\-7e"}
  2667. export XZ_OPT
  2668. .ft R
  2669. .fi
  2670. .RE
  2671. .RE
  2672. .
  2673. .SH "LZMA UTILS COMPATIBILITY"
  2674. The command line syntax of
  2675. .B xz
  2676. is practically a superset of
  2677. .BR lzma ,
  2678. .BR unlzma ,
  2679. and
  2680. .B lzcat
  2681. as found from LZMA Utils 4.32.x.
  2682. In most cases, it is possible to replace
  2683. LZMA Utils with XZ Utils without breaking existing scripts.
  2684. There are some incompatibilities though,
  2685. which may sometimes cause problems.
  2686. .
  2687. .SS "Compression preset levels"
  2688. The numbering of the compression level presets is not identical in
  2689. .B xz
  2690. and LZMA Utils.
  2691. The most important difference is how dictionary sizes
  2692. are mapped to different presets.
  2693. Dictionary size is roughly equal to the decompressor memory usage.
  2694. .RS
  2695. .PP
  2696. .TS
  2697. tab(;);
  2698. c c c
  2699. c n n.
  2700. Level;xz;LZMA Utils
  2701. \-0;256 KiB;N/A
  2702. \-1;1 MiB;64 KiB
  2703. \-2;2 MiB;1 MiB
  2704. \-3;4 MiB;512 KiB
  2705. \-4;4 MiB;1 MiB
  2706. \-5;8 MiB;2 MiB
  2707. \-6;8 MiB;4 MiB
  2708. \-7;16 MiB;8 MiB
  2709. \-8;32 MiB;16 MiB
  2710. \-9;64 MiB;32 MiB
  2711. .TE
  2712. .RE
  2713. .PP
  2714. The dictionary size differences affect
  2715. the compressor memory usage too,
  2716. but there are some other differences between
  2717. LZMA Utils and XZ Utils, which
  2718. make the difference even bigger:
  2719. .RS
  2720. .PP
  2721. .TS
  2722. tab(;);
  2723. c c c
  2724. c n n.
  2725. Level;xz;LZMA Utils 4.32.x
  2726. \-0;3 MiB;N/A
  2727. \-1;9 MiB;2 MiB
  2728. \-2;17 MiB;12 MiB
  2729. \-3;32 MiB;12 MiB
  2730. \-4;48 MiB;16 MiB
  2731. \-5;94 MiB;26 MiB
  2732. \-6;94 MiB;45 MiB
  2733. \-7;186 MiB;83 MiB
  2734. \-8;370 MiB;159 MiB
  2735. \-9;674 MiB;311 MiB
  2736. .TE
  2737. .RE
  2738. .PP
  2739. The default preset level in LZMA Utils is
  2740. .B \-7
  2741. while in XZ Utils it is
  2742. .BR \-6 ,
  2743. so both use an 8 MiB dictionary by default.
  2744. .
  2745. .SS "Streamed vs. non-streamed .lzma files"
  2746. The uncompressed size of the file can be stored in the
  2747. .B .lzma
  2748. header.
  2749. LZMA Utils does that when compressing regular files.
  2750. The alternative is to mark that uncompressed size is unknown
  2751. and use end-of-payload marker to indicate
  2752. where the decompressor should stop.
  2753. LZMA Utils uses this method when uncompressed size isn't known,
  2754. which is the case, for example, in pipes.
  2755. .PP
  2756. .B xz
  2757. supports decompressing
  2758. .B .lzma
  2759. files with or without end-of-payload marker, but all
  2760. .B .lzma
  2761. files created by
  2762. .B xz
  2763. will use end-of-payload marker and have uncompressed size
  2764. marked as unknown in the
  2765. .B .lzma
  2766. header.
  2767. This may be a problem in some uncommon situations.
  2768. For example, a
  2769. .B .lzma
  2770. decompressor in an embedded device might work
  2771. only with files that have known uncompressed size.
  2772. If you hit this problem, you need to use LZMA Utils
  2773. or LZMA SDK to create
  2774. .B .lzma
  2775. files with known uncompressed size.
  2776. .
  2777. .SS "Unsupported .lzma files"
  2778. The
  2779. .B .lzma
  2780. format allows
  2781. .I lc
  2782. values up to 8, and
  2783. .I lp
  2784. values up to 4.
  2785. LZMA Utils can decompress files with any
  2786. .I lc
  2787. and
  2788. .IR lp ,
  2789. but always creates files with
  2790. .B lc=3
  2791. and
  2792. .BR lp=0 .
  2793. Creating files with other
  2794. .I lc
  2795. and
  2796. .I lp
  2797. is possible with
  2798. .B xz
  2799. and with LZMA SDK.
  2800. .PP
  2801. The implementation of the LZMA1 filter in liblzma
  2802. requires that the sum of
  2803. .I lc
  2804. and
  2805. .I lp
  2806. must not exceed 4.
  2807. Thus,
  2808. .B .lzma
  2809. files, which exceed this limitation, cannot be decompressed with
  2810. .BR xz .
  2811. .PP
  2812. LZMA Utils creates only
  2813. .B .lzma
  2814. files which have a dictionary size of
  2815. .RI "2^" n
  2816. (a power of 2) but accepts files with any dictionary size.
  2817. liblzma accepts only
  2818. .B .lzma
  2819. files which have a dictionary size of
  2820. .RI "2^" n
  2821. or
  2822. .RI "2^" n " + 2^(" n "\-1)."
  2823. This is to decrease false positives when detecting
  2824. .B .lzma
  2825. files.
  2826. .PP
  2827. These limitations shouldn't be a problem in practice,
  2828. since practically all
  2829. .B .lzma
  2830. files have been compressed with settings that liblzma will accept.
  2831. .
  2832. .SS "Trailing garbage"
  2833. When decompressing,
  2834. LZMA Utils silently ignore everything after the first
  2835. .B .lzma
  2836. stream.
  2837. In most situations, this is a bug.
  2838. This also means that LZMA Utils
  2839. don't support decompressing concatenated
  2840. .B .lzma
  2841. files.
  2842. .PP
  2843. If there is data left after the first
  2844. .B .lzma
  2845. stream,
  2846. .B xz
  2847. considers the file to be corrupt unless
  2848. .B \-\-single\-stream
  2849. was used.
  2850. This may break obscure scripts which have
  2851. assumed that trailing garbage is ignored.
  2852. .
  2853. .SH NOTES
  2854. .
  2855. .SS "Compressed output may vary"
  2856. The exact compressed output produced from
  2857. the same uncompressed input file
  2858. may vary between XZ Utils versions even if
  2859. compression options are identical.
  2860. This is because the encoder can be improved
  2861. (faster or better compression)
  2862. without affecting the file format.
  2863. The output can vary even between different
  2864. builds of the same XZ Utils version,
  2865. if different build options are used.
  2866. .PP
  2867. The above means that once
  2868. .B \-\-rsyncable
  2869. has been implemented,
  2870. the resulting files won't necessarily be rsyncable
  2871. unless both old and new files have been compressed
  2872. with the same xz version.
  2873. This problem can be fixed if a part of the encoder
  2874. implementation is frozen to keep rsyncable output
  2875. stable across xz versions.
  2876. .
  2877. .SS "Embedded .xz decompressors"
  2878. Embedded
  2879. .B .xz
  2880. decompressor implementations like XZ Embedded don't necessarily
  2881. support files created with integrity
  2882. .I check
  2883. types other than
  2884. .B none
  2885. and
  2886. .BR crc32 .
  2887. Since the default is
  2888. .BR \-\-check=crc64 ,
  2889. you must use
  2890. .B \-\-check=none
  2891. or
  2892. .B \-\-check=crc32
  2893. when creating files for embedded systems.
  2894. .PP
  2895. Outside embedded systems, all
  2896. .B .xz
  2897. format decompressors support all the
  2898. .I check
  2899. types, or at least are able to decompress
  2900. the file without verifying the
  2901. integrity check if the particular
  2902. .I check
  2903. is not supported.
  2904. .PP
  2905. XZ Embedded supports BCJ filters,
  2906. but only with the default start offset.
  2907. .
  2908. .SH EXAMPLES
  2909. .
  2910. .SS Basics
  2911. Compress the file
  2912. .I foo
  2913. into
  2914. .I foo.xz
  2915. using the default compression level
  2916. .RB ( \-6 ),
  2917. and remove
  2918. .I foo
  2919. if compression is successful:
  2920. .RS
  2921. .PP
  2922. .nf
  2923. .ft CR
  2924. xz foo
  2925. .ft R
  2926. .fi
  2927. .RE
  2928. .PP
  2929. Decompress
  2930. .I bar.xz
  2931. into
  2932. .I bar
  2933. and don't remove
  2934. .I bar.xz
  2935. even if decompression is successful:
  2936. .RS
  2937. .PP
  2938. .nf
  2939. .ft CR
  2940. xz \-dk bar.xz
  2941. .ft R
  2942. .fi
  2943. .RE
  2944. .PP
  2945. Create
  2946. .I baz.tar.xz
  2947. with the preset
  2948. .B \-4e
  2949. .RB ( "\-4 \-\-extreme" ),
  2950. which is slower than the default
  2951. .BR \-6 ,
  2952. but needs less memory for compression and decompression (48\ MiB
  2953. and 5\ MiB, respectively):
  2954. .RS
  2955. .PP
  2956. .nf
  2957. .ft CR
  2958. tar cf \- baz | xz \-4e > baz.tar.xz
  2959. .ft R
  2960. .fi
  2961. .RE
  2962. .PP
  2963. A mix of compressed and uncompressed files can be decompressed
  2964. to standard output with a single command:
  2965. .RS
  2966. .PP
  2967. .nf
  2968. .ft CR
  2969. xz \-dcf a.txt b.txt.xz c.txt d.txt.lzma > abcd.txt
  2970. .ft R
  2971. .fi
  2972. .RE
  2973. .
  2974. .SS "Parallel compression of many files"
  2975. On GNU and *BSD,
  2976. .BR find (1)
  2977. and
  2978. .BR xargs (1)
  2979. can be used to parallelize compression of many files:
  2980. .RS
  2981. .PP
  2982. .nf
  2983. .ft CR
  2984. find . \-type f \e! \-name '*.xz' \-print0 \e
  2985. | xargs \-0r \-P4 \-n16 xz \-T1
  2986. .ft R
  2987. .fi
  2988. .RE
  2989. .PP
  2990. The
  2991. .B \-P
  2992. option to
  2993. .BR xargs (1)
  2994. sets the number of parallel
  2995. .B xz
  2996. processes.
  2997. The best value for the
  2998. .B \-n
  2999. option depends on how many files there are to be compressed.
  3000. If there are only a couple of files,
  3001. the value should probably be 1;
  3002. with tens of thousands of files,
  3003. 100 or even more may be appropriate to reduce the number of
  3004. .B xz
  3005. processes that
  3006. .BR xargs (1)
  3007. will eventually create.
  3008. .PP
  3009. The option
  3010. .B \-T1
  3011. for
  3012. .B xz
  3013. is there to force it to single-threaded mode, because
  3014. .BR xargs (1)
  3015. is used to control the amount of parallelization.
  3016. .
  3017. .SS "Robot mode"
  3018. Calculate how many bytes have been saved in total
  3019. after compressing multiple files:
  3020. .RS
  3021. .PP
  3022. .nf
  3023. .ft CR
  3024. xz \-\-robot \-\-list *.xz | awk '/^totals/{print $5\-$4}'
  3025. .ft R
  3026. .fi
  3027. .RE
  3028. .PP
  3029. A script may want to know that it is using new enough
  3030. .BR xz .
  3031. The following
  3032. .BR sh (1)
  3033. script checks that the version number of the
  3034. .B xz
  3035. tool is at least 5.0.0.
  3036. This method is compatible with old beta versions,
  3037. which didn't support the
  3038. .B \-\-robot
  3039. option:
  3040. .RS
  3041. .PP
  3042. .nf
  3043. .ft CR
  3044. if ! eval "$(xz \-\-robot \-\-version 2> /dev/null)" ||
  3045. [ "$XZ_VERSION" \-lt 50000002 ]; then
  3046. echo "Your xz is too old."
  3047. fi
  3048. unset XZ_VERSION LIBLZMA_VERSION
  3049. .ft R
  3050. .fi
  3051. .RE
  3052. .PP
  3053. Set a memory usage limit for decompression using
  3054. .BR XZ_OPT ,
  3055. but if a limit has already been set, don't increase it:
  3056. .RS
  3057. .PP
  3058. .nf
  3059. .ft CR
  3060. NEWLIM=$((123 << 20))\ \ # 123 MiB
  3061. OLDLIM=$(xz \-\-robot \-\-info\-memory | cut \-f3)
  3062. if [ $OLDLIM \-eq 0 \-o $OLDLIM \-gt $NEWLIM ]; then
  3063. XZ_OPT="$XZ_OPT \-\-memlimit\-decompress=$NEWLIM"
  3064. export XZ_OPT
  3065. fi
  3066. .ft R
  3067. .fi
  3068. .RE
  3069. .
  3070. .SS "Custom compressor filter chains"
  3071. The simplest use for custom filter chains is
  3072. customizing a LZMA2 preset.
  3073. This can be useful,
  3074. because the presets cover only a subset of the
  3075. potentially useful combinations of compression settings.
  3076. .PP
  3077. The CompCPU columns of the tables
  3078. from the descriptions of the options
  3079. .BR "\-0" " ... " "\-9"
  3080. and
  3081. .B \-\-extreme
  3082. are useful when customizing LZMA2 presets.
  3083. Here are the relevant parts collected from those two tables:
  3084. .RS
  3085. .PP
  3086. .TS
  3087. tab(;);
  3088. c c
  3089. n n.
  3090. Preset;CompCPU
  3091. \-0;0
  3092. \-1;1
  3093. \-2;2
  3094. \-3;3
  3095. \-4;4
  3096. \-5;5
  3097. \-6;6
  3098. \-5e;7
  3099. \-6e;8
  3100. .TE
  3101. .RE
  3102. .PP
  3103. If you know that a file requires
  3104. somewhat big dictionary (for example, 32\ MiB) to compress well,
  3105. but you want to compress it quicker than
  3106. .B "xz \-8"
  3107. would do, a preset with a low CompCPU value (for example, 1)
  3108. can be modified to use a bigger dictionary:
  3109. .RS
  3110. .PP
  3111. .nf
  3112. .ft CR
  3113. xz \-\-lzma2=preset=1,dict=32MiB foo.tar
  3114. .ft R
  3115. .fi
  3116. .RE
  3117. .PP
  3118. With certain files, the above command may be faster than
  3119. .B "xz \-6"
  3120. while compressing significantly better.
  3121. However, it must be emphasized that only some files benefit from
  3122. a big dictionary while keeping the CompCPU value low.
  3123. The most obvious situation,
  3124. where a big dictionary can help a lot,
  3125. is an archive containing very similar files
  3126. of at least a few megabytes each.
  3127. The dictionary size has to be significantly bigger
  3128. than any individual file to allow LZMA2 to take
  3129. full advantage of the similarities between consecutive files.
  3130. .PP
  3131. If very high compressor and decompressor memory usage is fine,
  3132. and the file being compressed is
  3133. at least several hundred megabytes, it may be useful
  3134. to use an even bigger dictionary than the 64 MiB that
  3135. .B "xz \-9"
  3136. would use:
  3137. .RS
  3138. .PP
  3139. .nf
  3140. .ft CR
  3141. xz \-vv \-\-lzma2=dict=192MiB big_foo.tar
  3142. .ft R
  3143. .fi
  3144. .RE
  3145. .PP
  3146. Using
  3147. .B \-vv
  3148. .RB ( "\-\-verbose \-\-verbose" )
  3149. like in the above example can be useful
  3150. to see the memory requirements
  3151. of the compressor and decompressor.
  3152. Remember that using a dictionary bigger than
  3153. the size of the uncompressed file is waste of memory,
  3154. so the above command isn't useful for small files.
  3155. .PP
  3156. Sometimes the compression time doesn't matter,
  3157. but the decompressor memory usage has to be kept low, for example,
  3158. to make it possible to decompress the file on an embedded system.
  3159. The following command uses
  3160. .B \-6e
  3161. .RB ( "\-6 \-\-extreme" )
  3162. as a base and sets the dictionary to only 64\ KiB.
  3163. The resulting file can be decompressed with XZ Embedded
  3164. (that's why there is
  3165. .BR \-\-check=crc32 )
  3166. using about 100\ KiB of memory.
  3167. .RS
  3168. .PP
  3169. .nf
  3170. .ft CR
  3171. xz \-\-check=crc32 \-\-lzma2=preset=6e,dict=64KiB foo
  3172. .ft R
  3173. .fi
  3174. .RE
  3175. .PP
  3176. If you want to squeeze out as many bytes as possible,
  3177. adjusting the number of literal context bits
  3178. .RI ( lc )
  3179. and number of position bits
  3180. .RI ( pb )
  3181. can sometimes help.
  3182. Adjusting the number of literal position bits
  3183. .RI ( lp )
  3184. might help too, but usually
  3185. .I lc
  3186. and
  3187. .I pb
  3188. are more important.
  3189. For example, a source code archive contains mostly US-ASCII text,
  3190. so something like the following might give
  3191. slightly (like 0.1\ %) smaller file than
  3192. .B "xz \-6e"
  3193. (try also without
  3194. .BR lc=4 ):
  3195. .RS
  3196. .PP
  3197. .nf
  3198. .ft CR
  3199. xz \-\-lzma2=preset=6e,pb=0,lc=4 source_code.tar
  3200. .ft R
  3201. .fi
  3202. .RE
  3203. .PP
  3204. Using another filter together with LZMA2 can improve
  3205. compression with certain file types.
  3206. For example, to compress a x86-32 or x86-64 shared library
  3207. using the x86 BCJ filter:
  3208. .RS
  3209. .PP
  3210. .nf
  3211. .ft CR
  3212. xz \-\-x86 \-\-lzma2 libfoo.so
  3213. .ft R
  3214. .fi
  3215. .RE
  3216. .PP
  3217. Note that the order of the filter options is significant.
  3218. If
  3219. .B \-\-x86
  3220. is specified after
  3221. .BR \-\-lzma2 ,
  3222. .B xz
  3223. will give an error,
  3224. because there cannot be any filter after LZMA2,
  3225. and also because the x86 BCJ filter cannot be used
  3226. as the last filter in the chain.
  3227. .PP
  3228. The Delta filter together with LZMA2
  3229. can give good results with bitmap images.
  3230. It should usually beat PNG,
  3231. which has a few more advanced filters than simple
  3232. delta but uses Deflate for the actual compression.
  3233. .PP
  3234. The image has to be saved in uncompressed format,
  3235. for example, as uncompressed TIFF.
  3236. The distance parameter of the Delta filter is set
  3237. to match the number of bytes per pixel in the image.
  3238. For example, 24-bit RGB bitmap needs
  3239. .BR dist=3 ,
  3240. and it is also good to pass
  3241. .B pb=0
  3242. to LZMA2 to accommodate the three-byte alignment:
  3243. .RS
  3244. .PP
  3245. .nf
  3246. .ft CR
  3247. xz \-\-delta=dist=3 \-\-lzma2=pb=0 foo.tiff
  3248. .ft R
  3249. .fi
  3250. .RE
  3251. .PP
  3252. If multiple images have been put into a single archive (for example,
  3253. .BR .tar ),
  3254. the Delta filter will work on that too as long as all images
  3255. have the same number of bytes per pixel.
  3256. .
  3257. .SH "SEE ALSO"
  3258. .BR xzdec (1),
  3259. .BR xzdiff (1),
  3260. .BR xzgrep (1),
  3261. .BR xzless (1),
  3262. .BR xzmore (1),
  3263. .BR gzip (1),
  3264. .BR bzip2 (1),
  3265. .BR 7z (1)
  3266. .PP
  3267. XZ Utils: <https://tukaani.org/xz/>
  3268. .br
  3269. XZ Embedded: <https://tukaani.org/xz/embedded.html>
  3270. .br
  3271. LZMA SDK: <https://7-zip.org/sdk.html>