lzma-native

Node.js interface to the native liblzma compression library (.xz file format, among others)

This package provides interfaces for compression and decompression of .xz (and legacy .lzma) files, both stream-based and string-based.

Example usage

Installation

Simply install lzma-native via npm:

$ npm install --save lzma-native

Note: As of version 1.0.0, this module provides pre-built binaries for multiple Node.js versions and all major OS using node-pre-gyp, so for 99 % of users no compiler toolchain is necessary. Please create an issue here if you have any trouble installing this module.

Note: lzma-native@2.x requires a Node version >= 4. If you want to support Node 0.10 or 0.12, you can feel free to use lzma-native@1.x.

For streams

If you don’t have any fancy requirements, using this library is quite simple:

var lzma = require('lzma-native');

var compressor = lzma.createCompressor();
var input = fs.createReadStream('README.md');
var output = fs.createWriteStream('README.md.xz');

input.pipe(compressor).pipe(output);

For decompression, you can simply use lzma.createDecompressor().

Both functions return a stream where you can pipe your input in and read your (de)compressed output from.

For simple strings/Buffers

If you want your input/output to be Buffers (strings will be accepted as input), this even gets a little simpler:

lzma.compress('Banana', function(result) {
    console.log(result); // <Buffer fd 37 7a 58 5a 00 00 01 69 22 de 36 02 00 21 ...>
});

Again, replace lzma.compress with lzma.decompress and you’ll get the inverse transformation.

lzma.compress() and lzma.decompress() will return promises and you don’t need to provide any kind of callback (Example code).

API

Compatibility implementations

Apart from the API described here, lzma-native implements the APIs of the following other LZMA libraries so you can use it nearly as a drop-in replacement:

• [node-xz][node-xz] via lzma.Compressor and lzma.Decompressor
• [LZMA-JS][LZMA-JS] via lzma.LZMA().compress and lzma.LZMA().decompress, though without actual support for progress functions and returning Buffer objects instead of integer arrays. (This produces output in the .lzma file format, not the .xz format!)

Multi-threaded encoding

Since version 1.5.0, lzma-native supports liblzma’s built-in multi-threading encoding capabilities. To make use of them, set the threads option to an integer value: lzma.createCompressor({ threads: n });. You can use value of 0 to use the number of processor cores. This option is only available for the easyEncoder (the default) and streamEncoder encoders.

Note that, by default, encoding will take place in Node’s libuv thread pool regardless of this option, and setting it when multiple encoders are running is likely to affect performance negatively.

Reference

Encoding strings and Buffer objects * compress() – Compress strings and Buffers * decompress() – Decompress strings and Buffers * LZMA().compress() ([LZMA-JS][LZMA-JS] compatibility) * LZMA().decompress() ([LZMA-JS][LZMA-JS] compatibility)

Creating streams for encoding * createCompressor() – Compress streams * createDecompressor() – Decompress streams * createStream() – (De-)Compression with advanced options * Compressor() ([node-xz][node-xz] compatibility) * Decompressor() ([node-xz][node-xz] compatibility)

.xz file metadata * isXZ() – Test Buffer for .xz file format * parseFileIndex() – Read .xz file metadata * parseFileIndexFD() – Read .xz metadata from a file descriptor

Miscellaneous functions * crc32() – Calculate CRC32 checksum * checkSize() – Return required size for specific checksum type * easyDecoderMemusage() – Expected memory usage * easyEncoderMemusage() – Expected memory usage * rawDecoderMemusage() – Expected memory usage * rawEncoderMemusage() – Expected memory usage * versionString() – Native library version string * versionNumber() – Native library numerical version identifier

Encoding strings and Buffer objects

lzma.compress(), lzma.decompress()

• lzma.compress(string, [opt, ]on_finish)
• lzma.decompress(string, [opt, ]on_finish)

Param	Type	Description
string	Buffer / String	Any string or buffer to be (de)compressed (that can be passed to stream.end(…))
[opt]	Options / int	Optional. See options
on_finish	Callback	Will be invoked with the resulting Buffer as the first parameter when encoding is finished, and as on_finish(null, err) in case of an error.

These methods will also return a promise that you can use directly.

Example code:

lzma.compress('Bananas', 6, function(result) {
    lzma.decompress(result, function(decompressedResult) {
        assert.equal(decompressedResult.toString(), 'Bananas');
    });
});

Example code for promises:

lzma.compress('Bananas', 6).then(function(result) {
    return lzma.decompress(result);
}).then(function(decompressedResult) {
    assert.equal(decompressedResult.toString(), 'Bananas');
}).catch(function(err) {
    // ...
});

lzma.LZMA().compress(), lzma.LZMA().decompress()

• lzma.LZMA().compress(string, mode, on_finish[, on_progress])
• lzma.LZMA().decompress(string, on_finish[, on_progress])

(Compatibility; See [LZMA-JS][LZMA-JS] for the original specs.)

Note that the result of compression is in the older LZMA1 format (.lzma files). This is different from the more universally used LZMA2 format (.xz files) and you will have to take care of possible compatibility issues with systems expecting .xz files.

Param	Type	Description
string	Buffer / String / Array	Any string, buffer, or array of integers or typed integers (e.g. Uint8Array)
mode	int	A number between 0 and 9, indicating compression level
on_finish	Callback	Will be invoked with the resulting Buffer as the first parameter when encoding is finished, and as on_finish(null, err) in case of an error.
on_progress	Callback	Indicates progress by passing a number in [0.0, 1.0]. Currently, this package only invokes the callback with 0.0 and 1.0.

These methods will also return a promise that you can use directly.

This does not work exactly as described in the original [LZMA-JS][LZMA-JS] specification: * The results are Buffer objects, not integer arrays. This just makes a lot more sense in a Node.js environment. * on_progress is currently only called with 0.0 and 1.0.

Example code:

lzma.LZMA().compress('Bananas', 4, function(result) {
    lzma.LZMA().decompress(result, function(decompressedResult) {
        assert.equal(decompressedResult.toString(), 'Bananas');
    });
});

For an example using promises, see compress().

Creating streams for encoding

lzma.createCompressor(), lzma.createDecompressor()

• lzma.createCompressor([options])
• lzma.createDecompressor([options])

Param	Type	Description
[options]	Options / int	Optional. See options

Return a [duplex][duplex] stream, i.e. a both readable and writable stream. Input will be read, (de)compressed and written out. You can use this to pipe input through this stream, i.e. to mimick the xz command line util, you can write:

var compressor = lzma.createCompressor();

process.stdin.pipe(compressor).pipe(process.stdout);

The output of compression will be in LZMA2 format (.xz files), while decompression will accept either format via automatic detection.

lzma.Compressor(), lzma.Decompressor()

• lzma.Compressor([preset], [options])
• lzma.Decompressor([options])

(Compatibility; See [node-xz][node-xz] for the original specs.)

These methods handle the .xz file format.

Param	Type	Description
[preset]	int	Optional. See options.preset
[options]	Options	Optional. See options

Return a [duplex][duplex] stream, i.e. a both readable and writable stream. Input will be read, (de)compressed and written out. You can use this to pipe input through this stream, i.e. to mimick the xz command line util, you can write:

var compressor = lzma.Compressor();

process.stdin.pipe(compressor).pipe(process.stdout);

lzma.createStream()

• lzma.createStream(coder, options)

Param	Type	Description
[coder]	string	Any of the supported coder names, e.g. "easyEncoder" (default) or "autoDecoder".
[options]	Options / int	Optional. See options

Return a [duplex][duplex] stream for (de-)compression. You can use this to pipe input through this stream.

The available coders are (the most interesting ones first):

• easyEncoder Standard LZMA2 (.xz file format) encoder. Supports options.preset and options.check options.
• autoDecoder Standard LZMA1/2 (both .xz and .lzma) decoder with auto detection of file format. Supports options.memlimit and options.flags options.
• aloneEncoder Encoder which only uses the legacy .lzma format. Supports the whole range of LZMA options.

Less likely to be of interest to you, but also available:

• aloneDecoder Decoder which only uses the legacy .lzma format. Supports the options.memlimit option.
• rawEncoder Custom encoder corresponding to lzma_raw_encoder (See the native library docs for details). Supports the options.filters option.
• rawDecoder Custom decoder corresponding to lzma_raw_decoder (See the native library docs for details). Supports the options.filters option.
• streamEncoder Custom encoder corresponding to lzma_stream_encoder (See the native library docs for details). Supports options.filters and options.check options.
• streamDecoder Custom decoder corresponding to lzma_stream_decoder (See the native library docs for details). Supports options.memlimit and options.flags options.

Options

<a name="api-opt