Swiftpack.co - Package - tsolomko/SWCompression

SWCompression

Swift 4 GitHub license Build Status

A framework with (de)compression algorithms and functions for processing various archives and containers.

What is this?

SWCompression — is a framework with a collection of functions for:

  1. Decompression (and sometimes compression) using different algorithms.
  2. Reading (and sometimes writing) archives of different formats.
  3. Reading containers such as ZIP, TAR and 7-Zip.

It also works both on Apple platforms and Linux.

All features are listed in the tables below. "TBD" means that feature is planned but not implemented (yet).

| | Deflate | BZip2 | LZMA/LZMA2 | | ------------- | ------- | ----- | ---------- | | Decompression | ✅ | ✅ | ✅ | | Compression | ✅ | ✅ | TBD |

| | Zlib | GZip | XZ | | ----- | ---- | ---- | --- | | Read | ✅ | ✅ | ✅ | | Write | ✅ | ✅ | TBD |

| | ZIP | TAR | 7-Zip | | ----- | --- | --- | ----- | | Read | ✅ | ✅ | ✅ | | Write | TBD | TBD | TBD |

Also, SWCompression is written with Swift only.

Installation

SWCompression can be integrated into your project using Swift Package Manager, CocoaPods or Carthage.

Swift Package Manager

Add SWCompression to you package dependencies and specify it as a dependency for your target, e.g.:

import PackageDescription

let package = Package(
    name: "PackageName",
    dependencies: [
        .package(url: "https://github.com/tsolomko/SWCompression.git",
                 from: "4.0.0")
    ],
    targets: [
        .target(
            name: "TargetName",
            dependencies: ["SWCompression"]
        )
    ]
)

More details you can find in Swift Package Manager's Documentation.

CocoaPods

Add pod 'SWCompression', '~> 4.0' and use_frameworks! to your Podfile.

To complete installation, run pod install.

If you need only some parts of framework, you can install only them using sub-podspecs. Available subspecs:

  • SWCompression/BZip2
  • SWCompression/Deflate
  • SWCompression/Gzip
  • SWCompression/LZMA
  • SWCompression/LZMA2
  • SWCompression/SevenZip
  • SWCompression/TAR
  • SWCompression/XZ
  • SWCompression/Zlib
  • SWCompression/ZIP

"Optional Dependencies"

Both ZIP and 7-Zip containers have a single compression method which is most likely to be used, for compression of data inside them. This is Deflate for ZIP and LZMA/LZMA2 for 7-Zip. Thus, SWCompression/ZIP subspec have SWCompression/Deflate subspec as a dependency and SWCompression/LZMA subspec is a dependency for SWCompression/SevenZip.

But both of these formats support other compression methods, and some of them are implemented in SWCompression. For CocoaPods configurations there are some sort of 'optional dependencies' for such compression methods.

"Optional dependency" in this context means that SWCompression/ZIP or SWCompression/7-Zip will support particular compression methods only if a corresponding subspec is expicitly specified in your Podfile and installed.

List of "optional dependecies":

  • For SWCompression/ZIP:
    • SWCompression/BZip2
    • SWCompression/LZMA
  • For SWCompression/SevenZip:
    • SWCompression/BZip2
    • SWCompression/Deflate

Note: If you use Swift Package Manager or Carthage you always have everything (ZIP and 7-Zip are built with Deflate, BZip2 and LZMA/LZMA2 support).

Carthage

Add to your Cartfile github "tsolomko/SWCompression" ~> 4.0.

Then run carthage update.

Finally, drag and drop SWCompression.framework from Carthage/Build folder into the "Embedded Binaries" section on your targets' "General" tab in Xcode.

SWCompression depends on BitByteData framework, so Carthage will also download this dependency, and you should drag and drop BitByteData.framework file into the "Embedded Binaries" as well.

Usage

Basic Example

If you'd like to decompress "deflated" data just use:

// let data = <Your compressed data>
let decompressedData = try? Deflate.decompress(data: data)

However, it is unlikely that you will encounter deflated data outside of any archive. So, in case of GZip archive you should use:

let decompressedData = try? GzipArchive.unarchive(archiveData: data)

Handling Errors

Most SWCompression functions can throw an error and you are responsible for handling them. If you look at list of available error types and their cases, you may be frightened by their number. However, most of these cases (such as XZError.wrongMagic) exist for diagnostic purposes.

Thus, you only need to handle the most common type of error for your archive/algorithm. For example:

do {
    // let data = <Your compressed data>
    let decompressedData = try XZArchive.unarchive(archive: data)
} catch let error as XZError {
    <handle XZ related error here>
} catch let error {
    <handle all other errors here>
}

Or, if you don't care about errors at all, use try?.

Documentation

Every function or type of SWCompression's public API is documented. This documentation can be found at its own website.

Sophisticated example

There is a small command-line program, "swcomp", which is included in this repository in "Sources/swcomp". To build it you need to uncomment several lines in "Package.swift" and run swift build -c release.

Contributing

Whether you find a bug, have a suggestion, idea or something else, please create an issue on GitHub.

In case you have encoutered a bug, it would be especially helpful if you attach a file (archive, etc.) that caused the bug to happen.

If you'd like to contribute code, please create a pull request on GitHub.

Note: If you are considering working on SWCompression, please note that Xcode project (SWCompression.xcodeproj) was created manually and you shouldn't use swift package generate-xcodeproj command.

Executing tests locally

If you'd like to run tests on your computer, you need to do an additional step after cloning this repository:

git submodule update --init --recursive

This command downloads files which are used for testing. These files are stored in a separate repository. Git LFS is used for storing them which is the reason for having them in the separate repository, since Swift Package Manager have some problems with Git LFS-enabled repositories (installing git-lfs locally with --skip-smudge option is required to solve these problems).

Note: You can also use "Utils/prepare-workspace-macos.sh" script from the repository, which not only downloads test files but also downloads dependencies.

Performace

Usage of whole module optimizations is recommended for best performance. These optimizations are enabled by default for Release configurations.

Tests Results document contains results of performance testing of various functions.

Why?

First of all, existing solutions for work with compression, archives and containers have some problems. They might not support some particular compression algorithms or archive formats and they all have different APIs, which sometimes can be slightly "unfriendly" to users. This project attempts to provide missing (and sometimes existing) functionality through unified API, which is easy to use and remember.

Secondly, it may be important to have a compression framework written completely in Swift, without relying on either system libraries or solutions implemented in different languages. Additionaly, since SWCompression is written fully in Swift without Objective-C, it can also be compiled on Linux.

Future plans

  • Performance...
  • Better Deflate compression.
  • Something else...

References

Github

link
Stars: 19
Help us keep the lights on

Dependencies

Used By

Total: 0

Releases

4.1.1 - Feb 11, 2018

  • Fixed incorrect value of TarEntryInfo.name in certain situations. Comment: This problem was occurring when file name was long enough to not fit into "name" field of TAR's header, but no PAX or GNU extensions were used, so the "prefix" field was used instead.
  • Updated documentation for TarEntryInfo.

4.1.0 - Jan 31, 2018

  • Functionality related to reading and writing bits and bytes was published as a separate project, BitByteData. As a result SWCompression now has BitByteData as a dependency.
  • Several performance improvements have been made. See updated test results for more information.
  • Added a workaround to prevent situations when some BZip2 archives created by SWCompression could not be opened by original BZip2 implementation. Comment: It seems that some BZip2 implementations (including original one) don't account for the fact that the initial Run-Length Encoding can actually cause expansion of the input data. To prevent this from happening SWCompression now uses BZip2 blocks smaller than intended by the format.
  • Modification time stored in ZIP's "native" field is now calculated relative to current system's calendar and time zone. Comment: Since standard ZIP's fields don't have any way to save time zone data it is more likely that archivers stored modification time relative to current system's settings rather than UTC-relative (which was assumed by SWCompression before). Behaviour of NTFS timestamp and Extended timestamp extra fields hasn't been changed.
  • Additions to swcomp command-line tool:
    • Added -1...-9 options to bz2 command which specifies what block size to use for BZip2 compression.
    • Added -i (--info) option to gz command which prints GZip's header.
    • Added comp-deflate and comp-bz2 subcommands to perf-test command which can be used for performance testing of Deflate and BZip2 compression.
    • Updated descriptions of -e options for zip, tar and 7z commands.
    • Now sets permissions for extracted files only if they are greater than 0.

4.1.0-test.2 - Jan 29, 2018

  • Performance improvements:
    • Preallocate byte arrays when output size is known.
    • Inverse Burrows-Wheeler transform without sorting.
    • Store ntfs reference date as a static constant.
  • "Fixed" a problem when some BZip2 archives created by SWCompression could not be opened by original BZip2 implementation.
  • Modification time stored in ZIP's "native" field is now calculated relative to current system's calendar and time zone.
  • Added options (-1...-9) to swcomp bz2 -c for specifying block size .
  • Added option (-i, --info) to swcomp gz for printing GZip's header.
  • Added commands (comp-deflate, comp-bz2) to swcomp for performance testing of Deflate and BZip2 compression.
  • Corrected descriptions of extract options of swcomp's container-related commands.
  • swcomp now sets permissions for extracted files only if they are greater than 0.

4.1.0-test.1 - Jan 3, 2018

The first part of changes planned for 4.1.0 is extraction of some code from SWCompression into a separate project. This was done recently and you can now check out the new project, BitByteData.

The purpose of this test release is to test integration of this new external dependency into SWCompression.

4.0.1 - Nov 25, 2017

  • Starting with this update, git tags for releases no longer have "v" prefix, since absence of such prefix is a common practice among Swift developers.
  • Fixed incorrectly thrown XZError.wrongDataSize without actually trying to decompress anything.
  • Fixed crash when opening 7-Zip containers with more than 255 entries with empty streams.
  • Reduced memory usage by Deflate compression.
  • Improved performance in some extreme cases (e.g. containers with an enormous amount of entries).
  • No longer verify if ZIP string field needs UTF-8 encoding, if language encoding flag is set for the entry.
  • Added "perf-test" command to swcomp, which is used for measuring performance. See Results document for the new tests results.