Swiftpack.co - Package - SwiftyTesseract/SwiftyTesseract

SwiftyTesseract

SPM compatible swift-version platforms CI

Table of Contents

Version Compatibility

| Swift Version | Platforms Supported | SwiftyTesseract Version | | ------------- | :-------------------------: | -------------------------: | | 5.3 | iOS macOS Linux | develop (soon to be 4.0.0) | | 5.0 - 5.2 | iOS | 3.x.x | | 4.2 | iOS | 2.x.x | | 4.0 - 4.1 | iOS | 1.x.x |

Develop should be considered unstable and API breaking changes could happen at any time. If you need to utilize some changes contained in develop, adding the specific commit is highly recommended:

.package(
    url: "https://github.com/SwiftyTesseract/SwiftyTesseract.git",
    // This is just an example of a commit hash, do not just copy and paste this into your Package.swift
    .revision("0e0c6aca147add5d5750ecb7810837ef4fd10fc2")
)

Only track develop if you are working on applications that absolutely need the changes contained in it. For example, if you are currently working on targeting iOS 14, there is an issue with an ambiguous initializer in the RecognitionBlock implementation that only affects that platform. This has been resolved in develop, but will not be merged into master and tagged for release until the Xcode 12 GM becomes available.

SwiftyTesseract class renamed to Tesseract

The SwiftyTesseract class name felt a bit verbose and is more descriptive of the project than the class itself. To disambiguate between Google's Tesseract project and SwiftyTesseract's Tesseract class, all mentions of the class will be displayed as a code snippet: Tesseract.

Using SwiftyTesseract in Your Project

Import the module

import SwiftyTesseract

There are two ways to quickly instantiate SwiftyTesseract without altering the default values. With one language:

let tesseract = Tesseract(language: .english)

Or with multiple languages:

let tesseract = Tesseract(languages: [.english, .french, .italian])

Performing OCR

Platform Agnostic

Pass an instance of Data derived from an image to performOCR(on:)

let imageData = try Data(contentsOf: urlOfYourImage)
let result: Result<String, Tesseract.Error> = tesseract.performOCR(on: imageData)

Combine

Pass an instance of Data derived from an image to performOCRPublisher(on:)

let imageData = try Data(contentsOf: urlOfYourImage)
let result: AnyPublisher<String, Tesseract.Error> = tesseract.performOCRPublisher(on: imageData)

UIKit

Pass a UIImage to the performOCR(on:) or performOCRPublisher(on:) methods:

let image = UIImage(named: "someImageWithText.jpg")!
let result: Result<String, Error> = tesseract.performOCR(on: image)
let publisher: AnyPublisher<String, Error> = tesseract.performOCRPublisher(on: image)

AppKit

Pass a NSImage to the performOCR(on:) or performOCRPublisher(on:) methods:

let image = NSImage(named: "someImageWithText.jpg")!
let result: Result<String, Error> = tesseract.performOCR(on: image)
let publisher: AnyPublisher<String, Error> = tesseract.performOCRPublisher(on: image)

Conclusion

For people who just want a synchronous call, the performOCR(on:) method provides a Result<String, Error> return value and blocks on the thread it is called on.

The performOCRPublisher(on:) publisher is available for ease of performing OCR in a background thread and receiving results on the main thread like so (only available on iOS 13.0+ and macOS 10.15+):

let cancellable = tesseract.performOCRPublisher(on: image)
  .subscribe(on: backgroundQueue)
  .receive(on: DispatchQueue.main)
  .sink(
    receiveCompletion: { completion in 
      // do something with completion
    },
    receiveValue: { string in
      // do something with string
    }
  )

The publisher provided by performOCRPublisher(on:) is a cold publisher, meaning it does not perform any work until it is subscribed to.

Extensibility

The major downside to the pre-4.0.0 API was it's lack of extensibility. If a user needed to set a variable or perform an operation that existed in the Google Tesseract API but didn't exist on the SwiftyTesseract API, their options were to fork the project or create a PR. These have been remedied by creating an extensible API for Tesseract variables and Tesseract functions.

Tesseract Variable Configuration

Starting in 4.0.0, all public instance variables of Tesseract have been removed in favor of a more extensible and declaritive API:

let tesseract = Tesseract(language: .english) {
  set(.disallowlist, "@#$%^&*")
  set(.minimumCharacterHeight, .integer(35))
  set(.preserveInterwordSpaces, .true)
}
// or
let tesseract = Tesseract(language: .english)
tesseract.configure {
  set(.disallowlist, "@#$%^&*")
  set(.minimumCharacterHeight, .integer(35))
  set(.preserveInterwordSpaces, .true)
}

The pre-4.0.0 API looks like this:

let swiftyTesseract = SwiftyTesseract(languge: .english)
swiftyTesseract.blackList = "@#$%^&*"
swiftyTesseract.minimumCharacterHeight = 35
swiftyTesseract.preserveInterwordSpaces = true

Tesseract.Variable

Tesseract.Variable is a new struct introduced in 4.0.0. It's definition is quite simple:

extension Tesseract {
  public struct Variable: RawRepresentable {
    public init(rawValue: String) {
      self.init(rawValue)
    }
    
    public init(_ rawValue: String) {
      self.rawValue = rawValue
    }
    
    public let rawValue: String
  }
}

// Extensions containing the previous API variables available as members of SwiftyTesseract
public extension Tesseract.Variable {
  static let allowlist = Tesseract.Variable("tessedit_char_whitelist")
  static let disallowlist = Tesseract.Variable("tessedit_char_blacklist")
  static let preserveInterwordSpaces = Tesseract.Variable("preserve_interword_spaces")
  static let minimumCharacterHeight = Tesseract.Variable("textord_min_xheight")
  static let oldCharacterHeight = Tesseract.Variable("textord_old_xheight")
}

The problem here is that the library doesn't cover all the cases. What if you wanted to set Tesseract to only recognize numbers? You may be able to set the allowlist to only recognize numerals, but the Google Tesseract API already has a variable that does that: "classify_bln_numeric_mode".

Extending the library to make use of that variable could look something like this:

tesseract.configure {
  set(Tesseract.Variable("classify_bln_numeric_mode"), .true)
}
// or extend Tesseract.Variable to get a clean trailing dot syntax
extension Tesseract.Variable {
  static let numericMode = Tesseract.Variable("classify_bln_numeric_mode")
}

tesseract.configure {
  set(.numericMode, .true)
}

perform(action:)

Another issue that I've seen come up several times is "Can you impelement X Tesseract feature" as a feature request. This has the same implications as the old property-based accessors for setting Tesseract variables. This allows users full access to the Tesseract API in a thread-safe manner.

This comes with one major caveat: You will be completely responsible for managing memory when dealing with the Tessearct API directly.

All of the library methods provided on Tesseract other than Tesseract.perform(action:) and Tesseract.configure(_:) are implemented as extensions using only Tesseract.perform(action:) to access the pointer created during initialization. To see this in action see the implementation of performOCR(on:) in Sources/SwiftyTesseract/Tesseract+OCR.swift

As an example, let's implement issue #66 using perform(action:):

import SwiftyTesseract
import libtesseract

public typealias PageSegmentationMode = TessPageSegMode
public extension PageSegmentationMode {
  static let osdOnly = PSM_OSD_ONLY
  static let autoOsd = PSM_AUTO_OSD
  static let autoOnly = PSM_AUTO_ONLY
  static let auto = PSM_AUTO
  static let singleColumn = PSM_SINGLE_COLUMN
  static let singleBlockVerticalText = PSM_SINGLE_BLOCK_VERT_TEXT
  static let singleBlock = PSM_SINGLE_BLOCK
  static let singleLine = PSM_SINGLE_LINE
  static let singleWord = PSM_SINGLE_WORD
  static let circleWord = PSM_CIRCLE_WORD
  static let singleCharacter = PSM_SINGLE_CHAR
  static let sparseText = PSM_SPARSE_TEXT
  static let sparseTextOsd = PSM_SPARSE_TEXT_OSD
  static let count = PSM_COUNT
}

public extension Tesseract {
  var pageSegmentationMode: PageSegmentationMode {
    get {
      perform { tessPointer in
        TessBaseAPIGetPageSegMode(tessPointer)
      }
    }
    set {
      perform { tessPointer in
        TessBaseAPISetPageSegMode(tessPointer, newValue)
      }
    }
  }
}

// usage
tesseract.pageSegmentationMode = .singleColumn

If you don't care about all the implentation needed to make your call-site feel "Swifty" (who doesn't love leading dot syntax?!) you could implement it simply like this:

import SwiftyTesseract
import libtesseract

extension Tesseract {
  var pageSegMode: TessPageSegMode {
    get {
      perform { tessPointer in
        TessBaseAPIGetPageSegMode(tessPointer)
      }
    }
    set {
      perform { tessPointer in
        TessBaseAPISetPageSegMode(tessPointer, newValue)
      }
    }
  }
}

// usage
tesseract.pageSegMode = PSM_SINGLE_COLUMN

(The information for what to implement for this example was found in the Tesseract documentation)

Now that this API is available, additions to the API surface of the library will be very selective. There should no longer be any restrictions to users of the library given the extensibility.

A Note on Initializer Defaults

The full signature of the primary Tesseract initializer is

public init Tesseract(
  languages: [RecognitionLanguage], 
  dataSource: LanguageModelDataSource = Bundle.main, 
  engineMode: EngineMode = .lstmOnly
)

The bundle parameter is required to locate the tessdata folder. This will only need to be changed if SwiftyTesseract is not being implemented in your application bundle. The engine mode dictates the type of .traineddata files to put into your tessdata folder. .lstmOnly was chosen as a default due to the higher speed and reliability found during testing, but could potentially vary depending on the language being recognized as well as the image itself. See Which Language Training Data Should You Use? for more information on the different types of .traineddata files that can be used with SwiftyTesseract

libtesseract

Tesseract and it's dependencies are now built and distributed as an xcframework under the SwiftyTesseract/libtesseract repository for Apple platforms. Any issues regarding the build configurations for those should be raised under that repository.

Installation

Swift Package Manager is now the only supported dependency manager for bringing SwiftyTesseract into your project.

Apple Platforms

// Package.swift
// swift-tools-version:5.3
// The swift-tools-version declares the minimum version of Swift required to build this package.
import PackageDescription

let package = Package(
  name: "AwesomePackage",
  platforms: [
    // These are the minimum versions libtesseract supports
    .macOS(.v10_13),
    .iOS(.v11),
  ],
  products: [
    .library(
      name: "AwesomePackage",
      targets: ["AwesomePackage"]
    ),
  ],
  dependencies: [
    .package(url: "https://github.com/SwiftyTesseract/SwiftyTesseract.git", from: "4.0.0")
  ],
  targets: [
    .target(
      name: "AwesomePackage",
      dependencies: ["SwiftyTesseract"]
    ),
  ]
)

Linux

// Package.swift
// swift-tools-version:5.3
// The swift-tools-version declares the minimum version of Swift required to build this package.
import PackageDescription

let package = Package(
  name: "AwesomePackage",
  products: [
    .library(
      name: "AwesomePackage",
      targets: ["AwesomePackage"]
    ),
  ],
  dependencies: [
    .package(url: "https://github.com/SwiftyTesseract/SwiftyTesseract.git", from: "4.0.0")
  ],
  targets: [
    .target(
      name: "AwesomePackage",
      dependencies: ["SwiftyTesseractLinux"]
    ),
  ]
)

Additional configuration

Shipping language training files as part of an application bundle

  1. Download the appropriate language training files from the tessdata, tessdata_best, or tessdata_fast repositories.
  2. Place your language training files into a folder on your computer named tessdata
  3. Drag the folder into your project. You must enure that "Create folder references" is selected or Tesseract will not be succesfully instantiated. tessdata_folder_example

Shipping language training files as part of a Swift Package

If you choose to keep the language training data files under source control, you will want to copy your tessdata directory as a package resource:

let package = Package(
  // Context omitted for brevity. The full Package.swift for this example
  // can be found in Examples/VaporExample/Package.swift
  targets: [
    .target(
      name: "App",
      dependencies: [
        .product(name: "Vapor", package: "vapor"),
        "SwiftyTesseract"
      ],
      // The path relative to your Target directory. In this example, the path
      // relative to the source root would be Sources/App/tessdata
      resources: [.copy("tessdata")],
    )
  ]
)

If you prefer not to keep the language training data files under source control see the instructions for using a custom location below.

Custom Location

Thanks to Minitour, developers now have more flexibility in where and how the language training files are included for Tesseract to use. This may be beneficial if your application supports multiple languages but you do not want your application bundle (or git repo) to contain all the possible training files needed to perform OCR (each language training file can range from 1 MB to 15 MB). You will need to provide conformance to the following protocol:

public protocol LanguageModelDataSource {
  var pathToTrainedData: String { get }
}

Then pass it to the Tesseract initializer:

let customDataSource = CustomDataSource()
let tesseract = Tesseract(
  language: .english, 
  dataSource: customDataSource, 
  engineMode: .lstmOnly
)

See the testDataSourceFromFiles() test in SwiftyTesseractTests.swift (located near the end of the file) for an example on how this can be done.

Language Training Data Considerations

There are three different types of .traineddata files that can be used in SwiftyTesseract: tessdata, tessdata_best, or tessdata_fast that correspond to SwiftyTesseract EngineModes .tesseractOnly, .lstmOnly, and .tesseractLstmCombined. .tesseractOnly uses the legacy Tesseract engine and can only use language training files from the tessdata repository. During testing of SwiftyTesseract, the .tesseractOnly engine mode was found to be the least reliable. .lstmOnly uses a long short-term memory recurrent neural network to perform OCR and can use language training files from either tessdata_best, tessdata_fast, or tessdata repositories. During testing, tessdata_best was found to provide the most reliable results at the cost of speed, while tessdata_fast provided results that were comparable to tessdata (when used with .lstmOnly) and faster than both tessdata and tessdata_best. .tesseractLstmCombined can only use language files from the tessdata repository, and the results and speed seemed to be on par with tessdata_best. For most cases, .lstmOnly along with the tessdata_fast language training files will likely be the best option, but this could vary depending on the language and application of SwiftyTesseract in your project.

Linux Specific Configuration

You will need to install libtesseract-dev (must be a 4.x.x release) and libleptonica-dev on the host system before running any application that has a dependency on SwiftyTesseract. For Ubuntu (or Debian based distributions) that may look like this:

apt-get install -yq libtesseract-dev libleptonica-dev

Custom Trained Data

The steps required are the same as the instructions provided in additional configuration. To utilize custom .traineddata files, simply use the .custom(String) case of RecognitionLanguage:

let swiftyTesseract = Tesseract(language: .custom("custom-traineddata-file-prefix"))

For example, if you wanted to use the MRZ code optimized OCRB.traineddata file provided by Exteris/tesseract-mrz, the instance of Tesseract would be created like this:

let swiftyTesseract = Tesseract(language: .custom("OCRB"))

You may also include the first party Tesseract language training files with custom training files:

let swiftyTesseract = Tesseract(languages: [.custom("OCRB"), .english])

Recognition Results

When it comes to OCR, the adage "garbage in, garbage out" applies. SwiftyTesseract is no different. The underlying Tesseract engine will process the image and return anything that it believes is text. For example, giving SwiftyTesseract this image raw_unprocessed_image yields the following:

a lot of jibbersh...
‘o 1 $ : M |
© 1 3 1; ie oI
LW 2 = o .C P It R <0f
O — £988 . 18 |
SALE + . < m m & f f |
7 Abt | | . 3 I] R I|
3 BE? | is —bB (|
* , § Be x I 3 |
...a lot more jibberish

You can see that it picked SALE out of the picture, but everything else surrounding it was still attempted to be read regardless of orientation. It is up to the individual developer to determine the appropriate way to edit and transform the image to allow SwiftyTesseract to render text in a way that yields predictable results. Originally, SwiftyTesseract was intended to be an out-of-the-box solution, however, the logic that was being added into the project made too many assumptions, nor did it seem right to force any particular implementation onto potential adoptors. SwiftyTesseractRTE provides a ready-made solution that can be implemented in a project with a few lines of code that should suit most needs and is a better place to start if the goal for your project is to get OCR into an application with little effort.

Contributions Welcome

SwiftyTesseract does not currently implement the full Tesseract API, so if there is functionality that you would like implemented, create an issue and open a pull request! Please see Contributing to SwiftyTesseract for the full guidelines on creating issues and opening pull requests to the project.

Documentation

Official documentation for SwiftyTesseract can be found here

Attributions

SwiftyTesseract would not be possible without the work done by the Tesseract team.

See the Attributions section in the libtesseract repo for a full list of vendored dependencies and their licenses.

Github

link
Stars: 122

Used By

Total: 0

Releases

Release 3.1.2 - 2020-04-26 13:27:46