Swiftpack.co - Package - patrick-zippenfenig/ClickHouseNIO

ClickHouseNIO

Swift 5 SPM Platforms codebeat badge CircleCI

High performance Swift ClickHouse client based on SwiftNIO 2. It is inspired by the ClickHouse source code (C++), but written in pure Swift.

Features:

  • Asynchronous Swift NIO implementation. Perfect for concurrent APIs.
  • Native Swift data types support. Per table column a simple [Float], [Int] or [String] array can be used.
  • Simple query(), command() and insert() operations

This client provides raw query capabilities. Connection pooling or relational abstraction may be implemented on top of this library. For connection pooling and integration into Vapor use, ClickHouseVapor.

Installation:

  1. Add ClickHouseNIO as a dependency to your Package.swift
  dependencies: [
    .package(url: "https://github.com/patrick-zippenfenig/ClickHouseNIO.git", from: "1.0.0")
  ],
  targets: [
    .target(name: "MyApp", dependencies: ["ClickHouseNIO"])
  ]
  1. Build your project:
$ swift build

Usage

  1. Connect to a ClickHouseServer. The client requires a eventLoop which is usually provided by frameworks which use SwiftNIO. We also use wait() for simplicity, but it is discouraged for production code.
import NIO
import ClickHouseNIO

let config = try ClickHouseConfiguration(
    hostname: "localhost", 
    port: 9000, 
    user: "default", 
    password: "admin", 
    database: "default")
  
let eventLoopGroup = MultiThreadedEventLoopGroup(numberOfThreads: 1)  
let connection = try ClickHouseConnection.connect(configuration: config, on: eventLoopGroup.next()).wait()
  1. Send commands without data returned. In this example to drop a table. Some DROP or CREATE commands actually do return data. In this case use query() instead of command().
try connection.command(sql: "DROP TABLE IF EXISTS test").wait()
  1. Create a table
let sql = """
CREATE TABLE test
(
    id String,
    string FixedString(4)
)
ENGINE = MergeTree() PRIMARY KEY id ORDER BY id
"""
try connection.command(sql: sql).wait()
  1. Insert data. ClickHouseColumn represents a column with an array. String, Float, Double, UUID and Integers are supported.
let data = [
    ClickHouseColumn("id", ["1","๐ŸŽ…โ˜ƒ๐Ÿงช","234"]),
    ClickHouseColumn("string", ["๐ŸŽ…โ˜ƒ๐Ÿงช","a","awfawfawf"])
]

try! connection.insert(into: "test", data: data).wait()
  1. Query data and cast is to the exptected array
try! conn.connection.query(sql: "SELECT * FROM test").map { res in
    guard let str = res.columns.first(where: {$0.name == "string"})!.values as? [String] else {
        fatalError("Column `string`, was not a String array")
    }
    XCTAssertEqual(str, ["๐ŸŽ…โ˜ƒ", "awfawfa", "a"])

    guard let id = res.columns.first(where: {$0.name == "id"})!.values as? [String] else {
        fatalError("Column `id`, was not a String array")
    }
    XCTAssertEqual(id, ["1", "234", "๐ŸŽ…โ˜ƒ๐Ÿงช"])
}.wait()

Secure TLS connections

For TLS encrypted connections to the ClickHouse server, a tlsConfiguration attribute can be set in the configuration. Usually port 9440 is used. certificateVerification: .none disables certificate verification for self signed certificates. TLS connections use BoringSSL with SwiftNIO SSL.

let tls = TLSConfiguration.forClient(certificateVerification: .none)

let config = try ClickHouseConfiguration(
    hostname: "localhost", 
    port: 9440, 
    user: "default", 
    password: "admin", 
    database: "default",
    tlsConfiguration: tls)

TODO

  • Data message decoding is not optimal, because we grow the buffer until the whole message fits. This could result in reduced performance, the first time a very large query is executed.
  • Advanced queries, that report the current progress of the query. This could be interesting in the context of Websockets.
  • extremes feature from ClickHouse to report min/max/mean metrics along the actual data result
  • swift metrics support

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Please make sure to update tests as appropriate.

License

MIT

Github

link
Stars: 6

Used By

Total: 0

Releases

1.1.0 TLS encryption support -

TLS encrypted connections are now supported!

For TLS encrypted connections to the ClickHouse server, a tlsConfiguration attribute can be set in the configuration. Usually port 9440 is used. certificateVerification: .none disables certificate verification for self signed certificates. TLS connections use BoringSSL with SwiftNIO SSL.

let tls = TLSConfiguration.forClient(certificateVerification: .none)

let config = try ClickHouseConfiguration(
    hostname: "localhost", 
    port: 9440, 
    user: "default", 
    password: "admin", 
    database: "default",
    tlsConfiguration: tls)

1.0.1 ClickHouse server 20.x bugfix -

Fix a small compatibility issue with ClickHouse server 20.x. For DROP TABLE statements a progress message is now send which was unexpected for this library.

See https://github.com/patrick-zippenfenig/ClickHouseNIO/commit/789deba0e5e06530ec86c6d2bb7c33f201b5e950

1.0.0 First release -

First version declared stable. Before it was in use at meteoblue for a while.