Swiftpack.co - Package - Balancingrock/BRBON

BRBON

An Binary Object Notation by Balancing Rock.

BRBON is an in-memory storage manager that has been optimized for speed. The result is a fast loading and fast access binary object manager. Loading is fast because the entire BRBON structure is loaded at once, and does not need conversion before use. Access is fast because all access is vectored and optimizations have been build in to speed up name comparisons. On top of that, static hierarchies can use an access mechanism (Portals) that allow near instantenious access of the underlying data.

The name BRBON appies to both the specification and the API presented in this project.

BRBON is used by the Swiftfire webserver project.

The API Reference manual.

Status

The BRBON Item specification is considered stable, no changes are planned or expected. Extensions are planned but will be compatible to the current definitions.

The API passes all tests. But is still missing a lot of conveniance operators and even some core operations. However it is usable and future development is expected to be backwards compatible.

The BRBON Block specification is not finished (and lacking from the provided documentation). Blocks are intended for communication and very large data structures. The (finished) Item specification deals with access of objects in a single file, on a single computer.

Description

BRBON is a binary storage format specification. It started out as a binary version for JSON but the requirement for speed has rendered some JSON aspects obsolete. Still, the JSON origins can be recognised in the object oriented approach.

Talking points:

  • Vectored access: The format includes vectors that can be used to quickly traverse the entire structure (in both directions).
  • Named objects: Objects may be named for identification, a hash value is included for faster name recognition.
  • Aligned to 8 byte boundaries: Most of the elements in the structure are aligned to 8 byte boundaries. This ensures alignment for al types.
  • All standard types are supported: Bool, Int8, Int16, Int32, Int64, UInt8, UInt16, UInt32, UInt64, Float32, Float64, String, Binary.
  • Collection types: Array, Dictionary, Sequence, Table.
  • Special types: Null, Font, RGB(A), UUID, CrcString, CrcBinary
  • Array’s are packed for optimal storage density.
  • The maximum Item capacity is 2GB (Int32.max). (The unfinished Block specification will include the capability for larger structures)

Two alternatives for BRBON have been considered: BSON and BinSON. However both were found lacking with respect to the capabilities for high speed (memory mapped) access.

Usage

Overview

BRBON is best viewed as a memory area that is formatted according to the BRBON specification. The reference API can be used to read-from and write-to this area. The API keeps the area formatted conform the BRBON specification.

All objects in storage are wrapped in an Item. There is one top level item (a container type) that contains the entire hierarchy. The top level item needs a memory manager wich is called the ItemManager.

To start a BRBON hierarchy an ItemManager must be created containg a top level container item. This can be done using one of the factory methods of ItemManager. Once the item manager is created the hierarchy can be created using the root item of that manager.

BRBON can be used ad-hoc, i.e.create the data structure as needed when needed. However using it ad-hoc can cause some slowdowns when writing data to the BRBON area. This is because increasing the size of an item may cause an expansion in the data structure necessitating a renewed memory allocation and (worst case) a memory-to-memory copy of the complete BRBON area. While it is true that with today's HW performance this is usually not a big deal, it is better to specify a data structure completely before filling it. In this aspect BRBON is similar to a database.

BRBON is strongly typed. A type must be read as the type that was stored.

Typical example

First create a BRBON ItemManager with a root item of a container type (dictionary, array, sequence or table). Then start reading writing or inspecting items using the root item in the manager. An example:

let dict = ItemManager.createDictionary()
dict.root["User"] = "John"
dict.root["Cars"] = parkingSpots
let a = dict.root["Cars"].uint16 // Assume that parkingSpots was an UInt16

Portals

Any item can be looked up by the path to that item. However every lookup requires traversing the hierarchy. Even though this is very quick, this can be made faster still by using Portals.

A Portal is a pointer to an item in the hierarchy. By using portals the need for traversing the hierarchy dissappears.

let dict = ItemManager.createDictionary()
dict.root["User"] = "John"
dict.root["Cars"] = parkingSpots
let a = dict.root["Cars"].portal // lookup
...
let oldParkingSpots = a.uint16 // no lookup
a = newParkingSpots // no lookup

The API will maintain the pointer -used by a portal- to stay in synch with mutations made to the hierarchy. I.e. the portal will always point to the correct memory location even when the hierarchy is mutated. Only when an item is removed (or one of its ancestors is removed) the portal will become invalid. Invalid portals return nil for most operations.

Version history

1.3.0

  • Changed Result to ResultCode to avoid confusion with Swift.Result
  • Simplified the usage of ResultCode

1.2.4

  • Updated BRUtils version.

1.2.3

  • Wrapped all ptest var/functions in conditional compilation

1.2.2

  • Bugfix for item size updates that failed in container items in table fileds when their size was increased
  • Bugfix in an assert statement (no runtime relevance)
  • Added PTEST compiler condition to add runtime pointer tests before/after use/update/access. Aid in debugging, not recommended for release.
  • A few tests were updated

1.2.1

  • Bugfix for table output

1.2.0

  • Added CustomStringConvertible and CustomDebugStringConvertible to Portal

1.1.0

  • Bugfix: increasing the size of a table caused failures for named tables
  • Added test to invoke bug and show that it is resolved

1.0.1

  • Documentation updates

1.0.0

  • Upped to 1.0.0 for Swiftfire

Github

link
Stars: 0

Dependencies

Used By

Total: 0

Releases

Release 1.0.0 - 2019-08-18 11:49:36

Swiftfire -and all its supporting packages- have been issued in their release 1.0.0

BRBON has not been updated for this release.

Major improvement - 2018-06-06 11:33:34

The code has been reviewed and many improvements have been made. This also means that the reference API is not compatible with previous releases. Also the specification itself was updated to v0.4, adding Font and Color types.

Migrated to SPM 4 - 2018-04-09 11:05:07

Switched to Swift Package Manager 4

Migration to Swift 4 - 2018-04-08 07:27:54

Migrated to Swift 4 Fixed some bugs when using the table field assignments.

Fixed access levels, added headers. - 2018-03-31 16:00:18

Fixed some access level problems. Added file headers.

New type: UUID - 2018-03-31 15:59:34

Added support for a new type: UUID.

Stable BRBON specification, usable API - 2018-03-31 07:08:27

The BRBON specification has stabilised and no further changes are expected. The block definitions are still open though. The item level specification is however fully usable.

The API has been completely overhauled. Some operations are still missing, and further convenience operators are expected to be implemented in future versions. Despite that the current implementation is quite usable.

In between changes - 2018-03-21 11:14:19

While version 0.3.0 seems to work, it is not stable yet. Specifically the BRBON structure specification will change once more. So this release is for study purposes only.

Changes, addition, bugfix - 2017-12-24 13:28:44

Changed the init parameter fixedByteCount to the more appropriate fixedItemByteCount. Added initialiser with fixedStorageArea parameter. Fixed bug on binary item type (was not inline with documentation). Fixed test-bug for Sequence and Dictionary.

Updated dependency - 2017-12-23 12:21:17

Dependency is not to github instead of local directory

Initial release - 2017-11-25 12:19:39

Initial release.