Swiftpack.co - Package - ABridoux/scout
Swiftpack.co is a collection of thousands of indexed Swift packages. Search packages.


Swift package
Swift Package Manager
Install
Mac + Linux

Scout

This library aims to make specific formats data values reading and writing simple when the data format is not known at build time. It was inspired by SwiftyJson and all the projects that followed, while trying to cover more ground, like Xml or Plist. It unifies writing and reading for those different formats. Getting a value in a Json format would be the same as getting a value in a Xml format.

Summary

Why?

With the Foundation libraries to encode/decode Json and Plist, one could ask: why would someone need Scout? Simple answer: there are still cases where you do not know the data format. Sometimes, you will just want to read a single value from a Plist file, and you do not want to create the the struct to decode this file. Or you simply cannot know the data format at build time.

Context

I have been working with many Mac admins recently, and many had to deal with Json, Plist and Xml data. While some were using a format-specific library like jq to parse Json, others were using awk. Each approach is valid, though it comes with some tradeoffs.

Using a format-specific library

You can use a library for each format. But I am not aware today of a library that unifies all of them. So, what you learnt with jq cannot be reused to parse Plist data. You would have to learn to use PlistBuddy or the defaults command. With Scout, you can parse the same way Json, Plist and Xml data.

Using a generic text-processing tool

Don't get me wrong, awk is a wonderful tool. It can do so many things. But it is not that easy to learn. And you have to find a way to parse each different format. Scout is really easy to use.


Features

  • CRUD functions for JSON, Plist and XML data format
    • Set a key name
    • Force a type
    • Dictionary and array count
    • Dictionary keys
    • Delete array or dictionary when deleting all its values
    • Array slicing for read and delete commands
    • Dictionary filtering for read and delete commands
  • Use paths to specify the target value
  • Stream or file input
  • Find best match in case of a typo
  • Syntax highlighting
  • CSV export for arrays and dictionaries of arrays
  • Folding at a depth level
  • Auto-completion for commands

Insights

See the wiki (Swift package, Command-line) to see in details how to use those features.

CRUD functions for JSON, Plist and XML data format

  • add a value (Create)
  • read a value (Read)
  • set a value (Update)
  • delete a value (Delete)
Set key name

Set a key name rather than its value.

Try to force a type

Prevent the automatic inferring of a type and try to force one when setting or adding a value.

Dictionary and array count

Get a dictionary or an array count with the [#] symbol

Dictionary keys list

Get a dictionary keys list with the {#} symbol. Bash/Zsh: useful when combined with csv-sep " " to iterate over the keys:

keys=(`scout read -i People.json "people{#}" —csv-sep " "`)

for key in $keys;  do
	scout read -i People.json ”people.$key”;
done
Delete arrays or dictionaries when left empty

With the delete command, it is possible to specify that a dictionary or an array should be deleted when all its keys are also being deleted.

Array slicing

Specify a slice of an array to read it or to delete it with [lower:upper] syntax. Omitting lower bound ~ 0, omitting upper bound ~ last index. Works with negative indexes like [-4:-3] to specify a slice from the last 5th to the last 3rd element. With negative slice, omitting the upper bound ~ last index like [-3:] to get the last 4 elements of the array.

Dictionary filtering

Specify a regular expression between sharp signs '#' to filter the keys of a dictionary, like people.#h.*# to target all the keys starting with "h" in the dictionary 'people'. A key is a valid match when it is entirely validated by the regular expression.

Use paths to specify the value to target

A path is a sequence of keys or symbols to navigate through the data.

Stream or file input

Set the input as a file with the input option -i | --input or as the last process/command output with a pipe:

scout "path.to.value" -i File
# is the same as
cat File | scout "path.to.value"

Find best match in case of a typo

Scout uses the Jaro-Winkler distance to indicate which key is the closest to an unresolved key.

Syntax highlighting

Scout will highlight the output when reading or outputting (with the verbose flag) a dictionary or an array value. This is done with the Lux library. You can try it with the following command.

curl --silent "https://api.github.com/repos/ABridoux/scout/releases/latest" | scout

Another example with one of the playground files and the following command:

scout -i People.plist "people.Robert.age=2" -v

When dealing with large files (although it is not recommended to ouput large files in the terminal), colorising the ouput might bring to slowdowns. You can deactivate the colorisation with the flag --no-color or --nc.

Customise colors

You can specify your own color set as explained here. Also, some presets for the macOS terminal default styles can be found in the Highlight presets folder

CSV export

Export data when dealing with arrays or a dictionary of arrays. Default separator ';' or customisable.

Folding

Fold arrays or dictionaries at a certain depth level to make the data more readable

Auto-completion of commands

When auto-completion is enabled on the shell, use scout install-completion-script, then the source command if needed to get auto-completion for scout commands.


Installation

Command Line

Homebrew

Use the following command.

brew install ABridoux/formulae/scout

It will download the notarized executable from the latest release

Download

You can download the latest version of the executable from the releases. Note that the executable is notarized. Also, a notarized scout package is provided.

After having unzipped the file, you can install it if you want to.

$ install scout /usr/local/bin/

Here is a command which downloads the latest version of the program and install it in /usr/local/bin. Run it to download and install the latest version of the program. It erases the current version you may have. The last line is optional and installs the script to auto-complete the commands.

curl -LO https://github.com/ABridoux/scout/releases/latest/download/scout.zip && \
unzip scout.zip && \
rm scout.zip && \
install scout /usr/local/bin && \
rm scout
Note
  • To find all scout versions, please browse the releases page.
  • When deploying a package (with a MDM for example), it might be useful to add the version to the name. To get scout latest version: simply run scout --version (scout version version < 2.0.0) to get your installed scout version, or curl --silent "https://api.github.com/repos/ABridoux/scout/releases/latest" | scout tag_name to get the latest version available on the Github repository.

Git

Use the following lines to clone the repository and to install scout (requires Swift 5.2 toolchain to be installed). You can check the Makefile to see the commands used to build and install the executable. The last line is optional and lets you install the script to auto-complete the commands.

$ git clone https://github.com/ABridoux/scout
$ cd scout
$ make

The program should be install in /usr/local/bin. You can then remove the repository if you do not want to keep it:

$ cd ..
$ rm -r Scout

Auto-completion

You can run scout install-completion-script to install the script to auto-complete commands depending on your shell. After this command, you might want to run the source command for the changes to be effective.

Bash: source ~/.bashrc
Zsh: source ~/.zshrc

Swift package

Start by importing the package in your file Packages.swift.

let package = Package (
    ...
    dependencies: [
        .package(url: "https://github.com/ABridoux/scout", from: "1.0.0")
    ],
    ...
)

You can then import Scout in a file.


Usage

Playground

You can find and try examples with one file People using the different available formats in the Playground folder. The folder contains a Commands.md file so that you can see how to use the same commands to parse the different formats.

Examples and explanations (wiki)

Command-line

Swift


Special thanks

To parse and edit XML data, as the standard library does not offer a simple way to do it, Scout uses the wonderful library of Marko Tadić: AEXML. He has done an amazing work. And if several XML parsing and writing libraries exist today, I would definitely recommend his. Marko, you might never read those lines, but thank you!

Thanks also to the team at Apple behind the ArgumentParser library. They have done an incredible work to make command line tools in Swift easy to implement.

Finally, thanks to Thijs Xhaflaire and Armin Briegel for your ideas and your helpful feedback.

References

Font used for the logo: Ver Army by Damien Gosset.


Contributing

Scout is open-source and under a MIT license. If you want to make a change or to add a new feature, please open a Pull Request. You can learn more about contributing on this wiki page. Also, feel free to report a bug, an error or even a typo.

Github

link
Stars: 64

Dependencies

Releases

2.0.0 - 2020-09-05T12:13:51

Added

  • Auto-completion for commands [#94]
  • Array slicing [#66]
  • Array and dictionaries folding at a certain level [#107]
  • Delete empty dictionaries and arrays when left empty [#109]
  • Dictionary filtering [#112]
  • CSV export [#103]
  • Commands.md new examples [#117]
  • CLT scout command help license [#117]
  • Get a dictionary keys [#121]
  • CLT read command new --output option to export the read data or the CSV export into a file.

Changed

  • In-line documentation updated [#117]
  • Color bool flag changed for an enumerable flag [#117]
  • ArgumentParser updated to 0.3.0 [#117]
  • CLT Version command changed for the --version ParsableCommand parameter [#117]

Removed

  • -v|--verbose flag removed. Breaking change. The delete, set, and add commands will output the data by default when no --output or --modify options are specified.

Linux distribution

Make sure to install Swift on Linux to be able to run the program.

2.0.0 Beta 01 - 2020-08-27T16:51:37

Added

  • Auto-completion for commands [#94]
  • Array slicing [#66]
  • Array and dictionaries folding at a certain level [#107]
  • Delete empty dictionaries and arrays when left empty [#109]
  • Dictionary filtering [#112]
  • CSV export [#103]
  • Commands.md new examples [#117]
  • CLT scout command help license [#117]
  • Get a dictionary keys [#121]
  • CLT read command new --output option to export the read data or the CSV export into a file.

Changed

  • In-line documentation updated [#117]
  • Color bool flag changed for an enumerable flag [#117]
  • ArgumentParser updated to 0.3.0 [#117]
  • CLT Version command changed for the --version ParsableCommand parameter [#117]

Removed

  • -v|--verbose flag removed. Breaking change. The delete, set, and add commands will output the data by default when no --output or --modify options are specified.

Hotfix: setting a value always as a String - 2020-08-09T11:53:00

Fixed

  • Setting a value was always writing a String rather than inferring the type for JSON and Plist [#96]

Documentation updated - 2020-07-27T11:23:06

Added

  • File header [#89]

Changed

  • Documentation updated [#89]

Array & Dictionary count - 2020-07-24T21:32:55

Added

  • Get array or dictionary count with PathElement.count after an array or a dictionary. The count is specified with [#] for the command-line tool [#76]

Changed

  • PathElement changed for an enum, with a PathElementRepresentable to initiate a Path [#79]
  • Path changed for a struct conforming to Collection and Equatable [#76]
  • Path explorer CRUD functions moved to extensions [#85]

Deprecated

  • PathExplorerFactory will be removed in 2.0.0

Errors to standard error output - 2020-07-06T18:41:24

Changed

  • Errors now sent to the standard error output with an error code different from 0 [#74]
  • Documentation updated [#74]

Fixed

  • JSON empty string colorisation [#72]

JSON escaped quote and documentation refactoring - 2020-07-04T10:30:00

Added

  • --no-color flag to prevent colorisation when outputting

Changed

  • Moved the documentaion in a doc command

Fixed

  • JSON escaped quotes [#68]

Lux updated to 0.2.1 - 2020-06-19T13:45:58

Changed

  • Lux updated to 0.2.1 to handle tag characters in quotes

Key propositions and custom highlight colors - 2020-05-28T13:35:24

Added

  • Key proposition when subscript key error [#52]
  • Custom highight colors in a plist file [#51]

Highlight output - 2020-05-20T18:33:44

Visible when outputting a dictionary or array value.

Xml dictionary and array output - 2020-03-30T16:42:11

Fixed

  • PathExplorerXml string value not empty because of new line.

CLT read command crash when other type the, string - 2020-03-30T12:09:45

Fixed

  • CLT read command when other type than string was not working

- 2020-03-29T14:56:40

Added

  • License
  • CLT: output a dictionary or an array rather than return an error
  • Json: backslashes removed when outputing the string

Root nested arrays - 2020-03-29T09:30:52

Fixed

  • Root element with nested arrays: [0][2][1].firstKey

Better subscript error description. CLT type enforcing. - 2020-03-28T16:03:25

Added

  • Reading path for subscript errors. A subscript error now shows precisely where the error occurs.
  • CLT force type. Possibility to try to force a type when setting/adding a value. ~25~ for reals, <25> for integers and ?Yes? for booleans.
  • Possibility to initialise a boolean with string values like 'y", "NO", "t", "True"...
  • PathExplorer generic get functions to try to convert to a KeyAllowedType type.
  • The newly added PKG and the Zip files are notarized

Fixed

  • It was not possible to initialise a Path starting with an array subscript like '[1].key1.key2'

Added nested array support and CLT modify option - 2020-03-23T23:56:52

Added

  • Github test action
  • Nested array support: array[0][2]...
  • CLT [-m | --modify] option to read and write the data from/to the same file

Fixed

  • Xml value adding when key already existed was not working

Several setting and adding actions bug fixes - 2020-03-22T17:06:03

Added

  • PathExplorer format value to indicate the data format
  • Playground files to try the CLT
  • Get a last element in an array at the end with the negative index
  • Setting a value in an array at the end with the negative index

Fixed

  • Custom separator to initialise a path now working
  • Initialise and convert to aKeyAllowedTypeKey now uses CustomStringConvertible to try the String option
  • Negative index to initialise a path now working
  • Array value setting was not working if the value was not a string
  • Inserting a value in an empty array was possible
  • Inserting a value in a Xml only worked when the element was the root element

- 2020-03-19T23:15:38

Added

  • SwiftLint file to execute SwiftLint analysis
  • CLT brackets for key names containing the separator

Changed

  • CLT path to read the values: the separator was changed from -> to .
  • CLT path to set the values: the separator was changed from : to =
  • CLT array subscript. Removed the separator e.g. array.[index] to array[index]

- 2020-03-19T12:19:23

Added

  • Possiblity to try to force the type of a value when setting or adding
  • Command-line tool options to force the string value
  • Command-line tool version command.
  • More in-line documentation
  • Readme instructions to use Homebrew

Changed

  • Refractored the PathExplorerSerialization and PahExplorerxml

Fixed

  • Command-line tool ""----input"" long option to specify a file input fixed to "--input"

Hotfix: updated Makefile and Package.swift - 2020-03-17T09:55:04

Instructions to download and use the executable - 2020-03-16T18:58:11

- 2020-03-16T18:23:48

Fixed oversights.

- 2020-03-16T18:12:47

CRUD operations with following formats: Json, Plist and Xml.