Scout 
This library aims to make specific formats data values reading and writing simple when the data format is not known at build time. It was inspired by SwiftyJson and all the projects that followed, while trying to cover more ground, like Xml or Plist. It unifies writing and reading for those different formats. Getting a value in a Json format would be the same as getting a value in a Xml format.
Summary
Why?
With the Foundation libraries to encode/decode Json and Plist, one could ask: why would someone need Scout? Simple answer: there are still cases where you do not know the data format. Sometimes, you will just want to read a single value from a Plist file, and you do not want to create the the struct
to decode this file. Or you simply cannot know the data format at build time.
Context
I have been working with many Mac admins recently, and many had to deal with Json, Plist and Xml data. While some were using a format-specific library like jq to parse Json, others were using awk. Each approach is valid, though it comes with some tradeoffs.
Using a format-specific library
You can use a library for each format. But I am not aware today of a library that unifies all of them. So, what you learnt with jq cannot be reused to parse Plist data. You would have to learn to use PlistBuddy or the defaults command. With Scout, you can parse the same way Json, Plist and Xml data.
Using a generic text-processing tool
Don't get me wrong, awk is a wonderful tool. It can do so many things. But it is not that easy to learn. And you have to find a way to parse each different format. Scout is really easy to use.
Features
- CRUD functions for JSON, Plist and XML data format
- Set a key name
- Force a type
- Dictionary and array count
- Dictionary keys
- Delete array or dictionary when deleting all its values
- Array slicing for read and delete commands
- Dictionary filtering for read and delete commands
- Use paths to specify the target value
- Stream or file input
- Find best match in case of a typo
- Syntax highlighting
- CSV export for arrays and dictionaries of arrays
- Folding at a depth level
- Auto-completion for commands
Insights
See the wiki (Swift package, Command-line) to see in details how to use those features.
CRUD functions for JSON, Plist and XML data format
- add a value (Create)
- read a value (Read)
- set a value (Update)
- delete a value (Delete)
Set key name
Set a key name rather than its value.
Try to force a type
Prevent the automatic inferring of a type and try to force one when setting or adding a value.
Dictionary and array count
Get a dictionary or an array count with the [#]
symbol
Dictionary keys list
Get a dictionary keys list with the {#}
symbol.
Bash/Zsh: useful when combined with csv-sep " "
to iterate over the keys:
keys=(`scout read -i People.json "people{#}" —csv-sep " "`)
for key in $keys; do
scout read -i People.json ”people.$key”;
done
Delete arrays or dictionaries when left empty
With the delete command, it is possible to specify that a dictionary or an array should be deleted when all its keys are also being deleted.
Array slicing
Specify a slice of an array to read it or to delete it with [lower:upper]
syntax. Omitting lower bound ~ 0, omitting upper bound ~ last index. Works with negative indexes like [-4:-3]
to specify a slice from the last 5th to the last 3rd element. With negative slice, omitting the upper bound ~ last index like [-3:]
to get the last 4 elements of the array.
Dictionary filtering
Specify a regular expression between sharp signs '#' to filter the keys of a dictionary, like people.#h.*#
to target all the keys starting with "h" in the dictionary 'people'. A key is a valid match when it is entirely validated by the regular expression.
Use paths to specify the value to target
A path is a sequence of keys or symbols to navigate through the data.
Stream or file input
Set the input as a file with the input option -i | --input
or as the last process/command output with a pipe:
scout "path.to.value" -i File
# is the same as
cat File | scout "path.to.value"
Find best match in case of a typo
Scout uses the Jaro-Winkler distance to indicate which key is the closest to an unresolved key.
Syntax highlighting
Scout will highlight the output when reading or outputting (with the verbose flag) a dictionary or an array value. This is done with the Lux library. You can try it with the following command.
curl --silent "https://api.github.com/repos/ABridoux/scout/releases/latest" | scout
Another example with one of the playground files and the following command:
scout -i People.plist "people.Robert.age=2" -v
When dealing with large files (although it is not recommended to ouput large files in the terminal), colorising the ouput might bring to slowdowns. You can deactivate the colorisation with the flag --no-color
or --nc
.
Customise colors
You can specify your own color set as explained here. Also, some presets for the macOS terminal default styles can be found in the Highlight presets folder
CSV export
Export data when dealing with arrays or a dictionary of arrays. Default separator ';' or customisable.
Folding
Fold arrays or dictionaries at a certain depth level to make the data more readable
Auto-completion of commands
When auto-completion is enabled on the shell, use scout install-completion-script
, then the source
command if needed to get auto-completion for scout commands.
Installation
Command Line
Homebrew
Use the following command.
brew install ABridoux/formulae/scout
It will download the notarized executable from the latest release
Download
You can download the latest version of the executable from the releases. Note that the executable is notarized. Also, a notarized scout package is provided.
After having unzipped the file, you can install it if you want to.
$ install scout /usr/local/bin/
Here is a command which downloads the latest version of the program and install it in /usr/local/bin. Run it to download and install the latest version of the program. It erases the current version you may have. The last line is optional and installs the script to auto-complete the commands.
curl -LO https://github.com/ABridoux/scout/releases/latest/download/scout.zip && \
unzip scout.zip && \
rm scout.zip && \
install scout /usr/local/bin && \
rm scout
Note
- To find all scout versions, please browse the releases page.
- When deploying a package (with a MDM for example), it might be useful to add the version to the name. To get scout latest version: simply run
scout --version
(scout version
version < 2.0.0) to get your installed scout version, orcurl --silent "https://api.github.com/repos/ABridoux/scout/releases/latest" | scout tag_name
to get the latest version available on the Github repository.
Git
Use the following lines to clone the repository and to install scout (requires Swift 5.2 toolchain to be installed). You can check the Makefile to see the commands used to build and install the executable. The last line is optional and lets you install the script to auto-complete the commands.
$ git clone https://github.com/ABridoux/scout
$ cd scout
$ make
The program should be install in /usr/local/bin. You can then remove the repository if you do not want to keep it:
$ cd ..
$ rm -r Scout
Auto-completion
You can run scout install-completion-script
to install the script to auto-complete commands depending on your shell. After this command, you might want to run the source
command for the changes to be effective.
Bash: source ~/.bashrc
Zsh: source ~/.zshrc
Swift package
Start by importing the package in your file Packages.swift.
let package = Package (
...
dependencies: [
.package(url: "https://github.com/ABridoux/scout", from: "1.0.0")
],
...
)
You can then import Scout
in a file.
Usage
Playground
You can find and try examples with one file People using the different available formats in the Playground folder. The folder contains a Commands.md file so that you can see how to use the same commands to parse the different formats.
Examples and explanations (wiki)
Special thanks
To parse and edit XML data, as the standard library does not offer a simple way to do it, Scout uses the wonderful library of Marko Tadić: AEXML. He has done an amazing work. And if several XML parsing and writing libraries exist today, I would definitely recommend his. Marko, you might never read those lines, but thank you!
Thanks also to the team at Apple behind the ArgumentParser library. They have done an incredible work to make command line tools in Swift easy to implement.
Finally, thanks to Thijs Xhaflaire and Armin Briegel for your ideas and your helpful feedback.
References
Font used for the logo: Ver Army by Damien Gosset.
Contributing
Scout is open-source and under a MIT license. If you want to make a change or to add a new feature, please open a Pull Request. You can learn more about contributing on this wiki page. Also, feel free to report a bug, an error or even a typo.
Github
link |
Stars: 64 |
Related Packages
You may find interesting
Dependencies
Releases
2.0.0 - 2020-09-05T12:13:51
Added
- Auto-completion for commands [#94]
- Array slicing [#66]
- Array and dictionaries folding at a certain level [#107]
- Delete empty dictionaries and arrays when left empty [#109]
- Dictionary filtering [#112]
- CSV export [#103]
- Commands.md new examples [#117]
- CLT scout command help license [#117]
- Get a dictionary keys [#121]
- CLT read command new
--output
option to export the read data or the CSV export into a file.
Changed
- In-line documentation updated [#117]
- Color bool flag changed for an enumerable flag [#117]
- ArgumentParser updated to 0.3.0 [#117]
- CLT Version command changed for the
--version
ParsableCommand
parameter [#117]
Removed
-v|--verbose
flag removed. Breaking change. The delete, set, and add commands will output the data by default when no--output
or--modify
options are specified.
Linux distribution
Make sure to install Swift on Linux to be able to run the program.
2.0.0 Beta 01 - 2020-08-27T16:51:37
Added
- Auto-completion for commands [#94]
- Array slicing [#66]
- Array and dictionaries folding at a certain level [#107]
- Delete empty dictionaries and arrays when left empty [#109]
- Dictionary filtering [#112]
- CSV export [#103]
- Commands.md new examples [#117]
- CLT scout command help license [#117]
- Get a dictionary keys [#121]
- CLT read command new
--output
option to export the read data or the CSV export into a file.
Changed
- In-line documentation updated [#117]
- Color bool flag changed for an enumerable flag [#117]
- ArgumentParser updated to 0.3.0 [#117]
- CLT Version command changed for the
--version
ParsableCommand
parameter [#117]
Removed
-v|--verbose
flag removed. Breaking change. The delete, set, and add commands will output the data by default when no--output
or--modify
options are specified.
Hotfix: setting a value always as a String - 2020-08-09T11:53:00
Fixed
- Setting a value was always writing a String rather than inferring the type for JSON and Plist [#96]
Documentation updated - 2020-07-27T11:23:06
Added
- File header [#89]
Changed
- Documentation updated [#89]
Array & Dictionary count - 2020-07-24T21:32:55
Added
- Get array or dictionary count with
PathElement.count
after an array or a dictionary. Thecount
is specified with[#]
for the command-line tool [#76]
Changed
PathElement
changed for an enum, with aPathElementRepresentable
to initiate aPath
[#79]Path
changed for a struct conforming toCollection
andEquatable
[#76]- Path explorer CRUD functions moved to extensions [#85]
Deprecated
PathExplorerFactory
will be removed in 2.0.0
Errors to standard error output - 2020-07-06T18:41:24
Changed
- Errors now sent to the standard error output with an error code different from 0 [#74]
- Documentation updated [#74]
Fixed
- JSON empty string colorisation [#72]
JSON escaped quote and documentation refactoring - 2020-07-04T10:30:00
Added
--no-color
flag to prevent colorisation when outputting
Changed
- Moved the documentaion in a
doc
command
Fixed
- JSON escaped quotes [#68]
Lux updated to 0.2.1 - 2020-06-19T13:45:58
Changed
- Lux updated to 0.2.1 to handle tag characters in quotes
Key propositions and custom highlight colors - 2020-05-28T13:35:24
Added
- Key proposition when subscript key error [#52]
- Custom highight colors in a plist file [#51]
Highlight output - 2020-05-20T18:33:44
Visible when outputting a dictionary or array value.
Xml dictionary and array output - 2020-03-30T16:42:11
Fixed
PathExplorerXml
string value not empty because of new line.
CLT read command crash when other type the, string - 2020-03-30T12:09:45
Fixed
- CLT read command when other type than string was not working
- 2020-03-29T14:56:40
Added
- License
- CLT: output a dictionary or an array rather than return an error
- Json: backslashes removed when outputing the string
Root nested arrays - 2020-03-29T09:30:52
Fixed
- Root element with nested arrays:
[0][2][1].firstKey
Better subscript error description. CLT type enforcing. - 2020-03-28T16:03:25
Added
- Reading path for subscript errors. A subscript error now shows precisely where the error occurs.
- CLT force type. Possibility to try to force a type when setting/adding a value.
~25~
for reals,<25>
for integers and?Yes?
for booleans. - Possibility to initialise a boolean with string values like 'y", "NO", "t", "True"...
PathExplorer
genericget
functions to try to convert to aKeyAllowedType
type.- The newly added PKG and the Zip files are notarized
Fixed
- It was not possible to initialise a
Path
starting with an array subscript like '[1].key1.key2'
Added nested array support and CLT modify option - 2020-03-23T23:56:52
Added
- Github test action
- Nested array support:
array[0][2]...
- CLT [-m | --modify] option to read and write the data from/to the same file
Fixed
- Xml value adding when key already existed was not working
Several setting and adding actions bug fixes - 2020-03-22T17:06:03
Added
- PathExplorer
format
value to indicate the data format - Playground files to try the CLT
- Get a last element in an array at the end with the negative index
- Setting a value in an array at the end with the negative index
Fixed
- Custom separator to initialise a path now working
- Initialise and convert to a
KeyAllowedTypeKey
now usesCustomStringConvertible
to try theString
option - Negative index to initialise a path now working
- Array value setting was not working if the value was not a string
- Inserting a value in an empty array was possible
- Inserting a value in a Xml only worked when the element was the root element
- 2020-03-19T23:15:38
Added
- SwiftLint file to execute SwiftLint analysis
- CLT brackets for key names containing the separator
Changed
- CLT path to read the values: the separator was changed from
->
to.
- CLT path to set the values: the separator was changed from
:
to=
- CLT array subscript. Removed the separator e.g.
array.[index]
toarray[index]
- 2020-03-19T12:19:23
Added
- Possiblity to try to force the type of a value when setting or adding
- Command-line tool options to force the string value
- Command-line tool
version
command. - More in-line documentation
- Readme instructions to use Homebrew
Changed
- Refractored the
PathExplorerSerialization
andPahExplorerxml
Fixed
- Command-line tool ""----input"" long option to specify a file input fixed to "--input"
Hotfix: updated Makefile and Package.swift - 2020-03-17T09:55:04
Instructions to download and use the executable - 2020-03-16T18:58:11
- 2020-03-16T18:23:48
Fixed oversights.
- 2020-03-16T18:12:47
CRUD operations with following formats: Json, Plist and Xml.