Swiftpack.co - Package - vapor/leaf-kit

Github

link
Stars: 19

Dependencies

Used By

Total: 0

Releases

LeafKit 1.0.0 -

LeafKit 1.0.0

1.0.0 Zeta 1 -

This release reverses the Tau changes and reflects what (should) be the final release for LeafKit 1.0.0 barring any glaring bugs. If you want to keep using Tau, we strongly encourage you to do so! You can find it (currently) in the fork here

Parse & Serialize Bug Fixes -

This patch was authored and released by @tdotclare.
  • Properly respect invariant flag on entities during parsing to avoid pre-resolving
  • Correct behavior on nested inline variable/define resolution
  • Correct behavior of declaring variables referencing previous stack definition
  • Improved error handling on define style mismatches in resolution

LeafKit 1.0.0-tau -

tau is the final prerelease of LeafKit 1.0 and Leaf4.

This is a massive rebuild from the previous prerelease version and entirely replaces the architecture for extending the language; as such, there is no direct transition path for converting LeafTag objects to the new architecture directly.

Migration documents are coming soon to address how that functionality has improved and changed

If you currently use the Leaf4 pre-release versions, pin to the final rc branch in your Swift Package Manifest to avoid updating to this state if you are not ready to transition:

dependencies: [
  .package(url: "https://github.com/vapor/leaf-kit.git", .exact("1.0.0-rc.1.17")),
  .package(url: "https://github.com/vapor/leaf.git", .exact("4.0.0-rc.1.4"))
]

Leaf4 is a dynamic language templating engine for (and inspired by) Swift with a unique hybrid design to allow significant extensibility, customization, performance, and optimizations for a broad range of applications. Leaf templates function primarily as the View component of Model-View-Controller architectures.

As the successor to Leaf3, it greatly expands the language's capabilities and introduces significant changes that are oriented towards: simplfying integration of Leaf into applications; broader use beyond web-templating; robust handling of compiling templates; and improved, more powerful, and safer extensibility of templates at runtime.

LeafKit is the core architecture of Leaf4

Leaf is the bindings of LeafKit to Vapor and Vapor-specific configuration.

Internal Bug Fix -

This patch was authored and released by @tdotclare.

Internal Only

Fixed malformed assert checks in LeafConfiguration

LeafConfiguration Encoding and Formatting Options -

This patch was authored and released by @tdotclare.

This release adds configuration methods for setting the default presentation output of data in rendered Leaf templates.

These options are all settable before LeafKit starts running.

Property Options

Stored property options in a LeafConfiguration may be used in various ways by a LeafRenderer which the configuration object was for, and changes to them after the object was provided to a specific LeafRenderer will have no affect.

.rootDirectory: String   // The default file directory used for file-system based `LeafSource`s                  

Static Options

Static options on LeafConfiguration are effectively constant once any LeafRenderer has been instantiated and attempts to change them will assert in Debug and silently fail in Release to prevent inconsistent behavior.

 // The global tag indicator for LeafKit
.tagIndicator: Character == "#"
 // Encoding used when a template is serialized
.encoding: String.Encoding == .utf8

// Formatters for converting the base internal data types to Strings for serialization
.boolFormatter: (Bool) -> String = { $0.description }     // Bool.description
.intFormatter: (Int) -> String = { $0.description }       // Int.description
.doubleFormatter: (Double) -> String = { $0.description } // Double.description
.nilFormatter: () -> String = { "" }                      // Empty string (Optional containing .none)
.voidFormatter: () -> String = { "" }                     // Empty string (Tag with no return value)
.stringFormatter: (String) -> String = { $0 }             // Identity return
.dataFormatter: (Data) -> String? =
        { String(data: $0, encoding: Self._encoding) }    // Data using .encoding

// Note: Array & Dictionaries elements will already have been converted to Strings

.arrayFormatter: ([String]) -> String =                   // Array: [element, ..., element]
        { "[\($0.map {"\"\($0)\""}.joined(separator: ", "))]" }
.dictFormatter: ([String: String]) -> String =            // Dictionary: [key: value, ..., key: value]
        { "[\($0.map { "\($0): \"\($1)\"" }.joined(separator: ", "))]" }

API Changes

  • Character.tagIndicator can no longer be directly set - it must be configured through LeafConfiguration

  • LeafData.NaturalType represents eight concrete Swift data types Leaf handles:

    • Instantiable data types: [Bool, Int, Double, String, Array, Dictionary, Data]
    • Non-instantiable: [Void]
  • LeafData static initializers from Swift data types now take Optional values:

    • If input is .none, the returned LeafData represents a Optional.none state of the specific type
    • .nil static initializer now requires a concrete type from LeafData.NaturalType be specified
  • LeafData objects present the following informational states:

    • celf : NaturalType of the object
    • isNil: whether the object contains .none
    • isCollection: if the object's type is array or dictionary.
    • isCastable(to type: NaturalType): whether the type is implicitly castable to a second type
    • isCoercible(to type: NaturalType): whether the type has an implicit one-way conersion path to the second type
    • NOTE both above methods are inherently true when the two types are the same type, or castable when asking if coercible.
    • hasUniformType: whether the object consists of one concrete type (true for all non-containers), false if it can be determined that it does not, and nil in unusual cases where an object is storing an internal dynamic data generator that returns a container itself.
    • uniformType: If the object can be determined to have a uniformType, returns that type or nil if not determinable.
  • LeafDataRepresentable adherence now requires .leafData return LeafData rather than LeafData?

    • Updated static initalizers mentioned above will automatically handle creating LeafData holding .none where otherwise a nil return would occur
    • Generic conformances for various Swift base types are updated to reflect this:
      • let invalidValue: Float80 = Float80.max
        let leafData = invalidValue.leafData
        // leafData now represents the equivalent of Double? containing nil
        
      • Default implementations for protocols FixedWidthInteger and BinaryFloatingPoint
      • Adherance via above for Bool, String, Int/Int8/Int32/Int64/UInt/UInt8/UInt16/UInt32/UInt64, Float/Double/Float80, Data, UUID (via String), Date (via Double), Array<LeafDataRepresentable>, Dictionary<String, LeafDataRepresentable>, Set<LeafDataRepresentable> via Array

Internal Only

Substantial internal changes to LeafData, LeafDataRepresentable Protocol, LeafDataStorage :

  • LeafData conversion between types is now explicitly handled by castable and coercible rules where underlying types are different. The default behaviors allow only implicit conversion via castable rules where a clearly established bi-directional rule exists between the two types.
  • LeafDataStorage signifcantly modified to allow above behaviors
    • Internal Lazy resolvable data generators must now state their concrete return type and whether they have variant return behavior; such generators will never be tested in any kind of comparison states to prevent side-effects - they will only be called when a template serializes.
    • Various state evaluation, data conversion, and related behaviors are moved internally to the enum definition to prevent confusing situations
  • Various internal changes reflecting the above modifications

Fixes

  • Throws an error if overflowing add/subtract on integer values

Fix Performance Regression -

This patch was authored and released by @tdotclare.

This release fixes a performance regression from an internal testing function being used by public render() calls

Performance Improvements -

This patch was authored and released by @tdotclare.

This release significantly improves performance of reading raw template files

Internal Notes

Template source documents being parsed previously involved mutating an array copy of the template source; now scans a constant String copy and pre-allocates chunked reading method [Character] storage

Improved Rendering Performance -

This patch was authored and released by @tdotclare.

This release substantially improves the rendering performance of Leaf on typical calls where the template is fully resolved and cached.

Performance Comparison

  • Linear test - 10 flat templates, 1 million render calls against them
  • Random test - 10 flat templates (layer 3), 20 templates referencing one of the flat templates (layer 2), 100 templates referencing two random layer 2 templates (layer 1). 1 million render calls against a random one of the 130 total templates.
  • Each test was run 50 times on an 4GHz i7 Retina iMac

Linear

Branch Min Avg Max Avg Baseline Avg CPU Time CPU Baseline
1.0.0rc-1.13 4.27s 4.72s 5.14s 44.83% 1m 14s 40.72%
1.0.0rc-1.12 10.4s 10.52s 11.4s 100% 3m 1s 100%

Random

Branch Min Avg Max Avg Baseline Avg CPU Time CPU Baseline
1.0.0rc-1.13 4.43s 4.82s 5.17s 49.9% 1m 18s 45.3%
1.0.0rc-1.12 9s 9.66s 10.61s 100% 2m 51s 100%

NOTE this is purely a pipeline measurement - the templates used are lightweight and require near-zero time to serialize

Set Default File Extension for `NIOLeafFiles` -

This patch was authored and released by @tdotclare.

This release adds an initialization parameter to NIOLeafFiles to allow setting the default extension to be used for Leaf files.

public struct NIOLeafFiles: LeafSource {
    ...
    /// Initialize `NIOLeafFiles` with a NIO file IO object, limit options, and sandbox/view dirs
    /// - Parameters:
    ///   - fileio: `NonBlockingFileIO` file object
    ///   - limits: Options for constraining which files may be read - see `NIOLeafFiles.Limit`
    ///   - sandboxDirectory: Full path of the lowest directory which may be escaped to
    ///   - viewDirectory: Full path of the default directory templates are relative to
    ///   - defaultExtension: The default extension inferred files will have (defaults to `leaf`)
    ///
    /// `viewDirectory` must be contained within (or overlap) `sandboxDirectory`
    public init(fileio: NonBlockingFileIO,
                limits: Limit = .default,
                sandboxDirectory: String = "/",
                viewDirectory: String = "/",
                defaultExtension: String = "leaf") {...}

Usage:

    // Use "leaf4" instead of "leaf" as the implied file extension
    let source: LeafSource = NIOLeafFiles(..., defaultExtension: "leaf4")

Internal Changes

  • Various documentation of public protocols/objects
  • General rearrangement of code for clarity

LeafSources, LeafSource and File Sandboxing/Limiting in NIOLeafFiles -

This patch was authored and released by @tdotclare.

LeafSources, LeafSource and File Sandboxing/Limiting in NIOLeafFiles

This update provides several notable changes to internal behaviors to how raw source templates for Leaf are located and read prior to parsing:

  • LeafSources stores multiple LeafSource*-adhering objects by name and maintains a default search order of which objects to attempt to read from
  • LeafSource (previously LeafFiles) represents any object with a directed behavior for interpreting a template name into its own reading space (eg, a filesystem or database)
  • NIOLeafFiles gains initialization configuration for sandboxing and reading-limit behavior

LeafSources Usage Pattern

LeafSources publishes three public attributes:

  • .all: Set<String> read-only set of all registered LeafSource objects
  • .searchOrder: [String] read-only array of LeafSources which will be checked if no specific source is specified
  • register(source key: String = "default", using source: LeafSource, searchable: Bool = true) will add the source by key name, and if searchable is set, will include it in the search order.

Keys must be unique, and search order cannot be changed after a source is added to prevent collisions or changes in behavior by modifying sources once LeafRenderer has been used to read any template, so register must be called in the order objects should be used.


NOTE

Hooks are not yet available for specifying a single source to use, but are intended to be added. render by default will progressively search the sources according to searchOrder, but a new method for directively using only a single named source (EG: render(template, from: source)) would force Leaf to only attempt to use that named source (failing if source can't provide it).

This will allow applications to directively secure templates so that, EG, internally called templates can be named as being in a specific source, while potentially publically written templates can be stored in another source (and prevented from accessing the secured templates by not including that source in the default search order.)


A default definition of LeafSources.singleSource(LeafSource) provides a convenience for creating a complete LeafSources object that mimics the previous behavior of LeafRenderer.files

Example usage:

// Will access templates exactly as before
let nioLeaf = NIOLeafFiles(fileio: app.fileio,
                           limits: [.requireExtensions],
                           sandboxDirectory: "/",
                           viewDirectory: app.directory.viewsDirectory)
let singleSource = LeafSources.singleSource(nioLeaf)

// Will search sourceOne, then SourceTwo if sourceOne can't provide requested template
// Will never search hiddenSource unless it is specifically requested via explicit request
let multipleSources = LeafSources()
try! multipleSources.register(using: sourceOne)
try! multipleSources.register(source: "sourceTwo", using: sourceTwo)
try! multipleSources.register(source: "hiddenSource", using: hiddenSource, searchable: false)

This results in a LeafSource with three specific sources of templates; a request for a template will first check sourceOne, then sourceTwo. hiddenSource will not be checked unless specifically named when attempting to render a template.

LeafSource Usage Pattern

LeafSource is a renaming of the LeafFiles protocol and updates the single signature call required to reflect generic concepts of a "template name" versus a filesystem path. escape may be meaningless for a particular source object, and can be safely ignored if so.

// Required signature format
func file(template: String, escape: Bool, on eventLoop: EventLoop) throws -> EventLoopFuture<ByteBuffer>
// Deprecated format
func file(path: String, on eventLoop: EventLoop) throws -> EventLoopFuture<ByteBuffer>

For objects with a concept of "escaping", an attempt to escape should throw LeafError(.illegalAccess()) so that LeafSources can fail a search entirely. EG; if the first configured source cannot provide the requested template, LeafSources will continue to the next source, but if the first throws illegal access, the entire render attempt will fail rather than going on to a potentially less-restricted source.


NOTE

template should be interpreted in a meaningful way by an adherent to LeafSource, so calls to render should generally refer to the template name as generically as possible and allow the specific LeafSource to expand or interpret it as appropriate for its internal store map; LeafRenderer no longer makes any attempt to expand or modify the template name prior to requesting it from the LeafSource:

// Previous: LeafFiles would get a request for "/#viewDirectory#/path/to/template.leaf"
render("path/to/template")
// Now: LeafSource gets a request for exactly "path/to/template"
render("path/to/template")

However, default implementations bridging the older call signature for LeafFiles will mimic the old LeafRenderer behavior and expand the template path using the old behavior (this does NOT protect against escaping or guarantee non-relative path requests)


NIOLeafFiles Usage Pattern

NIOLeafFiles gains the following initialization configuration for sandboxing and reading-limit behaviors:

  • viewDirectory: the default directory templates will be interpreted as relative to
  • sandboxDirectory: the highest level directory the object may read from
  • limits: (NIOLeafFiles.Limit): an OptionSet of behaviors:
    • .toSandbox: if set, files outside sandboxDirectory can't be read
    • .toVisibleFiles: if set, prevents reading any file/directory starting with . (or files inside such directories)
    • .requireExtensions: only files with an extension can be read
    • .onlyLeafExtensions: only files with .leaf extensions can be read
  • Default configuration of limits is [.toSandbox, .toVisibleFiles, .requireExtensions]
  • viewDirectory must be contained inside sandboxDirectory(or coincident to limit entirely to viewDirectory)

Example:

let templateFolder = "/web/app/Resources/"
NIOLeafFiles(fileio: fileio,
             limits: .default, // .toSandbox, .toVisibleFiles, .requireExtensions
             sandboxDirectory: templateFolder,
             viewDirectory: templateFolder + "Views/")
... // setup happens for LeafRenderer

renderer.render("a")       // Tries to render "/web/app/Resources/Views/a.leaf"
renderer.render("a.leaf")  // Tries to render "/web/app/Resources/Views/a.leaf"
renderer.render("../a")    // Tries to render "/web/app/Resources/a.leaf"
renderer.render("../../a") // Throws .illegalAccess for "/web/app/a.leaf"
renderer.render(".ssh")    // Throws .illegalAccess for ""/web/app/Resources/Views/.ssh"

Improved #export/#import Behavior -

This update provides significantly improved resolution of #export/#import.

API Changes

The unbodied usage of #export("exportkey", value) now supports any acceptable single parameter as the exported value. Example uses:

// Exports context variable
#export("isAdmin", admin)
// Exports an expression
#export("specialKey", username == "tdotclare")
// Exports a custom tag call
#export("baseURL", baseURL()) 

#import can now be used inside parameters, allowing behavior like:

#if(import("showSidebar")): <div id="sidebar"><div>  #endif

Observable Behavior Changes/Fixes

  • Fixes Issue #57 - #import syntaxes are now correctly replaced when nested inside other objects like loops

Internal Changes

  • Syntax.Conditional is now a chained-block struct instead of linked classes
  • Adds BodiedSyntax protocol to applicable Syntaxes, replacing monolithic extensions
    • Publish & maintain external or import dependency sets for adherants
    • Provide inlineRefs methods to adherents to self-rebuild
  • Smarter LeafAST resolution
  • Provides applicable import capabilities to ParameterDescription and Parameter

General File Cleanup -

This patch was authored and released by @tdotclare.

General housekeeping

  • Top level files are primary public interfaces to LeafKit -
    • LeafRenderer
    • LeafAST
    • LeafError
  • Files for internal objects/methods now grouped by general pipeline stage of rendering:
    • LeafSource/* for protocol/implementation of sources of raw templates (LeafFiles, NIOLeafFiles)
    • LeafLexer/* for LeafLexer and raw template and token structs
    • LeafSyntax/* for syntax and tag objects generated during Lex->Parse
    • LeafParser/* for LeafParser & other related files
    • LeafSerialize/* for LeafSerializer, LeafContext, LeafData objects
    • LeafCache/* for protocol/implementation of LeafAST caching (LeafCache, DefaultLeafCache

This release also includes various cleanups of whitespace, code formatting for readability, and improved documentation in places.

Deep Extension Resolution -

This patch was authored and released by @tdotclare.

DEEP RESOLVE

  • Significantly improved LeafAST resolution
  • Syntax items now recursively rebuild themselves to allow deeply-nested #extend tags to properly resolve
  • Simplify LeafAST resolution steps, correct tail recursion
  • Adds appropriate test cases

This is an ugly but nonetheless functional update to the referenced template resolution; ASTs will now recursively examine ALL Syntax objects during loading and resolution to inline templates no matter how deeply buried.

I used tons of inouts on associated value Enums, don't @ me. This is not ideal in terms of code beauty but it does what it purports to.

Resolves #50

Robustify - Improved/Extended Lexing Behavior & Error Handling -

This patch was authored and released by @tdotclare.

Major Changes

  • Converts LeafError from Enum to source-reporting Struct
  • Adjusts Lexer state change table for human clarity
  • Adds Lexer robustness for parameters with no whitespace boundaries
  • Adds Lexer parameter handling for bin/oct/hex constant Ints and hex Doubles
  • Adds Lexer internal methods for additional checking peek/pop sequences
  • Makes Parameter depth a state variable in Lexer instead of an enum property
  • Adjusts Character exts for additional granularity on token types
  • Adjusts Character exts for start/body validity of token types
  • Adds Character exts for bin/oct/hex numerics
  • Adjusts Parser behavior to allow decaying tagBodyIndicators to raw when tag is known to have no body
  • Adjusts Parser to allow replacing Lexed tokens when necessary for above
  • Re-enables a number of TestCase functions from Leaf3 and adjusts for Leaf4 syntax

Problems Solved

Better/clearer error handling properties

Much easier to follow state changes during lexing In most cases

Parameter processing is still complicated but other cases are clearly handled

Better Parameter handling with whitespace

EG, the varying inputs below all properly lex to the correct interpretation now. Before, the first three would inaccurately lex, and only the fourth would correctly lex the parameters to operator(not) variable(one) operator(||) operator(not) variable(two)

"#if(!one||!two)"
"#if(!one || !two)"
"#if(! one||! two)"
"#if(! one || ! two)"

Better handling of tagBodyIndicator

Previously syntax like #(index):#(value) would error because the colon was universally assumed to indicate the start of a body - now, cases where it's impossible for a tag to take a body (eg, anonymous functions for now) will mutate the tBI back to a raw colon. Next step to improving this is to make observers on tag and built-in control structures to allow parsing to inquire as to expected state (eg, a function may take two parameters and no body or one parameter and a body and both are acceptable)

Improved syntax options for constant numerics

You can specify bin/oct/hex Int and hex Double constants now in Swift-manner literals... eg

0b1111 // Constant Int
1_000_000 // Constant Int
0x0.50 // Constant Double

Why? Why not?

LeafRenderer.resolve Recursive Resolution of LeafAST -

This patch was authored and released by @tdotclare.

In cases where extend introduces new external references (eg, via import/export as in Issue #33 ), recursively resolve the AST until no new dependencies exist.

LeafAST - Substantial Refactoring of LeafKit processing chain -

API & Performance Changes

  • Replaces Unresolved/ResolvedDocument with LeafAST, a document structure capable of representing both unknown state and resolved state AST with public observers on its state.
  • Adjusts LeafRenderer and LeafCache to use LeafAST
    • Note: LeafCache protocol retains ResolvedDocument as an alias to LeafAST to prevent breaking changes in this release
  • Adds .insert method to LeafCache protocol with a replace parameter to clarify whether inserting an AST is allowed to replace the current cached AST if such a key already exists
  • Adds .remove method to LeafCache protocol to allow removing individual ASTs from the cache
  • Improves routine case rendering speed on flat ASTs
  • Collapses sequential .raw Syntaxes during AST resolution for faster serialization
  • Initial introduction of LeafError type for enhanced error handling
    • Note: - not stable, to be changed from Enum to Struct in next PR for codifying the type and allowing source document referencing
  • Changes DefaultLeafCache behavior to return provided AST or nil if caching is off

Bug Fixes

  • Prevents crash on malformed external Leaf documents that cause cyclical loops (eg, a.leaf extends b.leaf and b.leaf extends a.leaf)
  • Re-introduces index, isFirst, isLast variables inside loop bodies from Leaf 3
    • Note: - collision is possible between variable names if a key in the LeafData provided to a template shares the same name as the variables

Testing Changes

  • Add test cases for measuring speed of linear & randomly accessed templates
  • Add tests cases for cyclical error handling and missing templates
  • Make TestFiles threadsafe
  • Deactivate testTagIndicator for causing data race when actual MultiThreaded Event Loops are running in tests

Add Underscore as Valid in Parameter -

This patch was authored by @jagreenwood and released by @tanner0101.

Adds underscore to Character extension, then ORs .underscore in isValidInParameter (#41, fixes #39).

Add requirement methods to LeafContext -

This patch was authored by @tonyarnold and released by @tanner0101.

Adds methods to LeafContext to verify the existence, or non-existence of a context body, as well as to verify the context contains a specific number of parameters.

User info, tag config, and files protocol -

This patch was authored and released by @tanner0101.
  • Adds userInfo to LeafRenderer and LeafContext for passing user-defined data.

  • Adds tags property to LeafRenderer and defaultTags global for configuring custom tags.

  • Adds LeafFiles protocol for customizing how Leaf fetches files.

  • Publicizes properties on LeafContext as well as simplifying parameter access.

Fix #if Conditional Logix -

Fixes #if to support checking a value's truthiness like Leaf 3 did (#28, fixes #31).

  • Fix null handling for conditionals
  • Fix boolean reduction for future parameter lists with > 1 boolean values
  • Restored and updated leaf3 "testStringIf" test
  • Added tests for conditional and/or

This patch was authored by @b-straub and released by @tanner0101.

Release Candidate 1 -

Update to Swift 5.2 and macOS 10.15. Add additional CI + README updates.

Release candidates represent the final shift toward focusing on bug fixes and documentation. Breaking changes will only be accepted for critical issues. We expect a final release of this package shortly after Swift 5.2's release date.

Add support for nested key paths in loops -

Added support for using nested paths when specifying the right-hand expression of a for loop in Leaf templates.

For example with the following code:

struct Show: Content {
    let title: String
    let cast: [CastMember]
    
    struct CastMember: Content {
        let name: String
        let characterName: String
    }
}

let thirthyRock = Show(title: "30 Rock", cast: [
    Show.CastMember(name: "Tina Fey", characterName: "Liz Lemon"),
    Show.CastMember(name: "Alec Baldwin", characterName: "Jack Donaghy"),
    ...
])

...
req.view.render("ShowInfo", [thirthyRock])
...

It is now possible to use a Leaf template such as:

#for(show in shows):
    <h2>#(show.title)</h2>
    <h4>Cast</h2>
    #for(castMember in show.cast):
        <p>#(castMember.name) as #(castMember.characterName)</p>
    #endfor
#endfor

Add isEnabled Flag to LeafCache -

Adds a settable isEnabled flag to LeafCache to allow for caching to be turned on and off during runtime. (#29)

LeafKit 1.0.0 Beta 2 -

  • Fixed an issue with nested tags (#21)

  • Renamed LeafConfig to LeafConfiguration (#25)

  • Publicized LeafCache protocol (#25)

  • Enabled test discovery on Linux (#26)