Swiftpack.co - Package - Brainfinance/StackdriverLogging

StackdriverLogging

A SwiftLog LogHandler that logs GCP Stackdriver formatted JSON to a file.

For more information on Stackdriver structured logging, see: https://cloud.google.com/logging/docs/structured-logging and LogEntry

Dependencies

This Stackdriver LogHandler has a dependency on SwiftNIO which is used to create and save your new log entries in a non-blocking fashion.

How to install

Swift Package Manager

.package(url: "https://github.com/Brainfinance/StackdriverLogging.git", from:"2.0.0")

In your target's dependencies add "StackdriverLogging" e.g. like this:

.target(name: "App", dependencies: ["StackdriverLogging"]),

Bootstrapping

A factory is used to instantiate StackdriverLogHandler instances. Before bootstrapping your LoggingSystem, you must first call the StackdriverLogHandlerFactory.prepare(:) function with a StackdriverLoggingConfiguration, an NIO NonBlockingFileIO to write the logs asynchronously and an EventLoopGroup used to process new log entries in the background.

You are responsible for gracefully shutting down the NIO dependencies used to prepare the StackdriverLogHandlerFactory.

Here's an example of how this works:

let config = StackdriverLoggingConfiguration(logFilePath: "var/log/myapp.log", defaultLogLevel: "debug")

let threadPool = NIOThreadPool(numberOfThreads: NonBlockingFileIO.defaultThreadPoolSize)
threadPool.start()
let fileIO = NonBlockingFileIO(threadPool: threadPool)
let eventLoopGroup = MultiThreadedEventLoopGroup(numberOfThreads: NonBlockingFileIO.defaultThreadPoolSize)

StackdriverLogHandlerFactory.prepare(with: configuration,
                                     fileIO: fileIO,
                                     eventLoopGroup: eventLoopGroup)

LoggingSystem.bootstrap { label -> LogHandler in
    return StackdriverLogHandlerFactory.make()
}

Logging JSON values using Logger.MetadataValue

To log metadata values as JSON, simply log all JSON values other than String as a Logger.MetadataValue.stringConvertible and, instead of the usual conversion of your value to a String in the log entry, it will keep the original JSON type of your values whenever possible.

For example:

var logger = Logger(label: "Stackdriver")
logger[metadataKey: "jsonpayload-example-object"] = [
    "json-null": .stringConvertible(NSNull()),
    "json-bool": .stringConvertible(true),
    "json-integer": .stringConvertible(1),
    "json-float": .stringConvertible(1.5),
    "json-string": .string("Example"),
    "stackdriver-timestamp": .stringConvertible(Date()),
    "json-array-of-numbers": [.stringConvertible(1), .stringConvertible(5.8)],
    "json-object": [
        "key": "value"
    ]
]
logger.info("test")

Will log the non pretty-printed representation of:

{  
   "sourceLocation":{  
      "function":"boot(_:)",
      "file":"\/Sources\/App\/boot.swift",
      "line":25
   },
   "jsonpayload-example-object":{  
      "json-bool":true,
      "json-float":1.5,
      "json-string":"Example",
      "json-object":{  
         "key":"value"
      },
      "json-null":null,
      "json-integer":1,
      "json-array-of-numbers":[  
         1,
         5.8
      ],
      "stackdriver-timestamp":"2019-07-15T21:21:02.451Z"
   },
   "message":"test",
   "severity":"INFO"
}

Stackdriver logging agent + fluentd config

You must use this LogHandler in combination with the Stackdriver logging agent https://cloud.google.com/logging/docs/agent/installation and a matching json format google-fluentd config (/etc/google-fluentd/config.d/example.conf) to automatically send your JSON logs to Stackdriver.

Here's an example google-fluentd conf file that monitors a json based logfile and send new log entries to Stackdriver:

<source>
    @type tail
    # Format 'JSON' indicates the log is structured (JSON).
    format json
    # The path of the log file.
    path /var/log/example.log
    # The path of the position file that records where in the log file
    # we have processed already. This is useful when the agent
    # restarts.
    pos_file /var/lib/google-fluentd/pos/example-log.pos
    read_from_head true
    # The log tag for this log input.
    tag exampletag
</source>

Future

A Stackdriver gRPC API based implementation is being considered.

Github

link
Stars: 5

Used By

Total: 0

Releases

Manual NIO dependencies factory's preparation - 2019-11-25 15:44:56

The responsibility of creating the NonBlockingFileIO and the processing EventLoopGroup has been moved to the client so that client's can cleanly shut them down when their application is shuts down.

What this means is that the LogHandler factory must now be prepared with the NIO dependencies manually, they are no longer created automatically.

StackdriverLogHandlerFactory.prepare(with: configuration,
                                     fileIO: fileIO,
                                     eventLoopGroup: eventLoopGroup)

Logging timestamps automatically - 2019-11-19 22:08:19

Timestamps were only optionally attached to log entries, they are now always attached, otherwise, due to the async nature of the log writing, the timestamps shown by stackdriver are off.

Factory based instantiation + linux fixes - 2019-11-18 21:31:56

Changes:

  • Fix crash related to @autoreleasepool on linux.
  • Fix log entries source location filepaths being fully logged, now only the relevant filepath information is logged.
  • Add a configuration file from which you can control if timestamps are logged to your log file or not.
  • Switch to a factory based instantiation system.

The bootstrapping now uses a factory and a config file, it looks like this:

try! StackdriverLogHandlerFactory.prepare(with: .init(logFilePath: "/var/log/my-app.log", 
                                                      defaultLogLevel: .debug,
                                                      logTimestamps: true))
LoggingSystem.bootstrap { label in
    return StackdriverLogHandlerFactory.make()
}

- 2019-07-29 14:51:06

Remove the costly FileManager.fileExists verification when instantiating a StackdriverLogHandler, checking if a FileHandle file has been moved or deleted is not of the Logger's concern.

Non-blocking logging with SwiftNIO NonBlockingFileIO - 2019-07-18 19:28:47