Neuron is a swift package I developed to help learn how to make neural networks. It is far from perfect and I am still learning. There is A LOT to learn here and I've just scratched the surface. As of right now this package provides a way to get started in machine learning and neural networks.
Feel free to send me suggestions on how to improve this. I would be delighted to learn more!! You can also feel free to assign issues here as well. Run the unit tests as well to learn how the project works!
It is fairly simple to setup the neural network Brain
. This will be the only object you interface with.
private lazy var brain: Brain = {
let bias = 0.01
let nucleus = Nucleus(learningRate: 0.001,
bias: 0.001)
let brain = Brain(nucleus: nucleus,
epochs: 10,
lossFunction: .crossEntropy,
lossThreshold: 0.001,
initializer: .xavierNormal,
gradient: .sgd)
brain.add(.init(nodes: inputs, bias: bias)) //input layer
for _ in 0..<numOfHiddenLayers {
brain.add(.init(nodes: hidden, activation: .reLu, bias: bias))
}
brain.add(.init(nodes: outputs, bias: bias)) //output layer
brain.add(modifier: .softmax)
brain.add(optimizer: .adam())
brain.logLevel = .high
return brain
}()
The Brain
class is the main interactive class for dealing with the neural network.
The brain object also supports different log levels so you can see what's going on in the console.
brain.logLevel = .low
//show no logs
case none
//show only success logs
case low
//show only success and loading logs
case medium
//show all logs
case high
Nucleus
object that defines the learning rate and bias for the network.bias
it is NOT applied to the input layer.Nucleus
object takes in 2 properties learningRate
and bias
learningRate
- how quickly the node will adjust the weights of its inputs to fit the training model
0
and 1
.bias
- the offset of adjustment to the weight adjustment calculation.
0
and 1
. case meanSquareError
case crossEntropy
///Generates weights based on a normal gaussian distribution. Mean = 0 sd = 1
case xavierNormal
///Generates weights based on a uniform distribution
case xavierUniform
brain.add(optimizer:)
public enum Optimizer {
case adam(b1: Float = 0.9,
b2: Float = 0.999,
eps: Float = 1e-8)
}
public enum GradientDescent: Equatable {
case sgd
case mbgd(size: Int)
}
mbgd
you can specify the batch size.The brain object allows for adding layers in a module way through the add
function.
public func add(_ model: LobeModel)
The LobeModel
struct can be created with a simple initializer.
public init(nodes: Int,
activation: Activation = .none,
bias: Float = 0) {
self.nodes = nodes
self.activation = activation
self.bias = bias
}
Nodes
Activation
.input
the activation function will be ignoredBias
.input
the bias will be ignoredThe network also supports adding an output activation modifier such as softmax
public func add(modifier mod: OutputModifier) {
self.outputModifier = mod
}
add(modifier)
on the brain object will add the specified output activation to the output layer. case softmax
After adding all the specified layers and modifiers do not forget to call compile()
on the brain object. This will connect all the layers together using the proper initializer and get the network ready for training.
You can also train the Brain
object by passing an expected value.
public func train(data: [TrainingData],
validation: [TrainingData] = [],
complete: ((_ complete: Bool) -> ())? = nil)
data:
An array of TrainingData
objects to be used as the training data set.validation:
An array of TrainingData
objects to be used as the validation data set.complete
A block called when the network has finished training.public struct TrainingData {
public var data: [Float]
public var correct: [Float]
public init(data dat: [Float], correct cor: [Float]) {
self.data = dat
self.correct = cor
}
}
Brain can accept a pretrained model in the form of a .smodel
file. Basically a renamed JSON format, this is the format the Brain object will export as well.
public init?(model: PretrainedModel,
epochs: Int,
lossFunction: LossFunction = .crossEntropy,
lossThreshold: Float = 0.001,
initializer: Initializers = .xavierNormal) {
.smodel
file.public struct PretrainedModel: ModelBuilder {
public var fileURL: URL
public init(url file: URL) {
self.fileURL = file
}
...
The Brain object can export its current weights and setup as a .smodel
file. This can be used to import later when creating a Brain object that you wish to be fully trained based on the data.
brain.exportModelURL()
Will export a URL that links to the .smodel
file.
brain.exportModel()
Will export a ExportModel
that describes the network
Pass in new data to feed through the network and get a result.
let out = self.brain.feed(input: data)
[Float]
using the new inputs and the current weights, aka. feed forward.Using the Brain
object you can also get the result of the loss functions of each epoch as a CSV
file using the exportLoss
function on Brain
.
exportLoss(_ filename: String? = nil) -> URL?
filename:
Name of the file to save and export. defaults to loss-{timeIntervalSince1970}
Returns the url of the exported file if successful.
Example graph:
Cross Entropy + Softmax
Videos:
Cost Function vs Loss Function:
Gradient Clipping:
Activation Functions:
Backpropagation:
Validation:
Classification:
Weight Initialization
Optimizers
link |
Stars: 5 |
Last commit: 6 days ago |
Added ability to replace weights
Swiftpack is being maintained by Petr Pavlik | @ptrpavlik | @swiftpackco