Swiftpack.co - soto-project/soto-s3-file-transfer as Swift Package

Swiftpack.co is a collection of thousands of indexed Swift packages. Search packages.
See all packages published by soto-project.
soto-project/soto-s3-file-transfer 1.2.0
Simplifying upload and download from S3 using Soto the Swift SDK for AWS.
⭐️ 28
🕓 2 weeks ago
iOS watchOS tvOS linux macOS iOS
.package(url: "https://github.com/soto-project/soto-s3-file-transfer.git", from: "1.2.0")

Soto S3 Transfer

Make uploading and downloading of files to AWS S3 easy.


Soto S3 Transfer uses the Soto Swift SDK for AWS. You need to create a Soto S3 service object before you can use the S3 transfer manager. See Soto documentation for more guidance. You also need to supply the threadPoolProvider parameter which indicates where Soto S3 Transfer will get threads from to run the file loading and saving.

let client = AWSClient(httpClientProvider: .createNew)
let s3 = S3(client: client, region: .euwest1)
let s3FileTransfer = S3FileTransferManager(s3: s3, threadPoolProvider: .createNew)


Because S3FileTransferManager can hold a thread pool that requires explicit shutdown, it also requires explicit shutdown. Before the manager is deleted you need to call S3FileTransferManager.syncShutdown to ensure the thread pool has been shutdown correctly.

Upload to S3

Uploading files to S3 is done with one call. The call returns a NIO EventLoopFuture which will be fulfilled with the response when the operation is finished.

let uploadFuture = s3FileTransfer.copy(
    from: "/Users/me/images/test.jpg", 
    to: S3File(url: "s3://my-bucket/test.jpg")!

You can also upload a folder as follows

let uploadFuture = s3FileTransfer.copy(
    from: "/Users/me/images/", 
    to: S3Folder(url: "s3://my-bucket/images/")!

If you are uploading a folder multiple files will be uploaded in parallel. The number of upload tasks running concurrently defaults to 4 but you can control this by setting maxConcurrentTasks in the Configuration you supply to the initialization of S3FileTransferManager.

let s3Transfer = S3FileTransferManager(
    s3: s3, 
    threadPoolProvider: .createNew,
    configuration: .init(maxConcurrentTasks: 8)

Download from S3

Download is as simple as upload just swap the parameters around

let downloadFuture = s3FileTransfer.copy(
    from: S3File(url: "s3://my-bucket/test.jpg")!, 
    to: "/Users/me/images/test.jpg"
let downloadFolderFuture = s3FileTransfer.copy(
    from: S3Folder(url: "s3://my-bucket/images/")!, 
    to: "/Users/me/downloads/images/"

Copy from one S3 bucket to another

You can also copy from one S3 bucket to another by supplying two S3Files or two S3Folders

let copyFuture = s3FileTransfer.copy(
    from: S3File(url: "s3://my-bucket/test2.jpg")!, 
    to: S3File(url: "s3://my-bucket/test.jpg")!
let copyFolderFuture = s3FileTransfer.copy(
    from: S3Folder(url: "s3://my-bucket/images/")!, 
    to: S3Folder(url: "s3://my-other-bucket/images/")!)

Sync operations

There are sync versions of these operations as well. This will only copy files across if they are newer than the existing files. You can also have it delete files in the target folder if they don't exist in the source folder.

let uploadSyncFuture = s3FileTransfer.sync(
    from: "/Users/me/images/", 
    to: S3Folder(url: "s3://my-bucket/images")!,
    delete: true
let downloadFuture = s3FileTransfer.copy(
    from: S3Folder(url: "s3://my-bucket/images")!, 
    to: "/Users/me/downloads/images/",
    delete: false

Multipart upload

If uploads are above a certain size then the transfer manager will use multipart upload to upload the file to S3. You can control what this threshold is and the multipart part sizes by supplying a configuration at initialization of the manager. If you don't supply a configuration both of these values are set to 8MB.

let s3Transfer = S3FileTransferManager(
    s3: s3, 
    threadPoolProvider: .createNew,
    configuration: .init(multipartThreshold: 16*1024*1024, multipartPartSize: 16*1024*1024)


Stars: 28
Last commit: 2 weeks ago
jonrohan Something's broken? Yell at me @ptrpavlik. Praise and feedback (and money) is also welcome.

Release Notes

2 weeks ago

Minor Release Changes

  • Add resume function to resume a failed download. PR #33
  • Log level for file copy logging is now debug instead of info.

Patch Release Changes

  • Optimise the transfer and deletion lists when syncing. PR #34

Other Changes

  • Add a generate-contributors-list.sh script

Swiftpack is being maintained by Petr Pavlik | @ptrpavlik | @swiftpackco | API | Analytics