Soto Core
The core framework for Soto the Swift SDK for AWS. This is the underlying driver for executing requests to AWS, but you should be using one of the libraries provided by Soto instead of this!
Documentation for the core library can be found here. Documentation for Soto can be found here.
Contributing
We welcome and encourage contributions from all developers. Please read CONTRIBUTING.md for our contributing guidelines.
License
soto-core
is released under the Apache 2.0 license. See LICENSE for details.
Github
link |
Stars: 81 |
You may find interesting
Dependencies
Releases
v5.1.0 - 2021-01-04T16:05:43
Minor Changes
- Add support for
role_arn
,credential_source
andsource_profile
tags in~/.aws/credentials
and~/.aws/config
files. PR #408, #415 from @eneko - Added
AWSService.signHeaders
to provide signed headers for a request. Required when signing Elasticsearch queries. PR #414 - Added public
NullCredentialProvider.init
. - Add support for CustomCoding property wrappers of header and query values.
- Add
s3UseDualStackEndpoint
ands3UseTransferAcceleratedEndpoint
flags toAWSServiceConfig.Options
.
Patch Changes
- Retry requests on
HTTPClient.remoteConnectionClosed
error. - Percent encode non S3 URLs twice while signing. PR #414
- Percent encode "+" character in S3 URLs before signing. PR #414
- Fix sorting of query parameters in V4 Signer. Sort query params by key and if key is equal then by value. PR #414
v5.0.0 - 2020-12-07T07:23:42
Major changes
- Soto: The rename to Soto is complete. The package name is "soto-core" and core library is now "SotoCore".
AWSClient
has now been split from the service objects. So you can have oneAWSClient
that is used by many services.- HTTP Client: We are now using the swift server
async-http-client
instead of own custom http client. We also provide the ability for the user to replace this client with their own. - Credential providers: Abstracted credential acquisition to make it easier to write credential providers and control which credential providers you want to use.
- Swift server eco-system: On top of using
async-http-client
we have added support forswift-log
andswift-metrics
. - Crypto: There is no dependency on openssl libraries anymore. Instead we use
swift-crypto
on Linux andCommonCrypto
on macOS and iOS. - Request/Response encoding/decoding:
- We have added a
Codable
basedQueryEncoder
so all protocols now useCodable
for serialisation. - Added property wrappers for Collection types
Array
andDictionary
to define how they are serialised. We have also included property wrappers forDate
which has allowed us to removeTimestamp
and replace it withDate
. - Fixed many serialisation bugs.
- We have added a
- Request Signing: The V4 signer has been completely re-written and is available as a separate library. This also added support for signing streamed S3 requests.
- Streaming: We have added support for both streaming of request data and streaming of response data thus allowing the uploading and downloading of large files without requiring a large memory footprint. Added
AWSPayload
for raw payloads. Allows the user to supply a payload as aString
,Data
,ByteBuffer
or closure supplying a stream ofByteBuffers
. - Retrying Requests: AWSClient now retries requests that have failed. You can define how the client does this using a
RetryPolicy
object. - Errors: Errors thrown by SotoCore are now structs and not enums. They also hold additional data like the response code and http headers returned.
- Added
AWSService
protocol with support for signing URLs and creating edited versions of services. - Added new
Array.reduce
style paginator functions.
v5.0.0 Release Candidate 1.0 - 2020-11-23T11:39:01
Major version changes
RetryPolicy
is now provided with processed errors instead of a raw http response. PR #394Logger
andEventLoop
parameter order in public functionsAWSClient.execute
andAWSClient.paginate
has been swapped around. PR #399
Minor version changes
- Added errors
invalidSignature
,signatureDoesNotMatch
andunrecognizedClient
that are common across many service toAWSClientError
. - Added
RotatingCredential
struct so we don't need to keep re-implementing it. - Added
additionalFields
toAWSErrorContext
. Any additional fields on top of theerrorCode
andmessage
are stored here. PR #392 - Added
createNewWithEventLoopGroup
toAWSClient.HttpClientProvider
. PR #395 - Added
AWSClient.ClientError.notEnoughData
error, which is thrown when a streaming function indicates it has finished passing on data but it has not provided enough data. - Added
Region(awsRegionName:)
which verifies it has been passed an AWS region name. PR #397 by @sebsto SotoSignerV4
is now available as a library separate fromSotoCore
Patch version changes
- When calculating the delay before retrying a failed request, check the
Retry-After
header in the response. PR #394 - Retry when response is a
throttling
error. PR #394 - Pass
EventLoop
toAWSClient.invoke
. PR #398 HTTPClient
created byAWSClient
has a default connection timeout of 10 seconds- Fixed issues with S3 streaming code where data was not passed onto the HTTP streaming code and where already streamed data was not de-allocated. PR #400
- Request signer should sort header values by key not key=value. PR #401
v4.7.2 - 2020-10-28T17:32:51
- Fixed crash in XML parser when given invalid content
v5.0.0 Beta 3.0 - 2020-10-26T10:03:29
Major version changes
- Extended
AWSErrorType
protocol to include context structure which includes message, http status code and headers. PR #389
Minor version changes
- Add
Array.reduce
style paginator functions. PR #385 - Make
SotoTestUtils
a library so it is available to other Packages. PR #386
Patch version changes
- Fixed percent encoding of URL paths, which was being removed. Also fixed encoding of "+" character. PR #390
v5.0.0 Beta 2.0 - 2020-10-06T15:53:13
Major version changes
AWSService.signURL
no longer has a default for the expiration timeexpires
. It has also been changed to aNIO.TimeAmount
from a integer. PR #373- Removed Date wrapping struct
Timestamp
. Dates are now decoded based on service protocol. This can be overidden with aDateFormatCoder
property wrapper. PR #383 - Renamed
Coding
toCustomCoding
,OptionalCoding
toOptionalCustomCoding
. PR #376 - Renamed
DefaultArrayCoder
toStandardArrayCoder
,DefaultDictionaryCoder
toStandardDictionaryCoder
. PR #376
Minor version changes
- Add ability to include headers when generating a signed URL. When you use the URL you will need to supply the exact same headers. PR #367
- Added
AWSService.with
which creates a new copy of a service with edited values. Values you can edit includemiddlewares
,timeout
,byteBufferAllocator
andoptions
. PR #377 Region
now conforms toCodable
. PR #375
Patch version changes
- Allow for JSON error messages with
message
key capitalized. This fixes issue with Lambda service errors being unhandled. PR #370 - Improve performance of
QueryEncoder
. PR #380 - Run benchmarks on all PRs. PR #381
v4.7.1 - 2020-09-21T07:57:29
Patch version changes
- Removes duplicate "/" in InstanceMetaDataServiceProvider request to avoid redirect. PR #371 from @t089
v5.0.0 Beta 1.0 - 2020-09-18T07:42:42
AWS SDK Swift has been renamed to Soto
Major version changes
- AWS SDK Swift Core has changed name and is now called SotoCore. The package name is "soto-core".
Body.json
now holds aByteBuffer
instead ofData
. PR #350- Add
offset
function parameter toAWSPayload.fileHandle
. PR #352 - Add 'context' parameter to
AWSServiceMiddleware.chain()
functions AWSClient.signURL
useHTTPMethod
instead of aString
for the http method.- Renamed
TimeStampFormatterCoder
toTimeStampFormatCoder
and made it internal. PR #364
Minor version changes
AWSClient.HTTPResponseError
is now public. This eases the writing of customRetryPolicy
objects.- Added
AWSService
a protocol for AWS services which provides helper vars to access the service region, endpoint and eventLoopGroup and a function for signing a URL for use by the service. PR #347 - Added
ByteBufferAllocator
toAWSServiceConfig
to be used throughoutAWSClient
. PR #351 - Add pagination functions that include a test for a moreResults flag.
- Add support to
TimeStampFormatCoder
for multiple time formats and add both with millisecond and without millisecond formats to theISO8601TimeStampCoder
. PR #364 - Make
AWSPayload.size
public
Patch version changes
- Provide common internal
execute
function used by all the publicexecute
functions. PR #349 - Fix bug in
DeferredCredentialProvider
wherecredential
can be accessed and written at same time. PR #357 - Add Glibc import at top of Expat.swift to aid cross compiler compilation
- Tilda expansion for config file loading on macOS sandboxed apps expands to the home folder, instead of folder inside container. PR #361 from @sebsto
- Fix v4 signer issue when the URL ends in a slash. This fixes signing issue for Lambda.ListFunctions. PR #363
v5.0.0 Alpha 6.0 - 2020-08-20T17:18:21
Major version changes
- Use AsyncHTTPClient for macOS and iOS builds. This requires v1.2.0 of AsyncHTTPClient.
NIOTSHTTPClient
has been removed from the project. PR #339 - Use Expat for XML parsing. This allows us to remove our dependency on FoundationXML. PR #342
- Add
RetryPolicyFactory
setup in similar manner to theCredentialProviderFactory
. Allows you to chooseRetryPolicy
using dot notation egAWSClient(retryPolicy: .jitter(), ...)
. Also allows us to remove a number of public symbols. PR #323 - AWSClient.ClientError is now a struct. PR #338
- Add progress callback to AWSPayload.fileHandle. PR #334
- Replace HTTP method string with enum
NIOHTTP1.HTTPMethod
. PR #329 - Add logging to
AWSClient
. Pass in a swift-logLogger
to output log information. PR #330, #331, #332 Partition
renamed toAWSPartition
PayloadOptions
renamed toAWSShapePayloadOptions
. PR #322
Minor version changes
- Add
AWSClient.credentialProvider.shutdown()
to shutdown AWSClient asynchronously. PR #327, #333 - Add HTTP client timeout value to
AWSServiceConfig
. PR #328 - Added ap-northeast-3 (Osaka Local) region.
AWSClient.credentialProvider
is now public.
Internal changes
- Use SwiftFormat for code formatting.
v4.7.0 - 2020-08-13T07:56:48
Add Region ap-northeast-3 (Asia Pacific (Osaka-Local))
v5.0.0 Alpha 5.1 - 2020-07-18T06:53:14
- Use full .package() enum in Package.swift for swift-crypto package. This appears to resolve issues with latest version of swift 5.3
v5.0.0 Alpha 5.0 - 2020-07-06T07:11:46
Major version changes:
- AWS Service configuration data is now stored in a
AWSServiceConfig
class. - Renamed
AWSClient.send
methods toexecute
. PR #309 from @Ro-M AWSClient
no longer owns the service configuration data. This is passed to theAWSClient
in theexecute
methods. This allows us to separate the client from the services and have one client serving multiple services. PR #309 from @Ro-M- AWS Credentials are supplied via a
CredentialProvider
. Possible credential provider parameters arestatic
: provide credentials directly,environment
: from environment variables,ec2
: from EC2 metadata,ecs
: from ECS metadata,configFile
: from~/.aws/credentials
file,empty
: no credentials orselector
: choose from a list of credential providers. If you don't provide aCredentialProvider
the default is.selector(.environment, .ecs, .ec2, .configFile())
. PR #258, #303, #314, #318, #306 - Added support for streaming response payloads. This is only available if you are using
AsyncHTTPClient
though. PR #236 - Added
AWSClient.syncShutdown()
to shutdownHTTPClient
and any activeCredentialProviders
. It is required to call this before deleting theAWSClient
. PR #293 - Move XML code to separate target
AWSXML
. PR #271 - Removed
AWSMemberEncoding.encoding
. Raw payload flag is now OptionSet entryPayloadOptions.raw
. PR #294 AWSRequest
uses NIOHTTPHeaders
instead of a dictionary for headers. PR #301AWSPayload.stream
returns aStreamReaderResult
instead of aByteBuffer
. Instead of returning an emptyByteBuffer
to indicate the stream is done you now return.end
. PR #316
Minor version changes:
- S3 streamed requests are now signed. PR #277
- Moved
AWSClient.validate
andAWSClient.createError
toAWSResponse
. PR #295 - Moved
AWSClient.createAWSRequest
intoAWSRequest.init
. PR #296 from @Ro-M AWSTestServer
andTestUtils
have been moved to separate targetAWSTestUtils
. PR #271- Requests are always signed using Authorization header. PR #272 by @Ro-M
- Retry time calculation is more in line with
aws-sdk-go
sdk. PR #282 - Set
User-Agent
header to "AWSSDKSwift/5.0". PR #288 - Add port to
host
header. PR #291 - Run middleware on
AWSResponse
inAWSClient.createError
Patch version changes:
- The list of characters allowed in queries has been edited to be inline with AWS docs. PR #281
- Replace all usages of
String.components
withString.split
. PR #286 - Provide better error messaging when we get an unparseable error. PR #287
- Fixed crash when XML parser returned no XML.
- Fixed head uri for NIOTSHTTPClient. It is no longer the full path. PR #298
v3.5.2 - 2020-07-03T10:20:18
Encode Foundation Data
as base64 when encoding queries. This fixes an issue with SES.SendRawEmail()
v4.6.0 - 2020-06-02T13:50:46
- Upped swift-nio version package requirements to ensure we get swift 5.3 versions
- Ensure EC2 labels all have their first letter uppercased
v5.0.0 Alpha 4.0 - 2020-05-21T09:37:37
Major version changes:
AWSPayload
is used for payloads in responses. PR #243- Use property wrappers to define how collections are encoded/decoded. PR #215
- Added ability to stream request payloads. Added
AWSPayload.stream
which has a function that providesByteBuffers
to a request. AddedAWSPayload.fileHandle
which streams a file for a request. - Added support for all AWS partitions. This gives access to china and government regions. PR #242
Region
is now a struct so we can add new regions without requiring a major version change. PR #246- Change
AWSClientError
andAWSServerError
to structs so they can be extended without a major version change. Removed some service specific errors from them. PR #247 - Renamed
RetryController
toRetryPolicy
and defaulted it toJitterRetry
which retries failed requests with a random wait time taken from an exponential increasing range.
Minor version changes:
AWSClient.httpClient
is now public- Added TimeStampCoder property wrappers that enforces how a
TimeStamp
would be encoded/decoded. PR #249 - Added RED (Request, Error, Durarion) metrics using
swift-metrics
. PR #245
Patch version changes:
- Check for os Linux instead of availability of swift-crypto when chosing between which crypto to run.
- Don't set Content-Type header if it is already set
- Fixed issue with decoding empty XML arrays
- Add query encoding specialisation for EC2. EC2 requires all query variables to have their first character capitalised. PR #253
- Added iOS CI
- Replace using
ProcessInfo
to get environment variables with the C functiongetenv
. PR #257
v4.5.1 - 2020-05-10T16:21:45
- Check to see if "Content-Type" header is set before setting default content type.
v5.0.0 Alpha 3.3 - 2020-05-09T10:39:01
- Fixed compilation issue when compiling with another library that has swift-crypto as a dependency.
v4.5.0 - 2020-05-01T13:44:31
Minor version changes:
- Added regions af-south-1 and eu-south-1
Patch version changes:
- Fixed issue where "x-amz-target" header wasn't getting set
v5.0.0 Alpha 3.2 - 2020-05-01T11:10:23
- Added region eu-south-1
v5.0.0 Alpha 3.1 - 2020-05-01T11:05:05
- Added region af-south-1
v5.0.0 Alpha 3.0 - 2020-05-01T10:58:01
Major version changes:
- Allow user to set the HTTP client for AWSClient. This has replaced setting the
EventLoopGroup
. This means you provide a HTTP client with your own setup. If you are using AsyncHTTPClient this includes proxy settings, response decompression and timeouts. TheEventLoopGroup
AWSClient uses will be the same one the HTTP client uses. PR #203 - Add
on: EventLoop
parameter to all the commands to force theEventLoop
AWSClient
will use. This has one restriction in that theEventLoop
has to come from theEventLoopGroup
being used by the client. PR #240 - Simplified the service protocol, reducing it to an enum. #241
Minor version changes:
- Group all service configuration data into one
ServiceConfig
object. #239
v5.0.0 Alpha 2.0 - 2020-04-18T08:16:40
Major version changes
- Split
AWSShape
intoAWSEncodableShape
andAWSDecodableShape
. PR #214 - Added
AWSShapeWithPayload
for shapes with a payload. PR #214 - Added
AWSPayload
for raw data payload objects. Internally this is aByteBuffer
, but it can be initialised withData
,String
orByteBuffer
. PR #218 - Reduced pagination token protocols down to a single protocol
AWSPaginationToken
with associated type. PR #231 XMLEncoder
,XMLDecoder
,DictionaryDecoder
andQueryEncoder
are no longer public.- Removed
URL.hostWithPort
extension. AWSLoggingMiddleware
is now a struct
Minor version changes
- Added ability to extract the response status code and serialize into
AWSShape
. #234 - The user can provide an arbitrarily named region now using
.other(myregion)
. PR #228 - Added query serialization of Dictionaries. #235
- Added support for retrying requests if they return an error. This is currently default to off.
Patch version changes
- Fixed an issue where in a rare case the query parameters were not being sorted correctly
v4.4.0 - 2020-04-15T15:57:26
- Upped swift-nio-ssl version requirement to 2.6.0 and above
v5.0.0 Alpha 1.0 - 2020-04-04T10:31:08
This is the first alpha release of v5.0.0. There will be a number of future alpha releases each which could include more breaking changes.
Major version changes:
- On Linux we are using the AsyncHTTPClient to serve our HTTP requests. On macOS, iOS we are using a
NIOTSHTTPClient
internal to aws-sdk-swift-core. This is a reduced version of the oldHTTPClient
with only NIOTransportServices support. You can get macOS to use the AsyncHTTPClient by providing aMultithreadedEventLoopGroup
to AWSClient. PR #141 - The URL/Header signing code has been completely rewritten and can be found here in a separate module AWSSignerV4.
- Added AWSCrypto module which wraps crypto functions. The Linux build uses swift-crypto while AWSCrypto provides macOS and iOS with drop in replacements, using CommonCrypto, for the functions we use. This is to avoid forcing all our users onto macOS 10.15 and iOS 13.0. PR #197
- Removed the typealias
Future = EventLoopFuture
. PR #229 CredentialProvider
has been renamed toCredential
.- Removed dependency on HypertextApplicationLanguage package. We don't need the majority of this code. The small bit we require has been implemented in aws-sdk-swift-core. PR #189
- Remove dependency on INIParser package. Instead we vendor the code from within aws-sdk-swift-core. PR #206
- Cleaned up
AWSShapeMember
. Removed unnecessary members 'required' and 'type', renamedAWSShapeMember
toAWSMemberEncoding
, renamed_members
to_encoding
.removed allString
extensions used byAWSShapeMember
code,getEncoding(forLocation: String)
is nowgetEncoding(forKey: CodingKey)
. Don't assume everyAWSShape
member has an equivalentAWSShapeMember
. PR #205 - Body.buffer() now holds a
ByteBuffer
instead ofData
PR #200 - Package name has changed from
AWSSDKSwiftCore
toaws-sdk-swift-core
to ease adding it as a dependency in swift 5.2 package files.
Minor version changes:
- Added Support for EC2 Instance Meta v2. PR #184
- HTTP Client is created at
AWSClient
initialization, so the same http client can be used throughout. - Queries are encoded using the
Codable
system. Now all request body encoding and response body decoding usesCodable
. PR #187 - Added encoding functions to
AWSShape
. This allows us to encode Payload objects directly and removes any requirement for aDictionaryEncoder
. So this has been removed from the project. PR #212
Patch version changes:
- Fixed XML container encoding not being reset while encoding/decoding arrays of non-trivial objects.
- Fixed errors from requests expecting a raw data result not being parsed correctly.
- Fixed various error parsing issues, where errors were inside additional XML nodes.
- Return EC2 Error codes.
- Allow for query values that are arrays. Serialize them as
entry=value1&entry=value2
.
v4.3.1 - 2020-03-25T23:00:04
Fixes for swift 5.2
v4.3.0 - 2020-03-24T16:03:53
- Added
AWSClient.region
to access region client is working in. - Fixed up XML error parsing. Add tests for situation where error is enclosed in an additional XML Node.
- Removed automatically setting "Accept" header to "/" as this was breaking some services
v4.2.4 - 2020-03-22T12:43:32
- Fixed Query encoding of
Foundation.Data
(PR from @vkill) - Fixed ordering of headers in canonicalRequest while signing requests. Make sure we order them after lowercasing them, and not before. Fixes issue with signing of requests from Lambda.invoke()
- Improved error handling for restjson services where the error code is in the response body.
- Improved error handling for EC2, by returning a
AWSResponseError
with the error code in it, instead of an unrecognised error.
v4.2.3 - 2020-03-09T17:58:12
- Mojave will not use the NIOTS version of the HTTPClient anymore as it crashes.
- Fix issue with parsing errors from raw payload responses
- Include response status code in
AWSError
v4.2.2 - 2020-02-10T22:46:02
- Update SwiftNIO version to bring in security update https://github.com/apple/swift-nio/releases/tag/2.13.1
- Update SwiftNIOSSL version to bring in security update https://github.com/apple/swift-nio-ssl/security/advisories/GHSA-9556-94c5-c3j8
v3.5.1 - 2020-02-10T22:45:06
Update SwiftNIO version to pull in security update https://github.com/apple/swift-nio/releases/tag/1.14.2