Getting started with S3

Asynchronous transfers with the TransferManager

The AWS Java SDK includes TransferManager, a “High level utility for managing transfers to Amazon S3.”

It provides a simple API for uploading and downloading content to and from Amazon S3. It should achieve improved throughput and resource usage, but just as important is the ease with which one can make asynchronous transfers.

To get started, create instances of the wrapped client and the transfer manager.

val client = new AmazonS3ScalaClient(myCredentials)

val transferManager = new TransferManager(myCredentials)

Alternatively, you can share the underlying S3Client.

val transferManager = new TransferManager(client.client)

These classes are thread safe, and single instances should be shared whenever possible.

There is a utility method in aws-wrap called FutureTransfer.listenFor, which is intended for working with Transfer objects. For example,

FutureTransfer.listenFor {
  transferManager.upload(bucketName, key, file)
}

The result type of this expression is Future[Upload]. (More precisely, it is the singleton type of the Transfer object that was passed as an argument to listenFor.)

It is extremely important to note that the future returned by listenFor always completes successfully, not matter whether the transfer completed successfully, failed, or was canceled. It will be completed as soon as any of these progress events occur. To determine the actual outcome of the transfer, you must interogate the transfer object itself, using the waitForCompletion or waitForException methods (or waitForUploadResult in the case of an Upload). While these methods are documented as ‘blocking’, if they are called only once the future has been completed, then they will succeed immediately, rather than block.