Monthly Archives: April 2017

Adding custom metadata on AWS S3 pre-signed url generation

While uploading file to S3, we sometime need to store some metadata associated with a file, such as Content-Type, custom metadata etc. Content-Type  metadata can be easily added as header while uploading to S3, but custom metadata needs a bit more work.

Here is the file upload flow for a mobile app:

S3Upload (1)

Figure: File Upload flow from mobile app to S3

After uploading file, we have a lambda which scales the image to a specific dimension determined by a custom metadata: ‘imageType’ (business type, not content-type) associated with file.

I have not found any useful documentation on how to put the metadata programmatically while signing the url request. After trying out, I found the following solution.

While asking for pre-signed url from app, app sends the ‘imageType’ (profile/other etc) input. Backend takes that field and sends as user-metadata to S3 for signing in the following way using field: “x-amz-meta-image-type”

private URL getSignedUri(final String objectKey, final HttpMethod httpMethod, final Map<String, String> customHeaders) {
    Instant expirationTime = Instant.now().plus(LINK_EXPIRATION_TIME, ChronoUnit.MILLIS); //1 hour.

    GeneratePresignedUrlRequest generatePresignedUrlRequest =
            new GeneratePresignedUrlRequest(s3StorageConfiguration.getBucketName(), objectKey)
                .withExpiration(Date.from(expirationTime))
                .withMethod(httpMethod);

    customHeaders.forEach((key, value) -> generatePresignedUrlRequest.putCustomRequestHeader(key, value));

    return amazonS3.generatePresignedUrl(generatePresignedUrlRequest);
}

The generated url will have something like: X-Amz-SignedHeaders=host%3Bx-amz-meta-image-type

Now from app, the file can be uploaded with setting header: x-amz-meta-image-type=profile or something

If the header x-amz-meta-* was not signed before, the upload with the header will throw the following exception:


There were headers present in the request which were not signed
x-amz-meta-image-type
...

Finally one caveat, in the lambda code, we need to get the meatadata without the x-amz-meta- part,

objectMetadata.getUserMetaDataOf("image-type"); //not using x-amz-meta-image-type

Advertisements