AWS S3 Client-side Encryption

How it works (KMS Managed Keys)

Overall the entire procedure isn’t incredibly complex, just not very well documented (unless my google skills are failing me). And I may be wrong, but the Java SDK decrypts files made with this and the library can decrypt the Java made files.

Decryption

Firstly get the object from S3, it’ll have various crypto goodies in the object’s metadata.

• metadata: x-amz-unencrypted-content-length - Resultant length of the plaintext

• metadata: x-amz-key-v2 - this is the base64’d kms encrypted aes key.

• metadata: x-amz-matdesc - JSON KMS encryption context, has which KMS key encrypted the aes key

• metadata: x-amz-iv - AES IVs

• metadata: x-amz-cek-alg - Which AES aglorithm was used, AES/CBC/PKCS5Padding or AES/GCM/NoPadding

• metadata: x-amz-tag-len - If using AES-GCM then this is a fixed value of 128 otherwise it is not present

• metadata: x-amz-wrap-alg - Always KMS when using KMS managed master keys

Send x-amz-key-v2 and x-amz-matdesc to KMS, that will return the decrypted AES key

Decode the file with either CBC or GCM based on x-amz-cek-alg.

If CBC was used, you’ll also need to remove the PKCS5Padding. This snippet would do that a = a[:-a[-1]], what it does is removes N bytes off the end of the bytestring, the padding is a fixed value (less than 256) which is the same number as how many bytes to remove (lookup PKCS5Padding).

If GCM was used, during the decryption a tag appened to the plaintext is also verified for some added protection.

Encryption

Simply enough, you do the majority of the above… backwards.

Call the generate_data_key KMS API (with the encryption context) to get both an encrypted AES key and decypted AES key. Generete IV’s. Encrypt your data. Assemble all the required metadata (use the KMS provided encrypted AES key for x-amz-key-v2), then push to S3.

How it works (Symmetric Keys)

The method is pretty similar. The encryption key is stored in x-amz-key, its encrypted with AES/ECB/PKCS5Padding :/ Then the object’s data is always encrypted with AES/CBC/PKCS5Padding which means no range downloads.

How it works (Asymmetric Keys)

Once again its pretty similar, but this time the encryption key is encrypted/decrypted with RSA/ECB/PKCS1Padding

CryptoContext Class

This class performs 2 main functions. It converts the objects encrypted key metadata into a decryption key and it will generate an encryption key with corresponding encrypted encryption key that’s base64 encoded.

For example when decrypting a file using KMS managed client side encryption. It would pass the encrypted key to KMS.decrypt along with the “material description” (x-amz-matdesc metadata header) and KMS will return the original AES key.

Similar for encrypting a file, it will call KMS to generate a data key, it will then return an AES key, appropiate material description metadata and a base64’d encrypted form of the AES key.

CryptoContext Class	Description
aioboto3.s3.cse.KMSCryptoContext	Performs CSE using KMS managed keys
aioboto3.s3.cse.AsymmetricCryptoContext	Performs CSE using public / private keys
aioboto3.s3.cse.SymmetricCryptoContext	Performs CSE using a single symmetric key

Example

import asyncio
import aioboto3
from aioboto3.s3.cse import S3CSE, KMSCryptoContext

async def main():
    ctx = KMSCryptoContext(keyid='alias/someKey', kms_client_args={'region_name': 'eu-central-1'})

    some_data = b'Some sensitive data for S3'

    async with S3CSE(crypto_context=ctx, s3_client_args={'region_name': 'eu-central-1'}) as s3_cse:
        # Upload some binary data
        await s3_cse.put_object(
            Body=some_data,
            Bucket='some-bucket',
            Key='encrypted_file',
        )

        response = await s3_cse.get_object(
            Bucket='some-bucket',
            Key='encrypted_file'
        )
        data = await response['Body'].read()
        print(data)

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

# Outputs:
#  b'Some sensitive data for S3'