AWS S3 Client-side Encryption¶
How it works (KMS Managed Keys)¶
Overall the entire procedure isn’t incredibly complex, just not very well documented (unless my google skills are failing me). And I may be wrong, but the Java SDK decrypts files made with this and the library can decrypt the Java made files.
Decryption¶
Firstly get the object from S3, it’ll have various crypto goodies in the object’s metadata.
metadata:
x-amz-unencrypted-content-length
- Resultant length of the plaintextmetadata:
x-amz-key-v2
- this is the base64’d kms encrypted aes key.metadata:
x-amz-matdesc
- JSON KMS encryption context, has which KMS key encrypted the aes keymetadata:
x-amz-iv
- AES IVsmetadata:
x-amz-cek-alg
- Which AES aglorithm was used, AES/CBC/PKCS5Padding or AES/GCM/NoPaddingmetadata:
x-amz-tag-len
- If using AES-GCM then this is a fixed value of 128 otherwise it is not presentmetadata:
x-amz-wrap-alg
- Always KMS when using KMS managed master keys
Send x-amz-key-v2
and x-amz-matdesc
to KMS, that will return the decrypted AES key
Decode the file with either CBC or GCM based on x-amz-cek-alg
.
If CBC was used, you’ll also need to remove the PKCS5Padding. This snippet would do that a = a[:-a[-1]]
, what it does is removes N bytes off the end of
the bytestring, the padding is a fixed value (less than 256) which is the same number as how many bytes to remove (lookup PKCS5Padding).
If GCM was used, during the decryption a tag appened to the plaintext is also verified for some added protection.
Encryption¶
Simply enough, you do the majority of the above… backwards.
Call the generate_data_key
KMS API (with the encryption context) to get both an encrypted AES key and decypted AES key.
Generete IV’s. Encrypt your data. Assemble all the required metadata (use the KMS provided encrypted AES key for x-amz-key-v2
), then push to S3.
How it works (Symmetric Keys)¶
The method is pretty similar. The encryption key is stored in x-amz-key
, its encrypted with AES/ECB/PKCS5Padding :/
Then the object’s data is always encrypted with AES/CBC/PKCS5Padding which means no range downloads.
How it works (Asymmetric Keys)¶
Once again its pretty similar, but this time the encryption key is encrypted/decrypted with RSA/ECB/PKCS1Padding
CryptoContext Class¶
This class performs 2 main functions. It converts the objects encrypted key metadata into a decryption key and it will generate an encryption key with corresponding encrypted encryption key that’s base64 encoded.
For example when decrypting a file using KMS managed client side encryption. It would pass the encrypted key to KMS.decrypt
along
with the “material description” (x-amz-matdesc
metadata header) and KMS will return the original AES key.
Similar for encrypting a file, it will call KMS to generate a data key, it will then return an AES key, appropiate material description metadata and a base64’d encrypted form of the AES key.
CryptoContext Class |
Description |
---|---|
|
Performs CSE using KMS managed keys |
|
Performs CSE using public / private keys |
|
Performs CSE using a single symmetric key |
Example¶
import asyncio
import aioboto3
from aioboto3.s3.cse import S3CSE, KMSCryptoContext
async def main():
ctx = KMSCryptoContext(keyid='alias/someKey', kms_client_args={'region_name': 'eu-central-1'})
some_data = b'Some sensitive data for S3'
async with S3CSE(crypto_context=ctx, s3_client_args={'region_name': 'eu-central-1'}) as s3_cse:
# Upload some binary data
await s3_cse.put_object(
Body=some_data,
Bucket='some-bucket',
Key='encrypted_file',
)
response = await s3_cse.get_object(
Bucket='some-bucket',
Key='encrypted_file'
)
data = await response['Body'].read()
print(data)
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
# Outputs:
# b'Some sensitive data for S3'