6.5 KiB
Server-Side Encryption with Customer-Provided Keys (SSE-C) Implementation
This document describes the implementation of SSE-C support in SeaweedFS, addressing the feature request from GitHub Discussion #5361.
Overview
SSE-C allows clients to provide their own encryption keys for server-side encryption of objects stored in SeaweedFS. The server encrypts the data using the customer-provided AES-256 key but does not store the key itself - only an MD5 hash of the key for validation purposes.
Implementation Details
Architecture
The SSE-C implementation follows a transparent encryption/decryption pattern:
- Upload (PUT/POST): Data is encrypted with the customer key before being stored
- Download (GET/HEAD): Encrypted data is decrypted on-the-fly using the customer key
- Metadata Storage: Only the encryption algorithm and key MD5 are stored as metadata
Key Components
1. Constants and Headers (weed/s3api/s3_constants/header.go
)
- Added AWS-compatible SSE-C header constants
- Support for both regular and copy-source SSE-C headers
2. Core SSE-C Logic (weed/s3api/s3_sse_c.go
)
- SSECustomerKey: Structure to hold customer encryption key and metadata
- SSECEncryptedReader: Streaming encryption with AES-256-CTR mode
- SSECDecryptedReader: Streaming decryption with IV extraction
- validateAndParseSSECHeaders: Shared validation logic (DRY principle)
- ParseSSECHeaders: Parse regular SSE-C headers
- ParseSSECCopySourceHeaders: Parse copy-source SSE-C headers
- Header validation and parsing functions
- Metadata extraction and response handling
3. Error Handling (weed/s3api/s3err/s3api_errors.go
)
- New error codes for SSE-C validation failures
- AWS-compatible error messages and HTTP status codes
4. S3 API Integration
- PUT Object Handler: Encrypts data streams transparently
- GET Object Handler: Decrypts data streams transparently
- HEAD Object Handler: Validates keys and returns appropriate headers
- Metadata Storage: Integrates with existing
SaveAmzMetaData
function
Encryption Scheme
- Algorithm: AES-256-CTR (Counter mode)
- Key Size: 256 bits (32 bytes)
- IV Generation: Random 16-byte IV per object
- Storage Format:
[IV][EncryptedData]
where IV is prepended to encrypted content
Metadata Storage
SSE-C metadata is stored in the filer's extended attributes:
x-amz-server-side-encryption-customer-algorithm: "AES256"
x-amz-server-side-encryption-customer-key-md5: "<md5-hash-of-key>"
API Compatibility
Required Headers for Encryption (PUT/POST)
x-amz-server-side-encryption-customer-algorithm: AES256
x-amz-server-side-encryption-customer-key: <base64-encoded-256-bit-key>
x-amz-server-side-encryption-customer-key-md5: <md5-hash-of-key>
Required Headers for Decryption (GET/HEAD)
Same headers as encryption - the server validates the key MD5 matches.
Copy Operations
Support for copy-source SSE-C headers:
x-amz-copy-source-server-side-encryption-customer-algorithm
x-amz-copy-source-server-side-encryption-customer-key
x-amz-copy-source-server-side-encryption-customer-key-md5
Error Handling
The implementation provides AWS-compatible error responses:
- InvalidEncryptionAlgorithmError: Non-AES256 algorithm specified
- InvalidArgument: Invalid key format, size, or MD5 mismatch
- Missing customer key: Object encrypted but no key provided
- Unnecessary customer key: Object not encrypted but key provided
Security Considerations
- Key Management: Customer keys are never stored - only MD5 hashes for validation
- IV Randomness: Fresh random IV generated for each object
- Transparent Security: Volume servers never see unencrypted data
- Key Validation: Strict validation of key format, size, and MD5
Testing
Comprehensive test suite covers:
- Header validation and parsing (regular and copy-source)
- Encryption/decryption round-trip
- Error condition handling
- Metadata extraction
- Code reuse validation (DRY principle)
- AWS S3 compatibility
Run tests with:
go test -v ./weed/s3api
## Usage Example
### Upload with SSE-C
```bash
# Generate a 256-bit key
KEY=$(openssl rand -base64 32)
KEY_MD5=$(echo -n "$KEY" | base64 -d | openssl dgst -md5 -binary | base64)
# Upload object with SSE-C
curl -X PUT "http://localhost:8333/bucket/object" \
-H "x-amz-server-side-encryption-customer-algorithm: AES256" \
-H "x-amz-server-side-encryption-customer-key: $KEY" \
-H "x-amz-server-side-encryption-customer-key-md5: $KEY_MD5" \
--data-binary @file.txt
Download with SSE-C
# Download object with SSE-C (same key required)
curl "http://localhost:8333/bucket/object" \
-H "x-amz-server-side-encryption-customer-algorithm: AES256" \
-H "x-amz-server-side-encryption-customer-key: $KEY" \
-H "x-amz-server-side-encryption-customer-key-md5: $KEY_MD5"
Integration Points
Existing SeaweedFS Features
- Filer Metadata: Extends existing metadata storage
- Volume Servers: No changes required - store encrypted data transparently
- S3 API: Integrates seamlessly with existing handlers
- Versioning: Compatible with object versioning
- Multipart Upload: Ready for multipart upload integration
Future Enhancements
- SSE-S3: Server-managed encryption keys
- SSE-KMS: External key management service integration
- Performance Optimization: Hardware acceleration for encryption
- Compliance: Enhanced audit logging for encrypted objects
File Changes Summary
weed/s3api/s3_constants/header.go
- Added SSE-C header constantsweed/s3api/s3_sse_c.go
- Core SSE-C implementation (NEW)weed/s3api/s3_sse_c_test.go
- Comprehensive test suite (NEW)weed/s3api/s3err/s3api_errors.go
- Added SSE-C error codesweed/s3api/s3api_object_handlers.go
- GET/HEAD with SSE-C supportweed/s3api/s3api_object_handlers_put.go
- PUT with SSE-C supportweed/server/filer_server_handlers_write_autochunk.go
- Metadata storage
Compliance
This implementation follows the AWS S3 SSE-C specification for maximum compatibility with existing S3 clients and tools.
Performance Impact
- Encryption Overhead: Minimal CPU impact with efficient AES-CTR streaming
- Memory Usage: Constant memory usage via streaming encryption/decryption
- Storage Overhead: 16 bytes per object for IV storage
- Network: No additional network overhead