Browse Source

fix: correct chunk size in encrypted uploads (fixes #8151) (#8154)

* fix: correct chunk size in encrypted uploads

When uploading with encryption enabled (-encryptVolumeData flag),
the chunk metadata was incorrectly storing the encrypted data size
instead of the original uncompressed size.

This caused data corruption when reading files back because:
1. Encrypted data is larger than original data
2. After decryption and decompression, the actual data is smaller
3. Size validation in readEncryptedUrl would fail
4. Files appeared truncated with null bytes

The fix ensures uploadResult.Size uses clearDataLen (the original
uncompressed size) in both cipher and non-cipher code paths,
matching the expected behavior.

Fixes: #8151

* test: add comprehensive tests for encrypted upload size fix

Add unit tests to verify that encrypted uploads correctly store the
original uncompressed data size, not the encrypted size.

Tests cover:
- Cipher key generation and encryption/decryption roundtrip
- Compression + encryption roundtrip with size preservation
- Different data sizes
- Compression detection and effectiveness
- Size behavior demonstration showing the bug and fix

All tests verify that metadata stores the original size (19 bytes in
the key example) not the encrypted size (75 bytes), which is critical
for preventing data corruption on read operations.

Issue: #8151

* remove emojis

* Update upload_content_test.go

* Delete upload_content_test.go
pull/8026/merge
Chris Lu 4 days ago
committed by GitHub
parent
commit
c52d3d1229
No known key found for this signature in database GPG Key ID: B5690EEEBB952194
  1. 2
      weed/operation/upload_content.go

2
weed/operation/upload_content.go

@ -292,7 +292,7 @@ func (uploader *Uploader) doUploadData(ctx context.Context, data []byte, option
uploadResult.Name = option.Filename uploadResult.Name = option.Filename
uploadResult.Mime = option.MimeType uploadResult.Mime = option.MimeType
uploadResult.CipherKey = cipherKey uploadResult.CipherKey = cipherKey
uploadResult.Size = uint32(len(data))
uploadResult.Size = uint32(clearDataLen)
if contentIsGzipped { if contentIsGzipped {
uploadResult.Gzip = 1 uploadResult.Gzip = 1
} }

Loading…
Cancel
Save