* s3: support STREAMING-AWS4-HMAC-SHA256-PAYLOAD-TRAILER for signed chunked uploads with checksums
When AWS SDK v2 clients upload with both chunked encoding and checksum
validation enabled, they use the x-amz-content-sha256 header value of
STREAMING-AWS4-HMAC-SHA256-PAYLOAD-TRAILER instead of the simpler
STREAMING-AWS4-HMAC-SHA256-PAYLOAD.
This caused the chunked reader to not be properly activated, resulting
in chunk-signature metadata being stored as part of the file content.
Changes:
- Add streamingSignedPayloadTrailer constant for the new header value
- Update isRequestSignStreamingV4() to recognize this header
- Update newChunkedReader() to handle this streaming type
- Update calculateSeedSignature() to accept this header
- Add unit test for signed streaming upload with trailer
Fixes issue where Quarkus/AWS SDK v2 uploads with checksum validation
resulted in corrupted file content containing chunk-signature data.
* address review comments: add trailer signature to test, fix constant alignment
* test: separate canonical trailer text (\n) from on-wire format (\r\n)
* test: add negative test for invalid trailer signature
* refactor: check HTTP method first in streaming auth checks (fail-fast)
* test: handle crc32 Write error return for completeness
* refactor: extract createTrailerStreamingRequest helper to reduce test duplication
* fmt
* docs: clarify test comment about trailer signature validation status
* refactor: calculate chunk data length dynamically instead of hardcoding
* Update weed/s3api/chunked_reader_v4_test.go
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* fix: use current time for signatures instead of hardcoded past date
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* fix listing objects
* add more list testing
* address comments
* fix next marker
* fix isTruncated in listing
* fix tests
* address tests
* Update s3api_object_handlers_multipart.go
* fixes
* store json into bucket content, for tagging and cors
* switch bucket metadata from json to proto
* fix
* Update s3api_bucket_config.go
* fix test issue
* fix test_bucket_listv2_delimiter_prefix
* Update cors.go
* skip special characters
* passing listing
* fix test_bucket_list_delimiter_prefix
* ok. fix the xsd generated go code now
* fix cors tests
* fix test
* fix test_bucket_list_unordered and test_bucket_listv2_unordered
do not accept the allow-unordered and delimiter parameter combination
* fix test_bucket_list_objects_anonymous and test_bucket_listv2_objects_anonymous
The tests test_bucket_list_objects_anonymous and test_bucket_listv2_objects_anonymous were failing because they try to set bucket ACL to public-read, but SeaweedFS only supported private ACL.
Updated PutBucketAclHandler to use the existing ExtractAcl function which already supports all standard S3 canned ACLs
Replaced the hardcoded check for only private ACL with proper ACL parsing that handles public-read, public-read-write, authenticated-read, bucket-owner-read, bucket-owner-full-control, etc.
Added unit tests to verify all standard canned ACLs are accepted
* fix list unordered
The test is expecting the error code to be InvalidArgument instead of InvalidRequest
* allow anonymous listing( and head, get)
* fix test_bucket_list_maxkeys_invalid
Invalid values: max-keys=blah → Returns ErrInvalidMaxKeys (HTTP 400)
* updating IsPublicRead when parsing acl
* more logs
* CORS Test Fix
* fix test_bucket_list_return_data
* default to private
* fix test_bucket_list_delimiter_not_skip_special
* default no acl
* add debug logging
* more logs
* use basic http client
remove logs also
* fixes
* debug
* Update stats.go
* debugging
* fix anonymous test expectation
anonymous user can read, as configured in s3 json.
Chris Lu
Tom Crasset
Chris Lu