seaweedfs

Chris Lu 23fc3f2621 Fix AWS SDK Signature V4 with STS credentials (issue #7941 ) (#7944 ) * Add documentation for issue #7941 fix * ensure auth * rm FIX_ISSUE_7941.md * Integrate STS session token validation into V4 signature verification - Check for X-Amz-Security-Token header in verifyV4Signature - Call validateSTSSessionToken for STS requests - Skip regular access key lookup and expiration check for STS sessions * Fix variable scoping in verifyV4Signature for STS session token validation * Add ErrExpiredToken error for better AWS S3 compatibility with STS session tokens * Support STS session token in query parameters for presigned URLs * Fix nil pointer dereference in validateSTSSessionToken * Enhance STS token validation with detailed error diagnostics and logging * Fix missing credentials in STSSessionClaims.ToSessionInfo() * test: Add comprehensive STS session claims validation tests - TestSTSSessionClaimsToSessionInfo: Validates basic claims conversion - TestSTSSessionClaimsToSessionInfoCredentialGeneration: Verifies credential generation - TestSTSSessionClaimsToSessionInfoPreservesAllFields: Ensures all fields are preserved - TestSTSSessionClaimsToSessionInfoEmptyFields: Tests handling of empty/nil fields - TestSTSSessionClaimsToSessionInfoCredentialExpiration: Validates expiration handling All tests pass with proper timing tolerance for credential generation. * perf: Reuse CredentialGenerator instance for STS session claims Optimize ToSessionInfo() to reuse a package-level defaultCredentialGenerator instead of allocating a new CredentialGenerator on every call. This reduces allocation overhead since this method is called frequently during signature verification (potentially once per request). The CredentialGenerator is stateless and deterministic, making it safe to reuse across concurrent calls without synchronization. * refactor: Surface credential generation errors and remove sensitive logging Two improvements to error handling and security: 1. weed/iam/sts/session_claims.go: - Add logging for credential generation failures in ToSessionInfo() - Wrap errors with context (session ID) to aid debugging - Use glog.Warningf() to surface errors instead of silently swallowing them - Add fmt import for error wrapping 2. weed/s3api/auth_signature_v4.go: - Remove debug logging of actual access key IDs (glog.V(2) call) - Security improvement: avoid exposing sensitive access keys even at debug level - Keep warning-level logging that shows only count of available keys This ensures credential generation failures are observable while protecting sensitive authentication material from logs. * test: Verify deterministic credential generation in session claims tests Update TestSTSSessionClaimsToSessionInfoCredentialGeneration to properly verify deterministic credential generation: - Remove misleading comment about 'randomness' - parts of credentials ARE deterministic - Add assertions that AccessKeyId is identical for same SessionId (hash-based, deterministic) - Add assertions that SessionToken is identical for same SessionId (hash-based, deterministic) - Verify Expiration matches when SessionId is identical - Document that SecretAccessKey is NOT deterministic (uses random.Read) - Truncate expiresAt to second precision to avoid timing issues This test now properly verifies that the deterministic components of credential generation work correctly while acknowledging the cryptographic randomness of the secret access key. * test(sts): Assert credentials expiration relative to now in credential expiration tests Replace wallclock assertions comparing tc.expiresAt to time.Now() (which only verified test setup) with assertions that check sessionInfo.Credentials.Expiration relative to time.Now(), thus exercising the code under test. Include clarifying comment for intent. * feat(sts): Add IsExpired helpers and use them in expiration tests - Add Credentials.IsExpired() and SessionInfo.IsExpired() in new file session_helpers.go. - Update TestSTSSessionClaimsToSessionInfoCredentialExpiration to use helpers for clearer intent. * test: revert test-only IsExpired helpers; restore direct expiration assertions Remove session_helpers.go and update TestSTSSessionClaimsToSessionInfoCredentialExpiration to assert against sessionInfo.Credentials.Expiration directly as requested by reviewer., * fix(s3api): restore error return when access key not found Critical fix: The previous cleanup of sensitive logging inadvertently removed the error return statement when access key lookup fails. This caused the code to continue and call isCredentialExpired() on nil pointer, crashing the server. This explains EOF errors in CORS tests - server was panicking on requests with invalid keys. * fix(sts): make secret access key deterministic based on sessionId CRITICAL FIX: The secret access key was being randomly generated, causing signature verification failures when the same session token was used twice: 1. AssumeRoleWithWebIdentity generates random secret key X 2. Client signs request using secret key X 3. Server validates token, regenerates credentials via ToSessionInfo() 4. ToSessionInfo() calls generateSecretAccessKey(), which generates random key Y 5. Server tries to verify signature using key Y, but signature was made with X 6. Signature verification fails (SignatureDoesNotMatch) Solution: Make generateSecretAccessKey() deterministic by using SHA256 hash of 'secret-key:' + sessionId, just like generateAccessKeyId() already does. This ensures: - AssumeRoleWithWebIdentity generates deterministic secret key from sessionId - ToSessionInfo() regenerates the same secret key from the same sessionId - Client signature verification succeeds because keys match Fixes: AWS SDK v2 CORS tests failing with 'ExpiredToken' errors Affected files: - weed/iam/sts/token_utils.go: Updated generateSecretAccessKey() signature and implementation to be deterministic - Updated GenerateTemporaryCredentials() to pass sessionId parameter Tests: All 54 STS tests pass with this fix * test(sts): add comprehensive secret key determinism test coverage Updated tests to verify that secret access keys are now deterministic: 1. Updated TestSTSSessionClaimsToSessionInfoCredentialGeneration: - Changed comment from 'NOT deterministic' to 'NOW deterministic' - Added assertion that same sessionId produces identical secret key - Explains why this is critical for signature verification 2. Added TestSecretAccessKeyDeterminism (new dedicated test): - Verifies secret key is identical across multiple calls with same sessionId - Verifies access key ID and session token are also identical - Verifies different sessionIds produce different credentials - Includes detailed comments explaining why determinism is critical These tests ensure that the STS implementation correctly regenerates deterministic credentials during signature verification. Without determinism, signature verification would always fail because the server would use different secret keys than the client used to sign. * refactor(sts): add explicit zero-time expiration handling Improved defensive programming in IsExpired() methods: 1. Credentials.IsExpired(): - Added explicit check for zero-time expiration (time.Time{}) - Treats uninitialized credentials as expired - Prevents accidentally treating uninitialized creds as valid 2. SessionInfo.IsExpired(): - Added same explicit zero-time check - Treats uninitialized sessions as expired - Protects against bugs where sessions might not be properly initialized This is important because time.Now().After(time.Time{}) returns true, but explicitly checking for zero time makes the intent clear and helps catch initialization bugs during code review and debugging. * refactor(sts): remove unused IsExpired() helper functions The session_helpers.go file contained two unused IsExpired() methods: - Credentials.IsExpired() - SessionInfo.IsExpired() These were never called anywhere in the codebase. The actual expiration checks use: - isCredentialExpired() in weed/s3api/auth_credentials.go (S3 auth) - Direct time.Now().After() checks Removing unused code improves code clarity and reduces maintenance burden. * fix(auth): pass STS session token to IAM authorization for V4 signature auth CRITICAL FIX: Session tokens were not being passed to the authorization check when using AWS Signature V4 authentication with STS credentials. The bug: 1. AWS SDK sends request with X-Amz-Security-Token header (V4 signature) 2. validateSTSSessionToken validates the token, creates Identity with PrincipalArn 3. authorizeWithIAM only checked X-SeaweedFS-Session-Token (JWT auth header) 4. Since it was empty, fell into 'static V4' branch which set SessionToken = '' 5. AuthorizeAction returned ErrAccessDenied because SessionToken was empty The fix (in authorizeWithIAM): - Check X-SeaweedFS-Session-Token first (JWT auth) - If empty, fallback to X-Amz-Security-Token header (V4 STS auth) - If still empty, check X-Amz-Security-Token query param (presigned URLs) - When session token is found with PrincipalArn, use 'STS V4 signature' path - Only use 'static V4' path when there's no session token This ensures: - JWT Bearer auth with session tokens works (existing path) - STS V4 signature auth with session tokens works (new path) - Static V4 signature auth without session tokens works (existing path) Logging updated to distinguish: - 'JWT-based IAM authorization' - 'STS V4 signature IAM authorization' (new) - 'static V4 signature IAM authorization' (clarified) * test(s3api): add comprehensive STS session token authorization test coverage Added new test file auth_sts_v4_test.go with comprehensive tests for the STS session token authorization fix: 1. TestAuthorizeWithIAMSessionTokenExtraction: - Verifies X-SeaweedFS-Session-Token is extracted from JWT auth headers - Verifies X-Amz-Security-Token is extracted from V4 STS auth headers - Verifies X-Amz-Security-Token is extracted from query parameters (presigned URLs) - Verifies JWT tokens take precedence when both are present - Regression test for the bug where V4 STS tokens were not being passed to authorization 2. TestSTSSessionTokenIntoCredentials: - Verifies STS credentials have all required fields (AccessKeyId, SecretAccessKey, SessionToken) - Verifies deterministic generation from sessionId (same sessionId = same credentials) - Verifies different sessionIds produce different credentials - Critical for signature verification: same session must regenerate same secret key 3. TestActionConstantsForV4Auth: - Verifies S3 action constants are available for authorization checks - Ensures ACTION_READ, ACTION_WRITE, etc. are properly defined These tests ensure that: - V4 Signature auth with STS tokens properly extracts and uses session tokens - Session tokens are prioritized correctly (JWT > X-Amz-Security-Token header > query param) - STS credentials are deterministically generated for signature verification - The fix for passing STS session tokens to authorization is properly covered All 3 test functions pass (6 test cases total). * refactor(s3api): improve code quality and performance - Rename authorization path constants to avoid conflict with existing authType enum - Replace nested if/else with clean switch statement in authorizeWithIAM() - Add determineIAMAuthPath() helper for clearer intent and testability - Optimize key counting in auth_signature_v4.go: remove unnecessary slice allocation - Fix timing assertion in session_claims_test.go: use WithinDuration for symmetric tolerance These changes improve code readability, maintainability, and performance while maintaining full backward compatibility and test coverage. * refactor(s3api): use typed iamAuthPath for authorization path constants - Define iamAuthPath as a named string type (similar to existing authType enum) - Update constants to use explicit type: iamAuthPathJWT, iamAuthPathSTS_V4, etc. - Update determineIAMAuthPath() to return typed iamAuthPath - Improves type safety and prevents accidental string value misuse	1 week ago
..
main.go	Add Kafka Gateway (#7231)	3 months ago

Fix AWS SDK Signature V4 with STS credentials (issue #7941 ) (#7944 )

* Add documentation for issue #7941 fix

* ensure auth

* rm FIX_ISSUE_7941.md

* Integrate STS session token validation into V4 signature verification

- Check for X-Amz-Security-Token header in verifyV4Signature
- Call validateSTSSessionToken for STS requests
- Skip regular access key lookup and expiration check for STS sessions

* Fix variable scoping in verifyV4Signature for STS session token validation

* Add ErrExpiredToken error for better AWS S3 compatibility with STS session tokens

* Support STS session token in query parameters for presigned URLs

* Fix nil pointer dereference in validateSTSSessionToken

* Enhance STS token validation with detailed error diagnostics and logging

* Fix missing credentials in STSSessionClaims.ToSessionInfo()

* test: Add comprehensive STS session claims validation tests

- TestSTSSessionClaimsToSessionInfo: Validates basic claims conversion
- TestSTSSessionClaimsToSessionInfoCredentialGeneration: Verifies credential generation
- TestSTSSessionClaimsToSessionInfoPreservesAllFields: Ensures all fields are preserved
- TestSTSSessionClaimsToSessionInfoEmptyFields: Tests handling of empty/nil fields
- TestSTSSessionClaimsToSessionInfoCredentialExpiration: Validates expiration handling

All tests pass with proper timing tolerance for credential generation.

* perf: Reuse CredentialGenerator instance for STS session claims

Optimize ToSessionInfo() to reuse a package-level defaultCredentialGenerator
instead of allocating a new CredentialGenerator on every call. This reduces
allocation overhead since this method is called frequently during signature
verification (potentially once per request).

The CredentialGenerator is stateless and deterministic, making it safe to
reuse across concurrent calls without synchronization.

* refactor: Surface credential generation errors and remove sensitive logging

Two improvements to error handling and security:

1. weed/iam/sts/session_claims.go:
   - Add logging for credential generation failures in ToSessionInfo()
   - Wrap errors with context (session ID) to aid debugging
   - Use glog.Warningf() to surface errors instead of silently swallowing them
   - Add fmt import for error wrapping

2. weed/s3api/auth_signature_v4.go:
   - Remove debug logging of actual access key IDs (glog.V(2) call)
   - Security improvement: avoid exposing sensitive access keys even at debug level
   - Keep warning-level logging that shows only count of available keys

This ensures credential generation failures are observable while protecting
sensitive authentication material from logs.

* test: Verify deterministic credential generation in session claims tests

Update TestSTSSessionClaimsToSessionInfoCredentialGeneration to properly verify
deterministic credential generation:

- Remove misleading comment about 'randomness' - parts of credentials ARE deterministic
- Add assertions that AccessKeyId is identical for same SessionId (hash-based, deterministic)
- Add assertions that SessionToken is identical for same SessionId (hash-based, deterministic)
- Verify Expiration matches when SessionId is identical
- Document that SecretAccessKey is NOT deterministic (uses random.Read)
- Truncate expiresAt to second precision to avoid timing issues

This test now properly verifies that the deterministic components of credential
generation work correctly while acknowledging the cryptographic randomness of
the secret access key.

* test(sts): Assert credentials expiration relative to now in credential expiration tests

Replace wallclock assertions comparing tc.expiresAt to time.Now() (which only verified test setup)
with assertions that check sessionInfo.Credentials.Expiration relative to time.Now(), thus
exercising the code under test. Include clarifying comment for intent.

* feat(sts): Add IsExpired helpers and use them in expiration tests

- Add Credentials.IsExpired() and SessionInfo.IsExpired() in new file session_helpers.go.
- Update TestSTSSessionClaimsToSessionInfoCredentialExpiration to use helpers for clearer intent.

* test: revert test-only IsExpired helpers; restore direct expiration assertions

Remove session_helpers.go and update TestSTSSessionClaimsToSessionInfoCredentialExpiration to assert against sessionInfo.Credentials.Expiration directly as requested by reviewer.,

* fix(s3api): restore error return when access key not found

Critical fix: The previous cleanup of sensitive logging inadvertently removed
the error return statement when access key lookup fails. This caused the code
to continue and call isCredentialExpired() on nil pointer, crashing the server.

This explains EOF errors in CORS tests - server was panicking on requests
with invalid keys.

* fix(sts): make secret access key deterministic based on sessionId

CRITICAL FIX: The secret access key was being randomly generated, causing
signature verification failures when the same session token was used twice:

1. AssumeRoleWithWebIdentity generates random secret key X
2. Client signs request using secret key X
3. Server validates token, regenerates credentials via ToSessionInfo()
4. ToSessionInfo() calls generateSecretAccessKey(), which generates random key Y
5. Server tries to verify signature using key Y, but signature was made with X
6. Signature verification fails (SignatureDoesNotMatch)

Solution: Make generateSecretAccessKey() deterministic by using SHA256 hash
of 'secret-key:' + sessionId, just like generateAccessKeyId() already does.

This ensures:
- AssumeRoleWithWebIdentity generates deterministic secret key from sessionId
- ToSessionInfo() regenerates the same secret key from the same sessionId
- Client signature verification succeeds because keys match

Fixes: AWS SDK v2 CORS tests failing with 'ExpiredToken' errors
Affected files:
- weed/iam/sts/token_utils.go: Updated generateSecretAccessKey() signature
  and implementation to be deterministic
- Updated GenerateTemporaryCredentials() to pass sessionId parameter

Tests: All 54 STS tests pass with this fix

* test(sts): add comprehensive secret key determinism test coverage

Updated tests to verify that secret access keys are now deterministic:

1. Updated TestSTSSessionClaimsToSessionInfoCredentialGeneration:
   - Changed comment from 'NOT deterministic' to 'NOW deterministic'
   - Added assertion that same sessionId produces identical secret key
   - Explains why this is critical for signature verification

2. Added TestSecretAccessKeyDeterminism (new dedicated test):
   - Verifies secret key is identical across multiple calls with same sessionId
   - Verifies access key ID and session token are also identical
   - Verifies different sessionIds produce different credentials
   - Includes detailed comments explaining why determinism is critical

These tests ensure that the STS implementation correctly regenerates
deterministic credentials during signature verification. Without
determinism, signature verification would always fail because the
server would use different secret keys than the client used to sign.

* refactor(sts): add explicit zero-time expiration handling

Improved defensive programming in IsExpired() methods:

1. Credentials.IsExpired():
   - Added explicit check for zero-time expiration (time.Time{})
   - Treats uninitialized credentials as expired
   - Prevents accidentally treating uninitialized creds as valid

2. SessionInfo.IsExpired():
   - Added same explicit zero-time check
   - Treats uninitialized sessions as expired
   - Protects against bugs where sessions might not be properly initialized

This is important because time.Now().After(time.Time{}) returns true,
but explicitly checking for zero time makes the intent clear and helps
catch initialization bugs during code review and debugging.

* refactor(sts): remove unused IsExpired() helper functions

The session_helpers.go file contained two unused IsExpired() methods:
- Credentials.IsExpired()
- SessionInfo.IsExpired()

These were never called anywhere in the codebase. The actual expiration
checks use:
- isCredentialExpired() in weed/s3api/auth_credentials.go (S3 auth)
- Direct time.Now().After() checks

Removing unused code improves code clarity and reduces maintenance burden.

* fix(auth): pass STS session token to IAM authorization for V4 signature auth

CRITICAL FIX: Session tokens were not being passed to the authorization
check when using AWS Signature V4 authentication with STS credentials.

The bug:
1. AWS SDK sends request with X-Amz-Security-Token header (V4 signature)
2. validateSTSSessionToken validates the token, creates Identity with PrincipalArn
3. authorizeWithIAM only checked X-SeaweedFS-Session-Token (JWT auth header)
4. Since it was empty, fell into 'static V4' branch which set SessionToken = ''
5. AuthorizeAction returned ErrAccessDenied because SessionToken was empty

The fix (in authorizeWithIAM):
- Check X-SeaweedFS-Session-Token first (JWT auth)
- If empty, fallback to X-Amz-Security-Token header (V4 STS auth)
- If still empty, check X-Amz-Security-Token query param (presigned URLs)
- When session token is found with PrincipalArn, use 'STS V4 signature' path
- Only use 'static V4' path when there's no session token

This ensures:
- JWT Bearer auth with session tokens works (existing path)
- STS V4 signature auth with session tokens works (new path)
- Static V4 signature auth without session tokens works (existing path)

Logging updated to distinguish:
- 'JWT-based IAM authorization'
- 'STS V4 signature IAM authorization' (new)
- 'static V4 signature IAM authorization' (clarified)

* test(s3api): add comprehensive STS session token authorization test coverage

Added new test file auth_sts_v4_test.go with comprehensive tests for the
STS session token authorization fix:

1. TestAuthorizeWithIAMSessionTokenExtraction:
   - Verifies X-SeaweedFS-Session-Token is extracted from JWT auth headers
   - Verifies X-Amz-Security-Token is extracted from V4 STS auth headers
   - Verifies X-Amz-Security-Token is extracted from query parameters (presigned URLs)
   - Verifies JWT tokens take precedence when both are present
   - Regression test for the bug where V4 STS tokens were not being passed to authorization

2. TestSTSSessionTokenIntoCredentials:
   - Verifies STS credentials have all required fields (AccessKeyId, SecretAccessKey, SessionToken)
   - Verifies deterministic generation from sessionId (same sessionId = same credentials)
   - Verifies different sessionIds produce different credentials
   - Critical for signature verification: same session must regenerate same secret key

3. TestActionConstantsForV4Auth:
   - Verifies S3 action constants are available for authorization checks
   - Ensures ACTION_READ, ACTION_WRITE, etc. are properly defined

These tests ensure that:
- V4 Signature auth with STS tokens properly extracts and uses session tokens
- Session tokens are prioritized correctly (JWT > X-Amz-Security-Token header > query param)
- STS credentials are deterministically generated for signature verification
- The fix for passing STS session tokens to authorization is properly covered

All 3 test functions pass (6 test cases total).

* refactor(s3api): improve code quality and performance

- Rename authorization path constants to avoid conflict with existing authType enum
- Replace nested if/else with clean switch statement in authorizeWithIAM()
- Add determineIAMAuthPath() helper for clearer intent and testability
- Optimize key counting in auth_signature_v4.go: remove unnecessary slice allocation
- Fix timing assertion in session_claims_test.go: use WithinDuration for symmetric tolerance

These changes improve code readability, maintainability, and performance while
maintaining full backward compatibility and test coverage.

* refactor(s3api): use typed iamAuthPath for authorization path constants

- Define iamAuthPath as a named string type (similar to existing authType enum)
- Update constants to use explicit type: iamAuthPathJWT, iamAuthPathSTS_V4, etc.
- Update determineIAMAuthPath() to return typed iamAuthPath
- Improves type safety and prevents accidental string value misuse

1 week ago

main.go

Add Kafka Gateway (#7231)

3 months ago