* request_id: add shared request middleware
* s3err: preserve request ids in responses and logs
* iam: reuse request ids in XML responses
* sts: reuse request ids in XML responses
* request_id: drop legacy header fallback
* request_id: use AWS-style request id format
* iam: fix AWS-compatible XML format for ErrorResponse and field ordering
- ErrorResponse uses bare <RequestId> at root level instead of
<ResponseMetadata> wrapper, matching the AWS IAM error response spec
- Move CommonResponse to last field in success response structs so
<ResponseMetadata> serializes after result elements
- Add randomness to request ID generation to avoid collisions
- Add tests for XML ordering and ErrorResponse format
* iam: remove duplicate error_response_test.go
Test is already covered by responses_test.go.
* address PR review comments
- Guard against typed nil pointers in SetResponseRequestID before
interface assertion (CodeRabbit)
- Use regexp instead of strings.Index in test helpers for extracting
request IDs (Gemini)
* request_id: prevent spoofing, fix nil-error branch, thread reqID to error writers
- Ensure() now always generates a server-side ID, ignoring client-sent
x-amz-request-id headers to prevent request ID spoofing. Uses a
private context key (contextKey{}) instead of the header string.
- writeIamErrorResponse in both iamapi and embedded IAM now accepts
reqID as a parameter instead of calling Ensure() internally, ensuring
a single request ID per request lifecycle.
- The nil-iamError branch in writeIamErrorResponse now writes a 500
Internal Server Error response instead of returning silently.
- Updated tests to set request IDs via context (not headers) and added
tests for spoofing prevention and context reuse.
* sts: add request-id consistency assertions to ActionInBody tests
* test: update admin test to expect server-generated request IDs
The test previously sent a client x-amz-request-id header and expected
it echoed back. Since Ensure() now ignores client headers to prevent
spoofing, update the test to verify the server returns a non-empty
server-generated request ID instead.
* iam: add generic WithRequestID helper alongside reflection-based fallback
Add WithRequestID[T] that uses generics to take the address of a value
type, satisfying the pointer receiver on SetRequestId without reflection.
The existing SetResponseRequestID is kept for the two call sites that
operate on interface{} (from large action switches where the concrete
type varies at runtime). Generics cannot replace reflection there since
Go cannot infer type parameters from interface{}.
* Remove reflection and generics from request ID setting
Call SetRequestId directly on concrete response types in each switch
branch before boxing into interface{}, eliminating the need for
WithRequestID (generics) and SetResponseRequestID (reflection).
* iam: return pointer responses in action dispatch
* Fix IAM error handling consistency and ensure request IDs on all responses
- UpdateUser/CreatePolicy error branches: use writeIamErrorResponse instead
of s3err.WriteErrorResponse to preserve IAM formatting and request ID
- ExecuteAction: accept reqID parameter and generate one if empty, ensuring
every response carries a RequestId regardless of caller
* Clean up inline policies on DeleteUser and UpdateUser rename
DeleteUser: remove InlinePolicies[userName] from policy storage before
removing the identity, so policies are not orphaned.
UpdateUser: move InlinePolicies[userName] to InlinePolicies[newUserName]
when renaming, so GetUserPolicy/DeleteUserPolicy work under the new name.
Both operations persist the updated policies and return an error if
the storage write fails, preventing partial state.
* Places the CommonResponse struct at the end of all IAM responses, rather than the start.
* iam: fix error response request id layout
* iam: add XML ordering regression test
* iam: share request id generation
---------
Co-authored-by: Aaron Segal <aaron.segal@rpsolutions.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
* feat: Implement IAM managed policy operations (GetPolicy, ListPolicies, DeletePolicy, AttachUserPolicy, DetachUserPolicy)
- Add response type aliases in iamapi_response.go for managed policy operations
- Implement 6 handler methods in iamapi_management_handlers.go:
- GetPolicy: Lookup managed policy by ARN
- DeletePolicy: Remove managed policy
- ListPolicies: List all managed policies
- AttachUserPolicy: Attach managed policy to user, aggregating inline + managed actions
- DetachUserPolicy: Detach managed policy from user
- ListAttachedUserPolicies: List user's attached managed policies
- Add computeAllActionsForUser() to aggregate actions from both inline and managed policies
- Wire 6 new DoActions switch cases for policy operations
- Add comprehensive tests for all new handlers
- Fixes#8506
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address PR review feedback for IAM managed policy operations
- Add parsePolicyArn() helper with proper ARN prefix validation, replacing
fragile strings.Split parsing in GetPolicy, DeletePolicy, AttachUserPolicy,
and DetachUserPolicy
- DeletePolicy now detaches the policy from all users and recomputes their
aggregated actions, preventing stale permissions after deletion
- Set changed=true for DeletePolicy DoActions case so identity updates persist
- Make PolicyId consistent: CreatePolicy now uses Hash(&policyName) matching
GetPolicy and ListPolicies
- Remove redundant nil map checks (Go handles nil map lookups safely)
- DRY up action deduplication in computeAllActionsForUser with addUniqueActions
closure
- Add tests for invalid/empty ARN rejection and DeletePolicy identity cleanup
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add integration tests for managed policy lifecycle (#8506)
Add two integration tests covering the user-reported use case where
managed policy operations returned 500 errors:
- TestS3IAMManagedPolicyLifecycle: end-to-end workflow matching the
issue report — CreatePolicy, ListPolicies, GetPolicy, AttachUserPolicy,
ListAttachedUserPolicies, idempotent re-attach, DeletePolicy while
attached (expects DeleteConflict), DetachUserPolicy, DeletePolicy,
and verification that deleted policy is gone
- TestS3IAMManagedPolicyErrorCases: covers error paths — nonexistent
policy/user for GetPolicy, DeletePolicy, AttachUserPolicy,
DetachUserPolicy, and ListAttachedUserPolicies
Also fixes DeletePolicy to reject deletion when policy is still attached
to a user (AWS-compatible DeleteConflictException), and adds the 409
status code mapping for DeleteConflictException in the error response
handler.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: nil map panic in CreatePolicy, add PolicyId test assertions
- Initialize policies.Policies map in CreatePolicy if nil (prevents panic
when no policies exist yet); also handle filer_pb.ErrNotFound like other
callers
- Add PolicyId assertions in TestGetPolicy and TestListPolicies to lock in
the consistent Hash(&policyName) behavior
- Remove redundant time.Sleep calls from new integration tests (startMiniCluster
already blocks on waitForS3Ready)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: PutUserPolicy and DeleteUserPolicy now preserve managed policy actions
PutUserPolicy and DeleteUserPolicy were calling computeAggregatedActionsForUser
(inline-only), overwriting ident.Actions and dropping managed policy actions.
Both now call computeAllActionsForUser which unions inline + managed actions.
Add TestManagedPolicyActionsPreservedAcrossInlineMutations regression test:
attaches a managed policy, adds an inline policy (verifies both actions present),
deletes the inline policy, then asserts managed policy actions still persist.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: PutUserPolicy verifies user exists before persisting inline policy
Previously the inline policy was written to storage before checking if the
target user exists in s3cfg.Identities, leaving orphaned policy data when
the user was absent. Now validates the user first, returning
NoSuchEntityException immediately if not found.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: prevent stale/lost actions on computeAllActionsForUser failure
- PutUserPolicy: on recomputation failure, preserve existing ident.Actions
instead of falling back to only the current inline policy's actions
- DeleteUserPolicy: on recomputation failure, preserve existing ident.Actions
instead of assigning nil (which wiped all permissions)
- AttachUserPolicy: roll back ident.PolicyNames and return error if
action recomputation fails, keeping identity consistent
- DetachUserPolicy: roll back ident.PolicyNames and return error if
GetPolicies or action recomputation fails
- Add doc comment on newTestIamApiServer noting it only sets s3ApiConfig
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Running mount outside of the cluster would not need to expose all the volume servers to outside of the cluster. The chunk read and write will go through the filer.
1. option for "weed s3 -filer.dir.buckets" to choose a folder for buckets
2. create a bucket
3. delete a bucket, recursively delete all metadata on filer
Chris Lu
Aaron
Tom Crasset
chrislu
Konstantin Lebedev
Rinat Shigapov
j.laycock
Chris Lu