When renaming a user via UpdateUser, also update ParentUser references
in service accounts to prevent them from becoming orphaned after the
next configuration reload.
Group changes propagate to S3 servers via filer subscription
(watching /etc/iam/groups/) rather than gRPC RPCs, since there
are no group-specific RPCs in the S3 cache protocol.
Return errors instead of logging and continuing when group files
cannot be read or unmarshaled. This prevents silently applying a
partial IAM config with missing group memberships or policies.
Move the nil check on identity before accessing identity.Name to
prevent panic. Also refine hasAttachedPolicies to only consider groups
that are enabled and have actual policies attached, so membership in
a no-policy group doesn't incorrectly trigger IAM authorization.
Replace scattered defers with single ordered t.Cleanup in each test
to ensure resources are torn down in reverse-creation order:
remove membership, detach policies, delete access keys, delete users,
delete groups, delete policies. Move bucket cleanup to parent test
scope and delete objects before bucket.
If PutPolicies fails after moving inline policies to the new username,
restore both the identity name and the inline policies map to their
original state to avoid a partial-write window.
Check managed policies from GetPolicies() instead of s3cfg.Policies
so dynamically created policies are found. Also add duplicate name
check to UpdateGroup rename.
Return ServiceFailure for credential manager errors instead of masking
them as NoSuchEntity. Also switch ListGroupsForUser to use s3cfg.Groups
instead of in-memory reverse index to avoid stale data. Add duplicate
name check to UpdateGroup rename.
The embedded IAM endpoint rejects anonymous requests. Replace
callIAMAPI with callIAMAPIAuthenticated that uses JWT bearer token
authentication via the test framework.
Add UpdateGroup action to enable/disable groups and rename groups
via the IAM API. This is a SeaweedFS extension (not in AWS SDK) used
by tests to toggle group disabled status.
Policies created via CreatePolicy through credentialManager are stored
in the credential store, not in s3cfg.Policies (which only has static
config policies). Change AttachGroupPolicy to use credentialManager.GetPolicy()
for policy existence validation.
Reject DeleteGroup when group has attached policies, matching the
existing members check. Also fix GetGroup error handling in
DeletePolicy to only skip ErrGroupNotFound, not all errors.
Scan identities and inline policies for newUserName before mutating,
returning EntityAlreadyExists if a collision is found. Reuse the
already-loaded policies instead of loading them again inside the loop.
Merge policy names from user's enabled groups into the IAMIdentity
used for authorization, so group-attached policies are evaluated
alongside user-attached policies.
Add Marker string field to GetGroupResult, ListGroupsResult,
ListAttachedGroupPoliciesResult, and ListGroupsForUserResult to
match AWS IAM pagination response format.
Add CreateGroup, GetGroup, DeleteGroup, ListGroups, and UpdateGroup
methods instead of relying on embedded interface fallthrough. Group
changes propagate via filer subscription so no RPC propagation needed.
- Memory store: clone groups on store/retrieve to prevent mutation
- Admin dash: deep copy groups before mutation, validate user/policy exists
- HTTP handlers: translate credential errors to proper HTTP status codes,
use *bool for Enabled field to distinguish missing vs false
- Groups templ: use data attributes + event delegation instead of inline
onclick for XSS safety, prevent stale async responses
- Fix XSS vulnerability in groups.templ: replace innerHTML string
concatenation with DOM APIs (createElement/textContent) for rendering
member and policy lists
- Use userGroups reverse index in embedded IAM ListGroupsForUser for
O(1) lookup instead of iterating all groups
- Add buildUserGroupsIndex helper in standalone IAM handlers; use it
in ListGroupsForUser and removeUserFromAllGroups for efficient lookup
- Add note about gRPC store load-modify-save race condition limitation
Add comprehensive integration tests for group CRUD, membership,
policy attachment, policy enforcement, disabled group behavior,
user deletion side effects, and multi-group membership. Add
"group" test type to CI matrix in s3-iam-tests workflow.
Add groups page with CRUD operations, member management, policy
attachment, and enable/disable toggle. Register routes in admin
handlers and add Groups entry to sidebar navigation.
When a user is deleted, remove them from all groups they belong to.
When a user is renamed, update group membership references. Applied
to both embedded and standalone IAM handlers.
Add groups and userGroups reverse index to IdentityAccessManagement.
Populate both maps during ReplaceS3ApiConfiguration and
MergeS3ApiConfiguration. Modify evaluateIAMPolicies to evaluate
policies from user's enabled groups in addition to user policies.
Update VerifyActionPermission to consider group policies when
checking hasAttachedPolicies.
Add group handlers (CreateGroup, DeleteGroup, GetGroup, ListGroups,
AddUserToGroup, RemoveUserFromGroup, AttachGroupPolicy, DetachGroupPolicy,
ListAttachedGroupPolicies, ListGroupsForUser) and wire into DoActions
dispatch. Also add helper functions for user/policy side effects.
Add XML response types for group management IAM actions:
CreateGroup, DeleteGroup, GetGroup, ListGroups, AddUserToGroup,
RemoveUserFromGroup, AttachGroupPolicy, DetachGroupPolicy,
ListAttachedGroupPolicies, ListGroupsForUser.
Add group management methods (CreateGroup, GetGroup, DeleteGroup,
ListGroups, UpdateGroup) to the CredentialStore interface with
implementations for memory, filer_etc, postgres, and grpc stores.
Wire group loading/saving into filer_etc LoadConfiguration and
SaveConfiguration.
* filer.sync: add exponential backoff on unexpected EOF during replication
When the source volume server drops connections under high traffic,
filer.sync retries aggressively (every 1-6s), hammering the already
overloaded source. This adds a longer exponential backoff (10s to 2min)
specifically for "unexpected EOF" errors, reducing pressure on the
source while still retrying indefinitely until success.
Also adds more logging throughout the replication path:
- Log source URL and error at V(0) when ReadPart or io.ReadAll fails
- Log content-length and byte counts at V(4) on success
- Log backoff duration in retry messages
Fixes#8542
* filer.sync: extract backoff helper and fix 2-minute cap
- Extract nextEofBackoff() and isEofError() helpers to deduplicate
the backoff logic between fetchAndWrite and uploadManifestChunk
- Fix the cap: previously 80s would double to 160s and pass the
< 2min check uncapped. Now doubles first, then clamps to 2min.
* filer.sync: log source URL instead of empty upload URL on read errors
UploadUrl is not populated until after the reader is consumed, so the
V(0) and V(4) logs were printing an empty string. Add SourceUrl field
to UploadOption and populate it from the HTTP response in fetchAndWrite.
* filer.sync: guard isEofError against nil error
* filer.sync: use errors.Is for EOF detection, fix log wording
- Replace broad substring matching ("read input", "unexpected EOF")
with errors.Is(err, io.ErrUnexpectedEOF) and errors.Is(err, io.EOF)
so only actual EOF errors trigger the longer backoff
- Fix awkward log phrasing: "interrupted replicate" → "interrupted
while replicating"
* filer.sync: remove EOF backoff from uploadManifestChunk
uploadManifestChunk reads from an in-memory bytes.Reader, so any EOF
errors there are from the destination side, not a broken source stream.
The long source-oriented backoff is inappropriate; let RetryUntil
handle destination retries at its normal cadence.
---------
Co-authored-by: Copilot <copilot@github.com>
* admin: remove Scheduler Settings cards, make Next Run full-width
Remove the two "Scheduler Settings" placeholder cards from the plugin
UI (overview page and scheduler tab). They only contained a text note
saying detection intervals are configured per job type, which is
self-evident from the per-job-type settings form.
Make the "Next Run" card full-width on the overview page since it no
longer shares a row with the removed card.
* plugin UI: promote Next Run to top summary card row
Move "Next Run" from a standalone card into the top row alongside
Workers, Active Jobs, and Activities as a compact stat card.
* admin: expose per-job-type detection interval in plugin UI
The detection_interval_seconds field was not editable in the admin UI.
collectAdminSettings() silently preserved the existing value, making it
impossible for users to change how often a job type checks for new work.
Users would change the global "Sleep Between Iterations" setting expecting
it to control job scheduling frequency, but that only controls the
scheduler loop's idle polling rate.
Add a "Detection Interval (s)" input to the per-job-type admin settings
form so users can actually configure it.
Fixes#8549
* admin: remove global Sleep Between Iterations setting
Now that per-job-type detection intervals are exposed in the UI, the
global IdleSleepSeconds setting is redundant and confusing. It only
controlled the scheduler loop's idle polling rate, which is always
overridden by earliestNextDetectionAt() when job types exist.
Replace the three usages with simpler alternatives:
- Scheduler loop sleep: use defaultSchedulerIdleSleep constant
- Initial delay for new job types: use policy.DetectionInterval/2
(more logical since it's already per-job-type)
- Status fallback: use the constant
The API endpoints are kept for backward compatibility but the UI
no longer exposes or calls them.
* admin: restore configurable idle sleep in scheduler loop
The EC integration test sets idle_sleep_seconds=1 via the scheduler
config API so the scheduler wakes quickly after workers connect. The
previous commit replaced this with a hardcoded 613s constant, causing
the scheduler to sleep through the entire test window.
Restore GetSchedulerConfig().IdleSleepDuration() in the scheduler loop
and status reporting. The UI removal of the setting is still correct —
the API endpoint remains for programmatic use (e.g., tests).
* admin: cap first-run initial delay to 5s instead of DetectionInterval/2
The initial delay for first-run job types was set to
policy.DetectionInterval/2, which creates unbounded first-run latency
(e.g., 1 hour for vacuum with a 2-hour detection interval). A small
fixed 5-second delay provides sufficient stagger without penalizing
startup time.
* admin: fix mobile sidebar menu inaccessible in portrait mode
The hamburger button only toggled the user dropdown, leaving the
sidebar navigation inaccessible on mobile devices in portrait mode.
Add a dedicated sidebar toggle button (visible only on mobile), give
the sidebar an id so Bootstrap collapse can target it, add a backdrop
overlay for the open state, and auto-close the sidebar when a nav
link is clicked.
Fixes#8550
* admin: address review feedback on mobile sidebar
- Remove redundant JS show/hide.bs.collapse listeners; CSS sibling
selector already handles backdrop visibility
- Use const instead of var for non-reassigned variables
- Move inline style on user icon to CSS class
* admin: add aria attributes to user-menu toggler, use CSS variable for navbar height
- Add aria-controls, aria-expanded, and aria-label to the user-menu
toggle button for assistive technology
- Extract hard-coded 56px navbar height into --navbar-height CSS
custom property used by sidebar and backdrop positioning
* admin: extract hideSidebar helper, use toggler visibility for breakpoint check
- Extract duplicated collapse-hide logic into a hideSidebar helper
- Replace hardcoded window.innerWidth < 768 with a check on the
sidebar toggler's computed display, decoupling JS from CSS breakpoints
- Add aria-expanded="false" to sidebar toggle button
---------
Co-authored-by: Copilot <copilot@github.com>
* plugin worker: add handler registry with job categories
Introduce a self-registration pattern for plugin worker job handlers.
Each handler can register itself via init() with a HandlerFactory that
declares its job type, category (default/heavy), CLI aliases, and a
builder function.
ResolveHandlerFactories accepts a mix of category names ("all",
"default", "heavy") and explicit job type names/aliases, returning the
matching factories. This enables workers to be configured by resource
profile rather than requiring explicit job type enumeration.
* plugin worker: register all handlers via init()
Each job handler now self-registers into the global handler registry
with its canonical job type, category, CLI aliases, and build function:
- vacuum: category=default
- volume_balance: category=default
- admin_script: category=default
- erasure_coding: category=heavy
- iceberg_maintenance: category=heavy
Adding a new job type now only requires adding the init() call in the
handler file itself — no other files need to be touched.
* plugin worker: replace hardcoded job type switch with registry
Remove buildPluginWorkerHandler, parsePluginWorkerJobTypes, and
canonicalPluginWorkerJobType from worker_runtime.go. The simplified
buildPluginWorkerHandlers now delegates to
pluginworker.ResolveHandlerFactories, which resolves category names
("all", "default", "heavy") and explicit job type names/aliases.
The default job type is changed from an explicit list to "all", so new
handlers registered via init() are automatically picked up.
Update all tests to use the new API.
* plugin worker: update CLI help text for job categories
Update the -jobType flag description and command examples to document
category support (all, default, heavy) alongside explicit job type names.
* plugin worker: address review feedback
- Add CategoryAll constant; use typed constants in tokenAsCategory
- Pre-allocate result slice in ResolveHandlerFactories
- Add vacuum aliases (vol.vacuum, volume.vacuum)
- List alias examples (ec, balance, iceberg) in -jobType flag help
- Create handlers aggregator package for subpackage blank imports so
new handler subpackages only need to be added in one place
- Make category tests relationship-based (subset/union checks) instead
of asserting exact handler counts
- Add clarifying comments to worker_test.go and mini_plugin_test.go
listing expected handler names next to count assertions
---------
Co-authored-by: Copilot <copilot@github.com>
Chris Lu
Chris Lu