* handle volume not found when backing up
* error handling on reading volume ttl and replication
* fix Inconsistent error handling: should continue to next location.
* adjust messages
* close volume
* refactor
* refactor
* proper v.Close()
* Rework parameters passing for functions within `ec.rebuild`
This simplifies the overall codebase and allows to cleanly handle parallelization via waitgroups.
* fix copy source
* add tests
* remove tests not useful
* fmt
* nil check
---------
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
* Rework parameters passing for functions within `volume.check.disk`.
We'll need to rework this logic to account for read-only volumes, and there're already way too many parameters shuffled around.
Grouping these into a single struct simplifies the overall codebase.
* similar fix
* Improved Error Handling in Tests
* propagate the errors
* edge cases
* edge case on modified time
* clean up
---------
Co-authored-by: chrislu <chris.lu@gmail.com>
* Unify the parameter to disable dry-run on weed shell commands to --apply (instead of --force).
* lint
* refactor
* Execution Order Corrected
* handle deprecated force flag
* fix help messages
* Refactoring]: Using flag.FlagSet.Visit()
* consistent with other commands
* Checks for both flags
* fix toml files
---------
Co-authored-by: chrislu <chris.lu@gmail.com>
* do delete expired entries on s3 list request
https://github.com/seaweedfs/seaweedfs/issues/6837
* disable delete expires s3 entry in filer
* pass opt allowDeleteObjectsByTTL to all servers
* delete on get and head
* add lifecycle expiration s3 tests
* fix opt allowDeleteObjectsByTTL for server
* fix test lifecycle expiration
* fix IsExpired
* fix locationPrefix for updateEntriesTTL
* fix s3tests
* resolv coderabbitai
* GetS3ExpireTime on filer
* go mod
* clear TtlSeconds for volume
* move s3 delete expired entry to filer
* filer delete meta and data
* del unusing func removeExpiredObject
* test s3 put
* test s3 put multipart
* allowDeleteObjectsByTTL by default
* fix pipline tests
* rm dublicate SeaweedFSExpiresS3
* revert expiration tests
* fix updateTTL
* rm log
* resolv comment
* fix delete version object
* fix S3Versioning
* fix delete on FindEntry
* fix delete chunks
* fix sqlite not support concurrent writes/reads
* move deletion out of listing transaction; delete entries and empty folders
* Revert "fix sqlite not support concurrent writes/reads"
This reverts commit 5d5da14e0e.
* clearer handling on recursive empty directory deletion
* handle listing errors
* strut copying
* reuse code to delete empty folders
* use iterative approach with a queue to avoid recursive WithFilerClient calls
* stop a gRPC stream from the client-side callback is to return a specific error, e.g., io.EOF
* still issue UpdateEntry when the flag must be added
* errors join
* join path
* cleaner
* add context, sort directories by depth (deepest first) to avoid redundant checks
* batched operation, refactoring
* prevent deleting bucket
* constant
* reuse code
* more logging
* refactoring
* s3 TTL time
* Safety check
---------
Co-authored-by: chrislu <chris.lu@gmail.com>
- Update matchesCollection to recognize 'default' as filter for empty collection
- Remove incorrect conversion of 'default' to empty string in handlers
- Fixes issue where ?collection=default would show all collections instead of just default collection
* Added continue statements after all state transitions in the state machine to ensure immediate state processing
* simplify
* remove redundant continue clause
* ensure wrong signature
* weed master -peers=none
* single master mode only when peers is none
* refactoring
* revert duplicated code
* revert
* Update weed/command/master.go
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* preventing "none" passed to other components if master is not started
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* adjust "weed benchmark" CLI to use readOnly/writeOnly
* consistently use "-master" CLI option
* If both -readOnly and -writeOnly are specified, the current logic silently allows it with -writeOnly taking precedence. This is confusing and could lead to unexpected behavior.
* fallback to check master
* clean up
* parsing
* refactor
* handle parse error
* return error
* avoid dup lookup
* use batch key
* dedup lookup logic
* address comments
* errors.Join(lookupErrors...)
* add a comment
* Fix: Critical data race in MasterClient vidMap
Fixes a critical data race where resetVidMap() was writing to the vidMap
pointer while other methods were reading it concurrently without synchronization.
Changes:
- Removed embedded *vidMap from MasterClient
- Added vidMapLock (sync.RWMutex) to protect vidMap pointer access
- Created safe accessor methods (GetLocations, GetDataCenter, etc.)
- Updated all direct vidMap accesses to use thread-safe methods
- Updated resetVidMap() to acquire write lock during pointer swap
The vidMap already has internal locking for its operations, but this fix
protects the vidMap pointer itself from concurrent read/write races.
Verified with: go test -race ./weed/wdclient/...
Impact:
- Prevents potential panics from concurrent pointer access
- No performance impact - uses RWMutex for read-heavy workloads
- Maintains backward compatibility through wrapper methods
* fmt
* Fix: Critical data race in MasterClient vidMap
Fixes a critical data race where resetVidMap() was writing to the vidMap
pointer while other methods were reading it concurrently without synchronization.
Changes:
- Removed embedded *vidMap from MasterClient struct
- Added vidMapLock (sync.RWMutex) to protect vidMap pointer access
- Created minimal public accessor methods for external packages:
* GetLocations, GetLocationsClone, GetVidLocations
* LookupFileId, LookupVolumeServerUrl
* GetDataCenter
- Internal code directly locks and accesses vidMap (no extra indirection)
- Updated resetVidMap() to acquire write lock during pointer swap
- Updated shell/commands.go to use GetDataCenter() method
Design Philosophy:
- vidMap already has internal locking for its map operations
- This fix specifically protects the vidMap *pointer* from concurrent access
- Public methods for external callers, direct locking for internal use
- Minimizes wrapper overhead while maintaining thread safety
Verified with: go test -race ./weed/wdclient/... (passes)
Impact:
- Prevents potential panics/crashes from data races
- Minimal performance impact (RWMutex for read-heavy workload)
- Maintains full backward compatibility
* fix more concurrent access
* reduce lock scope
* Optimize vidMap locking for better concurrency
Improved locking strategy based on the understanding that:
- vidMapLock protects the vidMap pointer from concurrent swaps
- vidMap has internal locks that protect its data structures
Changes:
1. Read operations: Grab pointer with RLock, release immediately, then operate
- Reduces lock hold time
- Allows resetVidMap to proceed sooner
- Methods: GetLocations, GetLocationsClone, GetVidLocations,
LookupVolumeServerUrl, GetDataCenter
2. Write operations: Changed from Lock() to RLock()
- RLock prevents pointer swap during operation
- Allows concurrent readers and other writers (serialized by vidMap's lock)
- Methods: addLocation, deleteLocation, addEcLocation, deleteEcLocation
Benefits:
- Significantly reduced lock contention
- Better concurrent performance under load
- Still prevents all race conditions
Verified with: go test -race ./weed/wdclient/... (passes)
* Further reduce lock contention in LookupVolumeIdsWithFallback
Optimized two loops that were holding RLock for extended periods:
Before:
- Held RLock during entire loop iteration
- Included string parsing and cache lookups
- Could block resetVidMap for significant time with large batches
After:
- Grab vidMap pointer with brief RLock
- Release lock immediately
- Perform all loop operations on local pointer
Impact:
- First loop: Cache check on initial volumeIds
- Second loop: Double-check after singleflight wait
Benefits:
- Minimal lock hold time (just pointer copy)
- resetVidMap no longer blocked by long loops
- Better concurrent performance with large volume ID lists
- Still thread-safe (vidMap methods have internal locks)
Verified with: go test -race ./weed/wdclient/... (passes)
* Add clarifying comments to vidMap helper functions
Added inline documentation to each helper function (addLocation, deleteLocation,
addEcLocation, deleteEcLocation) explaining the two-level locking strategy:
- RLock on vidMapLock prevents resetVidMap from swapping the pointer
- vidMap has internal locks that protect the actual map mutations
- This design provides optimal concurrency
The comments make it clear why RLock (not Lock) is correct and intentional,
preventing future confusion about the locking strategy.
* Improve encapsulation: Add shallowClone() method to vidMap
Added a shallowClone() method to vidMap to improve encapsulation and prevent
MasterClient from directly accessing vidMap's internal fields.
Changes:
1. Added vidMap.shallowClone() in vid_map.go
- Encapsulates the shallow copy logic within vidMap
- Makes vidMap responsible for its own state representation
- Documented that caller is responsible for thread safety
2. Simplified resetVidMap() in masterclient.go
- Uses tail := mc.vidMap.shallowClone() instead of manual field access
- Cleaner, more maintainable code
- Better adherence to encapsulation principles
Benefits:
- Improved code organization and maintainability
- vidMap internals are now properly encapsulated
- Easier to modify vidMap structure in the future
- More self-documenting code
Verified with: go test -race ./weed/wdclient/... (passes)
* Optimize locking: Reduce lock acquisitions and use helper methods
Two optimizations to further reduce lock contention and improve code consistency:
1. LookupFileIdWithFallback: Eliminated redundant lock acquisition
- Before: Two separate locks to get vidMap and dataCenter
- After: Single lock gets both values together
- Benefit: 50% reduction in lock/unlock overhead for this hot path
2. KeepConnected: Use GetDataCenter() helper for consistency
- Before: Manual lock/unlock to access DataCenter field
- After: Use existing GetDataCenter() helper method
- Benefit: Better encapsulation and code consistency
Impact:
- Reduced lock contention in high-traffic lookup path
- More consistent use of accessor methods throughout codebase
- Cleaner, more maintainable code
Verified with: go test -race ./weed/wdclient/... (passes)
* Refactor: Extract common locking patterns into helper methods
Eliminated code duplication by introducing two helper methods that encapsulate
the common locking patterns used throughout MasterClient:
1. getStableVidMap() - For read operations
- Acquires lock, gets pointer, releases immediately
- Returns stable snapshot for thread-safe reads
- Used by: GetLocations, GetLocationsClone, GetVidLocations,
LookupFileId, LookupVolumeServerUrl, GetDataCenter
2. withCurrentVidMap(f func(vm *vidMap)) - For write operations
- Holds RLock during callback execution
- Prevents pointer swap while allowing concurrent operations
- Used by: addLocation, deleteLocation, addEcLocation, deleteEcLocation
Benefits:
- Reduced code duplication (eliminated 48 lines of repetitive locking code)
- Centralized locking logic makes it easier to understand and maintain
- Self-documenting pattern through named helper methods
- Easier to modify locking strategy in the future (single point of change)
- Improved readability - accessor methods are now one-liners
Code size reduction: ~40% fewer lines for accessor/helper methods
Verified with: go test -race ./weed/wdclient/... (passes)
* consistent
* Fix cache pointer race condition with atomic.Pointer
Use atomic.Pointer for vidMap cache field to prevent data races
during cache trimming in resetVidMap. This addresses the race condition
where concurrent GetLocations calls could read the cache pointer while
resetVidMap is modifying it during cache chain trimming.
Changes:
- Changed cache field from *vidMap to atomic.Pointer[vidMap]
- Updated all cache access to use Load() and Store() atomic operations
- Updated shallowClone, GetLocations, deleteLocation, deleteEcLocation
- Updated resetVidMap to use atomic operations for cache trimming
* Merge: Resolve conflict in deleteEcLocation - keep atomic.Pointer and fix bug
Resolved merge conflict by combining:
1. Atomic pointer access pattern (from HEAD): cache.Load()
2. Correct method call (from fix): deleteEcLocation (not deleteLocation)
Resolution:
- Before (HEAD): cachedMap.deleteLocation() - WRONG, reintroduced bug
- Before (fix): vc.cache.deleteEcLocation() - RIGHT method, old pattern
- After (merged): cachedMap.deleteEcLocation() - RIGHT method, new pattern
This preserves both improvements:
✓ Thread-safe atomic.Pointer access pattern
✓ Correct recursive call to deleteEcLocation
Verified with: go test -race ./weed/wdclient/... (passes)
* Update vid_map.go
* remove shallow clone
* simplify
* Add nginx reverse proxy documentation for S3 API
Fixes#7407
Add comprehensive documentation and example configuration for using
nginx as a reverse proxy with SeaweedFS S3 API while maintaining AWS
Signature V4 authentication compatibility.
Changes:
- Add docker/nginx/README.md with detailed setup guide
- Add docker/nginx/s3-example.conf with working configuration
- Update docker/nginx/proxy.conf with important S3 notes
The documentation covers:
- Critical requirements for AWS Signature V4 authentication
- Common mistakes and why they break S3 authentication
- Complete working nginx configurations
- Debugging tips and troubleshooting
- Performance tuning recommendations
* Fix IPv6 host header formatting to match AWS SDK behavior
Follow-up to PR #7403
When a default port (80 for HTTP, 443 for HTTPS) is stripped from an
IPv6 address, the square brackets should also be removed to match AWS
SDK behavior for S3 signature calculation.
Reference: https://github.com/aws/aws-sdk-go-v2/blob/main/aws/signer/internal/v4/host.go
The AWS SDK's stripPort function explicitly removes brackets when
returning an IPv6 address without a port.
Changes:
- Update extractHostHeader to strip brackets from IPv6 addresses when
no port or default port is used
- Update test expectations to match AWS SDK behavior
- Add detailed comments explaining the AWS SDK compatibility requirement
This ensures S3 signature validation works correctly with IPv6 addresses
behind reverse proxies, matching AWS S3 canonical request format.
Fixes the issue raised in PR #7403 comment:
https://github.com/seaweedfs/seaweedfs/pull/7403#issuecomment-3471105438
* Update docker/nginx/README.md
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Add nginx reverse proxy documentation for S3 API
Fixes#7407
Add comprehensive documentation and example configuration for using
nginx as a reverse proxy with SeaweedFS S3 API while maintaining AWS
Signature V4 authentication compatibility.
Changes:
- Add docker/nginx/README.md with detailed setup guide
- Add docker/nginx/s3-example.conf with working configuration
- Update docker/nginx/proxy.conf with important S3 notes
The documentation covers:
- Critical requirements for AWS Signature V4 authentication
- Common mistakes and why they break S3 authentication
- Complete working nginx configurations
- Debugging tips and troubleshooting
- Performance tuning recommendations
Fix IPv6 host header formatting to match AWS SDK behavior
Follow-up to PR #7403
When a default port (80 for HTTP, 443 for HTTPS) is stripped from an
IPv6 address, the square brackets should also be removed to match AWS
SDK behavior for S3 signature calculation.
Reference: https://github.com/aws/aws-sdk-go-v2/blob/main/aws/signer/internal/v4/host.go
The AWS SDK's stripPort function explicitly removes brackets when
returning an IPv6 address without a port.
Changes:
- Update extractHostHeader to strip brackets from IPv6 addresses when
no port or default port is used
- Update test expectations to match AWS SDK behavior
- Add detailed comments explaining the AWS SDK compatibility requirement
This ensures S3 signature validation works correctly with IPv6 addresses
behind reverse proxies, matching AWS S3 canonical request format.
Fixes the issue raised in PR #7403 comment:
https://github.com/seaweedfs/seaweedfs/pull/7403#issuecomment-3471105438
* Revert "Merge branch 'fix-ipv6-brackets-default-port' of https://github.com/seaweedfs/seaweedfs into fix-ipv6-brackets-default-port"
This reverts commit cca3f3985f, reversing
changes made to 2b8f9de78e.
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* server can start when no network for local dev
* fixed superfluous response.WriteHeader call" warning
* adaptive based on last write time
* more doc
* refactoring
* Fix S3 bucket policy ARN validation to accept AWS ARNs and simplified formats
Fixes#7252
The bucket policy validation was rejecting valid AWS-style ARNs and
simplified resource formats, causing validation failures with the
error 'resource X does not match bucket X' even when they were
identical strings.
Changes:
- Updated validateResourceForBucket() to accept three formats:
1. AWS-style ARNs: arn:aws:s3:::bucket-name[/*|/path]
2. SeaweedFS ARNs: arn:seaweed:s3:::bucket-name[/*|/path]
3. Simplified formats: bucket-name[/*|/path]
- Added comprehensive test coverage for all three formats
- Added specific test cases from issue #7252 to prevent regression
This ensures compatibility with standard AWS S3 bucket policies
while maintaining support for SeaweedFS-specific ARN format.
* Refactor validateResourceForBucket to reduce code duplication
Simplified the validation logic by stripping ARN prefixes first,
then performing validation on the remaining resource path.
This reduces code duplication and improves maintainability while
maintaining identical functionality.
Addresses review feedback from Gemini Code Assist.
* Use strings.CutPrefix for cleaner ARN prefix stripping
Replace strings.HasPrefix checks with strings.CutPrefix for more
idiomatic Go code. This function is available in Go 1.20+ and
provides cleaner conditional logic with the combined check and
extraction.
Addresses review feedback from Gemini Code Assist.
chrislu
Chris Lu
Lisandro Pin
Konstantin Lebedev