seaweedfs

Chris Lu b53e50485f s3: warm bucket config cache on startup for multi-filer consistency (#7772 ) * s3: warm bucket config cache on startup for multi-filer consistency In multi-filer clusters, the bucket configuration cache (storing Object Lock, versioning, and other settings) was not being pre-populated on S3 API server startup. This caused issues where: 1. After server restart, Object Lock and versioning settings appeared lost until the bucket was accessed (lazy loading) 2. In multi-filer clusters, race conditions during bucket creation could result in inconsistent Object Lock configuration This fix warms the bucketConfigCache during BucketRegistry initialization, ensuring all bucket configurations (including Object Lock and versioning) are immediately available after restart without waiting for first access. The fix piggybacks on the existing BucketRegistry.init() which already iterates through all buckets, adding a call to update the config cache with each bucket's extended attributes. * s3: add visibility logging for bucket config cache warming - Add bucket count tracking during initialization - Log error if bucket listing fails - Log INFO message with count of warmed buckets on successful init This improves observability for the cache warming process and addresses review feedback about error handling visibility. * s3: fix bucket deletion not invalidating config cache Bug fix: The metadata subscription handler had an early return when NewEntry was nil, which skipped the onBucketMetadataChange call for bucket deletions. This caused deleted buckets to remain in the config cache. The fix moves onBucketMetadataChange before the nil check so it's called for all events (create, update, delete). The IAM and circuit breaker updates still require NewEntry content, so they remain after the check. * s3: handle config file deletions for IAM and circuit breaker Refactored the metadata subscription handlers to properly handle all event types (create, update, delete) for IAM and circuit breaker configs: - Renamed onIamConfigUpdate -> onIamConfigChange - Renamed onCircuitBreakerConfigUpdate -> onCircuitBreakerConfigChange - Both handlers now check for deletions (newEntry == nil && oldEntry != nil) - On config file deletion, reset to empty config by loading empty bytes - Simplified processEventFn to call all handlers unconditionally - Each handler checks for nil entries internally This ensures that deleting identity.json or circuit_breaker.json will clear the in-memory config rather than leaving stale data. * s3: restore NewParentPath handling for rename/move operations The directory resolution logic was accidentally removed. This restores the check for NewParentPath which is needed when files are renamed or moved - in such cases, NewParentPath contains the destination directory which should be used for directory matching in the handlers.	2 months ago
..
main.go	Add Kafka Gateway (#7231)	4 months ago

s3: warm bucket config cache on startup for multi-filer consistency (#7772 )

* s3: warm bucket config cache on startup for multi-filer consistency

In multi-filer clusters, the bucket configuration cache (storing Object Lock,
versioning, and other settings) was not being pre-populated on S3 API server
startup. This caused issues where:

1. After server restart, Object Lock and versioning settings appeared lost
   until the bucket was accessed (lazy loading)
2. In multi-filer clusters, race conditions during bucket creation could
   result in inconsistent Object Lock configuration

This fix warms the bucketConfigCache during BucketRegistry initialization,
ensuring all bucket configurations (including Object Lock and versioning)
are immediately available after restart without waiting for first access.

The fix piggybacks on the existing BucketRegistry.init() which already
iterates through all buckets, adding a call to update the config cache
with each bucket's extended attributes.

* s3: add visibility logging for bucket config cache warming

- Add bucket count tracking during initialization
- Log error if bucket listing fails
- Log INFO message with count of warmed buckets on successful init

This improves observability for the cache warming process and addresses
review feedback about error handling visibility.

* s3: fix bucket deletion not invalidating config cache

Bug fix: The metadata subscription handler had an early return when
NewEntry was nil, which skipped the onBucketMetadataChange call for
bucket deletions. This caused deleted buckets to remain in the config
cache.

The fix moves onBucketMetadataChange before the nil check so it's
called for all events (create, update, delete). The IAM and circuit
breaker updates still require NewEntry content, so they remain after
the check.

* s3: handle config file deletions for IAM and circuit breaker

Refactored the metadata subscription handlers to properly handle all
event types (create, update, delete) for IAM and circuit breaker configs:

- Renamed onIamConfigUpdate -> onIamConfigChange
- Renamed onCircuitBreakerConfigUpdate -> onCircuitBreakerConfigChange
- Both handlers now check for deletions (newEntry == nil && oldEntry != nil)
- On config file deletion, reset to empty config by loading empty bytes
- Simplified processEventFn to call all handlers unconditionally
- Each handler checks for nil entries internally

This ensures that deleting identity.json or circuit_breaker.json will
clear the in-memory config rather than leaving stale data.

* s3: restore NewParentPath handling for rename/move operations

The directory resolution logic was accidentally removed. This restores
the check for NewParentPath which is needed when files are renamed or
moved - in such cases, NewParentPath contains the destination directory
which should be used for directory matching in the handlers.

2 months ago

main.go

Add Kafka Gateway (#7231)

4 months ago