Cross-compile Rust volume server natively for amd64/arm64 using musl
targets in a separate job, then inject pre-built binaries into the
Docker build. This replaces the ~5-hour QEMU-emulated cargo build
with ~15 minutes of native cross-compilation.
The Dockerfile falls back to building from source when no pre-built
binary is found, preserving local build compatibility.
* fix(s3): skip directories before marker in ListObjectVersions pagination
ListObjectVersions was re-traversing the entire directory tree from the
beginning on every paginated request, only skipping entries at the leaf
level. For buckets with millions of objects in deep hierarchies, this
caused exponentially slower responses as pagination progressed.
Two optimizations:
1. Use keyMarker to compute a startFrom position at each directory level,
skipping directly to the relevant entry instead of scanning from the
beginning (mirroring how ListObjects uses marker descent).
2. Skip recursing into subdirectories whose keys are entirely before the
keyMarker.
Changes per-page cost from O(entries_before_marker) to O(tree_depth).
* test(s3): add integration test for deep-hierarchy version listing pagination
Adds TestVersioningPaginationDeepDirectoryHierarchy which creates objects
across 20 subdirectories at depth 6 (mimicking Veeam 365 backup layout)
and paginates through them with small maxKeys. Verifies correctness
(no duplicates, sorted order, all objects found) and checks that later
pages don't take dramatically longer than earlier ones — the symptom
of the pre-fix re-traversal bug. Also tests delimiter+pagination
interaction across subdirectories.
* test(s3): strengthen deep-hierarchy pagination assertions
- Replace timing warning (t.Logf) with a failing assertion (t.Errorf)
so pagination regressions actually fail the test.
- Replace generic count/uniqueness/sort checks on CommonPrefixes with
exact equality against the expected prefix slice, catching wrong-but-
sorted results.
* test(s3): use allKeys for exact assertion in deep-hierarchy pagination test
Wire the allKeys slice (previously unused dead code) into the version
listing assertion, replacing generic count/uniqueness/sort checks with
an exact equality comparison against the keys that were created.
* STS: add GetCallerIdentity support
Implement the AWS STS GetCallerIdentity action, which returns the
ARN, account ID, and user ID of the caller based on SigV4 authentication.
This is commonly used by AWS SDKs and CLI tools (e.g. `aws sts get-caller-identity`)
to verify credentials and determine the authenticated identity.
* test: remove trivial GetCallerIdentity tests
Remove the XML unmarshal test (we don't consume this response as input)
and the routing constant test (just asserts a literal equals itself).
* fix: route GetCallerIdentity through STS in UnifiedPostHandler and use stable UserId
- UnifiedPostHandler only dispatched actions starting with "AssumeRole" to STS,
so GetCallerIdentity in a POST body would fall through to the IAM path and
get AccessDenied for non-admin users. Add explicit check for GetCallerIdentity.
- Use identity.Name as UserId instead of credential.AccessKey, which is a
transient value and incorrect for STS assumed-role callers.
* fix(weed/worker/tasks/ec_balance): non-recursive reportProgress
* fix(ec_balance): call ReportProgressWithStage and include volumeID in log
The original fix replaced infinite recursion with a glog.Infof, but
skipped the framework progress callback. This adds the missing
ReportProgressWithStage call so the admin server receives EC balance
progress, and includes volumeID in the log for disambiguation.
---------
Co-authored-by: Chris Lu <chris.lu@gmail.com>
* fix(test): address flaky S3 distributed lock integration test
Two root causes:
1. Lock ring convergence race: After waitForFilerCount(2) confirms the
master sees both filers, there's a window where filer0's lock ring
still only contains itself (master's LockRingUpdate broadcast is
delayed by the 1s stabilization timer). During this window filer0
considers itself primary for ALL keys, so both filers can
independently grant the same lock.
Fix: Add waitForLockRingConverged() that acquires the same lock
through both filers and verifies mutual exclusion before proceeding.
2. Hash function mismatch: ownerForObjectLock used util.HashStringToLong
(MD5 + modulo) to predict lock owners, but the production DLM uses
CRC32 consistent hashing via HashRing. This meant the test could
pick keys that route to the same filer, not exercising the
cross-filer coordination it intended to test.
Fix: Use lock_manager.NewHashRing + GetPrimary() to match production
routing exactly.
* fix(test): verify lock denial reason in convergence check
Ensure the convergence check only returns true when the second lock
attempt is denied specifically because the lock is already owned,
avoiding false positives from transient errors.
* fix(test): check one key per primary filer in convergence wait
A single arbitrary key can false-pass: if its real primary is the filer
with the stale ring, mutual exclusion holds trivially because that filer
IS the correct primary. Generate one test key per distinct primary using
the same consistent-hash ring as production, so a stale ring on any
filer is caught deterministically.
* filer.sync: show active chunk transfers when sync progress stalls
When the sync watermark is not advancing, print each in-progress chunk
transfer with its file path, bytes received so far, and current status
(downloading, uploading, or waiting with backoff duration). This helps
diagnose which files are blocking progress during replication.
Closes#8542
* filer.sync: include last error in stall diagnostics
* filer.sync: fix data races in ChunkTransferStatus
Add sync.RWMutex to ChunkTransferStatus and lock around all field
mutations in fetchAndWrite. ActiveTransfers now returns value copies
under RLock so callers get immutable snapshots.
Add sync.RWMutex to ChunkTransferStatus and lock around all field
mutations in fetchAndWrite. ActiveTransfers now returns value copies
under RLock so callers get immutable snapshots.
When the sync watermark is not advancing, print each in-progress chunk
transfer with its file path, bytes received so far, and current status
(downloading, uploading, or waiting with backoff duration). This helps
diagnose which files are blocking progress during replication.
Closes#8542
The test port allocation had a TOCTOU race where GetFreePort() would
open a listener, grab the port number, then immediately close it.
When called repeatedly, the OS could recycle a just-released port,
causing two services (e.g. Filer and S3) to be assigned the same port.
Replace per-call GetFreePort() with batch AllocatePorts() that holds
all listeners open until every port is obtained, matching the pattern
already used in test/volume_server/framework/cluster.go.
* fix(s3): use recursive delete for .versions directory cleanup
When only delete markers remain in a .versions directory,
updateLatestVersionAfterDeletion tried to delete it non-recursively,
which failed with "fail to delete non-empty folder" because the delete
marker entries were still present. Use recursive deletion so the
directory and its remaining delete marker entries are cleaned up together.
* fix(s3): guard .versions directory deletion against truncated listings
When the version listing is truncated (>1000 entries), content versions
may exist beyond the first page. Skip the recursive directory deletion
in this case to prevent data loss.
* fix(s3): preserve delete markers in .versions directory
Delete markers must be preserved per S3 semantics — they are only
removed by an explicit DELETE with versionId. The previous fix would
recursively delete the entire .versions directory (including delete
markers) when no content versions were found.
Now the logic distinguishes three cases:
1. Content versions exist → update latest version metadata
2. Only delete markers remain (or listing truncated) → keep directory
3. Truly empty → safe to delete directory (non-recursive)
* fix(admin): respect urlPrefix in S3 bucket and S3Tables navigation links (#8884)
Several admin UI templates used hardcoded URLs (templ.SafeURL) instead of
dash.PUrl(ctx, ...) for navigation links, causing 404 errors when the
admin is deployed with --urlPrefix.
Fixed in: s3_buckets.templ, s3tables_buckets.templ, s3tables_tables.templ
* fix(admin): URL-escape bucketName in S3Tables navigation links
Add url.PathEscape(bucketName) for consistency and correctness in
s3tables_tables.templ (back-to-namespaces link) and s3tables_buckets.templ
(namespace link), matching the escaping already used in the table details link.
* S3: map canned ACL to file permissions and add configurable default file mode
S3 uploads were hardcoded to 0660 regardless of ACL headers. Now the
X-Amz-Acl header maps to Unix file permissions per-object:
- public-read, authenticated-read, bucket-owner-read → 0644
- public-read-write → 0666
- private, bucket-owner-full-control → 0660
Also adds -defaultFileMode / -s3.defaultFileMode flag to set a
server-wide default when no ACL header is present.
Closes#8874
* Address review feedback for S3 file mode feature
- Extract hardcoded 0660 to defaultFileMode constant
- Change parseDefaultFileMode to return error instead of calling Fatalf
- Add -s3.defaultFileMode flag to filer.go and mini.go (was missing)
- Add doc comment to S3Options about updating all four flag sites
- Add TestResolveFileMode with 10 test cases covering ACL mapping,
server default, and priority ordering
- concurrent_operations_test: Add retry loop for transient I/O errors
on file close during ConcurrentDirectoryOperations
- git_operations_test: Wait for pushed objects to become visible through
FUSE mount before cloning in Phase 3
The Rust weed-volume binary requires libgcc_s.so.1 for stack unwinding
(_Unwind_* symbols). Without it, the binary fails to load in the Alpine
container with "Error loading shared library libgcc_s.so.1".
* fix(s3): remove customer encryption key from SSE-C debug log
The debug log in validateAndParseSSECHeaders was logging the raw
customer-provided encryption key bytes in hex format (keyBytes=%x),
leaking sensitive key material to log output. Remove the key bytes
from the log statement while keeping the MD5 hash comparison info.
* Apply suggestion from @gemini-code-assist[bot]
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
The pprof crate uses Unix-only APIs (nix, libc::pthread_t,
libc::siginfo_t, etc.) that don't exist on Windows. Move it to
[target.'cfg(unix)'.dependencies] and gate all profiling/debug
module usage with #[cfg(unix)].
* Add manual trigger to Rust volume server release build workflow
When triggered manually via workflow_dispatch, binaries are uploaded as
downloadable workflow artifacts instead of release assets. On tag push
the existing release upload behavior is unchanged.
* Vendor OpenSSL for cross-compilation of Rust volume server
The aarch64-unknown-linux-gnu build fails because openssl-sys cannot
find OpenSSL via pkg-config when cross-compiling. Adding openssl with
the vendored feature builds OpenSSL from source, fixing the issue.
* Fix aarch64 cross-compilation: install libssl-dev:arm64 instead of vendoring OpenSSL
The vendored OpenSSL feature breaks the S3 tier unit test by altering
the TLS stack behavior. Instead, install the aarch64 OpenSSL dev
libraries and point the build at them via OPENSSL_DIR/LIB_DIR/INCLUDE_DIR.
* filer.sync: support per-cluster mTLS with -a.security and -b.security flags
When syncing between two clusters that use different certificate authorities,
a single security.toml cannot authenticate to both. Add -a.security and
-b.security flags so each filer can use its own security.toml for TLS.
Closes#8481
* security: fatal on failure to read explicitly provided security config
When -a.security or -b.security is specified, falling back to insecure
credentials on read error would silently bypass mTLS. Fatal instead.
* fix(filer.sync): use source filer's fromTsMs flag in initOffsetFromTsMs
A→B was using bFromTsMs and B→A was using aFromTsMs — these were
swapped. Each path should seed the target's offset with the source
filer's starting timestamp.
* security: return error from LoadClientTLSFromFile, resolve relative PEM paths
Change LoadClientTLSFromFile to return (grpc.DialOption, error) so
callers can handle failures explicitly instead of a silent insecure
fallback. Resolve relative PEM paths (grpc.ca, grpc.client.cert,
grpc.client.key) against the config file's directory.
* Fix Admin UI master list showing gRPC port instead of HTTP port for followers (#8867)
Raft stores server addresses as gRPC addresses. The Admin UI was using
these addresses directly via ToHttpAddress(), which cannot extract the
HTTP port from a plain gRPC address. Use GrpcAddressToServerAddress()
to properly convert gRPC addresses back to HTTP addresses.
* Use httpAddress consistently as masterMap key
Address review feedback: masterInfo.Address (HTTP form) was already
computed but the raw address was used as the map key, causing
potential key mismatches between topology and raft data.
* fix(s3api): fix AWS Signature V2 format and validation
* fix(s3api): Skip space after "AWS" prefix (+1 offset)
* test(s3api): add unit tests for Signature V2 authentication fix
* fix(s3api): simply comparing signatures
* validation for the colon extraction in expectedAuth
* fix(shell): avoid marking skipped or unplaced volumes as fixed
---------
Co-authored-by: chrislu <chris.lu@gmail.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
* Process .ecj deletions during EC decode and vacuum decoded volume (#8798)
When decoding EC volumes back to normal volumes, deletions recorded in
the .ecj journal were not being applied before computing the dat file
size or checking for live needles. This caused the decoded volume to
include data for deleted files and could produce false positives in the
all-deleted check.
- Call RebuildEcxFile before HasLiveNeedles/FindDatFileSize in
VolumeEcShardsToVolume so .ecj deletions are merged into .ecx first
- Vacuum the decoded volume after mounting in ec.decode to compact out
deleted needle data from the .dat file
- Add integration tests for decoding with non-empty .ecj files
* storage: add offline volume compaction helper
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* ec: compact decoded volumes before deleting shards
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* ec: address PR review comments
- Fall back to data directory for .ecx when idx directory lacks it
- Make compaction failure non-fatal during EC decode
- Remove misleading "buffer: 10%" from space check error message
* ec: collect .ecj from all shard locations during decode
Each server's .ecj only contains deletions for needles whose data
resides in shards held by that server. Previously, sources with no
new data shards to contribute were skipped entirely, losing their
.ecj deletion entries. Now .ecj is always appended from every shard
location so RebuildEcxFile sees the full set of deletions.
* ec: add integration tests for .ecj collection during decode
TestEcDecodePreservesDeletedNeedles: verifies that needles deleted
via VolumeEcBlobDelete are excluded from the decoded volume.
TestEcDecodeCollectsEcjFromPeer: regression test for the fix in
collectEcShards. Deletes a needle only on a peer server that holds
no new data shards, then verifies the deletion survives decode via
.ecj collection.
* ec: address review nits in decode and tests
- Remove double error wrapping in mountDecodedVolume
- Check VolumeUnmount error in peer ecj test
- Assert 404 specifically for deleted needles, fail on 5xx
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* dlm: replace modulo hashing with consistent hash ring
Introduce HashRing with virtual nodes (CRC32-based consistent hashing)
to replace the modulo-based hashKeyToServer. When a filer node is
removed, only keys that hashed to that node are remapped to the next
server on the ring, leaving all other mappings stable. This is the
foundation for backup replication — the successor on the ring is
always the natural takeover node.
* dlm: add Generation and IsBackup fields to Lock
Lock now carries IsBackup (whether this node holds the lock as a backup
replica) and Generation (a monotonic fencing token that increments on
each fresh acquisition, stays the same on renewal). Add helper methods:
AllLocks, PromoteLock, DemoteLock, InsertBackupLock, RemoveLock, GetLock.
* dlm: add ReplicateLock RPC and generation/is_backup proto fields
Add generation field to LockResponse for fencing tokens.
Add generation and is_backup fields to Lock message.
Add ReplicateLock RPC for primary-to-backup lock replication.
Add ReplicateLockRequest/ReplicateLockResponse messages.
* dlm: add async backup replication to DistributedLockManager
Route lock/unlock via consistent hash ring's GetPrimaryAndBackup().
After a successful lock or unlock on the primary, asynchronously
replicate the operation to the backup server via ReplicateFunc
callback. Single-server deployments skip replication.
* dlm: add ReplicateLock handler and backup-aware topology changes
Add ReplicateLock gRPC handler for primary-to-backup replication.
Revise OnDlmChangeSnapshot to handle three cases on topology change:
- Promote backup locks when this node becomes primary
- Demote primary locks when this node becomes backup
- Transfer locks when this node is neither primary nor backup
Wire up SetupDlmReplication during filer server initialization.
* dlm: expose generation fencing token in lock client
LiveLock now captures the generation from LockResponse and exposes it
via Generation() method. Consumers can use this as a fencing token to
detect stale lock holders.
* dlm: update empty folder cleaner to use consistent hash ring
Replace local modulo-based hashKeyToServer with LockRing.GetPrimary()
which uses the shared consistent hash ring for folder ownership.
* dlm: add unit tests for consistent hash ring
Test basic operations, consistency on server removal (only keys from
removed server move), backup-is-successor property (backup becomes
new primary when primary is removed), and key distribution balance.
* dlm: add integration tests for lock replication failure scenarios
Test cases:
- Primary crash with backup promotion (backup has valid token)
- Backup crash with primary continuing
- Both primary and backup crash (lock lost, re-acquirable)
- Rolling restart across all nodes
- Generation fencing token increments on new acquisition
- Replication failure (primary still works independently)
- Unlock replicates deletion to backup
- Lock survives server addition (topology change)
- Consistent hashing minimal disruption (only removed server's keys move)
* dlm: address PR review findings
1. Causal replication ordering: Add per-lock sequence number (Seq) that
increments on every mutation. Backup rejects incoming mutations with
seq <= current seq, preventing stale async replications from
overwriting newer state. Unlock replication also carries seq and is
rejected if stale.
2. Demote-after-handoff: OnDlmChangeSnapshot now transfers the lock to
the new primary first and only demotes to backup after a successful
TransferLocks RPC. If the transfer fails, the lock stays as primary
on this node.
3. SetSnapshot candidateServers leak: Replace the candidateServers map
entirely instead of appending, so removed servers don't linger.
4. TransferLocks preserves Generation and Seq: InsertLock now accepts
generation and seq parameters. After accepting a transferred lock,
the receiving node re-replicates to its backup.
5. Rolling restart test: Add re-replication step after promotion and
assert survivedCount > 0. Add TestDLM_StaleReplicationRejected.
6. Mixed-version upgrade note: Add comment on HashRing documenting that
all filer nodes must be upgraded together.
* dlm: serve renewals locally during transfer window on node join
When a new node joins and steals hash ranges from surviving nodes,
there's a window between ring update and lock transfer where the
client gets redirected to a node that doesn't have the lock yet.
Fix: if the ring says primary != self but we still hold the lock
locally (non-backup, matching token), serve the renewal/unlock here
rather than redirecting. The lock will be transferred by
OnDlmChangeSnapshot, and subsequent requests will go to the new
primary once the transfer completes.
Add tests:
- TestDLM_NodeDropAndJoin_OwnershipDisruption: measures disruption
when a node drops and a new one joins (14/100 surviving-node locks
disrupted, all handled by transfer logic)
- TestDLM_RenewalDuringTransferWindow: verifies renewal succeeds on
old primary during the transfer window
* dlm: master-managed lock ring with stabilization batching
The master now owns the lock ring membership. Instead of filers
independently reacting to individual ClusterNodeUpdate add/remove
events, the master:
1. Tracks filer membership in LockRingManager
2. Batches rapid changes with a 1-second stabilization timer
(e.g., a node drop + join within 1 second → single ring update)
3. Broadcasts the complete ring snapshot atomically via the new
LockRingUpdate message in KeepConnectedResponse
Filers receive the ring as a complete snapshot and apply it via
SetSnapshot, ensuring all filers converge to the same ring state
without intermediate churn.
This eliminates the double-churn problem where a rapid drop+join
would fire two separate ring mutations, each triggering lock
transfers and disrupting ownership on surviving nodes.
* dlm: track ring version, reject stale updates, remove dead code
SetSnapshot now takes a version parameter from the master. Stale
updates (version < current) are rejected, preventing reordered
messages from overwriting a newer ring state. Version 0 is always
accepted for bootstrap.
Remove AddServer/RemoveServer from LockRing — the ring is now
exclusively managed by the master via SetSnapshot. Remove the
candidateServers map that was only used by those methods.
* dlm: fix SelectLocks data race, advance generation on backup insert
- SelectLocks: change RLock to Lock since the function deletes map
entries, which is a write operation and causes a data race under RLock.
- InsertBackupLock: advance nextGeneration to at least the incoming
generation so that after failover promotion, new lock acquisitions
get a generation strictly greater than any replicated lock.
- Bump replication failure log from V(1) to Warningf for production
visibility.
* dlm: fix SetSnapshot race, test reliability, timer edge cases
- SetSnapshot: hold LockRing lock through both version update and
Ring.SetServers() so they're atomic. Prevents a concurrent caller
from seeing the new version but applying stale servers.
- Transfer window test: search for a key that actually moves primary
when filer4 joins, instead of relying on a fixed key that may not.
- renewLock redirect: pass the existing token to the new primary
instead of empty string, so redirected renewals work correctly.
- scheduleBroadcast: check timer.Stop() return value. If the timer
already fired, the callback picks up latest state.
- FlushPending: only broadcast if timer.Stop() returns true (timer
was still pending). If false, the callback is already running.
- Fix test comment: "idempotent" → "accepted, state-changing".
* dlm: use wall-clock nanoseconds for lock ring version
The lock ring version was an in-memory counter that reset to 0 on
master restart. A filer that had seen version 5 would reject version 1
from the restarted master.
Fix: use time.Now().UnixNano() as the version. This survives master
restarts without persistence — the restarted master produces a
version greater than any pre-restart value.
* dlm: treat expired lock owners as missing
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* dlm: reject stale lock transfers
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* dlm: order replication by generation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* dlm: bootstrap lock ring on reconnect
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* chore(weed/wdclient): prune unused functions
* chore(weed/wdclient): prune test-only functions and associated tests
* chore(weed/wdclient): remove dead cursor field
The cursor field and its initialization are no longer used after
the removal of getLocationIndex.
---------
Co-authored-by: Chris Lu <chris.lu@gmail.com>
* Use Unix sockets for gRPC between co-located services in weed server
Extends the Unix socket gRPC optimization (added for mini mode in #8856)
to `weed server`. Registers Unix socket paths for each service's gRPC
port before startup, so co-located services (master, volume, filer, S3)
communicate via Unix sockets instead of TCP loopback.
Only services actually started in this process get registered. The gRPC
port is resolved early (port + 10000 if unset) so the socket path is
known before any service dials another.
* Refactor gRPC Unix socket registration into a data-driven loop
* Fix stale admin lock metric when lock expires and is reacquired (#8857)
When a lock expired without an explicit unlock and a different client
acquired it, the old client's metric was never cleared, causing
multiple clients to appear as simultaneously holding the lock.
* Use DeleteLabelValues instead of Set(0) to remove stale metric series
Avoids cardinality explosion from accumulated stale series when
client names are dynamic.
* rename metadata events
* fix subscription filter to use NewEntry.Name for rename path matching
The server-side subscription filter constructed the new path using
OldEntry.Name instead of NewEntry.Name when checking if a rename
event's destination matches the subscriber's path prefix. This could
cause events to be incorrectly filtered when a rename changes the
file name.
* fix bucket events to handle rename of bucket directories
onBucketEvents only checked IsCreate and IsDelete. A bucket directory
rename via AtomicRenameEntry now emits a single rename event (both
OldEntry and NewEntry non-nil), which matched neither check. Handle
IsRename by deleting the old bucket and creating the new one.
* fix replicator to handle rename events across directory boundaries
Two issues fixed:
1. The replicator filtered events by checking if the key (old path)
was under the source directory. Rename events now use the old path
as key, so renames from outside into the watched directory were
silently dropped. Now both old and new paths are checked, and
cross-boundary renames are converted to create or delete.
2. NewParentPath was passed to the sink without remapping to the
sink's target directory structure, causing the sink to write
entries at the wrong location. Now NewParentPath is remapped
alongside the key.
* fix filer sync to handle rename events crossing directory boundaries
The early directory-prefix filter only checked resp.Directory (old
parent). Rename events now carry the old parent as Directory, so
renames from outside the source path into it were dropped before
reaching the existing cross-boundary handling logic. Check both old
and new directories against sourcePath and excludePaths so the
downstream old-key/new-key logic can properly convert these to
create or delete operations.
* fix metadata event path matching
* fix metadata event consumers for rename targets
* Fix replication rename target keys
Logical rename events now reach replication sinks with distinct source and target paths.\n\nHandle non-filer sinks as delete-plus-create on the translated target key, and make the rename fallback path create at the translated target key too.\n\nAdd focused tests covering non-filer renames, filer rename updates, and the fallback path.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix filer sync rename path scoping
Use directory-boundary matching instead of raw prefix checks when classifying source and target paths during filer sync.\n\nAlso apply excludePaths per side so renames across excluded boundaries downgrade cleanly to create/delete instead of being misclassified as in-scope updates.\n\nAdd focused tests for boundary matching and rename classification.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix replicator directory boundary checks
Use directory-boundary matching instead of raw prefix checks when deciding whether a source or target path is inside the watched tree or an excluded subtree.\n\nThis prevents sibling paths such as /foo and /foobar from being misclassified during rename handling, and preserves the earlier rename-target-key fix.\n\nAdd focused tests for boundary matching and rename classification across sibling/excluded directories.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix etc-remote rename-out handling
Use boundary-safe source/target directory membership when classifying metadata events under DirectoryEtcRemote.\n\nThis prevents rename-out events from being processed as config updates, while still treating them as removals where appropriate for the remote sync and remote gateway command paths.\n\nAdd focused tests for update/removal classification and sibling-prefix handling.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Defer rename events until commit
Queue logical rename metadata events during atomic and streaming renames and publish them only after the transaction commits successfully.\n\nThis prevents subscribers from seeing delete or logical rename events for operations that later fail during delete or commit.\n\nAlso serialize notification.Queue swaps in rename tests and add failure-path coverage.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Skip descendant rename target lookups
Avoid redundant target lookups during recursive directory renames once the destination subtree is known absent.\n\nThe recursive move path now inserts known-absent descendants directly, and the test harness exercises prefixed directory listing so the optimization is covered by a directory rename regression test.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Tighten rename review tests
Return filer_pb.ErrNotFound from the bucket tracking store test stub so it follows the FilerStore contract, and add a webhook filter case for same-name renames across parent directories.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix HardLinkId format verb in InsertEntryKnownAbsent error
HardLinkId is a byte slice. %d prints each byte as a decimal number
which is not useful for an identifier. Use %x to match the log line
two lines above.
* only skip descendant target lookup when source and dest use same store
moveFolderSubEntries unconditionally passed skipTargetLookup=true for
every descendant. This is safe when all paths resolve to the same
underlying store, but with path-specific store configuration a child's
destination may map to a different backend that already holds an entry
at that path. Use FilerStoreWrapper.SameActualStore to check per-child
and fall back to the full CreateEntry path when stores differ.
* add nil and create edge-case tests for metadata event scope helpers
* extract pathIsEqualOrUnder into util.IsEqualOrUnder
Identical implementations existed in both replication/replicator.go and
command/filer_sync.go. Move to util.IsEqualOrUnder (alongside the
existing FullPath.IsUnder) and remove the duplicates.
* use MetadataEventTargetDirectory for new-side directory in filer sync
The new-side directory checks and sourceNewKey computation used
message.NewParentPath directly. If NewParentPath were empty (legacy
events, older filer versions during rolling upgrades), sourceNewKey
would be wrong (/filename instead of /dir/filename) and the
UpdateEntry parent path rewrite would panic on slice bounds.
Derive targetDir once from MetadataEventTargetDirectory, which falls
back to resp.Directory when NewParentPath is empty, and use it
consistently for all new-side checks and the sink parent path.
* Use Unix sockets for gRPC between co-located services in mini mode
In `weed mini`, all services run in one process. Previously, inter-service
gRPC traffic (volume↔master, filer↔master, S3↔filer, worker↔admin, etc.)
went through TCP loopback. This adds a gRPC Unix socket registry in the pb
package: mini mode registers a socket path per gRPC port at startup, each
gRPC server additionally listens on its socket, and GrpcDial transparently
routes to the socket via WithContextDialer when a match is found.
Standalone commands (weed master, weed filer, etc.) are unaffected since
no sockets are registered. TCP listeners are kept for external clients.
* Handle Serve error and clean up socket file in ServeGrpcOnLocalSocket
Log non-expected errors from grpcServer.Serve (ignoring
grpc.ErrServerStopped) and always remove the Unix socket file
when Serve returns, ensuring cleanup on Stop/GracefulStop.
After git reset --hard on a FUSE mount, the kernel dcache can
transiently show the directory then drop it moments later. Add a
1-second stabilisation delay and re-verification in
resetToCommitWithRecovery and tryPullFromCommit so that recovery
retries if the entry vanishes in that window.
Chris Lu
Chris Lu
Lars Lehtonen
qzh
dependabot[bot]
msementsov