bfd59dd579
adjust import
3 days ago
ec155022e7
"golang.org/x/exp/slices" => "slices" and go fmt
3 days ago
a1a76ccb8c
Fix for DeleteMultipleObjectsHandler wrongly deleting parent folders ( #6380 )
What problem are we solving?
Fix : #6379
How are we solving the problem?
We check for the AllowEmptyFolders option prior to cascade
deleting parent folders in S3 DeleteMultipleObjectsHandler.
How is the PR tested?
We ran SeaweedFS in a Kubernetes Cluster with a joint Filer
and S3 server in one container, with leveldb2 as the filer storage,
and AllowEmptyFolders set to true.
When using the Distribution Registry as the S3 client, it calls the
DeleteMultipleObjectsHandler as part of the artifact upload process
(uploads to a temp location, then performs a copy and delete).
Without this fix, the deletion cascade deleted parent folder until
the entire contents of the bucket were gone.
With this fix, the existing content of the bucket remained, and the
newly uploaded content was added.
Checks
[ ] I have added unit tests if possible.
[ ] I will add related wiki document changes and link to this PR after merging.
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
3 days ago
4d91ec359b
Fix volume replica parallelization within `ec.encode`. ( #6377 )
See 826edd5d
3 days ago
ba0707af64
Allow configuring the maximum number of concurrent tasks for EC parallelization. ( #6376 )
Follow-up to b0210df0
4 days ago
44c48c929a
Parallelize volume replica operations within `ec.encode`. ( #6374 )
4 days ago
72af97162f
[shell] feat:stop vacuum immediately once volume.vacuum.disable was executed ( #6375 )
stop vacuum immediately once volume.vacuum.disable was executed
Co-authored-by: dsd <dsd2019@foxmail.com>
4 days ago
9fbc4ea417
Rework `shell.EcBalance()`'s waitgroup code into a standalone type. ( #6373 )
Rework `shell.EcBalance()`'s waitgroup with errors code into a standalone type.
We'll re-use this for other EC jobs - for example, volume creation. Also fixes
potential concurrency issues when collecting error results.
5 days ago
e77e50886e
mount metacache add ttl ( #6360 )
* fix:mount deadlock
* fix
* feat: metaCache ttl
* Update weed/command/mount.go
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
* fix InodeEntry
---------
Co-authored-by: zemul <zhouzemiao@ihuman.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
6 days ago
b2f26804a0
chore(deps): bump docker/setup-buildx-action from 3.7.1 to 3.8.0 ( #6369 )
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action ) from 3.7.1 to 3.8.0.
- [Release notes](https://github.com/docker/setup-buildx-action/releases )
- [Commits](c47758b77c
6 days ago
52cf759a96
chore(deps): bump golang.org/x/crypto from 0.30.0 to 0.31.0 ( #6370 )
Bumps [golang.org/x/crypto](https://github.com/golang/crypto ) from 0.30.0 to 0.31.0.
- [Commits](https://github.com/golang/crypto/compare/v0.30.0...v0.31.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/crypto
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
6 days ago
93da4fe4d5
chore(deps): bump wangyoucao577/go-release-action from 1.52 to 1.53 ( #6368 )
Bumps [wangyoucao577/go-release-action](https://github.com/wangyoucao577/go-release-action ) from 1.52 to 1.53.
- [Release notes](https://github.com/wangyoucao577/go-release-action/releases )
- [Commits](2aa2977ad6
6 days ago
fa2d28c998
chore(deps): bump helm/kind-action from 1.10.0 to 1.11.0 ( #6367 )
Bumps [helm/kind-action](https://github.com/helm/kind-action ) from 1.10.0 to 1.11.0.
- [Release notes](https://github.com/helm/kind-action/releases )
- [Commits](https://github.com/helm/kind-action/compare/v1.10.0...v1.11.0 )
---
updated-dependencies:
- dependency-name: helm/kind-action
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
6 days ago
641fbc7a80
chore(deps): bump cloud.google.com/go/storage from 1.45.0 to 1.48.0 ( #6366 )
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go ) from 1.45.0 to 1.48.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.45.0...spanner/v1.48.0 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
6 days ago
ab7b81bbad
chore(deps): bump github.com/schollz/progressbar/v3 from 3.16.0 to 3.17.1 ( #6365 )
chore(deps): bump github.com/schollz/progressbar/v3
Bumps [github.com/schollz/progressbar/v3](https://github.com/schollz/progressbar ) from 3.16.0 to 3.17.1.
- [Release notes](https://github.com/schollz/progressbar/releases )
- [Commits](https://github.com/schollz/progressbar/compare/v3.16.0...v3.17.1 )
---
updated-dependencies:
- dependency-name: github.com/schollz/progressbar/v3
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
6 days ago
cbc3afe7d1
chore(deps): bump go.mongodb.org/mongo-driver from 1.16.0 to 1.17.1 ( #6364 )
Bumps [go.mongodb.org/mongo-driver](https://github.com/mongodb/mongo-go-driver ) from 1.16.0 to 1.17.1.
- [Release notes](https://github.com/mongodb/mongo-go-driver/releases )
- [Commits](https://github.com/mongodb/mongo-go-driver/compare/v1.16.0...v1.17.1 )
---
updated-dependencies:
- dependency-name: go.mongodb.org/mongo-driver
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
6 days ago
b4bcedf064
chore(deps): bump github.com/getsentry/sentry-go from 0.29.1 to 0.30.0 ( #6363 )
Bumps [github.com/getsentry/sentry-go](https://github.com/getsentry/sentry-go ) from 0.29.1 to 0.30.0.
- [Release notes](https://github.com/getsentry/sentry-go/releases )
- [Changelog](https://github.com/getsentry/sentry-go/blob/master/CHANGELOG.md )
- [Commits](https://github.com/getsentry/sentry-go/compare/v0.29.1...v0.30.0 )
---
updated-dependencies:
- dependency-name: github.com/getsentry/sentry-go
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
6 days ago
7a92b2cbb9
chore(deps): bump github.com/fclairamb/ftpserverlib from 0.24.1 to 0.25.0 ( #6362 )
chore(deps): bump github.com/fclairamb/ftpserverlib
Bumps [github.com/fclairamb/ftpserverlib](https://github.com/fclairamb/ftpserverlib ) from 0.24.1 to 0.25.0.
- [Release notes](https://github.com/fclairamb/ftpserverlib/releases )
- [Commits](https://github.com/fclairamb/ftpserverlib/compare/v0.24.1...v0.25.0 )
---
updated-dependencies:
- dependency-name: github.com/fclairamb/ftpserverlib
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
6 days ago
c531e2eb0e
Update disk_location_ec.go ( #6359 )
Add lock for l.ecVolumes
7 days ago
7eb8257126
Update store.go ( #6358 )
Add lock for location.ecVolumes And single delete
7 days ago
9b48ce0613
Parallelize EC shards balancing within racks ( #6354 )
Parallelize EC shards balancing within racks.
7 days ago
926cfea3dc
Parallelize EC shards balancing across racks. ( #6352 )
1 week ago
b81def5e5c
Parallelize EC balancing for racks. ( #6351 )
1 week ago
d6f3e1970d
fix: filer may crash by bucketLastActiveTsNs concurrency access. ( #6350 )
1 week ago
0a4b1909a2
[shell] only apply the balancing for writable volumes ( #6346 )
1 week ago
9987a65e8a
fix: record and delete bucket metrics after inactive ( #6349 )
1 week ago
b0210df081
Begin implementing EC balancing parallelization support. ( #6342 )
* Begin implementing EC balancing parallelization support.
Impacts both `ec.encode` and `ec.balance`,
* Nit: improve type naming.
* Make the goroutine workgroup handler for `EcBalance()` a bit smarter/error-proof.
* Nit: unify naming for `ecBalancer` wait group methods with the rest of the module.
* Fix concurrency bug.
* Fix whitespace after Gitlab automerge.
* Delete stray TODO.
1 week ago
23ffbb083c
Limit EC re-balancing for `ec.encode` to relevant collections when a volume ID argument is provided. ( #6347 )
Limit EC re-balancing for `ec.encode` to relevant collections when a volume ID is provided.
1 week ago
6320036c56
Delete legacy balancing code for `ec.encode`. ( #6344 )
1 week ago
700b95304b
[shell] volume.list show only writable volumes ( #6338 )
* show only writable volumes
* fix import
2 weeks ago
c37281735e
volume.list avoid output empty data center and rack and disk info ( #6341 )
2 weeks ago
8c82c037b9
Unify the re-balancing logic for `ec.encode` with `ec.balance`. ( #6339 )
Among others, this enables recent changes related to topology aware
re-balancing at EC encoding time.
2 weeks ago
ff1392f7f4
[shell] use constant for hdd of type ( #6337 )
use constant for hdd of type
2 weeks ago
4f6c989309
chore(deps): bump golang.org/x/net from 0.31.0 to 0.32.0 ( #6330 )
Bumps [golang.org/x/net](https://github.com/golang/net ) from 0.31.0 to 0.32.0.
- [Commits](https://github.com/golang/net/compare/v0.31.0...v0.32.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/net
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 weeks ago
af4c51aee6
chore(deps): bump cloud.google.com/go/pubsub from 1.45.1 to 1.45.3 ( #6328 )
Bumps [cloud.google.com/go/pubsub](https://github.com/googleapis/google-cloud-go ) from 1.45.1 to 1.45.3.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.45.1...pubsub/v1.45.3 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/pubsub
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 weeks ago
3cd2f06279
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.69.0 to 1.71.0 ( #6329 )
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3
Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2 ) from 1.69.0 to 1.71.0.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases )
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.69.0...service/s3/v1.71.0 )
---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 weeks ago
e438fdc16c
chore(deps): bump github.com/hanwen/go-fuse/v2 from 2.7.1 to 2.7.2 ( #6331 )
Bumps [github.com/hanwen/go-fuse/v2](https://github.com/hanwen/go-fuse ) from 2.7.1 to 2.7.2.
- [Commits](https://github.com/hanwen/go-fuse/compare/v2.7.1...v2.7.2 )
---
updated-dependencies:
- dependency-name: github.com/hanwen/go-fuse/v2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 weeks ago
3680bbac56
chore(deps): bump golang.org/x/sys from 0.27.0 to 0.28.0 ( #6332 )
Bumps [golang.org/x/sys](https://github.com/golang/sys ) from 0.27.0 to 0.28.0.
- [Commits](https://github.com/golang/sys/compare/v0.27.0...v0.28.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/sys
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 weeks ago
67461ccd27
chore(deps): bump docker/build-push-action from 6.9.0 to 6.10.0 ( #6333 )
Bumps [docker/build-push-action](https://github.com/docker/build-push-action ) from 6.9.0 to 6.10.0.
- [Release notes](https://github.com/docker/build-push-action/releases )
- [Commits](4f58ea7922
2 weeks ago
522a25790a
Remove average constraints when selecting nodes/racks to balance EC shards into. ( #6325 )
2 weeks ago
914ff0e718
Replaced https.client.enabled with false in scaffold/security ( #6320 )
2 weeks ago
f3dde99796
adjust error message
2 weeks ago
34cdbdd279
Share common parameters for EC re-balancing functions under a single struct. ( #6319 )
TODO cleanup for https://github.com/seaweedfs/seaweedfs/discussions/6179 .
2 weeks ago
edef485333
Account for replication placement settings when balancing EC shards within the same rack. ( #6317 )
* Account for replication placement settings when balancing EC shards within racks.
* Update help contents for `ec.balance`.
* Add a few more representative test cases for `pickEcNodeToBalanceShardsInto()`.
3 weeks ago
351efa134d
Account for replication placement settings when balancing EC shards across racks. ( #6316 )
3 weeks ago
b65eb2ec45
[security] reload whiteList on http seerver ( #6302 )
* reload whiteList
* white_list add to scaffold
3 weeks ago
b2ba7d7408
Resolve replica placement for EC volumes from master server defaults. ( #6303 )
3 weeks ago
9a741a61b1
Display details upon failures to re-balance EC shards racks. ( #6299 )
3 weeks ago
559a1fd0f4
Improve EC shards rebalancing logic across nodes ( #6297 )
* Improve EC shards rebalancing logic across nodes.
- Favor target nodes with less preexisting shards, to ensure a fair distribution.
- Randomize selection when multiple possible target nodes are available.
- Add logic to account for replication settings when selecting target nodes (currently disabled).
* Fix minor test typo.
* Clarify internal error messages for `pickEcNodeToBalanceShardsInto()`.
4 weeks ago
88fa64a01a
change comment for expect header ( #6289 )
* remove case of expect
* Set the default value of the Expect header for compatibility.
4 weeks ago
chrislu
Warren Hodgkinson
Lisandro Pin
dsd
zemul
dependabot[bot]
coffeecloudgit
zouyixiong
Konstantin Lebedev
vadimartynov
steve.wei