ba1d82db90
Move `shell.ErrorWaitGroup` into a common file, to cleanly reuse across `weed shell` commands. ( #6780 )
Move `shell.ErrorWaitGroup` into a dedicated common file, to cleanly reuse across `weed shell` commands.
2 days ago
b0f7c33ce0
chore(deps): bump github.com/redis/go-redis/v9 from 9.7.3 to 9.8.0 ( #6781 )
Bumps [github.com/redis/go-redis/v9](https://github.com/redis/go-redis ) from 9.7.3 to 9.8.0.
- [Release notes](https://github.com/redis/go-redis/releases )
- [Changelog](https://github.com/redis/go-redis/blob/master/CHANGELOG.md )
- [Commits](https://github.com/redis/go-redis/compare/v9.7.3...v9.8.0 )
---
updated-dependencies:
- dependency-name: github.com/redis/go-redis/v9
dependency-version: 9.8.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 days ago
c993a43a91
chore(deps): bump golang.org/x/tools from 0.31.0 to 0.33.0 ( #6782 )
Bumps [golang.org/x/tools](https://github.com/golang/tools ) from 0.31.0 to 0.33.0.
- [Release notes](https://github.com/golang/tools/releases )
- [Commits](https://github.com/golang/tools/compare/v0.31.0...v0.33.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/tools
dependency-version: 0.33.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 days ago
6eaa35040b
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.106.1 to 3.108.3 ( #6783 )
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3
Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk ) from 3.106.1 to 3.108.3.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases )
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md )
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.106.1...v3.108.3 )
---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
dependency-version: 3.108.3
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 days ago
8da3079d22
chore(deps): bump actions/dependency-review-action from 4.6.0 to 4.7.0 ( #6784 )
Bumps [actions/dependency-review-action](https://github.com/actions/dependency-review-action ) from 4.6.0 to 4.7.0.
- [Release notes](https://github.com/actions/dependency-review-action/releases )
- [Commits](ce3cf9537a
2 days ago
45964c2f86
fix: ydb filer bugs ( #6778 )
* fix: ydb filer bugs
* fix(ydb): correct DeleteEntry log argument types
* fix(ydb): bucket creation & deletion logic
3 days ago
9c1048bacb
Add prefix listing in mongodb_store.go ( #6777 )
3 days ago
dddb0f0ae5
Fix update of `SeaweedFS_volumeServer_volumes` gauge metrics when EC shards are unmounted ( #6776 )
5 days ago
d8cc269294
feature: added ssl support for HCFS ( #6699 ) ( #6775 )
5 days ago
848d1f7c34
Improve safety for weed shell's `ec.encode`. ( #6773 )
Improve safety for weed shells `ec.encode`.
The current process for `ec.encode` is:
1. EC shards for a volume are generated and added to a single server
2. The original volume is deleted
3. EC shards get re-balanced across the entire topology
It is then possible to lose data between #2 and #3 , if the underlying volume storage/server/rack/DC
happens to fail, for whatever reason. As a fix, this MR reworks `ec.encode` so:
* Newly created EC shards are spread across all locations for the source volume.
* Source volumes are deleted only after EC shards are converted and balanced.
5 days ago
2ae5b480a6
Use the correct constant when computing the offset in SearchNeedleFromSortedIndex ( #6771 )
NeedleHeaderSize happen to have the same size as NeedleMapEntrySize, except when running the 5 bytes offset variant of Seaweedfs, because it does not contain OffsetSize. This causes ECX corruption on deletes, due to the drifting offset computation (offset is always computed on a basis of 16 bytes per record instead of 17 bytes)
Signed-off-by: Quentin Devos <4972091+Okhoshi@users.noreply.github.com>
5 days ago
97dad06ed8
Improve parallelization for `ec.encode` ( #6769 )
Improve parallelization for `ec.encode`.
Instead of processing one volume at at time, perform all EC conversion
steps (mark readonly -> generate EC shards -> delete volume -> remount) in
parallel for all of them.
This should substantially improve performance when EC encoding
entire collections.
6 days ago
582206b7d2
load configuration before get max_file_name_length ( #6766 )
7 days ago
7522f11fd7
chore(deps): bump github.com/go-sql-driver/mysql from 1.9.1 to 1.9.2 ( #6720 )
Bumps [github.com/go-sql-driver/mysql](https://github.com/go-sql-driver/mysql ) from 1.9.1 to 1.9.2.
- [Release notes](https://github.com/go-sql-driver/mysql/releases )
- [Changelog](https://github.com/go-sql-driver/mysql/blob/master/CHANGELOG.md )
- [Commits](https://github.com/go-sql-driver/mysql/compare/v1.9.1...v1.9.2 )
---
updated-dependencies:
- dependency-name: github.com/go-sql-driver/mysql
dependency-version: 1.9.2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 week ago
3c15c8ddad
fix: remove blocking match label ( #6760 )
matchLabels are immutable. Thus, matching against the version, which
changes with every release, blocks Helm upgrades. To resolve this, the
label is removed. See [1] where this was similarly done.
[1] https://github.com/seaweedfs/seaweedfs/pull/6090
1 week ago
93aed187e9
Add SFTP Server Support ( #6753 )
* Add SFTP Server Support
Signed-off-by: Mohamed Sekour <mohamed.sekour@exfo.com>
* fix s3 tests and helm lint
Signed-off-by: Mohamed Sekour <mohamed.sekour@exfo.com>
* increase helm chart version
* adjust version
---------
Signed-off-by: Mohamed Sekour <mohamed.sekour@exfo.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
1 week ago
a2c5510ae1
Addendum to issue #6733 ( #6756 )
add s3.ip.bind command line parameter to filer
1 week ago
a2473af05a
chore(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.29.13 to 1.29.14 ( #6757 )
chore(deps): bump github.com/aws/aws-sdk-go-v2/config
Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2 ) from 1.29.13 to 1.29.14.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases )
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json )
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.29.13...config/v1.29.14 )
---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/config
dependency-version: 1.29.14
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 week ago
51b0f9b7fd
chore(deps): bump github.com/prometheus/procfs from 0.16.0 to 0.16.1 ( #6758 )
Bumps [github.com/prometheus/procfs](https://github.com/prometheus/procfs ) from 0.16.0 to 0.16.1.
- [Release notes](https://github.com/prometheus/procfs/releases )
- [Commits](https://github.com/prometheus/procfs/compare/v0.16.0...v0.16.1 )
---
updated-dependencies:
- dependency-name: github.com/prometheus/procfs
dependency-version: 0.16.1
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 week ago
5aff086aa4
chore(deps): bump actions/dependency-review-action from 4.3.2 to 4.6.0 ( #6759 )
Bumps [actions/dependency-review-action](https://github.com/actions/dependency-review-action ) from 4.3.2 to 4.6.0.
- [Release notes](https://github.com/actions/dependency-review-action/releases )
- [Commits](0c155c5e85
1 week ago
e4e8505208
3.87
1 week ago
ed15357c7a
adds tls examples
related to https://github.com/seaweedfs/seaweedfs/pull/6738
1 week ago
21514e1fea
feat(redis): add mTLS support for Redis connection initialization ( #6738 )
* feat(redis): add mTLS support for Redis connection initialization
- Enhanced the Redis2Store initialization to support mutual TLS (mTLS) by adding configuration options for CA certificate, client certificate, and client key paths.
- Updated the Redis client setup to use TLS configuration when mTLS is enabled, ensuring secure connections to the Redis server.
* feat(redis): extend Redis3Store initialization to support mTLS
- Added configuration options for enabling mutual TLS (mTLS) in Redis3Store.
- Implemented logic to load client certificates and CA certificates for secure Redis connections.
- Updated the Redis client setup to utilize TLS configuration when mTLS is enabled.
---------
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
1 week ago
4e7948abd2
chore(deps): bump github.com/go-jose/go-jose/v4 from 4.0.4 to 4.0.5 ( #6750 )
Bumps [github.com/go-jose/go-jose/v4](https://github.com/go-jose/go-jose ) from 4.0.4 to 4.0.5.
- [Release notes](https://github.com/go-jose/go-jose/releases )
- [Changelog](https://github.com/go-jose/go-jose/blob/main/CHANGELOG.md )
- [Commits](https://github.com/go-jose/go-jose/compare/v4.0.4...v4.0.5 )
---
updated-dependencies:
- dependency-name: github.com/go-jose/go-jose/v4
dependency-version: 4.0.5
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 weeks ago
b45f43ded1
chore(deps): bump google.golang.org/api from 0.228.0 to 0.230.0 ( #6742 )
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.228.0 to 0.230.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.228.0...v0.230.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-version: 0.230.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 weeks ago
f14de131b3
chore(deps): bump docker/build-push-action from 6.15.0 to 6.16.0 ( #6743 )
Bumps [docker/build-push-action](https://github.com/docker/build-push-action ) from 6.15.0 to 6.16.0.
- [Release notes](https://github.com/docker/build-push-action/releases )
- [Commits](471d1dc4e0
2 weeks ago
d424cbbcd5
fix: S3 Feature: please add s3.idleTimeout command line parameter #6746 ( #6747 )
ildeTimeout command line parameter for s3
2 weeks ago
71382d3279
bump depdency-review-action 4.3.0 -> 4.3.2 ( #6732 )
2 weeks ago
0fbdedecc7
[s3] refactor duplicate put-object code paths ( #6727 )
2 weeks ago
affacb1b67
[s3] add tests for s3 over https using aws-cli ( #6731 )
* [s3] add tests for s3 over https using aws-cli
* fix typo
2 weeks ago
9f4be83e88
fix #6733 add s3.ip.bind command line parameter ( #6741 )
2 weeks ago
8d0e6f1ead
fix: volume.list volume info output not in order ( #6737 )
2 weeks ago
c8b0a04409
golang up version to 1.24 ( #6724 )
3 weeks ago
ea941462f9
[s3] use chunkedReader in PutObjectPartHandler to handle aws-chunked encoding ( #6725 )
3 weeks ago
8c1cb203fa
[s3] fix checksum algorithm CRC64NVMe ( #6722 )
3 weeks ago
cea34dc21a
Fix implementation of `master_pb.CollectionList` RPC call ( #6715 )
4 weeks ago
df6f230681
chore(deps): bump gocloud.dev/pubsub/natspubsub from 0.40.0 to 0.41.0 ( #6707 )
Bumps [gocloud.dev/pubsub/natspubsub](https://github.com/google/go-cloud ) from 0.40.0 to 0.41.0.
- [Release notes](https://github.com/google/go-cloud/releases )
- [Commits](https://github.com/google/go-cloud/compare/v0.40.0...v0.41.0 )
---
updated-dependencies:
- dependency-name: gocloud.dev/pubsub/natspubsub
dependency-version: 0.41.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 month ago
7a97bea3c0
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.104.1 to 3.106.1 ( #6705 )
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3
Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk ) from 3.104.1 to 3.106.1.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases )
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md )
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.104.1...v3.106.1 )
---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
dependency-version: 3.106.1
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 month ago
23fe412ee3
chore(deps): bump cloud.google.com/go/pubsub from 1.48.1 to 1.49.0 ( #6706 )
Bumps [cloud.google.com/go/pubsub](https://github.com/googleapis/google-cloud-go ) from 1.48.1 to 1.49.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.48.1...pubsub/v1.49.0 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/pubsub
dependency-version: 1.49.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 month ago
ae73bc9685
chore(deps): bump google.golang.org/grpc from 1.71.0 to 1.71.1 ( #6708 )
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go ) from 1.71.0 to 1.71.1.
- [Release notes](https://github.com/grpc/grpc-go/releases )
- [Commits](https://github.com/grpc/grpc-go/compare/v1.71.0...v1.71.1 )
---
updated-dependencies:
- dependency-name: google.golang.org/grpc
dependency-version: 1.71.1
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 month ago
396a602f86
* Fix s3 auth failed with X-Forwarded-Host and X-Forwarded-Port ( #6698 )
1 month ago
2e0e5c09cd
chore(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.29.12 to 1.29.13 ( #6692 )
chore(deps): bump github.com/aws/aws-sdk-go-v2/config
Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2 ) from 1.29.12 to 1.29.13.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases )
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json )
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.29.12...config/v1.29.13 )
---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/config
dependency-version: 1.29.13
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
1 month ago
ad138a0c0e
chore(deps): bump golang.org/x/tools from 0.30.0 to 0.31.0 ( #6691 )
Bumps [golang.org/x/tools](https://github.com/golang/tools ) from 0.30.0 to 0.31.0.
- [Release notes](https://github.com/golang/tools/releases )
- [Commits](https://github.com/golang/tools/compare/v0.30.0...v0.31.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/tools
dependency-version: 0.31.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
1 month ago
999ca04e64
chore(deps): bump github.com/spf13/viper from 1.19.0 to 1.20.1 ( #6690 )
Bumps [github.com/spf13/viper](https://github.com/spf13/viper ) from 1.19.0 to 1.20.1.
- [Release notes](https://github.com/spf13/viper/releases )
- [Commits](https://github.com/spf13/viper/compare/v1.19.0...v1.20.1 )
---
updated-dependencies:
- dependency-name: github.com/spf13/viper
dependency-version: 1.20.1
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
1 month ago
183ab487c6
chore(deps): bump golang.org/x/sync from 0.12.0 to 0.13.0 ( #6693 )
Bumps [golang.org/x/sync](https://github.com/golang/sync ) from 0.12.0 to 0.13.0.
- [Commits](https://github.com/golang/sync/compare/v0.12.0...v0.13.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/sync
dependency-version: 0.13.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
1 month ago
aa9121aa9d
Revert "[filer] void panic if s3opt.tlsCACertificate is nill" ( #6694 )
Revert "[filer] void panic if s3opt.tlsCACertificate is nill (#6685 )"
This reverts commit 4dca59ed27
1 month ago
4dca59ed27
[filer] void panic if s3opt.tlsCACertificate is nill ( #6685 )
* fix issues/6684
* add s3.cacert.file option for filer
1 month ago
cc67a0370c
chore(deps): bump cloud.google.com/go/pubsub from 1.48.0 to 1.48.1 ( #6686 )
Bumps [cloud.google.com/go/pubsub](https://github.com/googleapis/google-cloud-go ) from 1.48.0 to 1.48.1.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.48.0...pubsub/v1.48.1 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/pubsub
dependency-version: 1.48.1
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 month ago
43ffbfe608
chore(deps): bump gocloud.dev/pubsub/rabbitpubsub from 0.40.0 to 0.41.0 ( #6687 )
Bumps [gocloud.dev/pubsub/rabbitpubsub](https://github.com/google/go-cloud ) from 0.40.0 to 0.41.0.
- [Release notes](https://github.com/google/go-cloud/releases )
- [Commits](https://github.com/google/go-cloud/compare/v0.40.0...v0.41.0 )
---
updated-dependencies:
- dependency-name: gocloud.dev/pubsub/rabbitpubsub
dependency-version: 0.41.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 month ago
b10791f8a0
chore(deps): bump github.com/prometheus/procfs from 0.15.1 to 0.16.0 ( #6688 )
Bumps [github.com/prometheus/procfs](https://github.com/prometheus/procfs ) from 0.15.1 to 0.16.0.
- [Release notes](https://github.com/prometheus/procfs/releases )
- [Commits](https://github.com/prometheus/procfs/compare/v0.15.1...v0.16.0 )
---
updated-dependencies:
- dependency-name: github.com/prometheus/procfs
dependency-version: 0.16.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 month ago
Lisandro Pin
dependabot[bot]
SmoothDenis
Kuzmin Anton
orthoxerox
Quentin D.
Bruce Zou
Manuel Leonhardt
Mohamed Sekour
ARibster
chrislu
Jade Devin Cabatlao
Taehyung Lim
NyaMisty
Konstantin Lebedev
zuzuviewer