f2db746690
Introduce logic to resolve volume replica placement within EC rebalancing. ( #6254 )
* Rename `command_ec_encode_test.go` to `command_ec_common_test.go`.
All tests defined in this file are now for `command_ec_common.go`.
* Minor code cleanups.
- Fix broken `ec.balance` test.
- Rework integer ceiling division to not use floats, which can introduce precision errors.
* Introduce logic to resolve volume replica placement within EC rebalancing.
This will be used to make rebalancing logic topology-aware.
* Give shell.EcNode.dc a dedicated DataCenterId type.
1 month ago
7b3c0e937f
3.80
1 month ago
180f267867
chore(deps): bump github.com/linxGnu/grocksdb from 1.9.5 to 1.9.7 ( #6249 )
Bumps [github.com/linxGnu/grocksdb](https://github.com/linxGnu/grocksdb ) from 1.9.5 to 1.9.7.
- [Release notes](https://github.com/linxGnu/grocksdb/releases )
- [Commits](https://github.com/linxGnu/grocksdb/compare/v1.9.5...v1.9.7 )
---
updated-dependencies:
- dependency-name: github.com/linxGnu/grocksdb
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 month ago
b21f599cde
chore(deps): bump modernc.org/sqlite from 1.33.1 to 1.34.1 ( #6251 )
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite ) from 1.33.1 to 1.34.1.
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.33.1...v1.34.1 )
---
updated-dependencies:
- dependency-name: modernc.org/sqlite
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 month ago
3fba0adc9d
chore(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.28.0 to 1.28.4 ( #6252 )
chore(deps): bump github.com/aws/aws-sdk-go-v2/config
Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2 ) from 1.28.0 to 1.28.4.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases )
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.28.0...config/v1.28.4 )
---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/config
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 month ago
f8f162aa72
chore(deps): bump go.etcd.io/etcd/client/v3 from 3.5.16 to 3.5.17 ( #6253 )
Bumps [go.etcd.io/etcd/client/v3](https://github.com/etcd-io/etcd ) from 3.5.16 to 3.5.17.
- [Release notes](https://github.com/etcd-io/etcd/releases )
- [Commits](https://github.com/etcd-io/etcd/compare/v3.5.16...v3.5.17 )
---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/client/v3
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 month ago
6d3e38f439
Adjust threshold config ( #6246 )
* fix:mount deadlock
* fix
---------
Co-authored-by: zemul <zhouzemiao@ihuman.com>
1 month ago
4a0dc883cf
Fix/parse upload filename ( #6241 )
* fix: parse filename in PUT + refactor
* fix: master iu public url with http
* fix: better parsing and handle disposition header
* fix: take mime type from file extension if not set
1 month ago
8ebbf37e8e
Add a loop to wait for master and filer services in post-install-bucket-hook ( #6240 )
Add a loop to wait for master and filer services before creating buckets
1 month ago
008ac38ebc
fix java.lang.IllegalArgumentException: Comparison method violates its general contract! ( #6239 )
1 month ago
a5fe6e21bc
feat(filer.backup): add ignore errors option ( #6235 )
* feat(filer.backup): add ignore errors option
* feat(filer.backup): fix 404 error wrap
* feat(filer.backup): fix wrapping function
* feat(filer.backup): fix wrapping errors in genProcessFunction
* Update weed/command/filer_backup.go
* Update weed/command/filer_backup.go
* Update weed/command/filer_backup.go
---------
Co-authored-by: Max Denushev <denushev@tochka.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
1 month ago
3003c9e17e
added backward compatible CRC check and extra explanatory comments ( #6233 )
1 month ago
31d8907e77
chore(deps): bump github.com/golang-jwt/jwt/v4 from 4.5.0 to 4.5.1 ( #6231 )
Bumps [github.com/golang-jwt/jwt/v4](https://github.com/golang-jwt/jwt ) from 4.5.0 to 4.5.1.
- [Release notes](https://github.com/golang-jwt/jwt/releases )
- [Changelog](https://github.com/golang-jwt/jwt/blob/main/VERSION_HISTORY.md )
- [Commits](https://github.com/golang-jwt/jwt/compare/v4.5.0...v4.5.1 )
---
updated-dependencies:
- dependency-name: github.com/golang-jwt/jwt/v4
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 month ago
7b0a37312d
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.66.2 to 1.66.3 ( #6225 )
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3
Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2 ) from 1.66.2 to 1.66.3.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases )
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.66.2...service/s3/v1.66.3 )
---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 month ago
623fc3a4fc
chore(deps): bump github.com/aws/aws-sdk-go-v2 from 1.32.3 to 1.32.4 ( #6227 )
Bumps [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2 ) from 1.32.3 to 1.32.4.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases )
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.32.3...v1.32.4 )
---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 month ago
aa31c5422b
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.89.2 to 3.91.0 ( #6228 )
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3
Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk ) from 3.89.2 to 3.91.0.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases )
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md )
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.89.2...v3.91.0 )
---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 month ago
6232041c64
chore(deps): bump golang.org/x/tools from 0.26.0 to 0.27.0 ( #6229 )
Bumps [golang.org/x/tools](https://github.com/golang/tools ) from 0.26.0 to 0.27.0.
- [Release notes](https://github.com/golang/tools/releases )
- [Commits](https://github.com/golang/tools/compare/v0.26.0...v0.27.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/tools
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 month ago
72b14a451e
delete aborted ec shards from both source and target servers ( #6221 )
fix https://github.com/seaweedfs/seaweedfs/issues/6205#issuecomment-2465004586
1 month ago
3400962877
[helm] fix s3 servicemonitor label matching ( #6218 )
1 month ago
113c9ce6a8
remove the direct_io flag, as it is not well-supported on macOS
2 months ago
88a1eee31c
adjust logs
2 months ago
62c4980754
add debug info
2 months ago
9a5d3e7b31
[shell] add admin noLock for balance ( #6209 )
add admin noLock for balance
2 months ago
15fc58bfd9
remove dup code for section.reader access ( #6169 )
2 months ago
706d1a8e22
fix tests
2 months ago
90acfd9394
remove case when read request is out side of the file size
2 months ago
6161d0b05d
a bit optimization
2 months ago
98b519b113
fix FUSE mount on mac
2 months ago
efdebf712e
Refactor `ec.balance` logic into a `weeed/shell/command_ec_common.go`… ( #6195 )
* Refactor `ec.balance` logic into a `weeed/shell/command_ec_common.go` standalone function.
This is a prerequisite to unify the balance logic for `ec.balance` and `ec.encode'.
* s/Balance()/EcBalance()/g
2 months ago
dc784bf217
merge current message queue code changes ( #6201 )
* listing files to convert to parquet
* write parquet files
* save logs into parquet files
* pass by value
* compact logs into parquet format
* can skip existing files
* refactor
* refactor
* fix compilation
* when no partition found
* refactor
* add untested parquet file read
* rename package
* refactor
* rename files
* remove unused
* add merged log read func
* parquet wants to know the file size
* rewind by time
* pass in stop ts
* add stop ts
* adjust log
* minor
* adjust log
* skip .parquet files when reading message logs
* skip non message files
* Update subscriber_record.go
* send messages
* skip message data with only ts
* skip non log files
* update parquet-go package
* ensure a valid record type
* add new field to a record type
* Update read_parquet_to_log.go
* fix parquet file name generation
* separating reading parquet and logs
* add key field
* add skipped logs
* use in memory cache
* refactor
* refactor
* refactor
* refactor, and change compact log
* refactor
* rename
* refactor
* fix format
* prefix v to version directory
2 months ago
ffe908371d
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.61.2 to 1.66.2 ( #6197 )
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3
Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2 ) from 1.61.2 to 1.66.2.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases )
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.61.2...service/s3/v1.66.2 )
---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2 months ago
92366134e3
chore(deps): bump github.com/gocql/gocql from 1.6.0 to 1.7.0 ( #6196 )
Bumps [github.com/gocql/gocql](https://github.com/gocql/gocql ) from 1.6.0 to 1.7.0.
- [Release notes](https://github.com/gocql/gocql/releases )
- [Changelog](https://github.com/apache/cassandra-gocql-driver/blob/trunk/CHANGELOG.md )
- [Commits](https://github.com/gocql/gocql/compare/v1.6.0...v1.7.0 )
---
updated-dependencies:
- dependency-name: github.com/gocql/gocql
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
e2d30cc3bc
chore(deps): bump gocloud.dev/pubsub/natspubsub from 0.39.0 to 0.40.0 ( #6198 )
Bumps [gocloud.dev/pubsub/natspubsub](https://github.com/google/go-cloud ) from 0.39.0 to 0.40.0.
- [Release notes](https://github.com/google/go-cloud/releases )
- [Commits](https://github.com/google/go-cloud/compare/v0.39.0...v0.40.0 )
---
updated-dependencies:
- dependency-name: gocloud.dev/pubsub/natspubsub
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
d185e6beea
chore(deps): bump golang.org/x/tools from 0.25.0 to 0.26.0 ( #6199 )
Bumps [golang.org/x/tools](https://github.com/golang/tools ) from 0.25.0 to 0.26.0.
- [Release notes](https://github.com/golang/tools/releases )
- [Commits](https://github.com/golang/tools/compare/v0.25.0...v0.26.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/tools
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
f67b5b20ca
chore(deps): bump cloud.google.com/go/pubsub from 1.43.0 to 1.45.1 ( #6200 )
Bumps [cloud.google.com/go/pubsub](https://github.com/googleapis/google-cloud-go ) from 1.43.0 to 1.45.1.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.43.0...pubsub/v1.45.1 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/pubsub
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
a7973ed7d1
fix deadlock hang when broadcast to clients ( #6184 )
fix deadlock when broadcast to clients
when master thransfer leader, the old master will disconnect with all
filers and volumeServers, if the cluster is a big , the broadcast
messages may be more big than the max of the channel len 100, then if the
KeepConnect was not listen on the channel in disconnect, it will
deadlock. and the whole cluster will not serve!
2 months ago
0f2c3648dc
fix mount: cannot locate OSXFUSE
addressing https://github.com/seaweedfs/seaweedfs/discussions/6193
2 months ago
5bddf0c085
[shell] volume.balance collect volume servers by dc rack node ( #6191 )
* chore: balance by rack
* fix: rm check lock
* fix: selected racks
* fix: selected nodes
* fix: containts
* fix: one collectVolumeServersByDcRackNode
* fix: revert lock and add lock
* fix: panic test
* revert noLock
2 months ago
65fb8fad99
Update filer_multipart.go ( #6188 )
fix the #6177 bug
2 months ago
0060a2cf9c
Fix 6181/6182 ( #6183 )
* set larger buf size for LogBuffer
* jump to next day when no more entry found
* Update weed/filer/filer_notify_read.go
---------
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2 months ago
c29c912bdc
fix format ( #6185 )
unitest weed/shell fail
2 months ago
ad6240f066
fix parameter
2 months ago
8802843511
pass in bytes buffer to avoid possible race condition
related to https://github.com/seaweedfs/seaweedfs/issues/6180
2 months ago
3feb66d0a1
adding more debug message
related to https://github.com/seaweedfs/seaweedfs/issues/6180
2 months ago
228946369c
3.79
2 months ago
9873b033d1
backward compatible vif loading
2 months ago
f17048d32a
chore(deps): bump github.com/redis/go-redis/v9 from 9.6.2 to 9.7.0 ( #6174 )
Bumps [github.com/redis/go-redis/v9](https://github.com/redis/go-redis ) from 9.6.2 to 9.7.0.
- [Release notes](https://github.com/redis/go-redis/releases )
- [Changelog](https://github.com/redis/go-redis/blob/master/CHANGELOG.md )
- [Commits](https://github.com/redis/go-redis/compare/v9.6.2...v9.7.0 )
---
updated-dependencies:
- dependency-name: github.com/redis/go-redis/v9
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
cd789d869d
chore(deps): bump modernc.org/sqlite from 1.33.0 to 1.33.1 ( #6173 )
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite ) from 1.33.0 to 1.33.1.
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.33.0...v1.33.1 )
---
updated-dependencies:
- dependency-name: modernc.org/sqlite
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
1f9f6fae6b
chore(deps): bump cloud.google.com/go/storage from 1.43.0 to 1.45.0 ( #6172 )
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go ) from 1.43.0 to 1.45.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.43.0...pubsub/v1.45.0 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
c74dca10cb
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.77.1 to 3.89.2 ( #6171 )
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3
Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk ) from 3.77.1 to 3.89.2.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases )
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md )
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.77.1...v3.89.2 )
---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
Lisandro Pin
chrislu
dependabot[bot]
zemul
Kevin Bulteel
Omar Hamadeh
fo40225
Max Denushev
Eugeniy E. Mikhailov
Chris Lu
Nathan Dorsey
Konstantin Lebedev
Bruce
wyang
han