926cfea3dc
Parallelize EC shards balancing across racks. ( #6352 )
11 months ago
b81def5e5c
Parallelize EC balancing for racks. ( #6351 )
11 months ago
d6f3e1970d
fix: filer may crash by bucketLastActiveTsNs concurrency access. ( #6350 )
11 months ago
0a4b1909a2
[shell] only apply the balancing for writable volumes ( #6346 )
11 months ago
9987a65e8a
fix: record and delete bucket metrics after inactive ( #6349 )
11 months ago
b0210df081
Begin implementing EC balancing parallelization support. ( #6342 )
* Begin implementing EC balancing parallelization support.
Impacts both `ec.encode` and `ec.balance`,
* Nit: improve type naming.
* Make the goroutine workgroup handler for `EcBalance()` a bit smarter/error-proof.
* Nit: unify naming for `ecBalancer` wait group methods with the rest of the module.
* Fix concurrency bug.
* Fix whitespace after Gitlab automerge.
* Delete stray TODO.
11 months ago
23ffbb083c
Limit EC re-balancing for `ec.encode` to relevant collections when a volume ID argument is provided. ( #6347 )
Limit EC re-balancing for `ec.encode` to relevant collections when a volume ID is provided.
11 months ago
6320036c56
Delete legacy balancing code for `ec.encode`. ( #6344 )
11 months ago
700b95304b
[shell] volume.list show only writable volumes ( #6338 )
* show only writable volumes
* fix import
11 months ago
c37281735e
volume.list avoid output empty data center and rack and disk info ( #6341 )
11 months ago
8c82c037b9
Unify the re-balancing logic for `ec.encode` with `ec.balance`. ( #6339 )
Among others, this enables recent changes related to topology aware
re-balancing at EC encoding time.
11 months ago
ff1392f7f4
[shell] use constant for hdd of type ( #6337 )
use constant for hdd of type
11 months ago
522a25790a
Remove average constraints when selecting nodes/racks to balance EC shards into. ( #6325 )
11 months ago
914ff0e718
Replaced https.client.enabled with false in scaffold/security ( #6320 )
11 months ago
f3dde99796
adjust error message
11 months ago
34cdbdd279
Share common parameters for EC re-balancing functions under a single struct. ( #6319 )
TODO cleanup for https://github.com/seaweedfs/seaweedfs/discussions/6179 .
11 months ago
edef485333
Account for replication placement settings when balancing EC shards within the same rack. ( #6317 )
* Account for replication placement settings when balancing EC shards within racks.
* Update help contents for `ec.balance`.
* Add a few more representative test cases for `pickEcNodeToBalanceShardsInto()`.
11 months ago
351efa134d
Account for replication placement settings when balancing EC shards across racks. ( #6316 )
11 months ago
b65eb2ec45
[security] reload whiteList on http seerver ( #6302 )
* reload whiteList
* white_list add to scaffold
11 months ago
b2ba7d7408
Resolve replica placement for EC volumes from master server defaults. ( #6303 )
11 months ago
9a741a61b1
Display details upon failures to re-balance EC shards racks. ( #6299 )
11 months ago
559a1fd0f4
Improve EC shards rebalancing logic across nodes ( #6297 )
* Improve EC shards rebalancing logic across nodes.
- Favor target nodes with less preexisting shards, to ensure a fair distribution.
- Randomize selection when multiple possible target nodes are available.
- Add logic to account for replication settings when selecting target nodes (currently disabled).
* Fix minor test typo.
* Clarify internal error messages for `pickEcNodeToBalanceShardsInto()`.
11 months ago
88fa64a01a
change comment for expect header ( #6289 )
* remove case of expect
* Set the default value of the Expect header for compatibility.
11 months ago
e2e97db917
[master] avoid timeout when assigning for main request with filter by DC or rack ( #6291 )
* avoid timeout when assigning for main request with filter by DC or rack
https://github.com/seaweedfs/seaweedfs/issues/6290
* use constant NoWritableVolumes
11 months ago
fec88e64eb
[master] update LastLeaderChangeTime for hashicorp raft ( #6292 )
11 months ago
d43fa07f06
use readable bytes size string in shell output ( #6288 )
11 months ago
8836fa19b6
use ShouldGrowVolumesByDcAndRack ( #6280 )
11 months ago
167b50be88
fix missing register master metric MasterPickForWriteErrorCounter ( #6277 )
11 months ago
fd6c787c02
fix EcVolumes sorting in volume UI ( #6275 )
* Update store_ec.go
* fix: EcVolume sorting not work
* use stdlib
* revert
11 months ago
ccf1795e6f
wait a bit before getting the next volume id if the leader is recently elected
11 months ago
04081128a9
use math rand v2
11 months ago
54204bc89b
format error
11 months ago
ca499de1cb
Improve EC shards rebalancing logic across racks ( #6270 )
Improve EC shards rebalancing logic across racks.
- Favor target shards with less preexisting shards, to ensure a fair distribution.
- Randomize selection when multiple possible target shards are available.
- Add logic to account for replication settings when selecting target shards (currently disabled).
11 months ago
e56327e3b0
ExclusiveLocker only create one renew goroutine ( #6269 )
Co-authored-by: liguowei <liguowei@xinye.com>
11 months ago
aebf3952b7
filer sync: source path and exclude path support dir suffix ( #6268 )
filer sync: source path and exclude paht support dir suffix
Co-authored-by: liguowei <liguowei@xinye.com>
11 months ago
7fa4e5c2a1
[master] Revert back automatic volume grow in case of compensate for crowded volumes ( #6267 )
brought back the Automatic Volume Grow to compensate for crowded volumes
11 months ago
a143c888e5
[shell] don't require lock when there are no changes for volume.fix.replication ( #6266 )
* don't require lock when there are no changes
* revert takeAction
11 months ago
a49d9e020c
[master] avoid crowded more writable for auto grow ( #6214 )
avoid crowded more writable
https://github.com/seaweedfs/seaweedfs/issues/6121
11 months ago
75f5afa571
fix compilation
11 months ago
1c143cfcea
display actual offset value
11 months ago
c9f3448692
ReadAt may return io.EOF t end of file
related to https://github.com/seaweedfs/seaweedfs/issues/6219
11 months ago
be09d08eb6
adjust error message
11 months ago
6b2612690b
consistent error message if auth is not setup
fix https://github.com/seaweedfs/seaweedfs/issues/6262
11 months ago
254ed8897e
[shell] add noLock param for volume.move ( #6261 )
11 months ago
96f1a2e01a
adjust error message
12 months ago
e28f55eb08
typo
12 months ago
98f03862aa
rename
12 months ago
07cf8cf22d
minor
12 months ago
0d5393641e
Unify usage of shell.EcNode.dc as DataCenterId. ( #6258 )
12 months ago
bc7650bd61
adds more info on skipped volumes
12 months ago
Lisandro Pin
zouyixiong
Konstantin Lebedev
vadimartynov
chrislu
steve.wei
Trim21
Numblgw