Lisandro Pin
ba0707af64
Allow configuring the maximum number of concurrent tasks for EC parallelization. ( #6376 )
Follow-up to b0210df0
.
4 days ago
Lisandro Pin
b0210df081
Begin implementing EC balancing parallelization support. ( #6342 )
* Begin implementing EC balancing parallelization support.
Impacts both `ec.encode` and `ec.balance`,
* Nit: improve type naming.
* Make the goroutine workgroup handler for `EcBalance()` a bit smarter/error-proof.
* Nit: unify naming for `ecBalancer` wait group methods with the rest of the module.
* Fix concurrency bug.
* Fix whitespace after Gitlab automerge.
* Delete stray TODO.
1 week ago
Lisandro Pin
8c82c037b9
Unify the re-balancing logic for `ec.encode` with `ec.balance`. ( #6339 )
Among others, this enables recent changes related to topology aware
re-balancing at EC encoding time.
2 weeks ago
Lisandro Pin
522a25790a
Remove average constraints when selecting nodes/racks to balance EC shards into. ( #6325 )
2 weeks ago
Lisandro Pin
edef485333
Account for replication placement settings when balancing EC shards within the same rack. ( #6317 )
* Account for replication placement settings when balancing EC shards within racks.
* Update help contents for `ec.balance`.
* Add a few more representative test cases for `pickEcNodeToBalanceShardsInto()`.
3 weeks ago
Lisandro Pin
351efa134d
Account for replication placement settings when balancing EC shards across racks. ( #6316 )
3 weeks ago
Lisandro Pin
b2ba7d7408
Resolve replica placement for EC volumes from master server defaults. ( #6303 )
3 weeks ago
Lisandro Pin
efdebf712e
Refactor `ec.balance` logic into a `weeed/shell/command_ec_common.go`… ( #6195 )
* Refactor `ec.balance` logic into a `weeed/shell/command_ec_common.go` standalone function.
This is a prerequisite to unify the balance logic for `ec.balance` and `ec.encode'.
* s/Balance()/EcBalance()/g
2 months ago
chrislu
ec30a504ba
refactor
3 months ago
chrislu
701abbb9df
add IsResourceHeavy() to command interface
3 months ago
Dan
794a5aa67d
Add warning with hints to why a EC shard cant be moved ( #5790 )
5 months ago
Tobias Gurtzick
78dbac7702
fix(ec): volumes created by foreign collection due to bug in ec balance ( #4864 )
* fix(ec): ignore 0 byte data files
refers to parts of #4861
Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>
* fix(ec): ignore volumes not from the current collection during balance
fixes #4861
Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>
---------
Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>
1 year ago
chrislu
645ae8c57b
Revert "Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs ""
This reverts commit 8cb42c39
1 year ago
chrislu
8cb42c39ad
Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs "
This reverts commit 2e5aa06026
, reversing
changes made to 4d414f54a2
.
1 year ago
dependabot[bot]
a04bd4d26f
Bump github.com/rclone/rclone from 1.63.1 to 1.64.0 ( #4850 )
* Bump github.com/rclone/rclone from 1.63.1 to 1.64.0
Bumps [github.com/rclone/rclone](https://github.com/rclone/rclone ) from 1.63.1 to 1.64.0.
- [Release notes](https://github.com/rclone/rclone/releases )
- [Changelog](https://github.com/rclone/rclone/blob/master/RELEASE.md )
- [Commits](https://github.com/rclone/rclone/compare/v1.63.1...v1.64.0 )
---
updated-dependencies:
- dependency-name: github.com/rclone/rclone
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* API changes
* go mod
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
1 year ago
Ryan Russell
bd2dc6d641
refactor(shell): `Decending` -> `Descending` ( #3675 )
Signed-off-by: Ryan Russell <git@ryanrussell.org>
Signed-off-by: Ryan Russell <git@ryanrussell.org>
2 years ago
chrislu
26dbc6c905
move to https://github.com/seaweedfs/seaweedfs
2 years ago
chrislu
6793bc853c
help message when in simulation mode
3 years ago
justin
3551ca2fcf
enhancement: replace sort.Slice with slices.SortFunc to reduce reflection
3 years ago
chrislu
a2d3f89c7b
add lock messages
3 years ago
Chris Lu
119d5908dd
shell: do not need to lock to see volume -h
3 years ago
Chris Lu
e5fc35ed0c
change server address from string to a type
3 years ago
Chris Lu
ad70107eb4
ec balance among nodes with HDD
4 years ago
Chris Lu
36f95e50a9
avoid possible nil disk info
4 years ago
Chris Lu
f8446b42ab
this can compile now!!!
4 years ago
Chris Lu
2bc05cfbc8
fix typo
4 years ago
Chris Lu
a595916342
shell: add volumeServer.evacuate command
4 years ago
Chris Lu
73564e6a01
master: add cluster wide lock/unlock operation in weed shell
fix https://github.com/chrislusf/seaweedfs/issues/1286
5 years ago
Chris Lu
97ab8a1976
remove ctx if possible
5 years ago
Chris Lu
892e726eb9
avoid reusing context object
fix https://github.com/chrislusf/seaweedfs/issues/1182
5 years ago
Chris Lu
66acc2c84f
passing in averageShardsPerEcNode instead of expectedTotalEcShards
addressing https://github.com/chrislusf/seaweedfs/issues/1111
5 years ago
Chris Lu
40514c5362
add sortEcNodesByFreeslotsDecending and sortEcNodesByFreeslotsAscending
addressing https://github.com/chrislusf/seaweedfs/issues/1111
5 years ago
Chris Lu
9c2f3b1b0f
fix rack_not_found case
fix rack_not_found in https://github.com/chrislusf/seaweedfs/issues/1111
5 years ago
j.laycock
6fc6322c90
Change joeslay paths to chrislusf paths
5 years ago
j.laycock
595a1beff0
Swap imports to use joeslay
5 years ago
Chris Lu
57092d69ac
go fmt
6 years ago
Chris Lu
f9d8bd51ad
ec shard balancing
6 years ago
Chris Lu
9d9162ca35
ec.balance: collect dc rack info
6 years ago
Chris Lu
d344e0a035
fix ec related bugs
6 years ago
Chris Lu
ede876cfdb
periodic scripts exeuction from leader master
6 years ago
Chris Lu
0e49add2bc
adjust help message
6 years ago
Chris Lu
06133ae98f
correct count for free ec slots, avoid removing generated shards
6 years ago
Chris Lu
6cea23d091
purge duplicated ec shards
6 years ago
Chris Lu
b05456fe07
able to purge extra ec shard copies
6 years ago
Chris Lu
7e80b2b882
fix multiple bugs
6 years ago
Chris Lu
9ce6b949bf
adjust help message
6 years ago
Chris Lu
d6e2c877fa
ec.balance: use specific collection names
6 years ago
Chris Lu
ba18314aab
ec shard delete also check ec volumes, in addition to volumes
6 years ago
Chris Lu
1d111d6ce8
collection list normal and ec volumes
6 years ago
Chris Lu
866197eee3
print out the ec balancing plan
6 years ago