Lisandro Pin
b0210df081
Begin implementing EC balancing parallelization support. ( #6342 )
* Begin implementing EC balancing parallelization support.
Impacts both `ec.encode` and `ec.balance`,
* Nit: improve type naming.
* Make the goroutine workgroup handler for `EcBalance()` a bit smarter/error-proof.
* Nit: unify naming for `ecBalancer` wait group methods with the rest of the module.
* Fix concurrency bug.
* Fix whitespace after Gitlab automerge.
* Delete stray TODO.
2 weeks ago
Lisandro Pin
23ffbb083c
Limit EC re-balancing for `ec.encode` to relevant collections when a volume ID argument is provided. ( #6347 )
Limit EC re-balancing for `ec.encode` to relevant collections when a volume ID is provided.
2 weeks ago
Lisandro Pin
6320036c56
Delete legacy balancing code for `ec.encode`. ( #6344 )
2 weeks ago
Lisandro Pin
8c82c037b9
Unify the re-balancing logic for `ec.encode` with `ec.balance`. ( #6339 )
Among others, this enables recent changes related to topology aware
re-balancing at EC encoding time.
2 weeks ago
Lisandro Pin
0d5393641e
Unify usage of shell.EcNode.dc as DataCenterId. ( #6258 )
1 month ago
Chris Lu
72b14a451e
delete aborted ec shards from both source and target servers ( #6221 )
fix https://github.com/seaweedfs/seaweedfs/issues/6205#issuecomment-2465004586
1 month ago
wyang
c29c912bdc
fix format ( #6185 )
unitest weed/shell fail
2 months ago
chrislu
9105c6bdd1
fix format
2 months ago
chrislu
089d4316ef
ensure 2 volume space since actual need 1.4x volume size empty space
2 months ago
chrislu
6e388e29c9
correcting free volume count, factor it during ec encoding to ensure enough disk space available
fix https://github.com/seaweedfs/seaweedfs/issues/6163
2 months ago
chrislu
ec30a504ba
refactor
3 months ago
chrislu
701abbb9df
add IsResourceHeavy() to command interface
3 months ago
Max Denushev
d056c0ddf2
fix(volume): don't persist RO state in specific cases ( #6058 )
* fix(volume): don't persist RO state in specific cases
* fix(volume): writable always persist
3 months ago
NyaMisty
0c62d591e2
Ignore remote volume when selecting volumes in operation (ec.encode/volume.tier.upload) ( #5635 )
7 months ago
chrislu
2bc05f70e7
log full percentage
1 year ago
chrislu
0fd7222d65
default to skip if less than 4 nodes
1 year ago
chrislu
31b2751aff
clone volume locations in case they are changed
fix https://github.com/seaweedfs/seaweedfs/issues/4642
1 year ago
Konstantin Lebedev
25535e9c36
Delete volume is empty ( #4561 )
* use onlyEmpty for deleteVolume
https://github.com/seaweedfs/seaweedfs/issues/4559
* fix IsEmpty
* fix test
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2 years ago
chrislu
31bb91583f
fix bug when vid not found
fix https://github.com/seaweedfs/seaweedfs/issues/4193
2 years ago
chrislu
676e27c589
shell: stop long running jobs if lock is lost
2 years ago
chrislu
26dbc6c905
move to https://github.com/seaweedfs/seaweedfs
2 years ago
chrislu
bc888226fc
erasure coding: tracking encoded/decoded volumes
If an EC shard is created but not spread to other servers, the masterclient would think this shard is not located here.
3 years ago
chrislu
f18803424a
volume.balance: add delay during tight loop
fix https://github.com/chrislusf/seaweedfs/issues/2637
3 years ago
chrislu
9f9ef1340c
use streaming mode for long poll grpc calls
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
3 years ago
chrislu
a2d3f89c7b
add lock messages
3 years ago
Chris Lu
00ae965d8d
randomize a bit for ec shards distribution
3 years ago
Chris Lu
794375ca0a
adjust help message since both fullPercent and quietFor are needed.
3 years ago
Chris Lu
119d5908dd
shell: do not need to lock to see volume -h
3 years ago
Chris Lu
6cd1ce8b74
erasure coding: add cleanup step if anything goes wrong
3 years ago
Chris Lu
e5fc35ed0c
change server address from string to a type
3 years ago
Chris Lu
0f7d4556d8
shell: volume.tier.move makes up changes if volume move failed
3 years ago
Chris Lu
76c48ffe27
optional parallel copy ec shards
fix https://github.com/chrislusf/seaweedfs/issues/2048
4 years ago
Chris Lu
1b1c018165
adjust text
4 years ago
Chris Lu
1c233ad986
refactoring
4 years ago
Chris Lu
f8446b42ab
this can compile now!!!
4 years ago
Chris Lu
536202e4e0
minor
4 years ago
Chris Lu
ae1994cbc1
erasure coding: fix cases where there are no .ecj files
5 years ago
Chris Lu
75179544c3
logging
5 years ago
Chris Lu
73564e6a01
master: add cluster wide lock/unlock operation in weed shell
fix https://github.com/chrislusf/seaweedfs/issues/1286
5 years ago
Chris Lu
97ab8a1976
remove ctx if possible
5 years ago
Chris Lu
892e726eb9
avoid reusing context object
fix https://github.com/chrislusf/seaweedfs/issues/1182
5 years ago
Chris Lu
72a64a5cf8
use the same context object in order to retry
5 years ago
Chris Lu
3ebeae0c0b
ec encode distribute ec data and parity shards evenly
5 years ago
Chris Lu
dda5c6d3cb
fmt
5 years ago
Chris Lu
5f6b360eb7
adjust error message
5 years ago
j.laycock
6fc6322c90
Change joeslay paths to chrislusf paths
5 years ago
j.laycock
595a1beff0
Swap imports to use joeslay
5 years ago
Chris Lu
8afd8d35b3
master: followers can also lookup and redirect
improve scalability
5 years ago
Chris Lu
a3d1296ed9
go fmt
6 years ago
Chris Lu
6883f9e322
mark volume readonly before ec encoding
6 years ago