Konstantin Lebedev
2f72103c83
avoid load volume file with BytesOffset mismatch ( #3841 )
* avoid load volume file with BytesOffset mismatch
https://github.com/seaweedfs/seaweedfs/issues/2966
* set BytesOffset if has not VolumeInfoFile
* typos fail => failed
* exit if bytesOffset mismatch
2 years ago
Viktor Kuzmin
f8d3ff466d
Allow parallel volume loading from different dirs during startup. ( #3802 )
2 years ago
Konstantin Lebedev
301b678147
[volume] Add new volumes to HUP(reload) signal ( #3755 )
Add new volumes to HUP(reload) signal
2 years ago
Ryan Russell
277976bd76
refactor(storage): readability improvements ( #3703 )
Signed-off-by: Ryan Russell <git@ryanrussell.org>
Signed-off-by: Ryan Russell <git@ryanrussell.org>
2 years ago
famosss
d949a238b8
volume: add "readBufSize" option to customize read optimization ( #3702 )
* simplify a bit
* feat: volume: add "readBufSize" option to customize read optimization
* refactor : redbufSIze -> readBufferSize
* simplify a bit
* simplify a bit
2 years ago
chrislu
896a85d6e4
volume: add "hasSlowRead" option to customize read optimization
2 years ago
Eric Yang
b324a6536c
ADHOC: add read needle meta grpc ( #3581 )
* ADHOC: add read needle meta grpc
* add test
* nit
Co-authored-by: root <root@HQ-10MSTD3EY.roblox.local>
2 years ago
chrislu
7c6324b114
adjust log level
2 years ago
chrislu
3f3a1341d8
make CodeQL happy
2 years ago
chrislu
e0f4366f4c
more detailed error
related to https://github.com/seaweedfs/seaweedfs/issues/3528
2 years ago
Konstantin Lebedev
4f7a1f67cd
avoid race conditions for diskLocation.MaxVolumeCount ( #3526 )
2 years ago
chrislu
4d72a1ce6e
s3: remove deleted collection during metrics reporting
fix https://github.com/seaweedfs/seaweedfs/issues/3453
2 years ago
chrislu
26dbc6c905
move to https://github.com/seaweedfs/seaweedfs
2 years ago
Evgeny Kuzhelev
47c72e6f35
remove all (currently existing) collection volume metrics
2 years ago
Evgeny Kuzhelev
709c83716c
delete disk_size metrics when collection deleted
2 years ago
chrislu
056c480eb0
volume: checksum remove the hashing step in Value()
3 years ago
chrislu
ecef844dfc
stream read large files
3 years ago
guol-fnst
b12944f9c6
fix naming convention
notify volume server of duplicate directoris
improve searching efficiency
3 years ago
guol-fnst
8fab39e775
rename UUID file
fix typo
move locationUUID into DiskLocation
3 years ago
guol-fnst
de6aa9cce8
avoid duplicated volume directory
3 years ago
chrislu
8e9ad7db5a
fix bug deleting volume or unmount volume if a volume server has multiple directories
3 years ago
chrislu
a129bda7d9
sync data first before stopping
3 years ago
Chris Lu
e5fc35ed0c
change server address from string to a type
3 years ago
Chris Lu
8c6ff55226
add volume not found error type, to reduce error log
3 years ago
Chris Lu
05a648bb96
refactor: separating out remote.proto
3 years ago
Chris Lu
78e8ddf910
Only when tailing volume, the zero-ed cookie should skip checking.
This only happens when checkCookie == false and fsync == false.
3 years ago
Chris Lu
734c980040
volume: support concurrent download data size limit
3 years ago
Chris Lu
49c66e88a0
volume: change all writes to fsync during graceful stopping
fix https://github.com/chrislusf/seaweedfs/issues/2193
4 years ago
Chris Lu
56eb522b13
fix stats when a collection is deleted
4 years ago
bingoohuang
cf552417a7
minFreeSpace refactored
4 years ago
bingoohuang
31f1cdeac2
minFreeSpace argument allows size like 10GiB
4 years ago
Chris Lu
ca998328c2
do not add new volumes when below minFreeSpacePercent
fix https://github.com/chrislusf/seaweedfs/issues/2017
4 years ago
Chris Lu
cb423312a4
prevent nil volume message
4 years ago
Chris Lu
828f6e9f4d
volume: auto add missing vif files
fix https://github.com/chrislusf/seaweedfs/issues/1878
4 years ago
qieqieplus
5b16820924
fix: collection stats won't update if all volumes expired at same iteration
4 years ago
Chris Lu
3575d41009
go fmt
4 years ago
Chris Lu
ebf320ec22
avoid divided by zero
4 years ago
Chris Lu
f8446b42ab
this can compile now!!!
4 years ago
Chris Lu
7ce647f27e
support customizable disk type
4 years ago
bingoohuang
94ea3bd3a5
renaming NeedleMapType to NeedleMapKind
4 years ago
Chris Lu
738c4fd203
fix logging when error
address https://github.com/chrislusf/seaweedfs/pull/1685#issuecomment-747845601
4 years ago
Chris Lu
1d88865869
passing disk type along
4 years ago
Chris Lu
933f2f4cfd
find location with matching disk type
4 years ago
Chris Lu
94525aa0fd
allocate volume by disk type
4 years ago
Chris Lu
f6a419c26c
disk type configurable for each folder
4 years ago
Chris Lu
0d2ec832e2
rename from volumeType to diskType
4 years ago
Chris Lu
d156c74ec0
volume server set volume type and heartbeat to the master
4 years ago
Chris Lu
ae655033ac
adjust logging
4 years ago
Chris Lu
96c48bc8a8
fix test
4 years ago
Chris Lu
2c913dde04
volume: detect and drop volumes with disk IO error
from Jethro in slack:
is it possible to make the assign request a bit smarter? Currently I’m in the state that a disk failed but all assign request are being send to this volume. It would be cool if the master sees this and stopped using this volume.
e=HTTP(http://x:8089/913,045a782b63176edf ) not 200 but 500 Internal Server Error
Body={"size":740167,"error":"failed to write to local disk: write /mnt/v9/913.dat: input/output error","eTag":"ee4381e202212ff3aee647704c036689"}
e=HTTP(http://x:8089/913,045a782c90240077 ) not 200 but 500 Internal Server Error
Body={"size":792779,"error":"failed to write to local disk: write /mnt/v9/913.dat: input/output error","eTag":"c43463ccc11eb6eb2fc306f407a6a953"}
e=HTTP(http://x:8089/913,045a782e6b7901ea ) not 200 but 500 Internal Server Error
Body={"size":3962392,"error":"failed to write to local disk: write /mnt/v9/913.dat: input/output error","eTag":"04c91198e9b276c81f11dbf189af5d28"}
4 years ago