Lisandro Pin
f2db746690
Introduce logic to resolve volume replica placement within EC rebalancing. ( #6254 )
* Rename `command_ec_encode_test.go` to `command_ec_common_test.go`.
All tests defined in this file are now for `command_ec_common.go`.
* Minor code cleanups.
- Fix broken `ec.balance` test.
- Rework integer ceiling division to not use floats, which can introduce precision errors.
* Introduce logic to resolve volume replica placement within EC rebalancing.
This will be used to make rebalancing logic topology-aware.
* Give shell.EcNode.dc a dedicated DataCenterId type.
1 month ago
Kevin Bulteel
4a0dc883cf
Fix/parse upload filename ( #6241 )
* fix: parse filename in PUT + refactor
* fix: master iu public url with http
* fix: better parsing and handle disposition header
* fix: take mime type from file extension if not set
1 month ago
Eugeniy E. Mikhailov
3003c9e17e
added backward compatible CRC check and extra explanatory comments ( #6233 )
1 month ago
chrislu
9873b033d1
backward compatible vif loading
2 months ago
chrislu
ae5bd0667a
rename proto field from DestroyTime to expire_at_sec
For TTL volume converted into EC volume, this change may leave the volumes staying.
2 months ago
chrislu
d38456dc06
do not delete collection metrics when unmount a volume
fix https://github.com/seaweedfs/seaweedfs/issues/6139
2 months ago
dsd
1e13b6879c
fix(volume): to avoid duplicate write a same needle ( #6138 )
fix WriteNeedleBlob to avoid duplicate write a same needle
Co-authored-by: 邓书东 <shudong_deng@hhnb2024010108.intsig.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2 months ago
chrislu
f11567816e
Revert "fix: volume data integrity check ( #6118 )"
This reverts commit e12b9ab746
.
2 months ago
wusong
e12b9ab746
fix: volume data integrity check ( #6118 )
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2 months ago
chrislu
7dd480acad
handle missing leading zero padding in replica
3 months ago
LHHDZ
4dc33cc143
fix unclaimed spaces calculation when volumePreallocate is enabled ( #6063 )
the calculation of `unclaimedSpaces` only needs to subtract `unusedSpace` when `preallocate` is not enabled.
Signed-off-by: LHHDZ <shichanglin5@qq.com>
3 months ago
Max Denushev
d056c0ddf2
fix(volume): don't persist RO state in specific cases ( #6058 )
* fix(volume): don't persist RO state in specific cases
* fix(volume): writable always persist
3 months ago
chrislu
4ec6eab004
fix max volume count auto setting
related to https://github.com/seaweedfs/seaweedfs/issues/6041#issuecomment-2360448182
3 months ago
Konstantin Lebedev
6d4f5c94ac
[volume] remove truncate idx file if size not healthy ( #6043 )
3 months ago
Eugeniy E. Mikhailov
bc01f09e37
Do CRC check if the buffer contains the full needle data before it is sent ( #5980 )
4 months ago
Bruce
f9e141a412
persist readonly state to volume info ( #5977 )
4 months ago
Bruce
310d41998d
Skip volume data integrity check for volume which has remote file ( #5976 )
4 months ago
chrislu
8e4bffc66b
copy ec shards to disks already having ec volumes
fix https://github.com/seaweedfs/seaweedfs/issues/5615
4 months ago
wyang
361e7ebc74
fix volume.move io.EOF ( #5960 )
4 months ago
Eugeniy E. Mikhailov
220061573d
update metric for ErrorCRC if it happens during volume readNeedleDataInto ( #5957 )
4 months ago
chrislu
5535b60e8c
fix replication range check
fix for https://github.com/seaweedfs/seaweedfs/wiki/Replication#replication-string
4 months ago
chrislu
e8462ba3ad
prevent compacting on the same volume
4 months ago
augustazz
0b00706454
EC volume supports expiration and displays expiration message when executing volume.list ( #5895 )
* ec volume expire
* volume.list show DestroyTime
* comments
* code optimization
---------
Co-authored-by: xuwenfeng <xuwenfeng1@zto.com>
4 months ago
chrislu
fdf7193ae7
rename
4 months ago
chrislu
07f4998188
add dat file size into vif for EC
4 months ago
chrislu
3a2e21fee7
fix EC reading on nLargeBlockRows
fix https://github.com/seaweedfs/seaweedfs/issues/5465
4 months ago
Dan
bd54669d58
Detect underflow when calculating unused space ( #5758 )
* Detect underflow when calculating unused space
* Detect underflow when calculating unused space
6 months ago
Konstantin Lebedev
5ffacbb6ea
refactor all methods strings to const ( #5726 )
6 months ago
Taehyung Lim
4d0bf6ddd4
fixed fail to initialize existing ec volume when volume server has separate index dictory ( #5723 )
6 months ago
NyaMisty
f2a37933b1
Fix sorted needle file's write mode for (*SortedFileNeedleMap).Delete() ( #5633 )
7 months ago
Konstantin Lebedev
d389c5b27e
fix: recreate index include deleted files ( #5579 )
* fix: recreate index include deleted files
https://github.com/seaweedfs/seaweedfs/issues/5508
* fix: counting the number of files
* fix: log
7 months ago
Konstantin Lebedev
6f9267ebdf
fix: skip file size checking for volume with TTL ( #5541 )
fix: skip file size checking for volume with tll
https://github.com/seaweedfs/seaweedfs/issues/5508
8 months ago
Kevin Bulteel
a3dab1fcd0
feat: upload files directly without multipart/form-data ( #5539 )
8 months ago
steve.wei
0bdf121e51
rename VolumeServerVolumeGauge ( #5504 )
8 months ago
Konstantin Lebedev
5189a09de0
[volume] Reduce the number of buffers for uploading one chunk ( #5458 )
9 months ago
Henco Appel
2a88da4de7
fix: always close volume file ( #4530 ) ( #5459 )
If sync fails then close is never called. We should always be calling
close on the file.
Co-authored-by: Hendrik Appel <happel@europe.altair.com>
9 months ago
chrislu
a1816d21e9
fix compilation
9 months ago
dependabot[bot]
359b00596f
chore(deps): bump github.com/rclone/rclone from 1.65.2 to 1.66.0 ( #5387 )
* chore(deps): bump github.com/rclone/rclone from 1.65.2 to 1.66.0
Bumps [github.com/rclone/rclone](https://github.com/rclone/rclone ) from 1.65.2 to 1.66.0.
- [Release notes](https://github.com/rclone/rclone/releases )
- [Changelog](https://github.com/rclone/rclone/blob/master/RELEASE.md )
- [Commits](https://github.com/rclone/rclone/compare/v1.65.2...v1.66.0 )
---
updated-dependencies:
- dependency-name: github.com/rclone/rclone
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Fix compilation error with rclone update (#5395 )
Fix compilation error
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Damiano Albani <damiano.albani@gmail.com>
9 months ago
chrislu
1efa502dbb
go fmt
10 months ago
rustrover
ab70aa92da
remove repetitive words ( #5364 )
10 months ago
Seyed Mahdi Sadegh Shobeiri
ecc154fa9e
Add deleted bytes to total_disk_size ( #5161 )
10 months ago
Konstantin Lebedev
7187346cc1
avoid unexpected compact size ( #5272 )
https://github.com/seaweedfs/seaweedfs/issues/5215
10 months ago
chrislu
7c45992c79
when verifyNeedleIntegrity, do not truncate files
related to https://github.com/seaweedfs/seaweedfs/pull/5259
10 months ago
Justin Dhillon
a48e2ec45b
Fix Broken Links ( #5287 )
* https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-setprocessworkingsetsize
* https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-getprocessworkingsetsize
* remove https://github.com/h5bp/html5-boilerplate/blob/master/src/css/main.css
* https://github.com/AShiou/hof
10 months ago
Benoît Knecht
f6e39139d8
weed/storage: Fix volume info file permissions ( #5246 )
11 months ago
chrislu
cb50b720fd
Revert "Fix weed fix"
This reverts commit 6c1ce18541
.
11 months ago
skycope
316ae45795
can vacuum volume when size mismatch ( #5200 )
Co-authored-by: Yang Wang <yangwang@weride.ai>
11 months ago
skycope
0e8a54f6f6
fix write volume over size MaxPossibleVolumeSize ( #5190 )
Co-authored-by: Yang Wang <yangwang@weride.ai>
12 months ago
chrislu
050efe6c08
factor in existing ec volume count when estimating max volume count
related to https://github.com/seaweedfs/seaweedfs/issues/5191
12 months ago
Konstantin Lebedev
a7fc723ae0
chore: add status code for request_total metrics ( #5188 )
12 months ago