ea2637734a
refactor filer proto chunk variable from mtime to modified_ts_ns
2 years ago
764d9cb105
[voluche.chek.disk] needles older than the cutoff time are not missing yet ( #3922 )
needles older than the cutoff time are not missing yet
https://github.com/seaweedfs/seaweedfs/issues/3919
2 years ago
f9f499b8d5
[fix] add param for ignore error ( #3918 )
2 years ago
1e0d64c048
master: delete partially created replicated volumes during volume growth
fix https://github.com/seaweedfs/seaweedfs/discussions/3792#discussioncomment-3973120
2 years ago
c6f2899375
avoid fix missing of deleted needles ( #3909 )
2 years ago
51d462f204
ADHOC: volume fsck using append at ns ( #3906 )
* ADHOC: volume fsck using append at ns
* nit
* nit
Co-authored-by: root <root@HQ-10MSTD3EY.roblox.local>
2 years ago
34132b2c9f
fix: isTruncated returns problematic ( #3907 )
2 years ago
874fd197b5
feat: simplify a bit ( #3905 )
2 years ago
25e012d30b
fix: set user metadata key to lowercase ( #3894 )
* fix: set user metadata key to lowercase
* feat: simplify a bit
2 years ago
377870f4a9
keep system log data
2 years ago
97edb40275
Fix errinfo ( #3893 )
* types packages is imported more than onece
* Fix error response when format of --expires is wrong.
It MUST be in RFC 1123 date format.
2 years ago
d5364218b2
adjust help message
2 years ago
d880fc2bb3
fix merge
2 years ago
6253058d9d
ensure monotonic n.AppendAtNs in each place ( #3880 )
https://github.com/seaweedfs/seaweedfs/issues/3852
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2 years ago
452202f35b
ReadNeedleMeta recovery panic occurred ( #3883 )
ReadNeedleMeta panic occurred
https://github.com/seaweedfs/seaweedfs/issues/3882
2 years ago
e3ea544d40
skip reading needle if size is invalid
related to https://github.com/seaweedfs/seaweedfs/issues/3882
2 years ago
5294b6dd4d
remove volume server tcp client
2 years ago
f2d9049e6a
fix size variable
2 years ago
184fbb6c50
volume server: remote tier volumes only soft delete in local index
fix https://github.com/seaweedfs/seaweedfs/issues/3889
2 years ago
25471d579a
add md5 header when UploadData to replication in ReplicatedWrite ( #3881 )
2 years ago
758d70bdc0
3.32
2 years ago
de286fe662
shell: volume.move handles volume moved to cloud tier
fix https://github.com/seaweedfs/seaweedfs/issues/3803
2 years ago
52e0a88a15
shell: volume.tier.upload clean up replicated copies
fix https://github.com/seaweedfs/seaweedfs/issues/3804
2 years ago
e20f0dbd2d
avoid data race of TraverseBfs ( #3856 )
* avoid data race of TraverseBfs
* close is enough
avoid panic
I1014 12:29:59.207120 volume_loading.go:131 loading sorted db /tmp/sw/test2_19.sdx error: unexpected file /tmp/sw/test2_19.idx size: 255
I1014 12:29:59.207125 volume_loading.go:119 open to write file /tmp/sw/test4_26.idx
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x5260a4c]
goroutine 287 [running]:
github.com/seaweedfs/seaweedfs/weed/storage.(*SortedFileNeedleMap).Close(0x0)
/Users/tochka/GolandProjects/seaweedfs/weed/storage/needle_map_sorted_file.go:97 +0x2c
github.com/seaweedfs/seaweedfs/weed/storage.(*Volume).load.func1()
/Users/tochka/GolandProjects/seaweedfs/weed/storage/volume_loading.go:32 +0x8e
github.com/seaweedfs/seaweedfs/weed/storage.(*Volume).load(0xc001b36280, 0x1, 0x1, 0x0, 0x69228c0?)
/Users/tochka/GolandProjects/seaweedfs/weed/storage/volume_loading.go:205 +0x256c
github.com/seaweedfs/seaweedfs/weed/storage.NewVolume({0x7ffeefbff6e0, 0x7}, {0x7ffeefbff6e0, 0x7}, {0xc0009a9284, 0x5}, 0x13, 0x0, 0x0, 0x0, ...)
/Users/tochka/GolandProjects/seaweedfs/weed/storage/volume.go:62 +0x30f
github.com/seaweedfs/seaweedfs/weed/storage.(*DiskLocation).loadExistingVolume(0xc0006f40c0, {0x846c8d0, 0xc0009ce600}, 0x0?, 0x1)
/Users/tochka/GolandProjects/seaweedfs/weed/storage/disk_location.go:161 +0x4da
github.com/seaweedfs/seaweedfs/weed/storage.(*DiskLocation).concurrentLoadingVolumes.func2()
/Users/tochka/GolandProjects/seaweedfs/weed/storage/disk_location.go:201 +0xf9
created by github.com/seaweedfs/seaweedfs/weed/storage.(*DiskLocation).concurrentLoadingVolumes
/Users/tochka/GolandProjects/seaweedfs/weed/storage/disk_location.go:198 +0x150
2 years ago
ee38ab8581
avoid data race of LogBuffer isStopping ( #3859 )
2 years ago
7836f7574e
[volume.fsck] hotfix apply purging and add option verifyNeedle #3860 ( #3861 )
* fix apply purging and add verifyNeedle
* common readSourceNeedleBlob
* use consts
2 years ago
f476cf3403
fix volume_server.proto conflict ( #3858 )
fix conflict
https://github.com/seaweedfs/seaweedfs/pull/3841
2 years ago
5c2eb602c9
Revert "mount: remove mount directory from generating cache directory id"
This reverts commit 109dc7fdf6
2 years ago
2f72103c83
avoid load volume file with BytesOffset mismatch ( #3841 )
* avoid load volume file with BytesOffset mismatch
https://github.com/seaweedfs/seaweedfs/issues/2966
* set BytesOffset if has not VolumeInfoFile
* typos fail => failed
* exit if bytesOffset mismatch
2 years ago
f19c9e3d9d
Volume fsck by volume ( #3851 )
* refactor
* refactor args verbose and writer
* refactor readFilerFileIdFile
* fix filter by collectMtime
* skip system log collection
2 years ago
9c8678ded9
ensure monotonic n.AppendAtNs
fix https://github.com/seaweedfs/seaweedfs/issues/3852
2 years ago
dcd0743a35
remove unused ReadNeedleBlobRequest.needle_id
fix https://github.com/seaweedfs/seaweedfs/issues/3853
2 years ago
a5b867af69
don't skip empty value at `AssembleEntryWithAcp` ( #3855 )
* add acl helper functionalities
Signed-off-by: changlin.shi <changlin.shi@ly.com>
* add tests
Signed-off-by: changlin.shi <changlin.shi@ly.com>
* remove 0 when create map
Signed-off-by: changlin.shi <changlin.shi@ly.com>
* delete when empty at `AssembleEntryWithAcp`
`PutBucketAcl/PutObjectAcl` allow request with empty grants, `AssembleEntryWithAcp` shouldn't skip empty value
Signed-off-by: changlin.shi <changlin.shi@ly.com>
Signed-off-by: changlin.shi <changlin.shi@ly.com>
2 years ago
1f7e52c63e
vacuum metrics and force sync dst files ( #3832 )
2 years ago
f5d4952d73
filer: redis store reduce from 2 redis operations to 1 for updates.
2 years ago
f95c25e113
types packages is imported more than onece ( #3838 )
2 years ago
9a339a9cfb
default hasSlowRead to true
2 years ago
cacc3e883b
volume server:set the default value of "hasSlowRead" to true ( #3710 )
* simplify a bit
* feat: volume: add "readBufSize" option to customize read optimization
* refactor : redbufSIze -> readBufferSize
* simplify a bit
* simplify a bit
* volume server:set the default value of "hasSlowRead" to true
2 years ago
bf5e45b66a
add acl helper functionalities ( #3831 )
2 years ago
401315f337
master fix interruption through ctrl+c ( #3834 )
2 years ago
a05725aea6
filer: get directory metadata ( #3833 )
2 years ago
e55076c46f
cloud tier: add retry when copying data file
fix https://github.com/seaweedfs/seaweedfs/issues/3828
2 years ago
cea73ac008
serialize sqlite operations
fix https://github.com/seaweedfs/seaweedfs/issues/3827
2 years ago
dff85e9c71
fix error handling
2 years ago
0d817bc347
fix invalid memory address or nil pointer dereference on filer.sync
fix https://github.com/seaweedfs/seaweedfs/issues/3826
2 years ago
84c401e693
Optimiz leveldb metric ( #3830 )
* optimiz updating mapmetric for leveldb
* import loading leveldb
* add comments
2 years ago
d21e2f523d
split `ExtAcpKey` to `ExtAmzOwnerKey` and `ExtAmzAclKey` to avoid unn… ( #3824 )
split `ExtAcpKey` to `ExtAmzOwnerKey` and `ExtAmzAclKey` to avoid unnecessary `json.Unmarshal()` call
Signed-off-by: changlin.shi <changlin.shi@ly.com>
Signed-off-by: changlin.shi <changlin.shi@ly.com>
2 years ago
5b28c3f728
revert disabling FSync for non Mac ( #3814 )
2 years ago
3550692afc
change s3_account.go package to avoid cycle dependency ( #3813 )
2 years ago
0711870f43
3.31
2 years ago
chrislu
Konstantin Lebedev
Eric Yang
famosss
Guo Lei
liubaojiang
LHHDZ
zemul
Guo Lei