Patrick Schmidt
98dcec0ee2
Clean up old signature hash pools
1 year ago
Patrick Schmidt
cdd817edf9
Improve S3 request signing performance
This change is caching HMAC hashers for repeated use in subsequent
requests and chunks, so they don't have to be initialized from
scratch every time.
On my local computer this gives me ~5-6 times faster signature
calculation and ~5-6.5% more throughput in S3 requests. The smaller
the payload the better the throughput gets.
2 years ago
SmsS4
f61490966f
Add time to first byte metric for s3 ( #4768 )
* Add time to first byte metric for s3
* Change second to millisecond
1 year ago
chrislu
6792db21dc
ErrInvalidPart when completed parts has extra parts
fix https://github.com/seaweedfs/seaweedfs/discussions/4746
1 year ago
SmoothDenis
51bcc219ea
s3api should return 500 code from filer ( #4699 )
1 year ago
Henco Appel
89542db2d0
Fix S3 API multipart upload on Windows ( #4666 )
The file paths were being having '/' changed to '\' by filepath.Dir()
resulting in a file being created with '\' separators, but when trying
to read the files, the same wasn't happening.
Co-authored-by: Hendrik Appel <happel@europe.altair.com>
1 year ago
SmsS4
17e91d2917
Use filerGroup for s3 buckets collection prefix ( #4465 )
* Use filerGroup for s3 buckets collection prefix
* Fix templates
* Remove flags
* Remove s3CollectionPrefix
2 years ago
Konstantin Lebedev
d75a7b7f62
allow deleting only older empty dir without recursion ( #4430 )
2 years ago
Konstantin Lebedev
5614ad0000
fix s3test test_bucket_listv2_delimiter_prefix_ends_with_delimiter ( #4399 )
* fix s3test test_bucket_listv2_delimiter_prefix_ends_with_delimiter
* fix list with delimiter and start token
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2 years ago
Konstantin Lebedev
88b2bbadd1
try fix s3test test_bucket_listv2_delimiter_prefix ( #4396 )
2 years ago
Konstantin Lebedev
095cdb0c7f
fix test prefix is a directory ( #4393 )
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2 years ago
Konstantin Lebedev
44ad07276c
s3 fix get list of dir object key with slash suffix ( #4391 )
* s3 fix get list of dir object key with slash suffix
https://github.com/seaweedfs/seaweedfs/issues/3086
* list only entry dir eq prefix
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2 years ago
Konstantin Lebedev
130bc3e668
s3 fix get fake dir object key ( #4390 )
2 years ago
chrislu
6bfb91b539
update entry Attributes.Mtime = time.Now().Unix()
fix https://github.com/seaweedfs/seaweedfs/issues/4337
2 years ago
chrislu
5db9fcccd4
refactoring
2 years ago
zemul
0bf56298d5
fix chunk.ModifiedTsNs ( #4264 )
* fix
* fix mtime s > ns
---------
Co-authored-by: zemul <zhouzemiao@ihuman.com>
2 years ago
LHHDZ
db5515eada
Fix s3api_object_list_handlers returning contents less than the specified limit when more data actually exists ( #4240 )
Fix when the stored data is actually enough but s3api_object_list_handlers returns less than the specified limit
Signed-off-by: changlin.shi <changlin.shi@ly.com>
2 years ago
chrislu
79d68a81fd
text change
2 years ago
chrislu
999a126a3f
typo
2 years ago
chrislu
1c2364936a
detect invalid bucket name
fix https://github.com/seaweedfs/seaweedfs/issues/4143
2 years ago
chrislu
5de93fe442
refactoring
2 years ago
chrislu
81fdf3651b
grpc connection to filer add sw-client-id header
2 years ago
Guo Lei
5f993a5f20
Fix ide warning ( #4082 )
fix warning message from IDE
"redundant type from array, slice, or map composite literal"
2 years ago
LHHDZ
f84c13dabd
fix response not allowed ( #4057 )
2 years ago
LHHDZ
bb289d7d69
sort tags by key ( #4047 )
Signed-off-by: changlin.shi <changlin.shi@ly.com>
Signed-off-by: changlin.shi <changlin.shi@ly.com>
2 years ago
Konstantin Lebedev
2b910d1cf8
avoid recursive deleting newly created empty directories ( #4016 )
2 years ago
chrislu
70a4c98b00
refactor filer_pb.Entry and filer.Entry to use GetChunks()
for later locking on reading chunks
2 years ago
LHHDZ
39f0246deb
fix 'NoSuchUpload' when upload part ( #3968 )
2 years ago
famosss
df0662038c
feat:add a uuid after uploadid ( #3963 )
2 years ago
pavelzhurov
abe4a61659
Bug fix: empty key in DeleteMultipleObjects request caused bucket delete ( #3939 )
2 years ago
chrislu
8b9957d461
add back "/" prefix if it is missing in object
fix https://github.com/seaweedfs/seaweedfs/issues/3737
2 years ago
chrislu
ea2637734a
refactor filer proto chunk variable from mtime to modified_ts_ns
2 years ago
famosss
34132b2c9f
fix: isTruncated returns problematic ( #3907 )
2 years ago
famosss
874fd197b5
feat: simplify a bit ( #3905 )
2 years ago
famosss
25e012d30b
fix: set user metadata key to lowercase ( #3894 )
* fix: set user metadata key to lowercase
* feat: simplify a bit
2 years ago
Guo Lei
97edb40275
Fix errinfo ( #3893 )
* types packages is imported more than onece
* Fix error response when format of --expires is wrong.
It MUST be in RFC 1123 date format.
2 years ago
LHHDZ
a5b867af69
don't skip empty value at `AssembleEntryWithAcp` ( #3855 )
* add acl helper functionalities
Signed-off-by: changlin.shi <changlin.shi@ly.com>
* add tests
Signed-off-by: changlin.shi <changlin.shi@ly.com>
* remove 0 when create map
Signed-off-by: changlin.shi <changlin.shi@ly.com>
* delete when empty at `AssembleEntryWithAcp`
`PutBucketAcl/PutObjectAcl` allow request with empty grants, `AssembleEntryWithAcp` shouldn't skip empty value
Signed-off-by: changlin.shi <changlin.shi@ly.com>
Signed-off-by: changlin.shi <changlin.shi@ly.com>
2 years ago
LHHDZ
bf5e45b66a
add acl helper functionalities ( #3831 )
2 years ago
LHHDZ
d21e2f523d
split `ExtAcpKey` to `ExtAmzOwnerKey` and `ExtAmzAclKey` to avoid unn… ( #3824 )
split `ExtAcpKey` to `ExtAmzOwnerKey` and `ExtAmzAclKey` to avoid unnecessary `json.Unmarshal()` call
Signed-off-by: changlin.shi <changlin.shi@ly.com>
Signed-off-by: changlin.shi <changlin.shi@ly.com>
2 years ago
LHHDZ
3550692afc
change s3_account.go package to avoid cycle dependency ( #3813 )
2 years ago
Konstantin Lebedev
ec46a34f33
avoid increasing the coordinates of metric labels ( #3785 )
2 years ago
LHHDZ
e9584d9661
add ownership rest apis ( #3765 )
2 years ago
LHHDZ
3de1e19780
s3: sync bucket info from filer ( #3759 )
2 years ago
LHHDZ
aacdcc4cad
s3: add account ( #3753 )
associate `Account` and `Identity` by accountId
2 years ago
chrislu
31922b2bf2
s3 to watch specific directories
2 years ago
Ryan Russell
6f7ef8711a
docs(s3api): readability improvements ( #3696 )
Signed-off-by: Ryan Russell <git@ryanrussell.org>
Signed-off-by: Ryan Russell <git@ryanrussell.org>
2 years ago
Ryan Russell
ad3a3c8782
refactor(s3api_object_handlers): `deleteMultipleObjectsLimmit` -> `de… ( #3695 )
refactor(s3api_object_handlers): `deleteMultipleObjectsLimmit` -> `deleteMultipleObjectsLimit`
Signed-off-by: Ryan Russell <git@ryanrussell.org>
Signed-off-by: Ryan Russell <git@ryanrussell.org>
2 years ago
ImFantuan
b64411bda8
fix 0 size ( #3683 )
2 years ago
famosss
bb880d6253
IsTruncated returned by the ListMultipartUploads method is set to false by default ( #3682 )
* simplify a bit
* fix: IsTruncated returned by the ListMultipartUploads method is set to false by default
2 years ago
chrislu
21c0587900
go fmt
2 years ago