Lê Hoàng Nam
1f2dc54647
fix(s3api): make post api upload object default return 204 ( #5648 )
7 months ago
Riccardo Bertossa
f6e8a9bf9c
added s3 iam DeleteBucket permission management ( #5599 )
7 months ago
chrislu
1917f70f78
also delete parent folder if empty
fix https://github.com/seaweedfs/seaweedfs/issues/5567
8 months ago
chrislu
e6bfd061c7
AllowEmptyFolder checks during object deletion
8 months ago
chrislu
d521466a37
split file
8 months ago
chrislu
a1b59948cc
rename files
8 months ago
Konstantin Lebedev
3d3ee04eb9
[s3] Put bucket lifecycle configuration ( #5510 )
8 months ago
Konstantin Lebedev
33537ae29f
[s3] fix s3 test_multipart_get_part ( #5476 )
* try fix s3 test_multipart_get_part
* add passed s3 tests
* fix SeaweedFSUploadId
* rm spaces
* convert part request to range
* add passed s3 tests of multipart
8 months ago
clonefetch
9e07a87fcb
chore: fix function names in comment ( #5478 )
9 months ago
Konstantin Lebedev
3e25ed1b11
[s3] add s3 pass test_multipart_upload_size_too_small ( #5475 )
* add s3 pass test_multipart_upload_size_too_small
* refactor metric names
* return ErrNoSuchUpload if empty parts
* fix test
9 months ago
Konstantin Lebedev
35cba720a5
[s3] add s3 pass test_multipart_upload ( #5474 )
add s3 pass test_multipart_upload
9 months ago
Konstantin Lebedev
d42a04cceb
[s3] fix s3 test_multipart_resend_first_finishes_last ( #5471 )
* try fix s3 test
https://github.com/seaweedfs/seaweedfs/pull/5466
* add error handler metrics
* refactor
* refactor multipartExt
* delete bad entry parts
9 months ago
Nikita Borzykh
7aa25c1137
fix: correctly form partNumber from new entry.Name format in "listObjectParts" ( #5470 )
9 months ago
skycope
b19c9847c6
fix completed multiupload lost data ( #5460 )
If there are putObjectPart requests with the same uploadId during
completeMultiPart, it can result in data loss. putObjectPart requests
might be due to timeout retries.
Co-authored-by: Yang Wang <yangwang@weride.ai>
9 months ago
Nikita Borzykh
df9031e19a
fix: ensure complete gRPC stream consumption in doListFilerEntries to prevent context canceled errors. ( #5441 )
9 months ago
adasauce
61f4e40ad9
fix s3api: delimeter properly takes prefixes into account ( #5411 )
9 months ago
Konstantin Lebedev
170b63d6f8
[filer.backup] add param uploader_part_size for S3sink ( #5352 )
* fix: install cronie
* chore: refactor configure S3Sink
* chore: refactor cinfig
* add filer-backup compose file
* fix: X-Amz-Meta-Mtime and resolve with comments
* fix: attr mtime
* fix: MaxUploadPartst is reduced to the maximum allowable
* fix: env and force set max MaxUploadParts
* fix: env WEED_SINK_S3_UPLOADER_PART_SIZE_MB
10 months ago
adasauce
3e3e0fa240
feat: add support for S3 ListObjects and ListObjectsV2 ( #5350 )
10 months ago
7y-9
41ec9d7759
refactor: put the auth outside ( #5313 )
10 months ago
7y-9
f037c09c11
fix: only admin auth can delete S3 bucket ( #5312 )
10 months ago
sxlehua
ffe5f19aa0
Add s3 postpolicy support header ( #5280 )
* Add s3 postpolicy support header
* optimized code
11 months ago
sxlehua
0b49c16361
Adapt S3 POST ContentType ( #5275 )
11 months ago
Konstantin Lebedev
a7fc723ae0
chore: add status code for request_total metrics ( #5188 )
12 months ago
Konstantin Lebedev
4f6172f369
Fix missing VersionConfiguration node in get-bucket-versioning response ( #5162 )
* fix: s3 response for get bucket version
https://github.com/seaweedfs/seaweedfs/issues/5155
* fix: s3 response for get bucket version
https://github.com/seaweedfs/seaweedfs/issues/5155
12 months ago
chrislu
7a0d1e7fc1
minor
1 year ago
jerebear12
06343f8976
Set allowed origins in config ( #5109 )
* Add a way to use a JWT in an HTTP only cookie
If a JWT is not included in the Authorization header or a query string, attempt to get a JWT from an HTTP only cookie.
* Added a way to specify allowed origins header from config
* Removed unecessary log
* Check list of domains from config or command flag
* Handle default wildcard and change name of config value to cors
1 year ago
Konstantin Lebedev
5e41ab1370
s3 api add not implemented response for PutBucketVersioning
1 year ago
Konstantin Lebedev
7f6152f130
s3 api add default response for GetBucketVersioning
1 year ago
chrislu
612fd7206e
fix
1 year ago
Konstantin Lebedev
6c82d7ddfc
[s3] do reload s3 static config ( #4923 )
* do reload s3 config
* print error on reload s3 config
* print success msg
* Update weed/s3api/s3api_server.go
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
1 year ago
chrislu
cbc24c7b24
disconnect from old subscribers
1 year ago
chrislu
7b3c63a8af
handle wrapped errors
1 year ago
chrislu
4e493d596c
minor
1 year ago
chrislu
b02fdeabff
RetryForever => RetryUntil
1 year ago
chrislu
645ae8c57b
Revert "Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs ""
This reverts commit 8cb42c39
1 year ago
Konstantin Lebedev
f8b94cac0e
[s3acl] Step1: move s3account.AccountManager into to iam.S3ApiConfiguration ( #4859 )
* move s3account.AccountManager into to iam.S3ApiConfiguration and switch to Interface
https://github.com/seaweedfs/seaweedfs/issues/4519
* fix: test bucket acl default and
adjust the variable names
* fix: s3 api config test
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
1 year ago
Konstantin Lebedev
750a0ba1b2
[iam] Replace action read/write to readAcp/writeAcp for handlers with acl ( #4858 )
Replace action read/write to readAcp/writeAcp for handlers with acl query
https://github.com/seaweedfs/seaweedfs/issues/4519
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
1 year ago
Konstantin Lebedev
a46f873edd
[s3acl] Step 0: Put bucket ACL only responds success if the ACL is private. ( #4856 )
* Passing test:
test_bucket_acl_default
test_bucket_acl_canned_private_to_private
https://github.com/seaweedfs/seaweedfs/issues/4519
* Update weed/s3api/s3api_bucket_handlers.go
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
1 year ago
Konstantin Lebedev
d8b424d123
[s3] optimization iam lookup for reducing algorithm complexity ( #4857 )
optimization iam lookup for reducing algorithm complexity
https://github.com/seaweedfs/seaweedfs/issues/4519
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
1 year ago
chrislu
cf28108ea4
fix grpd dial option
1 year ago
chrislu
8cb42c39ad
Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs "
This reverts commit 2e5aa06026
, reversing
changes made to 4d414f54a2
.
1 year ago
dependabot[bot]
a04bd4d26f
Bump github.com/rclone/rclone from 1.63.1 to 1.64.0 ( #4850 )
* Bump github.com/rclone/rclone from 1.63.1 to 1.64.0
Bumps [github.com/rclone/rclone](https://github.com/rclone/rclone ) from 1.63.1 to 1.64.0.
- [Release notes](https://github.com/rclone/rclone/releases )
- [Changelog](https://github.com/rclone/rclone/blob/master/RELEASE.md )
- [Commits](https://github.com/rclone/rclone/compare/v1.63.1...v1.64.0 )
---
updated-dependencies:
- dependency-name: github.com/rclone/rclone
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* API changes
* go mod
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
1 year ago
Patrick Schmidt
98dcec0ee2
Clean up old signature hash pools
1 year ago
Patrick Schmidt
cdd817edf9
Improve S3 request signing performance
This change is caching HMAC hashers for repeated use in subsequent
requests and chunks, so they don't have to be initialized from
scratch every time.
On my local computer this gives me ~5-6 times faster signature
calculation and ~5-6.5% more throughput in S3 requests. The smaller
the payload the better the throughput gets.
2 years ago
SmsS4
f61490966f
Add time to first byte metric for s3 ( #4768 )
* Add time to first byte metric for s3
* Change second to millisecond
1 year ago
chrislu
6792db21dc
ErrInvalidPart when completed parts has extra parts
fix https://github.com/seaweedfs/seaweedfs/discussions/4746
1 year ago
SmoothDenis
51bcc219ea
s3api should return 500 code from filer ( #4699 )
1 year ago
Henco Appel
89542db2d0
Fix S3 API multipart upload on Windows ( #4666 )
The file paths were being having '/' changed to '\' by filepath.Dir()
resulting in a file being created with '\' separators, but when trying
to read the files, the same wasn't happening.
Co-authored-by: Hendrik Appel <happel@europe.altair.com>
1 year ago
SmsS4
17e91d2917
Use filerGroup for s3 buckets collection prefix ( #4465 )
* Use filerGroup for s3 buckets collection prefix
* Fix templates
* Remove flags
* Remove s3CollectionPrefix
2 years ago
Konstantin Lebedev
d75a7b7f62
allow deleting only older empty dir without recursion ( #4430 )
2 years ago