chrislu
31922b2bf2
s3 to watch specific directories
2 years ago
Ryan Russell
6f7ef8711a
docs(s3api): readability improvements ( #3696 )
Signed-off-by: Ryan Russell <git@ryanrussell.org>
Signed-off-by: Ryan Russell <git@ryanrussell.org>
2 years ago
Ryan Russell
ad3a3c8782
refactor(s3api_object_handlers): `deleteMultipleObjectsLimmit` -> `de… ( #3695 )
refactor(s3api_object_handlers): `deleteMultipleObjectsLimmit` -> `deleteMultipleObjectsLimit`
Signed-off-by: Ryan Russell <git@ryanrussell.org>
Signed-off-by: Ryan Russell <git@ryanrussell.org>
2 years ago
ImFantuan
b64411bda8
fix 0 size ( #3683 )
2 years ago
famosss
bb880d6253
IsTruncated returned by the ListMultipartUploads method is set to false by default ( #3682 )
* simplify a bit
* fix: IsTruncated returned by the ListMultipartUploads method is set to false by default
2 years ago
chrislu
21c0587900
go fmt
2 years ago
Ryan Russell
d54eb9966f
refactor: `Directory` readability ( #3665 )
2 years ago
Konstantin Lebedev
8c3040db81
avoid DATA RACE on S3Options.localFilerSocket ( #3571 )
* avoid DATA RACE on S3Options.localFilerSocket
https://github.com/seaweedfs/seaweedfs/issues/3552
* copy localSocket
2 years ago
chrislu
a54f30b6c6
s3: close response body with copying object
2 years ago
Konstantin Lebedev
f7aeb06544
s3: report metadata if the directory is explicitly created ( #3498 )
* replace mkdir to mkFile
* ContentLength must be zero
* revert mkDir
* Seaweedfs-Is-Directory-Key return metadata
2 years ago
askeipx
2e78a522ab
remove old raft servers if they don't answer to pings for too long ( #3398 )
* remove old raft servers if they don't answer to pings for too long
add ping durations as options
rename ping fields
fix some todos
get masters through masterclient
raft remove server from leader
use raft servers to ping them
CheckMastersAlive for hashicorp raft only
* prepare blocking ping
* pass waitForReady as param
* pass waitForReady through all functions
* waitForReady works
* refactor
* remove unneeded params
* rollback unneeded changes
* fix
2 years ago
chrislu
9fce75607d
s3: report http.StatusOK if the directory is explicitly created
fix https://github.com/seaweedfs/seaweedfs/issues/3457
2 years ago
famosss
7eb15b1969
fix:Handle preflight cors requests ( #3496 )
2 years ago
famosss
bdba3da2e4
Handle preflight cors requests ( #3481 )
2 years ago
chrislu
42c6e52513
s3: fix regression on HEAD directory operation
2 years ago
chrislu
4573c99ae3
fix tests
2 years ago
chrislu
2b580a7566
also migrate jsonpb
2 years ago
chrislu
eaeb141b09
move proto package
2 years ago
Andrey Triumfov
31faa6d43d
Remove duplicate slashes in object path to prevent 500 errors ( #3442 )
2 years ago
Chris Lu
e88392b50f
Fix s3 pagination ( #3436 )
* Revert previous changes
* s3: use cursor to track tree traversal
fix https://github.com/seaweedfs/seaweedfs/issues/3166
* special cases for empty prefix and empty directory
* use constants
* address empty folder
* undo local changes
* fix IsTruncated
* adjust counting directories
* fix cases when prefix is a directory
* s3: handle directory object
works for
aws --endpoint-url http://127.0.0.1:8333/ s3api list-objects-v2 --bucket test --prefix "fakedir"
2 years ago
chrislu
7457c746f0
s3: fix aws s3api head-object
2 years ago
chrislu
67814a5c79
refactor and fix strings.Split
2 years ago
Konstantin Lebedev
4d08393b7c
filer prefer volume server in same data center ( #3405 )
* initial prefer same data center
https://github.com/seaweedfs/seaweedfs/issues/3404
* GetDataCenter
* prefer same data center for ReplicationSource
* GetDataCenterId
* remove glog
2 years ago
chrislu
90db4b5a44
minor
2 years ago
chrislu
26dbc6c905
move to https://github.com/seaweedfs/seaweedfs
2 years ago
chrislu
64f3d6fb6e
metadata subscription uses client epoch
2 years ago
LHHDZ
c08f6d0ff7
File upload should succeed should return `ETag` instead of `Etag`
at `/go-1.18.1/src/net/textproto/header.go:39`, header is canonicalized by `textproto.CanonicalMIMEHeaderKey`
2 years ago
guosj
ab1b9697e6
supplement check duplicate accesskey
2 years ago
chrislu
69ef6459c3
S3 ListObjectsV2 fix pagination under bucket root
fix https://github.com/chrislusf/seaweedfs/issues/3166
3 years ago
Konstantin Lebedev
3d5f1cbc5e
fix pagination
3 years ago
Konstantin Lebedev
bcbdc4cb37
use const multipart uploads folder
avoid error bucket NotEmpty if multipart uploads folder exist
3 years ago
Lapshin Vitaliy
7c3d9d0535
remove println
3 years ago
Lapshin Vitaliy
61b2e3f9a2
fix return
3 years ago
Lapshin Vitaliy
606252472c
fix return
3 years ago
Lapshin Vitaliy
e969370913
fix parseTagging function and add test for validate tags
3 years ago
Lapshin Vitaliy
1a5981d583
fix test
3 years ago
chrislu
b9f7b6fb9a
adjust log message
3 years ago
石昌林
15bfc60a88
Remove the collection query param of s3api and let the collection be determined by the filer
3 years ago
石昌林
be5c901bea
Fix global counter key is incorrect & Add read lock when reading counter map to avoid memory problems
3 years ago
Lapshin Vitaliy
5f5fd0bc48
validate tags on copy object and add regex for validating tags
3 years ago
guol-fnst
1d77deccd0
AbortMultipartUploadHandler should return 204 instead of 200
https://docs.aws.amazon.com/AmazonS3/latest/API/API_AbortMultipartUpload.html
3 years ago
石昌林
9e036df356
remove go.uber.org/atomic
3 years ago
石昌林
3dd60529c5
some code optimizations
3 years ago
石昌林
37df209195
add some unit tests and some code optimizes
3 years ago
zzq09494
62f74f5e3c
add bucket label to s3 prometheus metrics
3 years ago
石昌林
78b3728169
add s3 circuit breaker support for 'simultaneous request count' and 'simultaneous request bytes' limitations
configure s3 circuit breaker by 'command_s3_circuitbreaker.go':
usage eg:
# Configure the number of simultaneous global (current s3api node) requests
s3.circuit.breaker -global -type count -actions Write -values 1000 -apply
# Configure the number of simultaneous requests for bucket x read and write
s3.circuit.breaker -buckets -type count -actions Read,Write -values 1000 -apply
# Configure the total bytes of simultaneous requests for bucket write
s3.circuit.breaker -buckets -type bytes -actions Write -values 100MiB -apply
# Disable circuit breaker config of bucket 'x'
s3.circuit.breaker -buckets x -enable false -apply
# Delete circuit breaker config of bucket 'x'
s3.circuit.breaker -buckets x -delete -apply
3 years ago
shichanglin5
f01dd27752
fix: When there is no access permission configured before startup, the authentication does not take effect after configuring the permission after startup
3 years ago
Konstantin Lebedev
49f058da85
fix s3 tests:
ranged_request_invalid_range
ranged_request_empty_object
3 years ago
Konstantin Lebedev
c07820178f
fix s3 tests
bucket_list_delimiter_prefix
bucket_list_delimiter_prefix_underscore
bucket_list_delimiter_prefix_ends_with_delimiter
3 years ago
creeew
02ae102731
fix filer.sync missing source srv uploaded files to target when target down
3 years ago