Konstantin Lebedev
25643cfbc6
fix: panic: assignment to entry in nil map on S3Sink.CreateEntry ( #5406 )
9 months ago
Konstantin Lebedev
170b63d6f8
[filer.backup] add param uploader_part_size for S3sink ( #5352 )
* fix: install cronie
* chore: refactor configure S3Sink
* chore: refactor cinfig
* add filer-backup compose file
* fix: X-Amz-Meta-Mtime and resolve with comments
* fix: attr mtime
* fix: MaxUploadPartst is reduced to the maximum allowable
* fix: env and force set max MaxUploadParts
* fix: env WEED_SINK_S3_UPLOADER_PART_SIZE_MB
10 months ago
chrislu
cc0c8c5f81
simplify
2 years ago
chrislu
87b70a6809
clean up
2 years ago
chrislu
c839ce1b19
s3 sink use s3 upload manager
fix https://github.com/seaweedfs/seaweedfs/issues/3531
2 years ago
chrislu
e3f40d538d
cleaner code
2 years ago
chrislu
11f99836c3
filer.backup: backup small files if the file is saved in filer (saveToFilerLimit > 0)
fix https://github.com/seaweedfs/seaweedfs/issues/3468
2 years ago
chrislu
26dbc6c905
move to https://github.com/seaweedfs/seaweedfs
2 years ago
elee
881a0fe806
ensure compatibility
3 years ago
elee
954ad98e0d
set canned acl on replication create
3 years ago
Chris Lu
7ce97b59d8
go fmt
3 years ago
Chris Lu
c08ac536ed
cloud drive: add support for Wasabi
* disable md5, sha256 checking to avoid reading one chunk twice
* single threaded upload to avoid chunk swapping (to be enhanced later)
3 years ago
Chris Lu
00c4e06caa
cloud drive: s3 configurable force path style
3 years ago
Chris Lu
f0cc130849
do not force path style for better compatibility
3 years ago
Chris Lu
45bffc92a8
filer.backup: fix cloud sinks when updating entry
4 years ago
Chris Lu
c9722dceb2
adjust log level
4 years ago
Chris Lu
984fdd6192
always use non bucket prefixing url
4 years ago
Chris Lu
9abb041763
filer source: support filerProxy mode
4 years ago
Chris Lu
678c54d705
data sink: add incremental mode
4 years ago
Chris Lu
d75bc62196
to read files that are quickly changing
4 years ago
Chris Lu
a8624c2e4f
read from alternative replica
related to https://github.com/chrislusf/seaweedfs/issues/1512
4 years ago
Chris Lu
387ab6796f
filer: cross cluster synchronization
4 years ago
Chris Lu
eb7929a971
rename filer2 to filer
4 years ago
Chris Lu
c647deace1
file size support set file length
use Attr.FileSize and TotalChunkSize to determine file size
4 years ago
Chris Lu
97d97f3528
go code can read and write chunk manifest
4 years ago
Chris Lu
4c498e73de
filer replication: add s3 endpoint
5 years ago
Chris Lu
ae2ee379c0
consistent 64bit size
5 years ago
Chris Lu
bec6ec7db6
go fmt
5 years ago
HongyanShen
81610ed006
fix : #1226
5 years ago
Chris Lu
2e3f6ad3a9
filer: remember content is gzipped or not
5 years ago
Chris Lu
13e215ee5c
filer: option to encrypt data on volume server
5 years ago
Chris Lu
97ab8a1976
remove ctx if possible
5 years ago
Chris Lu
892e726eb9
avoid reusing context object
fix https://github.com/chrislusf/seaweedfs/issues/1182
5 years ago
Chris Lu
d335f04de6
support env variables to overwrite toml file
5 years ago
Chris Lu
9711a6ffaa
WIP
5 years ago
j.laycock
6fc6322c90
Change joeslay paths to chrislusf paths
5 years ago
j.laycock
595a1beff0
Swap imports to use joeslay
5 years ago
Chris Lu
b3b42bc947
replicate need to include new entry path
6 years ago
Chris Lu
55bab1b456
add context.Context
6 years ago
Chris Lu
1bfb96f34d
optimization for reading whole chunk with gzip encoding
6 years ago
Chris Lu
8dfac6a4cf
working b2 sink
6 years ago
Chris Lu
3674ad9f8e
go fmt
6 years ago
Chris Lu
4c97ff3717
support AWS SQS as file change notification message queue
6 years ago
Chris Lu
08266b7256
go fmt
6 years ago
Chris Lu
a5b5887efc
add filer replication to aws s3
6 years ago
Chris Lu
f8c2704d2b
reformat
6 years ago
Chris Lu
56c5c7b1b6
add google cloud storage
6 years ago
Chris Lu
e8ef501f02
add s3 replication sink
6 years ago