Guang Jiong Lou
6c986e9d70
improve worm support ( #5983 )
* improve worm support
Signed-off-by: lou <alex1988@outlook.com>
* worm mode in filer
Signed-off-by: lou <alex1988@outlook.com>
* update after review
Signed-off-by: lou <alex1988@outlook.com>
* update after review
Signed-off-by: lou <alex1988@outlook.com>
* move to fs configure
Signed-off-by: lou <alex1988@outlook.com>
* remove flag
Signed-off-by: lou <alex1988@outlook.com>
* update after review
Signed-off-by: lou <alex1988@outlook.com>
* support worm hardlink
Signed-off-by: lou <alex1988@outlook.com>
* update after review
Signed-off-by: lou <alex1988@outlook.com>
* typo
Signed-off-by: lou <alex1988@outlook.com>
* sync filer conf
Signed-off-by: lou <alex1988@outlook.com>
---------
Signed-off-by: lou <alex1988@outlook.com>
4 months ago
Guang Jiong Lou
367e7638d7
fix invalid file read ( #6024 )
4 months ago
Bruce
5428229347
fix file read crash ( #6021 )
4 months ago
Aleksey Kosov
7340c62c47
changing FindEntry error handling in cassandra store ( #6015 )
4 months ago
Eugeniy E. Mikhailov
dab0bb8097
Feature limit caching to prescribed number of bytes per file ( #6009 )
* feature: we can check if a fileId is already in the cache
We using this to protect cache from adding the same needle to
the cache over and over.
* fuse mount: Do not start dowloader if needle is already in the cache
* added maxFilePartSizeInCache property to ChunkCache
If file very large only first maxFilePartSizeInCache bytes
are going to be put to the cache (subject to the needle size
constrains).
* feature: for large files put in cache no more than prescribed number of bytes
Before this patch only the first needle of a large file was intended for
caching. This patch uses maximum prescribed amount of bytes to be put in
cache. This allows to bypass default 2MB maximum for a file part stored
in the cache.
* added dummy mock methods to satisfy interfaces of ChunkCache
4 months ago
chrislu
f23015ac5b
weed mount caching
4 months ago
chrislu
93decd1114
remove unused
4 months ago
chrislu
69fcdd0840
adjust logging
4 months ago
zouyixiong
d8e18a8eb7
bug fixed to filer store elastic deleteEntry ( #5988 )
4 months ago
chrislu
18afdb15b6
Revert "weed mount, weed dav add option to force cache"
This reverts commit 7367b976b0
.
4 months ago
chrislu
7367b976b0
weed mount, weed dav add option to force cache
4 months ago
chrislu
d49ecde535
rename functions
5 months ago
chrislu
4fee496b49
conditional delete
5 months ago
wyang
0581ce6096
fix delete chunk failed if volumeSever specified grpc.port ( #5820 )
Co-authored-by: Yang Wang <yangwang@weride.ai>
5 months ago
chrislu
aed2a18223
get, set and add path conf
6 months ago
chrislu
a8fa78b892
refactoring
6 months ago
vadimartynov
86d92a42b4
Added tls for http clients ( #5766 )
* Added global http client
* Added Do func for global http client
* Changed the code to use the global http client
* Fix http client in volume uploader
* Fixed pkg name
* Fixed http util funcs
* Fixed http client for bench_filer_upload
* Fixed http client for stress_filer_upload
* Fixed http client for filer_server_handlers_proxy
* Fixed http client for command_fs_merge_volumes
* Fixed http client for command_fs_merge_volumes and command_volume_fsck
* Fixed http client for s3api_server
* Added init global client for main funcs
* Rename global_client to client
* Changed:
- fixed NewHttpClient;
- added CheckIsHttpsClientEnabled func
- updated security.toml in scaffold
* Reduce the visibility of some functions in the util/http/client pkg
* Added the loadSecurityConfig function
* Use util.LoadSecurityConfiguration() in NewHttpClient func
6 months ago
wusong
ce61a66b65
Fix mmap write fail ( #5791 )
* [mount] fix GetAttr blocks count
Signed-off-by: wang wusong <wangwusong@virtaitech.com>
* [mount] fix mmap read
Signed-off-by: wang wusong <wangwusong@virtaitech.com>
---------
Signed-off-by: wang wusong <wangwusong@virtaitech.com>
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
6 months ago
chrislu
4e7d8eb3f1
small refactoring
6 months ago
chrislu
2addcd1623
fix test
6 months ago
chrislu
590397229e
fix heap init
6 months ago
chrislu
06e3dd306f
clean up
6 months ago
chrislu
c59517e284
upgrade to github.com/redis/go-redis/v9
fix https://github.com/seaweedfs/seaweedfs/issues/4532
6 months ago
chrislu
33b11f92bc
Squashed commit of the following:
commit 465b47b8f7
Merge: f39272b54 9c440d472
Author: chrislu <chris.lu@gmail.com>
Date: Wed Jul 10 23:56:51 2024 -0700
Merge branch 'master' into merged-persisted-logs
commit f39272b5416bdd4d1714ce3e99aded0fe62e7d89
Author: chrislu <chris.lu@gmail.com>
Date: Wed Jul 10 23:49:26 2024 -0700
merge meta logs from multiple files
6 months ago
chrislu
d29fccc521
skip checking if it is the root directory
6 months ago
chrislu
0fdf7eca48
avoid possible nil case during other error cases
6 months ago
chrislu
d8dfadb617
ParallelProcessDirectoryStructure
need to use this for simpler parallel processing
6 months ago
chrislu
c030cb3ce9
bootstrap filer from one peer
6 months ago
chrislu
cb67137a03
optionally open the leveldb in readonly mode
6 months ago
chrislu
9ac1023362
check bucket name in sql backend
fix https://github.com/seaweedfs/seaweedfs/issues/5710
6 months ago
chrislu
464611f614
optionally skip deleting file chunks
7 months ago
chrislu
843e778875
refactor
7 months ago
chrislu
677cfb8ad1
refactor
7 months ago
vadimartynov
8aae82dd71
Added context for the MasterClient's methods to avoid endless loops ( #5628 )
* Added context for the MasterClient's methods to avoid endless loops
* Returned WithClient function. Added WithClientCustomGetMaster function
* Hid unused ctx arguments
* Using a common context for the KeepConnectedToMaster and WaitUntilConnected functions
* Changed the context termination check in the tryConnectToMaster function
* Added a child context to the tryConnectToMaster function
* Added a common context for KeepConnectedToMaster and WaitUntilConnected functions in benchmark
7 months ago
chrislu
dbfbabac55
simplify
7 months ago
dependabot[bot]
722a14ef57
chore(deps): bump github.com/ydb-platform/ydb-go-sdk-auth-environ from 0.2.0 to 0.4.2 ( #5621 )
* chore(deps): bump github.com/ydb-platform/ydb-go-sdk-auth-environ
Bumps [github.com/ydb-platform/ydb-go-sdk-auth-environ](https://github.com/ydb-platform/ydb-go-sdk-auth-environ ) from 0.2.0 to 0.4.2.
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk-auth-environ/blob/master/CHANGELOG.md )
- [Commits](https://github.com/ydb-platform/ydb-go-sdk-auth-environ/compare/v0.2.0...v0.4.2 )
---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk-auth-environ
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* fix compilation
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
7 months ago
sb
a3a00d9499
Feature/mongodb security ( #5602 )
8 months ago
sb
3fae87632f
add filer configuration options for username/password and tls ( #5601 )
8 months ago
dependabot[bot]
36a1cf0361
chore(deps): bump github.com/viant/ptrie from 0.3.1 to 1.0.1 ( #5552 )
* chore(deps): bump github.com/viant/ptrie from 0.3.1 to 1.0.1
Bumps [github.com/viant/ptrie](https://github.com/viant/ptrie ) from 0.3.1 to 1.0.1.
- [Release notes](https://github.com/viant/ptrie/releases )
- [Changelog](https://github.com/viant/ptrie/blob/master/CHANGELOG.md )
- [Commits](https://github.com/viant/ptrie/compare/v0.3.1...v1.0.1 )
---
updated-dependencies:
- dependency-name: github.com/viant/ptrie
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* fix compilation
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
8 months ago
Nikita Borzykh
0d04264494
Move file deletion logging to v2 verbisity ( #5538 )
8 months ago
Henco Appel
5c8e6014ba
fix: filer authenticate with with volume server ( #5480 )
9 months ago
chrislu
ccdd9cd8de
refactor
9 months ago
Nikita Borzykh
829c0da381
fix: adjust condition in prefixFilterEntries prevent infinite loop ( #5440 )
9 months ago
chrislu
36fec34c47
print only adapted url
fix https://github.com/seaweedfs/seaweedfs/issues/5424
10 months ago
Nikita Borzykh
9ca871af53
Move ListDirectoryEntries logic to ListDirectoryPrefixedEntries in etcd meta storage backend ( #5416 )
Move ListDirectoryEntries logic to ListDirectoryPrefixedEntries
10 months ago
Nikita Borzykh
1f3742850d
Add TLS support, initial etcd connection check, fix key_prefix handling for etcd backend meta storage ( #5403 )
10 months ago
chrislu
56ce040ab6
log errors
10 months ago
chrislu
604091a480
use stopChan to close previous filer peer meta subscription instances
10 months ago
chrislu
1efa502dbb
go fmt
10 months ago
chrislu
34f2b600ac
each log function adds a "done" return parameter
10 months ago