113c9ce6a8
remove the direct_io flag, as it is not well-supported on macOS
5 months ago
88a1eee31c
adjust logs
5 months ago
62c4980754
add debug info
5 months ago
6161d0b05d
a bit optimization
5 months ago
9369a88c5c
stop renaming worm files ( #6154 )
* stop renaming worm file
Signed-off-by: lou <alex1988@outlook.com>
* update after review
Signed-off-by: lou <alex1988@outlook.com>
* Update weed/server/filer_server_handlers_write.go
---------
Signed-off-by: lou <alex1988@outlook.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
6 months ago
07df5c53de
avoid nil
I1010 03:10:25.220345 mount_std.go:281 This is SeaweedFS version 30GB 3.76 82ed61c655
6 months ago
c0e36231ad
use only one metadata follow process
6 months ago
11e53dd8ef
skip error if conf file is not created
6 months ago
99ae99e3b9
fix nil filer conf
fix https://github.com/seaweedfs/seaweedfs/issues/6034
7 months ago
6c986e9d70
improve worm support ( #5983 )
* improve worm support
Signed-off-by: lou <alex1988@outlook.com>
* worm mode in filer
Signed-off-by: lou <alex1988@outlook.com>
* update after review
Signed-off-by: lou <alex1988@outlook.com>
* update after review
Signed-off-by: lou <alex1988@outlook.com>
* move to fs configure
Signed-off-by: lou <alex1988@outlook.com>
* remove flag
Signed-off-by: lou <alex1988@outlook.com>
* update after review
Signed-off-by: lou <alex1988@outlook.com>
* support worm hardlink
Signed-off-by: lou <alex1988@outlook.com>
* update after review
Signed-off-by: lou <alex1988@outlook.com>
* typo
Signed-off-by: lou <alex1988@outlook.com>
* sync filer conf
Signed-off-by: lou <alex1988@outlook.com>
---------
Signed-off-by: lou <alex1988@outlook.com>
7 months ago
367e7638d7
fix invalid file read ( #6024 )
7 months ago
a9c9e1bcb3
refactor
7 months ago
eb02946c97
support write once read many
fix https://github.com/seaweedfs/seaweedfs/issues/5954
7 months ago
18afdb15b6
Revert "weed mount, weed dav add option to force cache"
This reverts commit 7367b976b0
7 months ago
7367b976b0
weed mount, weed dav add option to force cache
7 months ago
9ca30e52d5
fuse mount handles ttl entries
fix https://github.com/seaweedfs/seaweedfs/issues/5527
8 months ago
e86b0bcaaa
simplify
8 months ago
57dc39c451
randomizing next file handle id
8 months ago
952afd810c
Fix dead lock ( #5815 )
* reduce locks to avoid dead lock
Flush->FlushData->uplloadPipeline.FluahAll
uploaderCount>0
goroutine 1 [sync.Cond.Wait, 71 minutes]:
sync.runtime_notifyListWait(0xc0007ae4d0, 0x0)
/usr/local/go/src/runtime/sema.go:569 +0x159
sync.(*Cond).Wait(0xc001a59290?)
/usr/local/go/src/sync/cond.go:70 +0x85
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).waitForCurrentWritersToComplete(0xc0002ee4d0)
/github/workspace/weed/mount/page_writer/upload_pipeline_lock.go:58 +0x32
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).FlushAll(0xc0002ee4d0)
/github/workspace/weed/mount/page_writer/upload_pipeline.go:151 +0x25
github.com/seaweedfs/seaweedfs/weed/mount.(*ChunkedDirtyPages).FlushData(0xc00087e840)
/github/workspace/weed/mount/dirty_pages_chunked.go:54 +0x29
github.com/seaweedfs/seaweedfs/weed/mount.(*PageWriter).FlushData(...)
/github/workspace/weed/mount/page_writer.go:50
github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).doFlush(0xc0006ad600, 0xc00030d380, 0x0, 0x0)
/github/workspace/weed/mount/weedfs_file_sync.go:101 +0x169
github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).Flush(0xc0006ad600, 0xc001a594a8?, 0xc0004c1ca0)
/github/workspace/weed/mount/weedfs_file_sync.go:59 +0x48
github.com/hanwen/go-fuse/v2/fuse.doFlush(0xc0000da870?, 0xc0004c1b08)
SaveContent -> MemChunk.RLock ->
ChunkedDirtyPages.saveChunkedFileIntervalToStorage
pages.fh.AddChunks([]*filer_pb.FileChunk{chunk})
fh.entryLock.Lock()
sync.(*RWMutex).Lock(0x0?)
/usr/local/go/src/sync/rwmutex.go:146 +0x31
github.com/seaweedfs/seaweedfs/weed/mount.(*FileHandle).AddChunks(0xc00030d380, {0xc00028bdc8, 0x1, 0x1})
/github/workspace/weed/mount/filehandle.go:93 +0x45
github.com/seaweedfs/seaweedfs/weed/mount.(*ChunkedDirtyPages).saveChunkedFileIntervalToStorage(0xc00087e840, {0x2be7ac0, 0xc00018d9e0}, 0x0, 0x121, 0x17e3c624565ace45, 0x1?)
/github/workspace/weed/mount/dirty_pages_chunked.go:80 +0x2d4
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*MemChunk).SaveContent(0xc0008d9130, 0xc0008093e0)
/github/workspace/weed/mount/page_writer/page_chunk_mem.go:115 +0x112
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).moveToSealed.func1()
/github/workspace/weed/mount/page_writer/upload_pipeline.go:187 +0x55
github.com/seaweedfs/seaweedfs/weed/util.(*LimitedConcurrentExecutor).Execute.func1()
/github/workspace/weed/util/limited_executor.go:38 +0x62
created by github.com/seaweedfs/seaweedfs/weed/util.(*LimitedConcurrentExecutor).Execute in goroutine 1
/github/workspace/weed/util/limited_executor.go:33 +0x97
On metadata update
fh.entryLock.Lock()
fh.dirtyPages.Destroy()
up.chunksLock.Lock => each sealed chunk.FreeReference => MemChunk.Lock
goroutine 134 [sync.RWMutex.Lock, 71 minutes]:
sync.runtime_SemacquireRWMutex(0xc0007c3558?, 0xea?, 0x3fb0800?)
/usr/local/go/src/runtime/sema.go:87 +0x25
sync.(*RWMutex).Lock(0xc0007c35a8?)
/usr/local/go/src/sync/rwmutex.go:151 +0x6a
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*MemChunk).FreeResource(0xc0008d9130)
/github/workspace/weed/mount/page_writer/page_chunk_mem.go:38 +0x2a
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*SealedChunk).FreeReference(0xc00071cdb0, {0xc0006ba1a0, 0x20})
/github/workspace/weed/mount/page_writer/upload_pipeline.go:38 +0xb7
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).Shutdown(0xc0002ee4d0)
/github/workspace/weed/mount/page_writer/upload_pipeline.go:220 +0x185
github.com/seaweedfs/seaweedfs/weed/mount.(*ChunkedDirtyPages).Destroy(0xc0008cea40?)
/github/workspace/weed/mount/dirty_pages_chunked.go:87 +0x17
github.com/seaweedfs/seaweedfs/weed/mount.(*PageWriter).Destroy(...)
/github/workspace/weed/mount/page_writer.go:78
github.com/seaweedfs/seaweedfs/weed/mount.NewSeaweedFileSystem.func3({0xc00069a6c0, 0x30}, 0x6?)
/github/workspace/weed/mount/weedfs.go:119 +0x17a
github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.NewMetaCache.func1({0xc00069a6c0?, 0xc00069a480?}, 0x4015b40?)
/github/workspace/weed/mount/meta_cache/meta_cache.go:37 +0x1c
github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents.func1(0xc000661810)
/github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:43 +0x570
* use locked entry everywhere
* modifiable remote entry
* skip locking after getting lock from fhLockTable
9 months ago
86d92a42b4
Added tls for http clients ( #5766 )
* Added global http client
* Added Do func for global http client
* Changed the code to use the global http client
* Fix http client in volume uploader
* Fixed pkg name
* Fixed http util funcs
* Fixed http client for bench_filer_upload
* Fixed http client for stress_filer_upload
* Fixed http client for filer_server_handlers_proxy
* Fixed http client for command_fs_merge_volumes
* Fixed http client for command_fs_merge_volumes and command_volume_fsck
* Fixed http client for s3api_server
* Added init global client for main funcs
* Rename global_client to client
* Changed:
- fixed NewHttpClient;
- added CheckIsHttpsClientEnabled func
- updated security.toml in scaffold
* Reduce the visibility of some functions in the util/http/client pkg
* Added the loadSecurityConfig function
* Use util.LoadSecurityConfiguration() in NewHttpClient func
9 months ago
3a82f5ffad
ensure metadata follow a specific folder
fix https://github.com/seaweedfs/seaweedfs/issues/5774
9 months ago
fce8fc1e16
Fix Issue #5649 ( #5652 )
10 months ago
d218fe54fa
go fmt
11 months ago
d5b2cee34a
log possible racing condition
fix https://github.com/seaweedfs/seaweedfs/issues/5412
1 year ago
2a4a9c6343
log possible racing condition
fix https://github.com/seaweedfs/seaweedfs/issues/5412
1 year ago
6aa804b368
lock instead of RLock, to prevent racing condition
https://github.com/seaweedfs/seaweedfs/discussions/5432#discussioncomment-8933608
1 year ago
a5582cc921
[mount] fix GetAttr blocks count ( #5409 )
1 year ago
06e723f629
Waiting for writes to complete with acquired locks may lead to deadlock, fixes #4952
1 year ago
310eec9a9c
skip if path changed
mitigate https://github.com/seaweedfs/seaweedfs/issues/4968
1 year ago
cbc24c7b24
disconnect from old subscribers
2 years ago
b02fdeabff
RetryForever => RetryUntil
2 years ago
186b1f029b
read source uses shared lock
2 years ago
31fc165715
mount switch to ordered lock requests
2 years ago
27af11f1e8
Revert "Revert "Merge branch 'master' into sub""
This reverts commit 0bb97709d4
2 years ago
0bb97709d4
Revert "Merge branch 'master' into sub"
This reverts commit 4d414f54a24827425146
2 years ago
9e5b1eef9b
avoid passe lock by value
2 years ago
f5c4b642bb
re-ordering
2 years ago
3852307e94
renaming
2 years ago
6c7fa567d4
add separate cache directory for write buffers
2 years ago
33b9192536
Fix large file chunks deleted by mistake ( #4678 )
2 years ago
0cb9ddd8ec
Fix data loss: add lock for metacache ( #4664 )
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2 years ago
c44a6552d2
Fix buff use ( #4628 )
2 years ago
68f1cc34a5
[mount]Fix copy file range ( #4407 )
* fix:mount deadlock
* fix
---------
Co-authored-by: zemul <zhouzemiao@ihuman.com>
2 years ago
0f653d2153
update fuse client log which should output the fh of FileHandle. ( #4404 )
2 years ago
0122e022ea
Mount concurrent read ( #4400 )
* fix:mount deadlock
* feat: concurrent read
* fix
* Remove useless code
* fix
---------
Co-authored-by: zemul <zhouzemiao@ihuman.com>
2 years ago
5db9fcccd4
refactoring
2 years ago
71b33faef0
Skip parent directory creation in mount ( #4310 )
A POSIX filesystem does not implicitly create parent directories when
they do not exist. Directories must be explicitly created and
permissions be set.
This also fixes a bug where asynchronous operations would create a
file in the filer before the parent directory was created. If the
file was a symlink or another special type of file the directory
would inherit that type and become unusable in the mounted FS.
2 years ago
79af6b41d4
[mount] Fix read file-data in entry.content ( #4207 )
2 years ago
fe573f26a8
memchunk save content twice ( #4174 )
2 years ago
665917fdf1
[mount]: delete redundant code ( #4173 )
2 years ago
chrislu
Guang Jiong Lou
Chris Lu
vadimartynov
Gaspare Iengo
wusong
Viktor Kuzmin
wang wusong
zemul
Chengyu Liu
Patrick Schmidt