seaweedfs

Commit Graph

Author	SHA1	Message	Date
Gerry Hernandez	e653de54b4	FUSE Mount: Fix buffer allocation during copy (#6863 ) Fix buffer allocation during FUSE copy	5 months ago
chrislu	c79e73aa2a	mount: complete fix for freebsd fix https://github.com/seaweedfs/seaweedfs/issues/6645	5 months ago
chrislu	bd4891a117	change version directory	5 months ago
Aleksey Kosov	283d9e0079	Add context with request (#6824 )	5 months ago
Aleksey Kosov	165af32d6b	added context to filer_client method calls (#6808 ) Co-authored-by: akosov <a.kosov@kryptonite.ru>	5 months ago
chrislu	1af0b282db	skip logs	7 months ago
Chris Lu	528702d30b	errors when reading uncached files from cloud drive (#6674 ) fix https://github.com/seaweedfs/seaweedfs/issues/6672	7 months ago
jang1lee	4ad669b2aa	Fixes files with TTL can not be read in a mounted folder (#6646 )	7 months ago
chrislu	f99a818ad2	deal with handle not found when flushing fix https://github.com/seaweedfs/seaweedfs/issues/6645#issuecomment-2737400492	7 months ago
chrislu	94bd8b39cf	refactor	7 months ago
jang1lee	f7f6e1158e	Fixes files with TTL are not listed in a mounted folder. (#6621 )	8 months ago
lizhengui007	61249d8dde	clear cache directory when mount exits (#6605 ) Signed-off-by: lizhengui <lizhengui@virtaitech.com> Co-authored-by: lizhengui <lizhengui@virtaitech.com>	8 months ago
Guang Jiong Lou	3b1ac77e1f	worm grace period and retention time support (#6404 ) Signed-off-by: lou <alex1988@outlook.com>	10 months ago
zemul	e77e50886e	mount metacache add ttl (#6360 ) * fix:mount deadlock * fix * feat: metaCache ttl * Update weed/command/mount.go Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> * fix InodeEntry --------- Co-authored-by: zemul <zhouzemiao@ihuman.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>	11 months ago
chrislu	c9f3448692	ReadAt may return io.EOF t end of file related to https://github.com/seaweedfs/seaweedfs/issues/6219	11 months ago
chrislu	113c9ce6a8	remove the direct_io flag, as it is not well-supported on macOS	12 months ago
chrislu	88a1eee31c	adjust logs	12 months ago
chrislu	62c4980754	add debug info	12 months ago
chrislu	6161d0b05d	a bit optimization	12 months ago
Guang Jiong Lou	9369a88c5c	stop renaming worm files (#6154 ) * stop renaming worm file Signed-off-by: lou <alex1988@outlook.com> * update after review Signed-off-by: lou <alex1988@outlook.com> * Update weed/server/filer_server_handlers_write.go --------- Signed-off-by: lou <alex1988@outlook.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>	1 year ago
chrislu	07df5c53de	avoid nil I1010 03:10:25.220345 mount_std.go:281 This is SeaweedFS version 30GB 3.76 `82ed61c655` linux amd64 ^[OR^[OR2024/10/10 03:12:56 Unimplemented opcode POLL panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x3983b41] goroutine 112 [running]: github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents.func2.mergeProceesors.1(0xc000513540) /github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:30 +0x41 github.com/seaweedfs/seaweedfs/weed/pb.WithFilerClientFollowMetadata.makeSubscribeMetadataFunc.func1({0x514a560, 0xc0009b06d0}) /github/workspace/weed/pb/filer_pb_tail.go:87 +0x293 github.com/seaweedfs/seaweedfs/weed/mount.(WFS).WithFilerClient.func1.1(0xc000d9e008) /github/workspace/weed/mount/wfs_filer_client.go:27 +0x68 github.com/seaweedfs/seaweedfs/weed/pb.WithGrpcClient(0xa9?, 0x9d1f90fb?, 0xc00098fbe0, {0xc000e1c078, 0x15}, 0x0?, {0xc000a76bd0?, 0x1?, 0x1? }) /github/workspace/weed/pb/grpc_client_server.go:155 +0x33a github.com/seaweedfs/seaweedfs/weed/mount.(WFS).WithFilerClient.func1() /github/workspace/weed/mount/wfs_filer_client.go:25 +0x14c github.com/seaweedfs/seaweedfs/weed/util.Retry({0x489a10c, 0xa}, 0xc00098fcf8) /github/workspace/weed/util/retry.go:16 +0xa7 github.com/seaweedfs/seaweedfs/weed/mount.(WFS).WithFilerClient(0x0?, 0x30?, 0xc00056a568?) /github/workspace/weed/mount/wfs_filer_client.go:18 +0x65 github.com/seaweedfs/seaweedfs/weed/pb.WithFilerClientFollowMetadata({0x50f5898, 0xc000e2c180}, 0xc00101e000, 0xc000d22030) /github/workspace/weed/pb/filer_pb_tail.go:51 +0x87 github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents.func2() /github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:115 +0xcb github.com/seaweedfs/seaweedfs/weed/util.RetryUntil({0x48b8b35, 0x11}, 0xc00098fef0, 0x4af0048) /github/workspace/weed/util/retry.go:64 +0x4e github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents(0xc000d1d980, 0x9d1f90fb, {0x50f5898, 0xc000e2c180}, {0x7ffdf6f98dca, 0x4}, 0x17fcf8b7c7ecf8cf, {0xc0000ac208, 0x1, 0x1}) /github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:113 +0x39b created by github.com/seaweedfs/seaweedfs/weed/mount.(WFS).StartBackgroundTasks in goroutine 1 /github/workspace/weed/mount/weedfs.go:150 +0x146	1 year ago
chrislu	c0e36231ad	use only one metadata follow process	1 year ago
chrislu	11e53dd8ef	skip error if conf file is not created	1 year ago
chrislu	99ae99e3b9	fix nil filer conf fix https://github.com/seaweedfs/seaweedfs/issues/6034	1 year ago
Guang Jiong Lou	6c986e9d70	improve worm support (#5983 ) * improve worm support Signed-off-by: lou <alex1988@outlook.com> * worm mode in filer Signed-off-by: lou <alex1988@outlook.com> * update after review Signed-off-by: lou <alex1988@outlook.com> * update after review Signed-off-by: lou <alex1988@outlook.com> * move to fs configure Signed-off-by: lou <alex1988@outlook.com> * remove flag Signed-off-by: lou <alex1988@outlook.com> * update after review Signed-off-by: lou <alex1988@outlook.com> * support worm hardlink Signed-off-by: lou <alex1988@outlook.com> * update after review Signed-off-by: lou <alex1988@outlook.com> * typo Signed-off-by: lou <alex1988@outlook.com> * sync filer conf Signed-off-by: lou <alex1988@outlook.com> --------- Signed-off-by: lou <alex1988@outlook.com>	1 year ago
Guang Jiong Lou	367e7638d7	fix invalid file read (#6024 )	1 year ago
chrislu	a9c9e1bcb3	refactor	1 year ago
chrislu	eb02946c97	support write once read many fix https://github.com/seaweedfs/seaweedfs/issues/5954	1 year ago
chrislu	18afdb15b6	Revert "weed mount, weed dav add option to force cache" This reverts commit `7367b976b0`.	1 year ago
chrislu	7367b976b0	weed mount, weed dav add option to force cache	1 year ago
chrislu	9ca30e52d5	fuse mount handles ttl entries fix https://github.com/seaweedfs/seaweedfs/issues/5527	1 year ago
chrislu	e86b0bcaaa	simplify	1 year ago
chrislu	57dc39c451	randomizing next file handle id	1 year ago
Chris Lu	952afd810c	Fix dead lock (#5815 ) * reduce locks to avoid dead lock Flush->FlushData->uplloadPipeline.FluahAll uploaderCount>0 goroutine 1 [sync.Cond.Wait, 71 minutes]: sync.runtime_notifyListWait(0xc0007ae4d0, 0x0) /usr/local/go/src/runtime/sema.go:569 +0x159 sync.(Cond).Wait(0xc001a59290?) /usr/local/go/src/sync/cond.go:70 +0x85 github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(UploadPipeline).waitForCurrentWritersToComplete(0xc0002ee4d0) /github/workspace/weed/mount/page_writer/upload_pipeline_lock.go:58 +0x32 github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(UploadPipeline).FlushAll(0xc0002ee4d0) /github/workspace/weed/mount/page_writer/upload_pipeline.go:151 +0x25 github.com/seaweedfs/seaweedfs/weed/mount.(ChunkedDirtyPages).FlushData(0xc00087e840) /github/workspace/weed/mount/dirty_pages_chunked.go:54 +0x29 github.com/seaweedfs/seaweedfs/weed/mount.(PageWriter).FlushData(...) /github/workspace/weed/mount/page_writer.go:50 github.com/seaweedfs/seaweedfs/weed/mount.(WFS).doFlush(0xc0006ad600, 0xc00030d380, 0x0, 0x0) /github/workspace/weed/mount/weedfs_file_sync.go:101 +0x169 github.com/seaweedfs/seaweedfs/weed/mount.(WFS).Flush(0xc0006ad600, 0xc001a594a8?, 0xc0004c1ca0) /github/workspace/weed/mount/weedfs_file_sync.go:59 +0x48 github.com/hanwen/go-fuse/v2/fuse.doFlush(0xc0000da870?, 0xc0004c1b08) SaveContent -> MemChunk.RLock -> ChunkedDirtyPages.saveChunkedFileIntervalToStorage pages.fh.AddChunks([]filer_pb.FileChunk{chunk}) fh.entryLock.Lock() sync.(RWMutex).Lock(0x0?) /usr/local/go/src/sync/rwmutex.go:146 +0x31 github.com/seaweedfs/seaweedfs/weed/mount.(FileHandle).AddChunks(0xc00030d380, {0xc00028bdc8, 0x1, 0x1}) /github/workspace/weed/mount/filehandle.go:93 +0x45 github.com/seaweedfs/seaweedfs/weed/mount.(ChunkedDirtyPages).saveChunkedFileIntervalToStorage(0xc00087e840, {0x2be7ac0, 0xc00018d9e0}, 0x0, 0x121, 0x17e3c624565ace45, 0x1?) /github/workspace/weed/mount/dirty_pages_chunked.go:80 +0x2d4 github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(MemChunk).SaveContent(0xc0008d9130, 0xc0008093e0) /github/workspace/weed/mount/page_writer/page_chunk_mem.go:115 +0x112 github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(UploadPipeline).moveToSealed.func1() /github/workspace/weed/mount/page_writer/upload_pipeline.go:187 +0x55 github.com/seaweedfs/seaweedfs/weed/util.(LimitedConcurrentExecutor).Execute.func1() /github/workspace/weed/util/limited_executor.go:38 +0x62 created by github.com/seaweedfs/seaweedfs/weed/util.(LimitedConcurrentExecutor).Execute in goroutine 1 /github/workspace/weed/util/limited_executor.go:33 +0x97 On metadata update fh.entryLock.Lock() fh.dirtyPages.Destroy() up.chunksLock.Lock => each sealed chunk.FreeReference => MemChunk.Lock goroutine 134 [sync.RWMutex.Lock, 71 minutes]: sync.runtime_SemacquireRWMutex(0xc0007c3558?, 0xea?, 0x3fb0800?) /usr/local/go/src/runtime/sema.go:87 +0x25 sync.(RWMutex).Lock(0xc0007c35a8?) /usr/local/go/src/sync/rwmutex.go:151 +0x6a github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(MemChunk).FreeResource(0xc0008d9130) /github/workspace/weed/mount/page_writer/page_chunk_mem.go:38 +0x2a github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(SealedChunk).FreeReference(0xc00071cdb0, {0xc0006ba1a0, 0x20}) /github/workspace/weed/mount/page_writer/upload_pipeline.go:38 +0xb7 github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(UploadPipeline).Shutdown(0xc0002ee4d0) /github/workspace/weed/mount/page_writer/upload_pipeline.go:220 +0x185 github.com/seaweedfs/seaweedfs/weed/mount.(ChunkedDirtyPages).Destroy(0xc0008cea40?) /github/workspace/weed/mount/dirty_pages_chunked.go:87 +0x17 github.com/seaweedfs/seaweedfs/weed/mount.(PageWriter).Destroy(...) /github/workspace/weed/mount/page_writer.go:78 github.com/seaweedfs/seaweedfs/weed/mount.NewSeaweedFileSystem.func3({0xc00069a6c0, 0x30}, 0x6?) /github/workspace/weed/mount/weedfs.go:119 +0x17a github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.NewMetaCache.func1({0xc00069a6c0?, 0xc00069a480?}, 0x4015b40?) /github/workspace/weed/mount/meta_cache/meta_cache.go:37 +0x1c github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents.func1(0xc000661810) /github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:43 +0x570 use locked entry everywhere * modifiable remote entry * skip locking after getting lock from fhLockTable	1 year ago
vadimartynov	86d92a42b4	Added tls for http clients (#5766 ) * Added global http client * Added Do func for global http client * Changed the code to use the global http client * Fix http client in volume uploader * Fixed pkg name * Fixed http util funcs * Fixed http client for bench_filer_upload * Fixed http client for stress_filer_upload * Fixed http client for filer_server_handlers_proxy * Fixed http client for command_fs_merge_volumes * Fixed http client for command_fs_merge_volumes and command_volume_fsck * Fixed http client for s3api_server * Added init global client for main funcs * Rename global_client to client * Changed: - fixed NewHttpClient; - added CheckIsHttpsClientEnabled func - updated security.toml in scaffold * Reduce the visibility of some functions in the util/http/client pkg * Added the loadSecurityConfig function * Use util.LoadSecurityConfiguration() in NewHttpClient func	1 year ago
chrislu	3a82f5ffad	ensure metadata follow a specific folder fix https://github.com/seaweedfs/seaweedfs/issues/5774	1 year ago
Gaspare Iengo	fce8fc1e16	Fix Issue #5649 (#5652 )	1 year ago
chrislu	d218fe54fa	go fmt	1 year ago
chrislu	d5b2cee34a	log possible racing condition fix https://github.com/seaweedfs/seaweedfs/issues/5412	2 years ago
chrislu	2a4a9c6343	log possible racing condition fix https://github.com/seaweedfs/seaweedfs/issues/5412	2 years ago
chrislu	6aa804b368	lock instead of RLock, to prevent racing condition https://github.com/seaweedfs/seaweedfs/discussions/5432#discussioncomment-8933608	2 years ago
wusong	a5582cc921	[mount] fix GetAttr blocks count (#5409 )	2 years ago
Viktor Kuzmin	06e723f629	Waiting for writes to complete with acquired locks may lead to deadlock, fixes #4952	2 years ago
chrislu	310eec9a9c	skip if path changed mitigate https://github.com/seaweedfs/seaweedfs/issues/4968	2 years ago
chrislu	cbc24c7b24	disconnect from old subscribers	2 years ago
chrislu	b02fdeabff	RetryForever => RetryUntil	2 years ago
chrislu	186b1f029b	read source uses shared lock	2 years ago
chrislu	31fc165715	mount switch to ordered lock requests	2 years ago
chrislu	27af11f1e8	Revert "Revert "Merge branch 'master' into sub"" This reverts commit `0bb97709d4`.	2 years ago
chrislu	0bb97709d4	Revert "Merge branch 'master' into sub" This reverts commit `4d414f54a2`, reversing changes made to `4827425146`.	2 years ago

1 2 3 4 5 ...

263 Commits (804979d68bdc1763878b91741e22c913fba62d73)