seaweedfs

Commit Graph

Author	SHA1	Message	Date
Nikita Mochalov	4b1ba7f5b2	Save disk space metrics immediately (#4740 )	2 years ago
Nikita Mochalov	e6a49dc533	Fix resource leaks (#4737 ) * Fix division by zero * Fix file handle leak * Fix file handle leak * Fix file handle leak * Fix goroutine leak	2 years ago
LHHDZ	3730772e05	refresh max volume count after adjust (#4662 ) Signed-off-by: changlin.shi <changlin.shi@ly.com>	2 years ago
chrislu	c278bac263	avoid nil needle map fix https://github.com/seaweedfs/seaweedfs/issues/4640	2 years ago
chrislu	14546f0ad8	minor	2 years ago
wusong	61553beba2	Fix DataBackend nil pointer (#4641 )	2 years ago
chrislu	2aa59ab37c	fix copying level db files during commi fix https://github.com/seaweedfs/seaweedfs/issues/4635	2 years ago
Damiano Albani	0d9ba8c612	Support templating name of files stored via Rclone backend (#4606 )	2 years ago
Bai Jie	3b88ab42aa	remove duplicate `fileCount` query (#4588 )	2 years ago
Bai Jie	44b9d72ef0	doIsEmpty() return error if v.DataBackend is nil (#4587 )	2 years ago
柏杰	0b0fb9b9e4	avoid data race read volume.IsEmpty (#4574 ) * avoid data race read volume.IsEmpty - avoid phantom read isEmpty for onlyEmpty - use `v.DataBackend.GetStat()` in v.dataFileAccessLock scope * add Destroy(onlyEmpty: true) test * add Destroy(onlyEmpty: false) test * remove unused `IsEmpty()` * change literal `8` to `SuperBlockSize`	2 years ago
Konstantin Lebedev	1e22d5caf2	fix get file stats for IsEmpty (#4576 )	2 years ago
Konstantin Lebedev	4527ead295	fix from comment delete volume is empty (#4573 ) * fix from coments https://github.com/seaweedfs/seaweedfs/pull/4561 * fix tests --------- Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>	2 years ago
Konstantin Lebedev	25535e9c36	Delete volume is empty (#4561 ) * use onlyEmpty for deleteVolume https://github.com/seaweedfs/seaweedfs/issues/4559 * fix IsEmpty * fix test --------- Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>	2 years ago
wusong	26f15d0079	Fix no more writable volumes by delay judgment (#4548 ) * fix nomore writables volumes while disk free space is sufficient by time delay * reset --------- Co-authored-by: wang wusong <wangwusong@virtaitech.com>	2 years ago
chrislu	e23f3d6eca	set df.File to nil after it is closed possibly fix https://github.com/seaweedfs/seaweedfs/issues/4530	2 years ago
chrislu	ff7b6d779e	avoid overwriting variables fix https://github.com/seaweedfs/seaweedfs/issues/4365	3 years ago
Damiano Albani	9f55c7c90e	Rclone storage backend (#4402 ) * Add Rclone storage backend * Support templating the name of files stored via Rclone * Enable Rclone accounting * Remove redundant type conversion * Provide progress information for Rclone download/upload operations * Log error when Rclone can't instantiate filesystem * Remove filename templating functionality for Rclone storage To (maybe) be later reintroduced as a generic functionality for all storage backends. * Remove S3 specific check * Move Rclone config initialisation to init() method	3 years ago
chrislu	0a22eea55d	collect ec shard from multiple locations fix https://github.com/seaweedfs/seaweedfs/issues/4365	3 years ago
chrislu	f5854d13df	fix test	3 years ago
chrislu	0454bb2d88	fix test when size is zero, the "LastModified" is not written	3 years ago
chrislu	c589e9837c	fix tests	3 years ago
zemul	bd6f2716d3	fix_read_tomb_needle_meta (#4263 ) Co-authored-by: zemul <zhouzemiao@ihuman.com>	3 years ago
James Hartig	81624de27b	Include name/mime in ReadAllNeedles (#4005 )	3 years ago
James Hartig	4c85da7844	Include meta in ReadAllNeedles (#3991 ) This is useful for doing backups on the data so we can accurately store the last modified time, the compression state, and verify the crc. Previously we were doing VolumeNeedleStatus and then an HTTP request which needlessly read from the dat file twice.	3 years ago
Guo Lei	5b905fb2b7	Lazy loading (#3958 ) * types packages is imported more than onece * lazy-loading * fix bugs * fix bugs * fix unit tests * fix test error * rename function * unload ldb after initial startup * Don't load ldb when starting volume server if ldbtimeout is set. * remove uncessary unloadldb * Update weed/command/server.go Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> * Update weed/command/volume.go Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: guol-fnst <goul-fnst@fujitsu.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>	3 years ago
Konstantin Lebedev	bf8a9d2db1	[volume.chek.disk] sync of deletions the fix (#3923 ) * sync of deletions the fix * avoid return if only partiallyDeletedNeedles * refactor sync deletions	3 years ago
Konstantin Lebedev	764d9cb105	[voluche.chek.disk] needles older than the cutoff time are not missing yet (#3922 ) needles older than the cutoff time are not missing yet https://github.com/seaweedfs/seaweedfs/issues/3919	3 years ago
Eric Yang	51d462f204	ADHOC: volume fsck using append at ns (#3906 ) * ADHOC: volume fsck using append at ns * nit * nit Co-authored-by: root <root@HQ-10MSTD3EY.roblox.local>	3 years ago
chrislu	d880fc2bb3	fix merge	3 years ago
Konstantin Lebedev	6253058d9d	ensure monotonic n.AppendAtNs in each place (#3880 ) https://github.com/seaweedfs/seaweedfs/issues/3852 Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>	3 years ago
Konstantin Lebedev	452202f35b	ReadNeedleMeta recovery panic occurred (#3883 ) ReadNeedleMeta panic occurred https://github.com/seaweedfs/seaweedfs/issues/3882	3 years ago
chrislu	e3ea544d40	skip reading needle if size is invalid related to https://github.com/seaweedfs/seaweedfs/issues/3882	3 years ago
chrislu	f2d9049e6a	fix size variable	3 years ago
chrislu	184fbb6c50	volume server: remote tier volumes only soft delete in local index fix https://github.com/seaweedfs/seaweedfs/issues/3889	3 years ago
Konstantin Lebedev	e20f0dbd2d	avoid data race of TraverseBfs (#3856 ) * avoid data race of TraverseBfs * close is enough avoid panic I1014 12:29:59.207120 volume_loading.go:131 loading sorted db /tmp/sw/test2_19.sdx error: unexpected file /tmp/sw/test2_19.idx size: 255 I1014 12:29:59.207125 volume_loading.go:119 open to write file /tmp/sw/test4_26.idx panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x5260a4c] goroutine 287 [running]: github.com/seaweedfs/seaweedfs/weed/storage.(SortedFileNeedleMap).Close(0x0) /Users/tochka/GolandProjects/seaweedfs/weed/storage/needle_map_sorted_file.go:97 +0x2c github.com/seaweedfs/seaweedfs/weed/storage.(Volume).load.func1() /Users/tochka/GolandProjects/seaweedfs/weed/storage/volume_loading.go:32 +0x8e github.com/seaweedfs/seaweedfs/weed/storage.(Volume).load(0xc001b36280, 0x1, 0x1, 0x0, 0x69228c0?) /Users/tochka/GolandProjects/seaweedfs/weed/storage/volume_loading.go:205 +0x256c github.com/seaweedfs/seaweedfs/weed/storage.NewVolume({0x7ffeefbff6e0, 0x7}, {0x7ffeefbff6e0, 0x7}, {0xc0009a9284, 0x5}, 0x13, 0x0, 0x0, 0x0, ...) /Users/tochka/GolandProjects/seaweedfs/weed/storage/volume.go:62 +0x30f github.com/seaweedfs/seaweedfs/weed/storage.(DiskLocation).loadExistingVolume(0xc0006f40c0, {0x846c8d0, 0xc0009ce600}, 0x0?, 0x1) /Users/tochka/GolandProjects/seaweedfs/weed/storage/disk_location.go:161 +0x4da github.com/seaweedfs/seaweedfs/weed/storage.(DiskLocation).concurrentLoadingVolumes.func2() /Users/tochka/GolandProjects/seaweedfs/weed/storage/disk_location.go:201 +0xf9 created by github.com/seaweedfs/seaweedfs/weed/storage.(DiskLocation).concurrentLoadingVolumes /Users/tochka/GolandProjects/seaweedfs/weed/storage/disk_location.go:198 +0x150	3 years ago
Konstantin Lebedev	2f72103c83	avoid load volume file with BytesOffset mismatch (#3841 ) * avoid load volume file with BytesOffset mismatch https://github.com/seaweedfs/seaweedfs/issues/2966 * set BytesOffset if has not VolumeInfoFile * typos fail => failed * exit if bytesOffset mismatch	3 years ago
chrislu	9c8678ded9	ensure monotonic n.AppendAtNs fix https://github.com/seaweedfs/seaweedfs/issues/3852	3 years ago
Konstantin Lebedev	1f7e52c63e	vacuum metrics and force sync dst files (#3832 )	3 years ago
Guo Lei	f95c25e113	types packages is imported more than onece (#3838 )	3 years ago
chrislu	e55076c46f	cloud tier: add retry when copying data file fix https://github.com/seaweedfs/seaweedfs/issues/3828	3 years ago
Guo Lei	84c401e693	Optimiz leveldb metric (#3830 ) * optimiz updating mapmetric for leveldb * import loading leveldb * add comments	3 years ago
Konstantin Lebedev	5b28c3f728	revert disabling FSync for non Mac (#3814 )	3 years ago
binbinshi	b7de4a967e	fix: compact_map get error mismatching cokie (#3748 ) * fix: compact_map get error * fix: CompactSection delete lock and move test to compact_map Co-authored-by: shibinbin <shibinbin@megvii.com>	3 years ago
Viktor Kuzmin	f8d3ff466d	Allow parallel volume loading from different dirs during startup. (#3802 )	3 years ago
Eric Yang	56c94cc08e	ADHOC: filter deleted files from idx file binary search (#3763 ) * ADHOC: filter deleted files from idx file binary search * remove unwanted check Co-authored-by: root <root@HQ-10MSTD3EY.roblox.local>	3 years ago
Konstantin Lebedev	301b678147	[volume] Add new volumes to HUP(reload) signal (#3755 ) Add new volumes to HUP(reload) signal	3 years ago
chrislu	06748f2771	remove unused code	3 years ago
Ryan Russell	277976bd76	refactor(storage): readability improvements (#3703 ) Signed-off-by: Ryan Russell <git@ryanrussell.org> Signed-off-by: Ryan Russell <git@ryanrussell.org>	3 years ago
famosss	d949a238b8	volume: add "readBufSize" option to customize read optimization (#3702 ) * simplify a bit * feat: volume: add "readBufSize" option to customize read optimization * refactor : redbufSIze -> readBufferSize * simplify a bit * simplify a bit	3 years ago

1 2 3 4 5 ...

808 Commits (dbcba75271a4617b5931c4779ca0f7c924369a8f)