seaweedfs

Commit Graph

Author	SHA1	Message	Date
Konstantin Lebedev	1f7e52c63e	vacuum metrics and force sync dst files (#3832 )	3 years ago
chrislu	e55076c46f	cloud tier: add retry when copying data file fix https://github.com/seaweedfs/seaweedfs/issues/3828	3 years ago
Konstantin Lebedev	5b28c3f728	revert disabling FSync for non Mac (#3814 )	3 years ago
Ryan Russell	277976bd76	refactor(storage): readability improvements (#3703 ) Signed-off-by: Ryan Russell <git@ryanrussell.org> Signed-off-by: Ryan Russell <git@ryanrussell.org>	3 years ago
chrislu	21c0587900	go fmt	3 years ago
chrislu	39340f7e42	cloud tier: s3 consume all read response body fix https://github.com/seaweedfs/seaweedfs/issues/3584	3 years ago
chrislu	93261f5199	Revert "Refactor for Sync method (#3426 )" This reverts commit `670cb759f8`. with the pr weed/storage () - (master) > go test -count=1 ./... ok github.com/seaweedfs/seaweedfs/weed/storage 18.486s ? github.com/seaweedfs/seaweedfs/weed/storage/backend [no test files] ok github.com/seaweedfs/seaweedfs/weed/storage/backend/memory_map 0.025s ? github.com/seaweedfs/seaweedfs/weed/storage/backend/s3_backend [no test files] ok github.com/seaweedfs/seaweedfs/weed/storage/erasure_coding 0.864s ? github.com/seaweedfs/seaweedfs/weed/storage/idx [no test files] ok github.com/seaweedfs/seaweedfs/weed/storage/needle 0.110s ok github.com/seaweedfs/seaweedfs/weed/storage/needle_map 24.414s ok github.com/seaweedfs/seaweedfs/weed/storage/super_block 0.203s ? github.com/seaweedfs/seaweedfs/weed/storage/types [no test files] ? github.com/seaweedfs/seaweedfs/weed/storage/volume_info [no test files] weed/storage () - (master) > weed/storage () - (master) > without the pr weed/storage () - (master) > weed/storage () - (master) > go test -count=1 ./... ok github.com/seaweedfs/seaweedfs/weed/storage 1.617s ? github.com/seaweedfs/seaweedfs/weed/storage/backend [no test files] ok github.com/seaweedfs/seaweedfs/weed/storage/backend/memory_map 0.026s ? github.com/seaweedfs/seaweedfs/weed/storage/backend/s3_backend [no test files] ok github.com/seaweedfs/seaweedfs/weed/storage/erasure_coding 0.906s ? github.com/seaweedfs/seaweedfs/weed/storage/idx [no test files] ok github.com/seaweedfs/seaweedfs/weed/storage/needle 0.202s ok github.com/seaweedfs/seaweedfs/weed/storage/needle_map 24.533s ok github.com/seaweedfs/seaweedfs/weed/storage/super_block 0.280s ? github.com/seaweedfs/seaweedfs/weed/storage/types [no test files] ? github.com/seaweedfs/seaweedfs/weed/storage/volume_info [no test files]	3 years ago
Rain Li	670cb759f8	Refactor for Sync method (#3426 )	3 years ago
chrislu	26dbc6c905	move to https://github.com/seaweedfs/seaweedfs	3 years ago
chrislu	52580743b9	stuck with file.Sync() starting to get weed/storage () - (master) > go test ./... I0630 12:19:54 65819 needle_map_metric_test.go:26] FileCount expected 10000 actual 10000 I0630 12:19:54 65819 needle_map_metric_test.go:27] DeletedSize expected 1648 actual 1648 I0630 12:19:54 65819 needle_map_metric_test.go:28] ContentSize expected 10000 actual 10000 I0630 12:19:54 65819 needle_map_metric_test.go:29] DeletedCount expected 1648 actual 1959 I0630 12:19:54 65819 needle_map_metric_test.go:30] MaxFileKey expected 10000 actual 10000 I0630 12:19:54 65819 volume_loading.go:136] loading index /var/folders/jv/0rlhdck17jzgt7b3hcnq63mc0000gp/T/TestCompaction909350722/001/1.idx to memory I0630 12:20:36 65819 volume_vacuum.go:98] Committing volume 1 vacuuming... panic: test timed out after 10m0s goroutine 61 [running]: testing.(M).startAlarm.func1() /usr/local/go/src/testing/testing.go:2029 +0x8e created by time.goFunc /usr/local/go/src/time/sleep.go:176 +0x32 goroutine 1 [chan receive, 10 minutes]: testing.(T).Run(0xc0000cd520, {0x1c09bde?, 0x5ab798ea5c102?}, 0x1c67bd8) /usr/local/go/src/testing/testing.go:1487 +0x37a testing.runTests.func1(0xc0000cd520?) /usr/local/go/src/testing/testing.go:1839 +0x6e testing.tRunner(0xc0000cd520, 0xc00039fcd8) /usr/local/go/src/testing/testing.go:1439 +0x102 testing.runTests(0xc0000dadc0?, {0x25b4460, 0x6, 0x6}, {0x2855108?, 0x40?, 0x25c4b80?}) /usr/local/go/src/testing/testing.go:1837 +0x457 testing.(M).Run(0xc0000dadc0) /usr/local/go/src/testing/testing.go:1719 +0x5d9 main.main() _testmain.go:57 +0x1aa goroutine 21 [chan receive]: github.com/chrislusf/seaweedfs/weed/glog.(loggingT).flushDaemon(0x0?) /Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/glog/glog.go:883 +0x6a created by github.com/chrislusf/seaweedfs/weed/glog.init.0 /Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/glog/glog.go:410 +0x1bf goroutine 55 [syscall]: syscall.syscall(0x44ac0f96?, 0xc00007ad80?, 0xc00098b480?, 0x10588df?) /usr/local/go/src/runtime/sys_darwin.go:22 +0x4e syscall.fcntl(0x100000001?, 0xc00098b4b0?, 0x1069a7c?) /usr/local/go/src/syscall/zsyscall_darwin_amd64.go:319 +0x30 internal/poll.(FD).Fsync.func1(...) /usr/local/go/src/internal/poll/fd_fsync_darwin.go:18 internal/poll.ignoringEINTR(...) /usr/local/go/src/internal/poll/fd_posix.go:74 internal/poll.(FD).Fsync(0xc00038f1e0?) /usr/local/go/src/internal/poll/fd_fsync_darwin.go:17 +0xfc os.(File).Sync(0xc00012a030) /usr/local/go/src/os/file_posix.go:168 +0x4e github.com/chrislusf/seaweedfs/weed/storage/backend.(DiskFile).Sync(...) /Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/backend/disk_file.go:84 github.com/chrislusf/seaweedfs/weed/storage.(Volume).makeupDiff(0xc0000bb440, {0xc000633a40, 0x52}, {0xc000633aa0, 0x52}, {0xc000633b00, 0x52}, {0xc000633b60, 0x52}) /Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/volume_vacuum.go:295 +0x12fa github.com/chrislusf/seaweedfs/weed/storage.(Volume).CommitCompact(0xc0000bb440) /Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/volume_vacuum.go:119 +0x3e8 github.com/chrislusf/seaweedfs/weed/storage.TestCompaction(0xc00025a000) /Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/volume_vacuum_test.go:89 +0x305 testing.tRunner(0xc00025a000, 0x1c67bd8) /usr/local/go/src/testing/testing.go:1439 +0x102 created by testing.(T).Run /usr/local/go/src/testing/testing.go:1486 +0x35f goroutine 56 [chan receive, 10 minutes]: github.com/chrislusf/seaweedfs/weed/storage.(Volume).startWorker.func1() /Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/volume_write.go:244 +0x41a created by github.com/chrislusf/seaweedfs/weed/storage.(*Volume).startWorker /Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/volume_write.go:234 +0x56 FAIL github.com/chrislusf/seaweedfs/weed/storage 600.194s ? github.com/chrislusf/seaweedfs/weed/storage/backend [no test files] ok github.com/chrislusf/seaweedfs/weed/storage/backend/memory_map (cached) ? github.com/chrislusf/seaweedfs/weed/storage/backend/s3_backend [no test files] ok github.com/chrislusf/seaweedfs/weed/storage/erasure_coding (cached) ? github.com/chrislusf/seaweedfs/weed/storage/idx [no test files] ok github.com/chrislusf/seaweedfs/weed/storage/needle (cached) ok github.com/chrislusf/seaweedfs/weed/storage/needle_map (cached) ok github.com/chrislusf/seaweedfs/weed/storage/super_block (cached) ? github.com/chrislusf/seaweedfs/weed/storage/types [no test files] ? github.com/chrislusf/seaweedfs/weed/storage/volume_info [no test files] FAIL	3 years ago
chrislu	3e20336bde	fix atomic add int64 fix https://github.com/chrislusf/seaweedfs/issues/3038#issuecomment-1123269831	4 years ago
chrislu	76148ce0f7	use storage_class for backend configuration fix https://github.com/chrislusf/seaweedfs/issues/3008	4 years ago
chrislu	1aae7a3f1b	volume.tier.upload progress starts negative #2992 fix https://github.com/chrislusf/seaweedfs/issues/2992	4 years ago
chrislu	192983b464	s3 backend support customizing storage class	4 years ago
Konstantin Lebedev	f53cff045f	checks disk file exist	4 years ago
Chris Lu	0c8dea9de8	go fmt	4 years ago
Chris Lu	c857cc7286	cloud tier: remove tagging since not all s3 vendors support this	4 years ago
Chris Lu	d04cdcf40d	s3 header add user agent	4 years ago
Chris Lu	f58ea6a2ee	add source name to error message	4 years ago
Chris Lu	7ce97b59d8	go fmt	4 years ago
Chris Lu	c08ac536ed	cloud drive: add support for Wasabi * disable md5, sha256 checking to avoid reading one chunk twice * single threaded upload to avoid chunk swapping (to be enhanced later)	4 years ago
Chris Lu	00c4e06caa	cloud drive: s3 configurable force path style	4 years ago
Chris Lu	f0cc130849	do not force path style for better compatibility	4 years ago
Chris Lu	400de380f4	volume server: support tcp direct put/get/delete	5 years ago
Chris Lu	d680676d45	skip already loaded backends	5 years ago
Chris Lu	2cca07b44c	cloud tier to non-AWS s3 gateways	5 years ago
Chris Lu	7635f6b9fa	disk file avoid file.Stat()	5 years ago
Chris Lu	a2383b3b12	Revert "Revert "avoid file.Stat()"" This reverts commit `7ef4c24f18`.	5 years ago
Chris Lu	7ef4c24f18	Revert "avoid file.Stat()" This reverts commit `98c93ca465`.	5 years ago
Chris Lu	98c93ca465	avoid file.Stat() avoid one Syscall, but did not help on performance though	5 years ago
Chris Lu	a1210d1e8d	Revert "volume: avoid file.stat(), file.seek() if possible during writes" This reverts commit `c78409a598`.	5 years ago
Chris Lu	c78409a598	volume: avoid file.stat(), file.seek() if possible during writes	5 years ago
Chris Lu	cfb9342a15	avoid concurrent map updates to viper	5 years ago
Chris Lu	d522df0ecd	adjust log level	5 years ago
Chris Lu	627b081b67	adjust logs	5 years ago
Chris Lu	7764e0465c	refactoring	6 years ago
Chris Lu	c8ca234773	refactoring	6 years ago
Chris Lu	81797a059a	volume: sync volume file right before compaction fix https://github.com/chrislusf/seaweedfs/issues/1237	6 years ago
Chris Lu	709f231e23	tiered storage: add s3 endpoint for private s3 implementation fix https://github.com/chrislusf/seaweedfs/issues/1238	6 years ago
Chris Lu	d335f04de6	support env variables to overwrite toml file	6 years ago
Chris Lu	2000284435	rename volume tier info to volume info	6 years ago
Chris Lu	48d28d3eb2	tier: support remote file attributes and remember the file extension	6 years ago
Chris Lu	eecad6062a	go fmt	6 years ago
Chris Lu	d960b3474a	tier storage: support downloading the remote dat files	6 years ago
Chris Lu	356bd1b629	adjust text	6 years ago
Chris Lu	10bd3c6b4b	refactoring	6 years ago
Chris Lu	2b8e20f122	remove unused functions	6 years ago
Chris Lu	70648d35ad	go fmt	6 years ago
Chris Lu	caae543a9f	fix test	6 years ago
Chris Lu	ec8de250e2	tiered storage: can copy to s3, read from s3 master not aware tiered volume yet, file assigning is not working yet	6 years ago

1 2

62 Commits (f0b2650adb09bfc6b667dddebb46487d39e58976)