seaweedfs

Commit Graph

Author	SHA1	Message	Date
Max Denushev	a5fe6e21bc	feat(filer.backup): add ignore errors option (#6235 ) * feat(filer.backup): add ignore errors option * feat(filer.backup): fix 404 error wrap * feat(filer.backup): fix wrapping function * feat(filer.backup): fix wrapping errors in genProcessFunction * Update weed/command/filer_backup.go * Update weed/command/filer_backup.go * Update weed/command/filer_backup.go --------- Co-authored-by: Max Denushev <denushev@tochka.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>	8 months ago
Eugeniy E. Mikhailov	3003c9e17e	added backward compatible CRC check and extra explanatory comments (#6233 )	9 months ago
Chris Lu	72b14a451e	delete aborted ec shards from both source and target servers (#6221 ) fix https://github.com/seaweedfs/seaweedfs/issues/6205#issuecomment-2465004586	9 months ago
chrislu	113c9ce6a8	remove the direct_io flag, as it is not well-supported on macOS	9 months ago
chrislu	88a1eee31c	adjust logs	9 months ago
chrislu	62c4980754	add debug info	9 months ago
Konstantin Lebedev	9a5d3e7b31	[shell] add admin noLock for balance (#6209 ) add admin noLock for balance	9 months ago
Bruce	15fc58bfd9	remove dup code for section.reader access (#6169 )	9 months ago
chrislu	706d1a8e22	fix tests	9 months ago
chrislu	90acfd9394	remove case when read request is out side of the file size	9 months ago
chrislu	6161d0b05d	a bit optimization	9 months ago
chrislu	98b519b113	fix FUSE mount on mac	9 months ago
Lisandro Pin	efdebf712e	Refactor `ec.balance` logic into a `weeed/shell/command_ec_common.go`… (#6195 ) * Refactor `ec.balance` logic into a `weeed/shell/command_ec_common.go` standalone function. This is a prerequisite to unify the balance logic for `ec.balance` and `ec.encode'. * s/Balance()/EcBalance()/g	9 months ago
Chris Lu	dc784bf217	merge current message queue code changes (#6201 ) * listing files to convert to parquet * write parquet files * save logs into parquet files * pass by value * compact logs into parquet format * can skip existing files * refactor * refactor * fix compilation * when no partition found * refactor * add untested parquet file read * rename package * refactor * rename files * remove unused * add merged log read func * parquet wants to know the file size * rewind by time * pass in stop ts * add stop ts * adjust log * minor * adjust log * skip .parquet files when reading message logs * skip non message files * Update subscriber_record.go * send messages * skip message data with only ts * skip non log files * update parquet-go package * ensure a valid record type * add new field to a record type * Update read_parquet_to_log.go * fix parquet file name generation * separating reading parquet and logs * add key field * add skipped logs * use in memory cache * refactor * refactor * refactor * refactor, and change compact log * refactor * rename * refactor * fix format * prefix v to version directory	9 months ago
wyang	a7973ed7d1	fix deadlock hang when broadcast to clients (#6184 ) fix deadlock when broadcast to clients when master thransfer leader, the old master will disconnect with all filers and volumeServers, if the cluster is a big , the broadcast messages may be more big than the max of the channel len 100, then if the KeepConnect was not listen on the channel in disconnect, it will deadlock. and the whole cluster will not serve!	9 months ago
Konstantin Lebedev	5bddf0c085	[shell] volume.balance collect volume servers by dc rack node (#6191 ) * chore: balance by rack * fix: rm check lock * fix: selected racks * fix: selected nodes * fix: containts * fix: one collectVolumeServersByDcRackNode * fix: revert lock and add lock * fix: panic test * revert noLock	9 months ago
han	65fb8fad99	Update filer_multipart.go (#6188 ) fix the #6177 bug	9 months ago
Bruce	0060a2cf9c	Fix 6181/6182 (#6183 ) * set larger buf size for LogBuffer * jump to next day when no more entry found * Update weed/filer/filer_notify_read.go --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>	9 months ago
wyang	c29c912bdc	fix format (#6185 ) unitest weed/shell fail	9 months ago
chrislu	ad6240f066	fix parameter	9 months ago
chrislu	8802843511	pass in bytes buffer to avoid possible race condition related to https://github.com/seaweedfs/seaweedfs/issues/6180	9 months ago
chrislu	3feb66d0a1	adding more debug message related to https://github.com/seaweedfs/seaweedfs/issues/6180	9 months ago
chrislu	228946369c	3.79	9 months ago
chrislu	9873b033d1	backward compatible vif loading	9 months ago
chrislu	9105c6bdd1	fix format	9 months ago
chrislu	ebbb35d533	3.78	9 months ago
chrislu	089d4316ef	ensure 2 volume space since actual need 1.4x volume size empty space	9 months ago
chrislu	6e388e29c9	correcting free volume count, factor it during ec encoding to ensure enough disk space available fix https://github.com/seaweedfs/seaweedfs/issues/6163	9 months ago
chrislu	8d6189bcc5	adjust output format	9 months ago
chrislu	2f3d820f52	rename proto field This should not have any impact.	9 months ago
chrislu	ae5bd0667a	rename proto field from DestroyTime to expire_at_sec For TTL volume converted into EC volume, this change may leave the volumes staying.	9 months ago
chrislu	1c2b10303a	remove X-Forwarded-For fix https://github.com/seaweedfs/seaweedfs/issues/6136	9 months ago
chrislu	73921ce4f6	adjust help message	9 months ago
Guang Jiong Lou	9369a88c5c	stop renaming worm files (#6154 ) * stop renaming worm file Signed-off-by: lou <alex1988@outlook.com> * update after review Signed-off-by: lou <alex1988@outlook.com> * Update weed/server/filer_server_handlers_write.go --------- Signed-off-by: lou <alex1988@outlook.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>	9 months ago
chrislu	d38456dc06	do not delete collection metrics when unmount a volume fix https://github.com/seaweedfs/seaweedfs/issues/6139	9 months ago
chrislu	6c7824c813	consistent sorted tags elastic,gocdk,rclone,sqlite,tikv,ydb	9 months ago
chrislu	0df4a4298b	adjust comment	9 months ago
Guang Jiong Lou	54b91ba2b2	fix permission check in put request (#6142 )	9 months ago
dsd	1e13b6879c	fix(volume): to avoid duplicate write a same needle (#6138 ) fix WriteNeedleBlob to avoid duplicate write a same needle Co-authored-by: 邓书东 <shudong_deng@hhnb2024010108.intsig.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>	9 months ago
chrislu	f11567816e	Revert "fix: volume data integrity check (#6118 )" This reverts commit `e12b9ab746`.	9 months ago
wusong	e12b9ab746	fix: volume data integrity check (#6118 ) Co-authored-by: wang wusong <wangwusong@virtaitech.com>	10 months ago
steve.wei	cfbe45c765	feat: add in-flight metric for s3/file/volume-server (#6120 )	10 months ago
chrislu	804e765397	remove unused	10 months ago
chrislu	6845e25318	set proxied request content length	10 months ago
chrislu	35fd1e1c9a	optimize memory usage for large number of volumes 1. unwrap the map to avoid extra map object creation 2. fix ec shard counting in UpdateEcShards	10 months ago
chrislu	b28b1a3402	3.77	10 months ago
chrislu	07df5c53de	avoid nil I1010 03:10:25.220345 mount_std.go:281 This is SeaweedFS version 30GB 3.76 `82ed61c655` linux amd64 ^[OR^[OR2024/10/10 03:12:56 Unimplemented opcode POLL panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x3983b41] goroutine 112 [running]: github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents.func2.mergeProceesors.1(0xc000513540) /github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:30 +0x41 github.com/seaweedfs/seaweedfs/weed/pb.WithFilerClientFollowMetadata.makeSubscribeMetadataFunc.func1({0x514a560, 0xc0009b06d0}) /github/workspace/weed/pb/filer_pb_tail.go:87 +0x293 github.com/seaweedfs/seaweedfs/weed/mount.(WFS).WithFilerClient.func1.1(0xc000d9e008) /github/workspace/weed/mount/wfs_filer_client.go:27 +0x68 github.com/seaweedfs/seaweedfs/weed/pb.WithGrpcClient(0xa9?, 0x9d1f90fb?, 0xc00098fbe0, {0xc000e1c078, 0x15}, 0x0?, {0xc000a76bd0?, 0x1?, 0x1? }) /github/workspace/weed/pb/grpc_client_server.go:155 +0x33a github.com/seaweedfs/seaweedfs/weed/mount.(WFS).WithFilerClient.func1() /github/workspace/weed/mount/wfs_filer_client.go:25 +0x14c github.com/seaweedfs/seaweedfs/weed/util.Retry({0x489a10c, 0xa}, 0xc00098fcf8) /github/workspace/weed/util/retry.go:16 +0xa7 github.com/seaweedfs/seaweedfs/weed/mount.(WFS).WithFilerClient(0x0?, 0x30?, 0xc00056a568?) /github/workspace/weed/mount/wfs_filer_client.go:18 +0x65 github.com/seaweedfs/seaweedfs/weed/pb.WithFilerClientFollowMetadata({0x50f5898, 0xc000e2c180}, 0xc00101e000, 0xc000d22030) /github/workspace/weed/pb/filer_pb_tail.go:51 +0x87 github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents.func2() /github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:115 +0xcb github.com/seaweedfs/seaweedfs/weed/util.RetryUntil({0x48b8b35, 0x11}, 0xc00098fef0, 0x4af0048) /github/workspace/weed/util/retry.go:64 +0x4e github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents(0xc000d1d980, 0x9d1f90fb, {0x50f5898, 0xc000e2c180}, {0x7ffdf6f98dca, 0x4}, 0x17fcf8b7c7ecf8cf, {0xc0000ac208, 0x1, 0x1}) /github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:113 +0x39b created by github.com/seaweedfs/seaweedfs/weed/mount.(WFS).StartBackgroundTasks in goroutine 1 /github/workspace/weed/mount/weedfs.go:150 +0x146	10 months ago
chrislu	82ed61c655	3.76	10 months ago
chrislu	c0e36231ad	use only one metadata follow process	10 months ago
Konstantin Lebedev	44b275879b	[s3] add {Get,Put,Delete}BucketTagging and PublicAccessBlock Handlers (#6088 ) * add {Get,Put,Delete}BucketTagging Handlers * s3 add skip bucket PublicAccessBlock handlers --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>	10 months ago

... 6 7 8 9 10 ...

7516 Commits (fix-versioning-listing-only)