6c986e9d70 
								
									
								
							
								 
							
						 
						
							
							
								
								improve worm support ( #5983 )  
							
							
 
							
							
							* improve worm support
Signed-off-by: lou <alex1988@outlook.com>
* worm mode in filer
Signed-off-by: lou <alex1988@outlook.com>
* update after review
Signed-off-by: lou <alex1988@outlook.com>
* update after review
Signed-off-by: lou <alex1988@outlook.com>
* move to fs configure
Signed-off-by: lou <alex1988@outlook.com>
* remove flag
Signed-off-by: lou <alex1988@outlook.com>
* update after review
Signed-off-by: lou <alex1988@outlook.com>
* support worm hardlink
Signed-off-by: lou <alex1988@outlook.com>
* update after review
Signed-off-by: lou <alex1988@outlook.com>
* typo
Signed-off-by: lou <alex1988@outlook.com>
* sync filer conf
Signed-off-by: lou <alex1988@outlook.com>
---------
Signed-off-by: lou <alex1988@outlook.com> 
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								15965f7c54 
								
									
								
							
								 
							
						 
						
							
							
								
								[shell] fix volume grow in shell ( #5992 )  
							
							
 
							
							
							* fix volume grow in shell
* revert add Async
* check available volume space
* create a VolumeGrowRequest and remove unnecessary fields 
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								282867a71e 
								
									
								
							
								 
							
						 
						
							
							
								
								Show volume server which failed to verify a chunk ( #5958 )  
							
							
								
 
							
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								766290411b 
								
									
								
							
								 
							
						 
						
							
							
								
								filer.remote.gateway not filer.remote.sync for remote.mount.buckets help string. ( #5934 )  
							
							
								
 
							
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								0b00706454 
								
									
								
							
								 
							
						 
						
							
							
								
								EC volume supports expiration and displays expiration message when executing volume.list ( #5895 )  
							
							
 
							
							
							* ec volume expire
* volume.list show DestroyTime
* comments
* code optimization
---------
Co-authored-by: xuwenfeng <xuwenfeng1@zto.com> 
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								8f1f1730e9 
								
							
								 
							
						 
						
							
							
								
								minor  
							
							
								
 
							
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								e14b6e02df 
								
							
								 
							
						 
						
							
							
								
								load ec volume id locations  
							
							
 
							
							
							fix https://github.com/seaweedfs/seaweedfs/issues/5713  
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								e825dd7496 
								
							
								 
							
						 
						
							
							
								
								fix opposite logic  
							
							
 
							
							
							fix https://github.com/seaweedfs/seaweedfs/issues/5871  
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								e069009f09 
								
							
								 
							
						 
						
							
							
								
								[remote.cache, remote.uncache] fix min/max age  
							
							
 
							
							
							fix https://github.com/seaweedfs/seaweedfs/issues/5871  
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								ef4c555e90 
								
									
								
							
								 
							
						 
						
							
							
								
								[shell] volume.grow  ( #5855 )  
							
							
								
 
							
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								8378a5b70b 
								
							
								 
							
						 
						
							
							
								
								rename  
							
							
								
 
							
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								31b89c1062 
								
									
								
							
								 
							
						 
						
							
							
								
								fsck: only check the appendNs of deleted needle ( #5841 )  
							
							
 
							
							
							increase fsck speed
Co-authored-by: Yang Wang <yangwang@weride.ai> 
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								c1bffca246 
								
									
								
							
								 
							
						 
						
							
							
								
								fix evacuate volume to different disk types ( #5530 )  
							
							
 
							
							
							Co-authored-by: wyang <wyang@wyangs-Air.lan> 
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								aed2a18223 
								
							
								 
							
						 
						
							
							
								
								get, set and add path conf  
							
							
								
 
							
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								86d92a42b4 
								
									
								
							
								 
							
						 
						
							
							
								
								Added tls for http clients ( #5766 )  
							
							
 
							
							
							* Added global http client
* Added Do func for global http client
* Changed the code to use the global http client
* Fix http client in volume uploader
* Fixed pkg name
* Fixed http util funcs
* Fixed http client for bench_filer_upload
* Fixed http client for stress_filer_upload
* Fixed http client for filer_server_handlers_proxy
* Fixed http client for command_fs_merge_volumes
* Fixed http client for command_fs_merge_volumes and command_volume_fsck
* Fixed http client for s3api_server
* Added init global client for main funcs
* Rename global_client to client
* Changed:
- fixed NewHttpClient;
- added CheckIsHttpsClientEnabled func
- updated security.toml in scaffold
* Reduce the visibility of some functions in the util/http/client pkg
* Added the loadSecurityConfig function
* Use util.LoadSecurityConfiguration() in NewHttpClient func 
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								794a5aa67d 
								
									
								
							
								 
							
						 
						
							
							
								
								Add warning with hints to why a EC shard cant be moved ( #5790 )  
							
							
								
 
							
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								182f6d00a9 
								
							
								 
							
						 
						
							
							
								
								always copy ecj files  
							
							
								
 
							
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								7ee1f520a4 
								
									
								
							
								 
							
						 
						
							
							
								
								volume.tier.upload: Fix deleting replicated volumes ( #5743 )  
							
							
 
							
							
							Co-authored-by: Rhys M <rhysm@protonmail.com> 
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								5ffacbb6ea 
								
									
								
							
								 
							
						 
						
							
							
								
								refactor all methods strings to const ( #5726 )  
							
							
								
 
							
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								34165013c8 
								
									
								
							
								 
							
						 
						
							
							
								
								chore: hot fix panic fs.verify ( #5696 )  
							
							
								
 
							
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								7988ee0805 
								
									
								
							
								 
							
						 
						
							
							
								
								[fs.verify] skip failed files if entry not found on filerStore ( #5693 )  
							
							
								
 
							
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								ece018b1a9 
								
									
								
							
								 
							
						 
						
							
							
								
								[fs.verify] getmeta data from filer log ( #5678 )  
							
							
 
							
							
							* initial
* fix: print entry path 
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								8aae82dd71 
								
									
								
							
								 
							
						 
						
							
							
								
								Added context for the MasterClient's methods to avoid endless loops ( #5628 )  
							
							
 
							
							
							* Added context for the MasterClient's methods to avoid endless loops
* Returned WithClient function. Added WithClientCustomGetMaster function
* Hid unused ctx arguments
* Using a common context for the KeepConnectedToMaster and WaitUntilConnected functions
* Changed the context termination check in the tryConnectToMaster function
* Added a child context to the tryConnectToMaster function
* Added a common context for KeepConnectedToMaster and WaitUntilConnected functions in benchmark 
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								4744889973 
								
									
								
							
								 
							
						 
						
							
							
								
								fix issue: sometimes volume.fsck report 'volume not found' ( #5537 )  
							
							
 
							
							
							* fix issue: sometimes volume.fsck report 'volume not found' when a volume server has multiple disk types
* rename variable
* adjust counters
---------
Co-authored-by: chrislu <chris.lu@gmail.com> 
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								579ebbdf60 
								
									
								
							
								 
							
						 
						
							
							
								
								Support concurrent volume.fsck & support disabling -cutoffTimeAgo to improve speed ( #5636 )  
							
							
								
 
							
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								0c62d591e2 
								
									
								
							
								 
							
						 
						
							
							
								
								Ignore remote volume when selecting volumes in operation (ec.encode/volume.tier.upload) ( #5635 )  
							
							
								
 
							
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								e1c671c028 
								
									
								
							
								 
							
						 
						
							
							
								
								Fix volume.tier.upload nil pointer panic ( #5634 )  
							
							
								
 
							
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								72f7b4e9b1 
								
									
								
							
								 
							
						 
						
							
							
								
								Replaces volume.unmount in the help of the volumeServer.leave ( #5619 )  
							
							
								
 
							
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								cfcf383ee7 
								
									
								
							
								 
							
						 
						
							
							
								
								fix wrong prefix example  
							
							
								
 
							
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								a9b057389f 
								
									
								
							
								 
							
						 
						
							
							
								
								chore: add missing concurrency help in fs.meta.load command ( #5609 )  
							
							
								
 
							
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								d218fe54fa 
								
							
								 
							
						 
						
							
							
								
								go fmt  
							
							
								
 
							
							
						 
						1 year ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								6e4b9181f5 
								
									
								
							
								 
							
						 
						
							
							
								
								fix "volume.fix.replication" move many replications only to one volumeServer ( #5522 )  
							
							
								
 
							
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								7a72677f98 
								
							
								 
							
						 
						
							
							
								
								add test  
							
							
								
 
							
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								67ead9b18f 
								
									
								
							
								 
							
						 
						
							
							
								
								fix(volume.fix.replication): adjust volume count, not free volume count ( #5479 )  
							
							
								
 
							
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								ca042bd067 
								
							
								 
							
						 
						
							
							
								
								simplify  
							
							
								
 
							
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								4dce1e9d29 
								
							
								 
							
						 
						
							
							
								
								Revert "remove github.com/golang/protobuf/proto"  
							
							
 
							
							
							This reverts commit 1516e0249f 
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								1516e0249f 
								
							
								 
							
						 
						
							
							
								
								remove github.com/golang/protobuf/proto  
							
							
								
 
							
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								5c0f01c0eb 
								
							
								 
							
						 
						
							
							
								
								purge cluster id  
							
							
								
 
							
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								872b5161e6 
								
									
								
							
								 
							
						 
						
							
							
								
								[shell] fs log purge ( #5344 )  
							
							
 
							
							
							* fix: install cronie
* chore: shell delete old logs
* Revert "fix: install cronie"
This reverts commit 920832b04b 
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								6181aa7594 
								
									
								
							
								 
							
						 
						
							
							
								
								fix: fs verify error counter ( #5261 )  
							
							
								
 
							
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								fa72761bb8 
								
							
								 
							
						 
						
							
							
								
								fix on finding lock owner  
							
							
								
 
							
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								56287bd07d 
								
									
								
							
								 
							
						 
						
							
							
								
								weed/shell: Cluster check other disk types ( #5245 )  
							
							
 
							
							
							* week/shell: Cluster check other disk types
The `cluster.check` command only took the empty (`""`) and `hdd` disk types
into consideration, but a cluster with only `ssd` or `nvme` disk types would be
equally valid.
This commit simply checks that _any_ disk type is defined, and that some
volumes are available for it.
Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
* weed/shell: Replace loop that copies slice
Use the following construct instead of a `for` loop:
```golang
x = append(x, y...)
```
See https://staticcheck.dev/docs/checks#S1011 .
Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
* weed/shell: Check disk types when filer is in use
Filer stores its metadata logs in generic (i.e. `""`) or HDD disk type volumes,
so make sure those disk types exist and have volumes associated with them when
Filer is deployed in the cluster.
Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
---------
Signed-off-by: Benoît Knecht <bknecht@protonmail.ch> 
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								1f08010ef0 
								
									
								
							
								 
							
						 
						
							
							
								
								weed/shell: Cleanup volume balance logic ( #5241 )  
							
							
								
 
							
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								a6aee847b9 
								
									
								
							
								 
							
						 
						
							
							
								
								weed/shell: Fix volume.balance logic ( #5238 )  
							
							
								
 
							
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								34a78ffad0 
								
							
								 
							
						 
						
							
							
								
								remove isForPublish from LookupTopicBrokers  
							
							
 
							
							
							also adds a return parameter: whether the topic exists or not 
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								183352c796 
								
									
								
							
								 
							
						 
						
							
							
								
								shell meta load add concurrency ( #4529 )  
							
							
 
							
							
							* fix: increase speed cmd fs meta load
* fix: add wg 
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								97236389e8 
								
									
								
							
								 
							
						 
						
							
							
								
								Add modifyTimeAgo to volume.fsck ( #5133 )  
							
							
 
							
							
							* Add modifyTimeAgo to volume.fsck
* Fix AppendAtNs 
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								54ba2c8868 
								
									
								
							
								 
							
						 
						
							
							
								
								Fix cutoffTimeAgo in findMissingChunksInFiler ( #5132 )  
							
							
								
 
							
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								580940bf82 
								
									
								
							
								 
							
						 
						
							
							
								
								Merge accumulated changes related to message queue ( #5098 )  
							
							
 
							
							
							* balance partitions on brokers
* prepare topic partition first and then publish, move partition
* purge unused APIs
* clean up
* adjust logs
* add BalanceTopics() grpc API
* configure topic
* configure topic command
* refactor
* repair missing partitions
* sequence of operations to ensure ordering
* proto to close publishers and consumers
* rename file
* topic partition versioned by unixTimeNs
* create local topic partition
* close publishers
* randomize the client name
* wait until no publishers
* logs
* close stop publisher channel
* send last ack
* comments
* comment
* comments
* support list of brokers
* add cli options
* Update .gitignore
* logs
* return io.eof directly
* refactor
* optionally create topic
* refactoring
* detect consumer disconnection
* sub client wait for more messages
* subscribe by time stamp
* rename
* rename to sub_balancer
* rename
* adjust comments
* rename
* fix compilation
* rename
* rename
* SubscriberToSubCoordinator
* sticky rebalance
* go fmt
* add tests
* balance partitions on brokers
* prepare topic partition first and then publish, move partition
* purge unused APIs
* clean up
* adjust logs
* add BalanceTopics() grpc API
* configure topic
* configure topic command
* refactor
* repair missing partitions
* sequence of operations to ensure ordering
* proto to close publishers and consumers
* rename file
* topic partition versioned by unixTimeNs
* create local topic partition
* close publishers
* randomize the client name
* wait until no publishers
* logs
* close stop publisher channel
* send last ack
* comments
* comment
* comments
* support list of brokers
* add cli options
* Update .gitignore
* logs
* return io.eof directly
* refactor
* optionally create topic
* refactoring
* detect consumer disconnection
* sub client wait for more messages
* subscribe by time stamp
* rename
* rename to sub_balancer
* rename
* adjust comments
* rename
* fix compilation
* rename
* rename
* SubscriberToSubCoordinator
* sticky rebalance
* go fmt
* add tests
* tracking topic=>broker
* merge
* comment 
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								d818ca9631 
								
							
								 
							
						 
						
							
							
								
								fix TestMisplacedChecking() and add test-case  
							
							
								
 
							
							
						 
						2 years ago