00c621abb8 
								
									
								
							
								 
							
						 
						
							
							
								
								Fix dumb typo in  08556257 ( #6844 )  
							
							
								
 
							
							
						 
						5 months ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								bed0a64693 
								
									
								
							
								 
							
						 
						
							
							
								
								New `needle_map.CompactMap()` implementation for reduced memory usage ( #6842 )  
							
							
 
							
							
							* Rework `needle_map.CompactMap()` to maximize memory efficiency.
* Use a memory-efficient structure for `CompactMap` needle value entries.
This slightly complicates the code, but makes a **massive** difference
in memory efficiency - preliminary results show a ~30% reduction in
heap usage, with no measurable performance impact otherwise.
* Clean up type for `CompactMap` chunk IDs.
* Add a small comment description for `CompactMap()`.
* Add the old version of `CompactMap()` for comparison purposes. 
							
						 
						5 months ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								b25561d0d7 
								
							
								 
							
						 
						
							
							
								
								3.89  
							
							
								
 
							
							
						 
						5 months ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								d40746f34e 
								
									
								
							
								 
							
						 
						
							
							
								
								fix insert beyond look back window ( #6838 )  
							
							
								
 
							
							
						 
						5 months ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								7204731749 
								
									
								
							
								 
							
						 
						
							
							
								
								Minor fix for the `CompactMap()` performance test. ( #6836 )  
							
							
 
							
							
							Per-entry memory usage is based on `TotalAllocs`, which is incorrect - that
value is a cummulative of heap usage, which doesn't decrease when objects
are freeed.
`Allocs` is instead an accurate represeentation of actual memory usage
at the time metrics are reported. 
							
						 
						5 months ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								9ffc8bcb54 
								
									
								
							
								 
							
						 
						
							
							
								
								Further improve memory usage of `needle_map.CompactMap()`. ( #6825 )  
							
							
								
 
							
							
						 
						5 months ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								2e1506c31e 
								
									
								
							
								 
							
						 
						
							
							
								
								Rewrite `needle_map.CompactMap()` for more efficient memory usage ( #6813 )  
							
							
								
 
							
							
						 
						5 months ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								cb50b720fd 
								
							
								 
							
						 
						
							
							
								
								Revert "Fix weed fix"  
							
							
 
							
							
							This reverts commit 6c1ce18541 
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								81f11883e3 
								
							
								 
							
						 
						
							
							
								
								go fmt  
							
							
								
 
							
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								6c1ce18541 
								
							
								 
							
						 
						
							
							
								
								Fix weed fix  
							
							
								
 
							
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								bf8a9d2db1 
								
									
								
							
								 
							
						 
						
							
							
								
								[volume.chek.disk] sync of deletions the fix ( #3923 )  
							
							
 
							
							
							* sync of deletions the fix
* avoid return if only partiallyDeletedNeedles
* refactor sync deletions 
							
						 
						3 years ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								764d9cb105 
								
									
								
							
								 
							
						 
						
							
							
								
								[voluche.chek.disk] needles older than the cutoff time are not missing yet ( #3922 )  
							
							
 
							
							
							needles older than the cutoff time are not missing yet
https://github.com/seaweedfs/seaweedfs/issues/3919  
							
						 
						3 years ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								1f7e52c63e 
								
									
								
							
								 
							
						 
						
							
							
								
								vacuum metrics and force sync dst files ( #3832 )  
							
							
								
 
							
							
						 
						3 years ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								b7de4a967e 
								
									
								
							
								 
							
						 
						
							
							
								
								fix: compact_map get error mismatching cokie ( #3748 )  
							
							
 
							
							
							* fix: compact_map get error
* fix: CompactSection delete lock and move test to compact_map
Co-authored-by: shibinbin <shibinbin@megvii.com> 
							
						 
						3 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								21c0587900 
								
							
								 
							
						 
						
							
							
								
								go fmt  
							
							
								
 
							
							
						 
						3 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								26dbc6c905 
								
							
								 
							
						 
						
							
							
								
								move to  https://github.com/seaweedfs/seaweedfs  
							
							
								
 
							
							
						 
						3 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								ac694f0c8f 
								
							
								 
							
						 
						
							
							
								
								rename parameter and reuse functions  
							
							
 
							
							
							rename milestone to  watermark 
							
						 
						3 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								509a9047db 
								
							
								 
							
						 
						
							
							
								
								test compact map with snowflake sequencer  
							
							
								
 
							
							
						 
						3 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								625fd16a2e 
								
							
								 
							
						 
						
							
							
								
								reduce upfront memory usage for low density volume  
							
							
								
 
							
							
						 
						3 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								7ce97b59d8 
								
							
								 
							
						 
						
							
							
								
								go fmt  
							
							
								
 
							
							
						 
						4 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								24e11d1e90 
								
							
								 
							
						 
						
							
							
								
								look back when adding to sorted values  
							
							
 
							
							
							look back when adding to sorted values, before adding it to overflow 
							
						 
						4 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								fc8dd58aea 
								
							
								 
							
						 
						
							
							
								
								volume: large_volume version has bug when using in memory index  
							
							
 
							
							
							fix https://github.com/chrislusf/seaweedfs/issues/2162  
							
						 
						4 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								b465095db1 
								
							
								 
							
						 
						
							
							
								
								shell: add volume.check.disk to fix inconsistency for replicated volumes  
							
							
 
							
							
							fix https://github.com/chrislusf/seaweedfs/issues/1923  
							
						 
						5 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								f2723c1bc8 
								
							
								 
							
						 
						
							
							
								
								do not idx file format  
							
							
 
							
							
							revert c9ab8d05fa 
							
						 
						5 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								c9ab8d05fa 
								
							
								 
							
						 
						
							
							
								
								fixes for reading deleted fid  
							
							
								
 
							
							
						 
						5 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								99ecf63276 
								
							
								 
							
						 
						
							
							
								
								go fmt  
							
							
								
 
							
							
						 
						5 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								3b4b1d4a77 
								
							
								 
							
						 
						
							
							
								
								fix tests  
							
							
								
 
							
							
						 
						5 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								fe01191b5b 
								
							
								 
							
						 
						
							
							
								
								support read option readDeleted=true  
							
							
								
 
							
							
						 
						5 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								6ccd7f0a4d 
								
							
								 
							
						 
						
							
							
								
								refactoring  
							
							
								
 
							
							
						 
						5 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								ee11d98650 
								
							
								 
							
						 
						
							
							
								
								refactoring  
							
							
								
 
							
							
						 
						5 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								6a92f0bc7a 
								
							
								 
							
						 
						
							
							
								
								refactoring to typed Size  
							
							
 
							
							
							Go is amazing with refactoring! 
							
						 
						5 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								f2e8ad6241 
								
							
								 
							
						 
						
							
							
								
								refactoring  
							
							
								
 
							
							
						 
						5 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								0871d2cff0 
								
							
								 
							
						 
						
							
							
								
								volume: fix memory leak during compaction  
							
							
 
							
							
							fix https://github.com/chrislusf/seaweedfs/issues/1222  
							
						 
						6 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								acf7ca7b93 
								
							
								 
							
						 
						
							
							
								
								volume: fix compaction  
							
							
								
 
							
							
						 
						6 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								9ff72f616a 
								
							
								 
							
						 
						
							
							
								
								go fmt  
							
							
								
 
							
							
						 
						6 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								d8b39fe92a 
								
							
								 
							
						 
						
							
							
								
								testing  
							
							
								
 
							
							
						 
						6 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								abffe857a1 
								
							
								 
							
						 
						
							
							
								
								change btree map to in memory level db  
							
							
								
 
							
							
						 
						6 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								6fc6322c90 
								
							
								 
							
						 
						
							
							
								
								Change joeslay paths to chrislusf paths  
							
							
								
 
							
							
						 
						6 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								595a1beff0 
								
							
								 
							
						 
						
							
							
								
								Swap imports to use joeslay  
							
							
								
 
							
							
						 
						6 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								ec75b2d761 
								
							
								 
							
						 
						
							
							
								
								volume: fix bug with 8000GB version if using in memory index  
							
							
 
							
							
							fix https://github.com/chrislusf/seaweedfs/issues/994  
							
						 
						6 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								3f9ecee40f 
								
							
								 
							
						 
						
							
							
								
								working with reading remote intervals  
							
							
								
 
							
							
						 
						7 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								87f63b9c08 
								
							
								 
							
						 
						
							
							
								
								generate ec01~ec14, generate ecx file with sorted needle values  
							
							
								
 
							
							
						 
						7 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								ac2727853f 
								
							
								 
							
						 
						
							
							
								
								fix needle map entry size  
							
							
								
 
							
							
						 
						7 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								e5506152c0 
								
							
								 
							
						 
						
							
							
								
								refactoring  
							
							
								
 
							
							
						 
						7 years ago