bed0a64693 
								
									
								
							
								 
							
						 
						
							
							
								
								New `needle_map.CompactMap()` implementation for reduced memory usage ( #6842 )  
							
							
 
							
							
							* Rework `needle_map.CompactMap()` to maximize memory efficiency.
* Use a memory-efficient structure for `CompactMap` needle value entries.
This slightly complicates the code, but makes a **massive** difference
in memory efficiency - preliminary results show a ~30% reduction in
heap usage, with no measurable performance impact otherwise.
* Clean up type for `CompactMap` chunk IDs.
* Add a small comment description for `CompactMap()`.
* Add the old version of `CompactMap()` for comparison purposes. 
							
						 
						5 months ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								d40746f34e 
								
									
								
							
								 
							
						 
						
							
							
								
								fix insert beyond look back window ( #6838 )  
							
							
								
 
							
							
						 
						5 months ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								9ffc8bcb54 
								
									
								
							
								 
							
						 
						
							
							
								
								Further improve memory usage of `needle_map.CompactMap()`. ( #6825 )  
							
							
								
 
							
							
						 
						5 months ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								2e1506c31e 
								
									
								
							
								 
							
						 
						
							
							
								
								Rewrite `needle_map.CompactMap()` for more efficient memory usage ( #6813 )  
							
							
								
 
							
							
						 
						5 months ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								81f11883e3 
								
							
								 
							
						 
						
							
							
								
								go fmt  
							
							
								
 
							
							
						 
						2 years ago  
				
					
						
							
							
								 
						
							
							
								
								
							
							
								
							
								b7de4a967e 
								
									
								
							
								 
							
						 
						
							
							
								
								fix: compact_map get error mismatching cokie ( #3748 )  
							
							
 
							
							
							* fix: compact_map get error
* fix: CompactSection delete lock and move test to compact_map
Co-authored-by: shibinbin <shibinbin@megvii.com> 
							
						 
						3 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								26dbc6c905 
								
							
								 
							
						 
						
							
							
								
								move to  https://github.com/seaweedfs/seaweedfs  
							
							
								
 
							
							
						 
						3 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								509a9047db 
								
							
								 
							
						 
						
							
							
								
								test compact map with snowflake sequencer  
							
							
								
 
							
							
						 
						3 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								3b4b1d4a77 
								
							
								 
							
						 
						
							
							
								
								fix tests  
							
							
								
 
							
							
						 
						5 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								6ccd7f0a4d 
								
							
								 
							
						 
						
							
							
								
								refactoring  
							
							
								
 
							
							
						 
						5 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								6a92f0bc7a 
								
							
								 
							
						 
						
							
							
								
								refactoring to typed Size  
							
							
 
							
							
							Go is amazing with refactoring! 
							
						 
						5 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								9ff72f616a 
								
							
								 
							
						 
						
							
							
								
								go fmt  
							
							
								
 
							
							
						 
						6 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								d8b39fe92a 
								
							
								 
							
						 
						
							
							
								
								testing  
							
							
								
 
							
							
						 
						6 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								6fc6322c90 
								
							
								 
							
						 
						
							
							
								
								Change joeslay paths to chrislusf paths  
							
							
								
 
							
							
						 
						6 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								595a1beff0 
								
							
								 
							
						 
						
							
							
								
								Swap imports to use joeslay  
							
							
								
 
							
							
						 
						6 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								87f63b9c08 
								
							
								 
							
						 
						
							
							
								
								generate ec01~ec14, generate ecx file with sorted needle values  
							
							
								
 
							
							
						 
						7 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								e5506152c0 
								
							
								 
							
						 
						
							
							
								
								refactoring  
							
							
								
 
							
							
						 
						7 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								7a4b234ea1 
								
							
								 
							
						 
						
							
							
								
								divide offset into higher and lower sections  
							
							
								
 
							
							
						 
						7 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								000ee725fc 
								
							
								 
							
						 
						
							
							
								
								refactor Offset into a struct of bytes  
							
							
								
 
							
							
						 
						7 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								09471b46f9 
								
							
								 
							
						 
						
							
							
								
								fix related tests  
							
							
								
 
							
							
						 
						7 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								36d13355bb 
								
							
								 
							
						 
						
							
							
								
								compact map fix bug when iterating  
							
							
								
 
							
							
						 
						7 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								42cb9b76c4 
								
							
								 
							
						 
						
							
							
								
								reduce needle map memory usage by 25%  
							
							
								
 
							
							
						 
						7 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								5741134ca0 
								
							
								 
							
						 
						
							
							
								
								change overflow from map to slice  
							
							
								
 
							
							
						 
						7 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								7e2031b18f 
								
							
								 
							
						 
						
							
							
								
								go fmt  
							
							
								
 
							
							
						 
						7 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								5b1fd374be 
								
							
								 
							
						 
						
							
							
								
								fix all tests  
							
							
								
 
							
							
						 
						7 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								d4d7ced922 
								
							
								 
							
						 
						
							
							
								
								refactoring: add type for needle id, offset  
							
							
 
							
							
							later the type size can possibly be adjusted 
							
						 
						7 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								82c3ccc8dd 
								
							
								 
							
						 
						
							
							
								
								add btree for volume index  
							
							
								
 
							
							
						 
						9 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								5ce6bbf076 
								
							
								 
							
						 
						
							
							
								
								directory structure change to work with glide  
							
							
 
							
							
							glide has its own requirements. My previous workaround caused me some
code checkin errors. Need to fix this. 
							
						 
						10 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								3185eebf2e 
								
							
								 
							
						 
						
							
							
								
								add test case for issue 52  
							
							
								
 
							
							
						 
						12 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								ed154053c8 
								
							
								 
							
						 
						
							
							
								
								switching to temporarily use glog library  
							
							
								
 
							
							
						 
						12 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								5071f528f6 
								
							
								 
							
						 
						
							
							
								
								testing compilation with remove package  
							
							
								
 
							
							
						 
						13 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								ab6fb13ad7 
								
							
								 
							
						 
						
							
							
								
								avoid the "src" folder  
							
							
								
 
							
							
						 
						13 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								cb4e8ec16b 
								
							
								 
							
						 
						
							
							
								
								re-organize code directory structure  
							
							
								
 
							
							
						 
						13 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								b0c7df0c3b 
								
							
								 
							
						 
						
							
							
								
								go fmt for all source codes  
							
							
								
 
							
							
						 
						13 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								bf9c4ed033 
								
							
								 
							
						 
						
							
							
								
								Revert "add cmd/dump - a dumper"  
							
							
 
							
							
							This reverts commit 5d2a1e8d48 
							
						 
						13 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								5d2a1e8d48 
								
							
								 
							
						 
						
							
							
								
								add cmd/dump - a dumper  
							
							
 
							
							
							Walk needed to be added to NeedleMap and CompactMap, to be able to add WalkKeys and WalkValues to volume. This is needed for iterating through all the stored needles in a volume - this was dump's purpose. 
							
						 
						13 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								6ce41e30a4 
								
							
								 
							
						 
						
							
							
								
								change to 3-digit replication types  
							
							
								
 
							
							
						 
						13 years ago  
				
					
						
							
							
								 
						
							
							
							
								
							
								2fe4371899 
								
							
								 
							
						 
						
							
							
								
								change to a more memory efficient map, implemented by several lists of  
							
							
 
							
							
							<key,offset,size> 
							
						 
						13 years ago