Lisandro Pin
00c621abb8
Fix dumb typo in 08556257
( #6844 )
2 weeks ago
Lisandro Pin
bed0a64693
New `needle_map.CompactMap()` implementation for reduced memory usage ( #6842 )
* Rework `needle_map.CompactMap()` to maximize memory efficiency.
* Use a memory-efficient structure for `CompactMap` needle value entries.
This slightly complicates the code, but makes a **massive** difference
in memory efficiency - preliminary results show a ~30% reduction in
heap usage, with no measurable performance impact otherwise.
* Clean up type for `CompactMap` chunk IDs.
* Add a small comment description for `CompactMap()`.
* Add the old version of `CompactMap()` for comparison purposes.
2 weeks ago
Chris Lu
b25561d0d7
3.89
3 weeks ago
Chris Lu
d40746f34e
fix insert beyond look back window ( #6838 )
3 weeks ago
Lisandro Pin
7204731749
Minor fix for the `CompactMap()` performance test. ( #6836 )
Per-entry memory usage is based on `TotalAllocs`, which is incorrect - that
value is a cummulative of heap usage, which doesn't decrease when objects
are freeed.
`Allocs` is instead an accurate represeentation of actual memory usage
at the time metrics are reported.
3 weeks ago
Lisandro Pin
9ffc8bcb54
Further improve memory usage of `needle_map.CompactMap()`. ( #6825 )
4 weeks ago
Lisandro Pin
2e1506c31e
Rewrite `needle_map.CompactMap()` for more efficient memory usage ( #6813 )
4 weeks ago
chrislu
cb50b720fd
Revert "Fix weed fix"
This reverts commit 6c1ce18541
.
1 year ago
chrislu
81f11883e3
go fmt
2 years ago
SmsS4
6c1ce18541
Fix weed fix
2 years ago
Konstantin Lebedev
bf8a9d2db1
[volume.chek.disk] sync of deletions the fix ( #3923 )
* sync of deletions the fix
* avoid return if only partiallyDeletedNeedles
* refactor sync deletions
3 years ago
Konstantin Lebedev
764d9cb105
[voluche.chek.disk] needles older than the cutoff time are not missing yet ( #3922 )
needles older than the cutoff time are not missing yet
https://github.com/seaweedfs/seaweedfs/issues/3919
3 years ago
Konstantin Lebedev
1f7e52c63e
vacuum metrics and force sync dst files ( #3832 )
3 years ago
binbinshi
b7de4a967e
fix: compact_map get error mismatching cokie ( #3748 )
* fix: compact_map get error
* fix: CompactSection delete lock and move test to compact_map
Co-authored-by: shibinbin <shibinbin@megvii.com>
3 years ago
chrislu
21c0587900
go fmt
3 years ago
chrislu
26dbc6c905
move to https://github.com/seaweedfs/seaweedfs
3 years ago
guol-fnst
ac694f0c8f
rename parameter and reuse functions
rename milestone to watermark
3 years ago
chrislu
509a9047db
test compact map with snowflake sequencer
3 years ago
chrislu
625fd16a2e
reduce upfront memory usage for low density volume
3 years ago
Chris Lu
7ce97b59d8
go fmt
4 years ago
Chris Lu
24e11d1e90
look back when adding to sorted values
look back when adding to sorted values, before adding it to overflow
4 years ago
Chris Lu
fc8dd58aea
volume: large_volume version has bug when using in memory index
fix https://github.com/chrislusf/seaweedfs/issues/2162
4 years ago
Chris Lu
b465095db1
shell: add volume.check.disk to fix inconsistency for replicated volumes
fix https://github.com/chrislusf/seaweedfs/issues/1923
4 years ago
Chris Lu
f2723c1bc8
do not idx file format
revert c9ab8d05fa
5 years ago
Chris Lu
c9ab8d05fa
fixes for reading deleted fid
5 years ago
Chris Lu
99ecf63276
go fmt
5 years ago
Chris Lu
3b4b1d4a77
fix tests
5 years ago
Chris Lu
fe01191b5b
support read option readDeleted=true
5 years ago
Chris Lu
6ccd7f0a4d
refactoring
5 years ago
Chris Lu
ee11d98650
refactoring
5 years ago
Chris Lu
6a92f0bc7a
refactoring to typed Size
Go is amazing with refactoring!
5 years ago
Chris Lu
f2e8ad6241
refactoring
5 years ago
Chris Lu
0871d2cff0
volume: fix memory leak during compaction
fix https://github.com/chrislusf/seaweedfs/issues/1222
5 years ago
Chris Lu
acf7ca7b93
volume: fix compaction
6 years ago
Chris Lu
9ff72f616a
go fmt
6 years ago
Chris Lu
d8b39fe92a
testing
6 years ago
Chris Lu
abffe857a1
change btree map to in memory level db
6 years ago
j.laycock
6fc6322c90
Change joeslay paths to chrislusf paths
6 years ago
j.laycock
595a1beff0
Swap imports to use joeslay
6 years ago
Chris Lu
ec75b2d761
volume: fix bug with 8000GB version if using in memory index
fix https://github.com/chrislusf/seaweedfs/issues/994
6 years ago
Chris Lu
3f9ecee40f
working with reading remote intervals
6 years ago
Chris Lu
87f63b9c08
generate ec01~ec14, generate ecx file with sorted needle values
6 years ago
Chris Lu
ac2727853f
fix needle map entry size
6 years ago
Chris Lu
e5506152c0
refactoring
6 years ago