Guo Lei
c57c79a0ab
optimiz commitig compact ( #3388 )
* optimiz vacuuming volume
* fix bugx
* rename parameters
* fix conflict
* change copyDataBasedOnIndexFile to an instance method
* close needlemap
* optimiz commiting Vacuum volume for leveldb index
* fix bugs
* fix leveldb loading bugs
* refactor
* fix leveldb loading bug
* add leveldb recovery
* add test case for levelDB
* modify test case to cover all the new branches
* use one tmpNm instead of two instances
* refactor
* refactor
* move setWatermark to the end
* add test for watermark and updating leveldb
* fix error logic
* refactor, add test
* check nil before close needlemapeer
add test case
fix metric bug
* add tests, fix bugs
* adjust log level
remove wrong test case
refactor
* avoid duplicate updating metric for leveldb index
2 years ago
Guo Lei
81e7e6ceb3
the statistical methord for fileCount should stay same ( #3477 )
the statistical methord for fileCount between should stay same
They all equal to the entry count in .idx file.
relate to commit(c7892bc
)
2 years ago
chrislu
26dbc6c905
move to https://github.com/seaweedfs/seaweedfs
2 years ago
guol-fnst
ac694f0c8f
rename parameter and reuse functions
rename milestone to watermark
2 years ago
Chris Lu
1ad3200094
skip seek() when index file writes
4 years ago
Chris Lu
f2723c1bc8
do not idx file format
revert c9ab8d05fa
4 years ago
Chris Lu
446e476a11
go fmt
4 years ago
Chris Lu
c9ab8d05fa
fixes for reading deleted fid
4 years ago
Chris Lu
6ccd7f0a4d
refactoring
4 years ago
Chris Lu
6a92f0bc7a
refactoring to typed Size
Go is amazing with refactoring!
4 years ago
Chris Lu
08e4702542
sync before closing index file
merge from 0181f87d91
5 years ago
Chris Lu
efd2f50ede
compaction changed to .idx based deletion
5 years ago
Chris Lu
abffe857a1
change btree map to in memory level db
5 years ago
j.laycock
6fc6322c90
Change joeslay paths to chrislusf paths
5 years ago
j.laycock
595a1beff0
Swap imports to use joeslay
5 years ago
Chris Lu
17ac1290c0
volume: load ec shards during heartbeats to master
6 years ago
Chris Lu
177f3add81
adjust memory layout to avoid possible nil pointer
6 years ago
Chris Lu
ac2727853f
fix needle map entry size
6 years ago
Chris Lu
e5506152c0
refactoring
6 years ago
Chris Lu
3e8a3a8fec
fix race detector found problems
6 years ago
Chris Lu
000ee725fc
refactor Offset into a struct of bytes
6 years ago
Chris Lu
7e2031b18f
go fmt
7 years ago
Chris Lu
d4d7ced922
refactoring: add type for needle id, offset
later the type size can possibly be adjusted
7 years ago
Chris Lu
aba1fe01b3
use constant NeedleIndexSize instead of the number 16
7 years ago
Chris Lu
458ada173e
go fmt
7 years ago
Chris Lu
1675243f29
maybe speed up a little when loading index
7 years ago
Chris Lu
82c3ccc8dd
add btree for volume index
8 years ago
Chris Lu
59022b6fe0
fix byte counter on loading index file
fix https://github.com/chrislusf/seaweedfs/issues/441
8 years ago
Chris Lu
13e7069eb9
keep track of total data file size
8 years ago
霍晓栋
096ffa9744
concurrent loading volume
8 years ago
Chris Lu
5ce6bbf076
directory structure change to work with glide
glide has its own requirements. My previous workaround caused me some
code checkin errors. Need to fix this.
9 years ago
chrislusf
86cd40fba8
Add "weed backup" command.
This is a pre-cursor for asynchronous replication.
10 years ago
Stuart P. Bentley
f0c2a2dcb3
Change all chrislusf/weed-fs links to point to chrislu/seaweedfs
10 years ago
chrislusf
020ba6c9a8
add leveldb support for needle map
This supposedly should reduce memory consumption. However, for tests
with millions of, this shows consuming more memories. Need to see
whether this will work out. If not, later boltdb will be tested.
10 years ago
chrislusf
9d8a6d2562
adjust log printing
10 years ago
Chris Lu
af416189f1
Cleanup error printing.
10 years ago
Brian McQueen
d56c748fa8
switch it back to chris's repo
10 years ago
Brian McQueen
c0dfdf4392
switch to my forked repo
10 years ago
Chris Lu
179d36ba0e
formatting code by: goimports -w=true .
10 years ago
wyy
4126280d55
use github.com/chrislusf instead of github.com/aszxqw
10 years ago
wyy
1cd19447e3
use github.com/aszxqw instead of code.google.com/p
10 years ago
Chris Lu
faf497feb8
Important Fix: Avoid overwriting the index file!!!
11 years ago
Chris Lu
3b5035c468
1. v0.54
2. go vet found many printing format errors
11 years ago
Chris Lu
51939efeac
1. volume server now sends master server its max file key, so that
master server does not need to store the sequence on disk any more
2. fix raft server's failure to init cluster during bootstrapping
11 years ago
Chris Lu
3dbebfd1e1
Thread-safe fixes:
1. avoid sharing []byte
2. switch to use ReadAt()
11 years ago
Chris Lu
cd10c277b2
can now delete a collection! Is this a dangerous feature? Only enabling
deleting "benchmark" collections for now.
11 years ago
Chris Lu
82b74c7940
issue 43 "go fmt" chagnes from "Ryan S. Brown" <sb@ryansb.com>
some basic changes to parse upload url
11 years ago
Chris Lu
ed154053c8
switching to temporarily use glog library
12 years ago
Chris Lu
50269b74ce
add dataCenter option when assign file keys
add dataCenter option when starting volume servers
some work related to freeze a volume. Not tested yet.
12 years ago
Chris Lu
fb635146a1
refactoring needle mapper interface to separate index file storage logic
out
12 years ago