chrislu
72faae91e1
implement read directory and read directory plus
3 years ago
chrislu
a10c28ba82
simplify
3 years ago
chrislu
f4d88862c4
can attr root directory
3 years ago
chrislu
180445f5a8
change to use fuse file system
3 years ago
chrislu
45a0fda9bd
need to follow https://github.com/hanwen/go-fuse/blob/master/fuse/api.go
3 years ago
chrislu
f87da798a4
to be re-written following fuse virtual file system
3 years ago
chrislu
9a913457dd
supports stats
3 years ago
chrislu
b6143de52a
mount with name
3 years ago
chrislu
7a0c35674c
clean up previously mounted folder
3 years ago
chrislu
c3f9d9fa2e
initial setup
3 years ago
Radtoo
fbe7ed7927
Revert the the flag to "reallyDeleteFromVolume".
3 years ago
Chris Lu
0c1f42f4eb
Merge pull request #2654 from chrislusf/dependabot/maven/other/java/hdfs3/org.apache.hadoop-hadoop-common-3.1.4
Bump hadoop-common from 3.1.1 to 3.1.4 in /other/java/hdfs3
3 years ago
Chris Lu
439f0f834d
Merge pull request #2655 from chrislusf/dependabot/maven/other/java/hdfs-over-ftp/org.apache.hadoop-hadoop-common-3.2.2
Bump hadoop-common from 3.2.1 to 3.2.2 in /other/java/hdfs-over-ftp
3 years ago
Chris Lu
47c04db9b3
Merge pull request #2656 from chrislusf/dependabot/maven/other/java/hdfs2/org.apache.hadoop-hadoop-common-2.10.1
Bump hadoop-common from 2.9.2 to 2.10.1 in /other/java/hdfs2
3 years ago
Chris Lu
91876bf628
Merge pull request #2657 from chrislusf/dependabot/maven/other/java/examples/org.apache.hadoop-hadoop-common-2.10.1
Bump hadoop-common from 2.9.2 to 2.10.1 in /other/java/examples
3 years ago
dependabot[bot]
7a5a717063
Bump hadoop-common from 2.9.2 to 2.10.1 in /other/java/examples
Bumps hadoop-common from 2.9.2 to 2.10.1.
---
updated-dependencies:
- dependency-name: org.apache.hadoop:hadoop-common
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
3 years ago
dependabot[bot]
f955096e92
Bump hadoop-common from 2.9.2 to 2.10.1 in /other/java/hdfs2
Bumps hadoop-common from 2.9.2 to 2.10.1.
---
updated-dependencies:
- dependency-name: org.apache.hadoop:hadoop-common
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
3 years ago
dependabot[bot]
2065715712
Bump hadoop-common from 3.2.1 to 3.2.2 in /other/java/hdfs-over-ftp
Bumps hadoop-common from 3.2.1 to 3.2.2.
---
updated-dependencies:
- dependency-name: org.apache.hadoop:hadoop-common
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
3 years ago
dependabot[bot]
1236efb039
Bump hadoop-common from 3.1.1 to 3.1.4 in /other/java/hdfs3
Bumps hadoop-common from 3.1.1 to 3.1.4.
---
updated-dependencies:
- dependency-name: org.apache.hadoop:hadoop-common
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
3 years ago
chrislu
eb39f78396
rocksdb: PORTABLE=1
3 years ago
chrislu
13cb609f60
add filer.toml for rocksdb to docker image for rocksdb
3 years ago
chrislu
21aaa4c1f1
ec.encode: calculate free ec slots based on (maxVolumeCount-volumeCount)
fix https://github.com/chrislusf/seaweedfs/issues/2642
3 years ago
chrislu
13f6ec1c4e
test checking ec distribution
3 years ago
chrislu
f34c2ff7c5
use embed txt
3 years ago
chrislu
c8c7c10c3f
volume.tier.move: avoid double counting
related to https://github.com/chrislusf/seaweedfs/issues/2637
3 years ago
chrislu
0901438bd2
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
3 years ago
chrislu
f18803424a
volume.balance: add delay during tight loop
fix https://github.com/chrislusf/seaweedfs/issues/2637
3 years ago
Chris Lu
51a5d731ce
Merge pull request #2646 from kmlebedev/graceTimeout
graceful stop cluster http server default timeout 60 sec
3 years ago
Konstantin Lebedev
22a3ef56f2
graceful stop cluster http server default timeout 60 sec
https://github.com/chrislusf/seaweedfs/issues/2630
3 years ago
chrislu
9860405974
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
3 years ago
chrislu
85c1615b43
filer read empty file may cause OOM in some cases
fix https://github.com/chrislusf/seaweedfs/issues/2641
3 years ago
Chris Lu
a2ac540ecc
Merge pull request #2645 from guol-fnst/fix_Precedence
fix preconditions
3 years ago
root
7f0c793083
fix preconditions according to https://tools.ietf.org/id/draft-ietf-httpbis-p4-conditional-26.html#preconditions
3 years ago
chrislu
b1cff07ab0
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
3 years ago
chrislu
81f86c381d
s3: avoid nil response
fix https://github.com/chrislusf/seaweedfs/issues/2636
3 years ago
Chris Lu
f46763d74f
Merge pull request #2644 from kmlebedev/vacum_fsync
updated needle with fsync
3 years ago
Konstantin Lebedev
ef541972f8
updated needle with fsync
3 years ago
Chris Lu
1b7fb3ce84
Merge pull request #2643 from kmlebedev/graceful_volume_stop
3 years ago
Konstantin Lebedev
fb97e234c9
skips compact if store is stopping
3 years ago
chrislu
9405eaefdb
filer.sync: fix replicating partially updated file
Run two servers with volumes and fillers:
server -dir=Server1alpha -master.port=11000 -filer -filer.port=11001 -volume.port=11002
server -dir=Server1sigma -master.port=11006 -filer -filer.port=11007 -volume.port=11008
Run Active-Passive filler.sync:
filer.sync -a localhost:11007 -b localhost:11001 -isActivePassive
Upload file to 11007 port:
curl -F file=@/Desktop/9.xml "http://localhost:11007/testFacebook/ "
If we request a file on two servers now, everything will be correct, even if we add data to the file and upload it again:
curl "http://localhost:11007/testFacebook/9.xml "
EQUALS
curl "http://localhost:11001/testFacebook/9.xml "
However, if we change the already existing data in the file (for example, we change the first line in the file, reducing its length), then this file on the second server will not be valid and will not be equivalent to the first file
Снимок экрана 2022-02-07 в 14 21 11
This problem occurs on line 202 in the filer_sink.go file. In particular, this is due to incorrect mapping of chunk names in the DoMinusChunks function. The names of deletedChunks do not match the chunks of existingEntry.Chunks, since the first chunks come from another server and have a different addressing (name) compared to the addressing on the server where the file is being overwritten.
Deleted chunks are not actually deleted on the server to which the file is replicated.
3 years ago
Radtoo
fbb14e0ea8
Adding separate toggle to purge absent vols
While this toggle is basically required to clean out entries for deleted volumes, having a separate description + toggling this separately seems like a good idea so people get a chance to check if their volumes are all mounted/connected as expected.
Also renamed forcePurge to just purge.
3 years ago
Radtoo
6b17f45da2
Also delete paths for which a volume is entirely absent.
3 years ago
Radtoo
fa0cfdfb7f
Added basic deletion capabilities to fsck.
3 years ago
chrislu
433fde4b18
move error to a separate file
This file contains metric names for all errors
The naming convention is ErrorSomeThing = "error.some.thing"
3 years ago
Chris Lu
a23fcb9a7c
Merge pull request #2634 from kmlebedev/errorMetrics
error metrics for filer and store
3 years ago
Chris Lu
247bbabda5
Merge pull request #2632 from lapshin-vitaly/s3api_errors
add s3api error for copy in file, not directory
3 years ago
chrislu
ced3b89395
add util package
3 years ago
chrislu
affe3c2c12
change to util.WriteFile
3 years ago
chrislu
76e297d64f
sync call to write file, avoid vif loading error
fix https://github.com/chrislusf/seaweedfs/issues/2633
3 years ago
Chris Lu
d8e6bd0a94
Merge pull request #2635 from zerospiel/rearrange_handlers
3 years ago